Appendix J — Frequently Asked Questions
Frequently Asked Questions
This page answers common questions about AI in public health, drawn from the handbook’s chapters and appendices. For detailed treatment of any topic, see the referenced chapters.
Quick navigation:
Getting Started
What is the role of AI in disease surveillance?
AI enhances disease surveillance through anomaly detection algorithms like CDC’s EARS and spatial-temporal clustering tools like SaTScan. Systems like BlueDot detected COVID-19 nine days before WHO’s announcement.
However, AI supplements rather than replaces traditional surveillance. Google Flu Trends failed by overestimating flu by 140% due to spurious correlations, demonstrating that AI must complement epidemiologist expertise, not replace it.
→ See: AI in Disease Surveillance and Outbreak Detection
What machine learning algorithm should I start with for public health data?
Start with logistic regression. It’s interpretable, works with modest data, and epidemiologists already understand it.
Progression:
- Logistic Regression - Your default starting point
- Random Forests - Best first “real” ML model
- Gradient Boosting (XGBoost) - Maximum accuracy for tabular data
- Deep Learning - Only for images, text, or truly complex patterns
For most public health spreadsheet data, gradient boosting beats neural networks. Save deep learning for medical imaging and clinical notes.
→ See: Machine Learning Fundamentals
Will AI replace epidemiologists?
No. AI augments epidemiological expertise but cannot replace it.
- Google Flu Trends failed because it lacked epidemiologist input
- ProMED-mail (human curators) succeeded where pure algorithmic surveillance failed
- Algorithms detect signals, but humans determine public health action
Feature engineering based on domain expertise outperforms algorithm sophistication. Epidemiologists have an advantage over pure data scientists.
→ See: AI in Healthcare: A Brief History
How do I get started with my first public health AI project?
Start small with a well-defined problem where you have clean data and clear success metrics.
Setup:
- Install Python with Anaconda
- Learn pandas for data manipulation
- Learn scikit-learn for modeling
- Begin with logistic regression on a problem you understand deeply
Focus on understanding the problem before optimizing the algorithm.
→ See: Building Your First Public Health AI Project
What programming language should I learn for public health AI?
Python is the dominant language for AI/ML with the richest ecosystem (scikit-learn, pandas, TensorFlow, PyTorch).
R remains strong for statistical analysis and visualization, especially in epidemiology.
Learn Python first if starting from scratch. If you already know R, you can do substantial ML work with tidymodels and caret. SQL is essential for data extraction.
→ See: Your AI Toolkit for Public Health
Evaluation & Validation
How do I evaluate an AI tool before deploying it in my health department?
Require external validation on your population with your data quality.
Checklist:
The Epic Sepsis Model was deployed in over 100 hospitals but had only 33% sensitivity. Deployment ≠ effectiveness.
→ See: Evaluating AI Systems for Healthcare, Appendix G: Vendor Evaluation
How accurate does an AI model need to be for public health use?
Accuracy alone is misleading. With imbalanced datasets common in public health (rare diseases, outbreak detection), a model predicting “no outbreak” 99% of the time could be 99% accurate but useless.
Focus on:
- Sensitivity - Catching real cases
- Specificity - Avoiding false alarms
- Positive Predictive Value - When you alert, are you right?
- Calibration - Do 80% confidence predictions occur 80% of the time?
Context determines which metric matters most.
→ See: Evaluating AI Systems for Healthcare
What are the biggest red flags when evaluating AI vendors?
- ❌ Internal validation only with no external studies
- ❌ “98% accuracy” on cherry-picked test sets
- ❌ “Deployed in over 150 hospitals” without outcome data
- ❌ Proprietary validation with no peer-reviewed publications
- ❌ “We don’t use race so it’s fair”
- ❌ Vague privacy claims without BAA documentation
Ask for external validation data, subgroup performance, and references from similar institutions.
→ See: Appendix G: Vendor Evaluation, Appendix E: AI Failures
Why do most AI projects in healthcare fail?
Technical excellence ≠ clinical adoption. The hardest problems deploying medical AI are legal, regulatory, social, and organizational, not algorithmic.
Common failure modes:
- Overfitting to training data
- Lack of external validation
- Poor workflow integration
- Alert fatigue from false positives
- Trust barriers from black box predictions
- Liability concerns
MYCIN matched infectious disease experts in the 1970s but was never used clinically due to these non-technical barriers.
→ See: Appendix E: AI Failures
Data & Technical Issues
What data quality do I need before building an AI model?
80% of effort in machine learning is data preparation, 20% is modeling.
Before building, check:
| Dimension | Question |
|---|---|
| Completeness | What’s missing and why? |
| Consistency | Same concepts coded differently? |
| Temporal Validity | Is data current? |
| Representativeness | Does training match deployment? |
| Label Quality | How was ground truth established? |
Most project failures trace back to data issues, not algorithm choices.
→ See: The Data Problem in Public Health AI
How much data do I need to train an AI model?
It depends on problem complexity and algorithm choice:
| Algorithm | Minimum Data |
|---|---|
| Logistic Regression | Hundreds |
| Random Forests | Thousands |
| Deep Learning | Tens of thousands to millions |
For rare events, you may need oversampling or synthetic data generation. A smaller, clean dataset often outperforms a larger, noisy one.
How do I handle missing data in public health datasets?
Missing data is rarely random in public health. Sicker patients often have more incomplete records.
- First: Understand why data is missing (MCAR, MAR, MNAR)
- For limited missingness: Consider multiple imputation
- For substantial missingness: Treat as informative; include missingness indicators as features
- Never: Simply delete rows unless missingness is truly random
Document your approach and test sensitivity to assumptions.
What is data drift and why does it matter for AI?
Data drift occurs when statistical properties of input data change over time, causing model performance to degrade.
COVID-19 broke nearly all surveillance and forecasting models because distributions shifted dramatically. AI models are not “set and forget”.
Requirements:
- Continuous monitoring
- Periodic retraining
- Recalibration procedures before deployment
→ See: AI Deployment in Healthcare
Can I use ChatGPT or other LLMs for public health work?
LLMs can assist with:
- Literature review
- Code generation
- Drafting communications
- Brainstorming
LLMs are not reliable for clinical decision-making or situations requiring factual accuracy. They hallucinate and lack domain expertise verification.
Rules:
- Use as assistants, not authorities
- Always verify outputs
- Never input patient-identifiable information without proper agreements
→ See: Large Language Models in Public Health
Ethics & Governance
How do I avoid algorithmic bias in public health AI?
Algorithmic bias creates systematic unfair outcomes. The OPTUM/UnitedHealth algorithm affected millions by using healthcare costs as a proxy for health needs, disadvantaging Black patients.
Prevention:
- Test performance across demographic subgroups
- Audit for proxy discrimination
- Involve diverse stakeholders in design
- Monitor deployed systems for disparate impact
“We don’t use race so it’s fair” is a red flag.
→ See: Ethics, Bias, and Equity in Healthcare AI
What privacy regulations apply to AI in public health?
| Regulation | Scope |
|---|---|
| HIPAA | Protected health information (US) |
| 42 CFR Part 2 | Substance use disorder records (US) |
| GDPR | Personal data (Europe) |
| State laws | May add requirements |
Key requirements:
- Minimum necessary data use
- Proper data use agreements
- De-identification where possible
- Transparency about AI decision-making
Cloud-based AI services require Business Associate Agreements.
→ See: Privacy, Security, and Governance for Health AI
What is the FDA’s role in regulating AI for health?
The FDA regulates AI/ML-based Software as a Medical Device (SaMD) through:
- 510(k) clearance
- De Novo classification
- Premarket Approval
Most diagnostic AI requires FDA clearance before clinical use. Public health applications (surveillance, forecasting) generally fall outside FDA jurisdiction unless they make individual clinical recommendations.
→ See: AI Policy and Governance in Healthcare
How do I explain AI predictions to public health decision-makers?
Focus on what the model does, not how it works technically.
Good: “The model identifies patterns similar to past outbreaks”
Avoid: “The gradient boosting ensemble optimizes log-loss”
Best practices:
- Provide uncertainty estimates, not just point predictions
- Use visualizations showing model reasoning
- Be transparent about limitations
- If you can’t explain why the model made a prediction, that’s a warning sign
Career & Learning
How do I build a career in public health AI?
Combine domain expertise with technical skills.
If you have public health training:
- Add programming (Python)
- Add statistics and machine learning
If you have computer science background:
- Add epidemiology coursework
- Add biostatistics
Key skills:
- Data manipulation (pandas, SQL)
- Machine learning (scikit-learn)
- Visualization
- Communication
Build a portfolio with real projects. The field needs people who understand both the public health problems and the technical solutions.
→ See: Appendix I: Career Guide
What’s the difference between outbreak detection and epidemic forecasting?
| Aspect | Outbreak Detection | Epidemic Forecasting |
|---|---|---|
| Question | Is there an outbreak now? | How big will it get? |
| Methods | Anomaly detection, surveillance | Time series, compartmental models |
| Difficulty | Hard | Harder |
| Reliability | Better | Lower (human behavior changes predictions) |
Short-term forecasts (1-4 weeks) are more reliable than long-term projections.
→ See: AI in Disease Surveillance, Epidemic Forecasting with AI
This FAQ covers the most common questions. For deeper exploration:
- Browse chapters for detailed coverage
- Check appendices for checklists and templates
- See the glossary for term definitions