Appendix K — Frequently Asked Questions
Public health AI decisions depend on intended use, evidence, local context, and the consequences of error. These answers provide practical starting points and link readers to the handbook’s detailed evaluation, implementation, and governance guidance.
After using this FAQ, readers should be able to:
- distinguish model performance from demonstrated public health benefit;
- identify the evidence and local testing required before deployment;
- avoid fixed algorithm, sample-size, and accuracy rules that ignore context; and
- recognize when privacy, equity, and human oversight require specialist review.
Surveillance and Evaluation
What role can AI play in disease surveillance?
AI can help prioritize signals, classify reports, detect anomalies, and combine information from multiple sources. It supplements rather than replaces established surveillance, laboratory confirmation, epidemiologic investigation, and public health judgment. BlueDot notified clients about the Wuhan outbreak on December 31, 2019, the same date WHO’s China Country Office identified the Wuhan notice; this should not be described as nine days before WHO knew of the outbreak.
Source: WHO’s five-year account of the first outbreak notice.
How should a health department evaluate an AI tool before deployment?
Define the intended use, users, population, workflow, comparator, and decision consequences first. Then require independent evidence, local validation, calibration, subgroup performance, security and privacy review, human-oversight procedures, failure-mode testing, and a prospective monitoring and retirement plan. A high AUC or a long customer list does not establish local effectiveness. See Evaluating AI Systems for Healthcare.
Which machine learning algorithm should a public health team start with?
Start with a simple, interpretable baseline appropriate to the outcome, often logistic or linear regression. Compare more complex models only when the task, data, and expected operational benefit justify them. Choose the final approach using validated discrimination, calibration, transportability, interpretability, maintenance burden, and workflow fit, not an assumption that one algorithm is universally best.
How accurate must an AI model be for public health use?
There is no universal accuracy threshold. The acceptable operating point depends on prevalence, consequences of false positives and false negatives, available response capacity, the comparator, and the model’s role in the workflow. Evaluate sensitivity, specificity, predictive values, calibration, subgroup performance, decision impact, and patient or population outcomes where applicable. See Performance Metrics for Public Health AI.
How much data is needed to train a public health AI model?
No fixed sample-size rule applies across models. Required data depend on outcome frequency, feature count, model complexity, label quality, clustering, missingness, intended subgroup analyses, and the precision needed for performance estimates. Use an analysis-specific sample-size or learning-curve assessment and reserve independent data for validation.
Equity, Language Models, and Privacy
How can algorithmic bias be assessed before deployment?
Assess whether the target, labels, proxies, sampling, missingness, access patterns, and workflow encode inequity. Report performance and calibration across relevant groups, examine allocation and downstream outcomes, involve affected communities, and test mitigations against the specific harm. Omitting protected attributes does not by itself prevent discrimination. See Ethics, Bias, and Equity.
Can public health teams use large language models for their work?
Large language models can assist with bounded tasks such as drafting, code explanation, document classification, and retrieval-supported summarization. Their outputs require verification against authoritative sources. Do not enter protected or sensitive data into an unapproved service, and do not use an unvalidated model as an autonomous source of clinical, legal, or public health decisions. See Large Language Models in Public Health.
Which privacy rules apply to AI in public health?
The answer depends on the organization, data, jurisdiction, program, and purpose. HIPAA applies to protected health information handled by covered entities and business associates; 42 CFR Part 2 can apply to certain substance use disorder records; state privacy, public health, research, employment, consumer, and cybersecurity laws may also apply. Conduct a role-specific legal and privacy review rather than assuming one rule covers every public health dataset. See Privacy, Security, and Governance.