The Public Health AI Handbook
Evaluating AI Tools for Public Health Practice
Welcome to The Public Health AI Handbook
AI tools often perform worse in the field than in development studies. Health departments, ministries of health, and public health agencies worldwide face the same challenge: separating tools that work from tools that were validated on clean data they will never have.
Three questions drive this handbook: Which AI tools actually perform in real-world public health settings? How do you evaluate a model when your data is messy, delayed, and incomplete? What happens when the algorithm is wrong and public health action follows?
This resource is continuously updated as new research emerges.
Important Disclaimers
This handbook is for educational and informational purposes only. It does not provide official public health guidance, operational protocols, or policy recommendations. It is not a substitute for guidance from public health authorities (CDC, WHO, state/local health departments).
Public health use remains context-specific. Officials should validate AI outputs before action, follow applicable privacy and data-governance requirements, and meet professional standards in their jurisdiction.
Information may become outdated as AI tools, evidence, and public health practice change. Verify recommendations with current public health guidelines and protocols before implementation.
This handbook does not provide legal or regulatory advice. Consult qualified legal counsel for questions about data governance, procurement, liability, and regulatory compliance.
Recent Updates
- 10 Oct Google’s Med-PaLM 2 evaluation: question-answering performance and clinical limits Read in the handbook
- 9 Oct HHS civil enforcement for 42 CFR Part 2 substance-use treatment records Read in the handbook
- 7 Oct OECD Expert-Led Madrid Action Plan for scaling AI in health Read in the handbook
- 6 Oct UK government accepts all 44 National Commission recommendations on healthcare AI Read in the handbook
- 5 Oct Rosenbaum: algorithm disclosure did not change flu-vaccine uptake Read in the handbook
Start Here
Read the evidence gaps first, then the core surveillance example.
- Executive Summary: Key findings and evidence gaps
- AI in Healthcare: A Brief History: Context for AI in public health
- Disease Surveillance and Outbreak Detection: Core AI application
Then continue to Evaluating AI Systems for Healthcare before adopting any tool.
For role-specific reading paths, see the Preface.
An AI assistant helping you should follow the Guidance for AI Agents.
Companion Handbooks
The Physician AI Handbook
Peer-Reviewed Evidence Across Every Medical Specialty
What works. What doesn’t. What happens when AI is wrong.
The Biosecurity Handbook
Biological Security in the AI Era
A guide for two communities that don’t speak the same language.
The Life Sciences AI Handbook
Evidence for AI in Biology and Drug Discovery
What works, what remains unproven, and what stands in the way.
Free to read and reuse under CC BY 4.0 · DOI 10.5281/zenodo.18263442 · How to cite
This handbook is open and continuously maintained. Support the Public Health AI Handbook →