Large Language Models in Public Health: Theory and Practice
Large language models can assist with synthesis and summarization, but unsupported output rates vary by model, task, prompt, source access, and scoring definition. Covered entities and business associates must not disclose protected health information to an LLM vendor unless HIPAA permits the disclosure and required safeguards are in place. A Business Associate Agreement is required when the vendor acts as a business associate. Consumer accounts should not receive PHI.
If you plan to use LLMs with ANY health-related data, read the Privacy and Security section BEFORE proceeding.
Never upload Protected Health Information (PHI) to consumer LLMs. HIPAA penalty tiers are adjusted and depend on the violation and culpability; do not use a fixed per-incident range as a decision rule (HHS HIPAA Enforcement). Even de-identified data may have residual privacy risks.
Key rule: Determine whether PHI is involved, whether the vendor is a business associate, whether a BAA and other safeguards are required, and whether organizational policy permits the use.
This chapter demystifies large language models for public health. You will learn to:
- Understand LLM technical foundations (tokenization, embeddings, transformers)
- Recognize LLM strengths (pattern completion) and failures (factual accuracy, reasoning)
- Apply LLMs to public health tasks (literature review, synthesis, protocols)
- Identify fundamental limitations (hallucination, lack of comprehension)
- Implement prompt engineering techniques for quality outputs
- Deploy validation strategies catching errors before propagation
- Navigate privacy requirements (never upload PHI to consumer LLMs)
- Select appropriate tools based on task, privacy, and cost
- Develop disciplined practices integrating LLMs without introducing risks
- Recognize when NOT to use LLMs (high-stakes decisions, precision requirements)
Prerequisites: Machine Learning Fundamentals.
This is an in-depth chapter covering both theory and practice. Choose your path:
For Practitioners (Practical focus): - Read: Introduction, Privacy & Security, Choosing Tools, Prompting, Use Cases - Skip or skim: Technical Foundations (or just read the key takeaway boxes) - Time: ~2-3 hours
For Administrators/Policy Makers: - Focus on: Privacy & Security, When NOT to Use LLMs, Organizational Implementation - Skim: Technical details and coding examples - Time: ~1.5-2 hours
For Technical Users & Researchers: - Read everything in sequence - Deep dive: Technical Foundations, Training Process, Advanced Prompting - Time: ~4-5 hours
All readers should read: - Introduction (The ChatGPT Moment) - Privacy and Security (non-negotiable) - When NOT to Use LLMs (critical boundaries) - Check Your Understanding (self-assessment)
This chapter builds on:
- Machine Learning Fundamentals (basic AI concepts)
- The Data Problem in Public Health AI (data quality, bias)
- Ethics, Bias, and Equity in Healthcare AI (ethical frameworks)
- Privacy, Security, and Governance for Health AI (HIPAA, GDPR basics)
You should be familiar with AI fundamentals, ethical considerations, and privacy frameworks.
What You’ll Learn
Large language models (LLMs) have transformed public health practice, from fundamental theory to practical implementation. Unlike specific AI techniques, LLMs represent a qualitative shift: natural language as a sufficient interface for powerful computation.
We cover how LLMs actually work (the technical foundations), what they can and cannot do (capabilities and limitations), and how to use them safely and effectively in your daily work while protecting privacy, ensuring accuracy, and maintaining professional standards.
We emphasize a safety-first approach: understanding constraints and risks before using capabilities. You’ll learn not just what LLMs can do, but critically, what they should not be used for in public health practice.
Demonstrated (supported by published evidence):
- LLMs accelerate literature review and synthesis tasks
- Prompt engineering significantly affects output quality
- Hallucination rates range from 3-27% depending on task and model
- Enterprise LLMs with BAAs can be HIPAA-compliant
- RAG systems reduce hallucination for document-grounded tasks
Theoretical (plausible but not yet demonstrated at scale):
- LLMs as clinical decision support (promising pilots, no large RCTs)
- Automated public health report generation without human review
- LLM-powered chatbots for patient triage (safety concerns unresolved)
- Reliable extraction of structured data from unstructured clinical notes
Beyond current capabilities:
- Zero observed hallucinations across all inputs has not been demonstrated
- LLMs that truly “understand” medical concepts (pattern matching, not reasoning)
- Safe autonomous LLM agents in clinical settings without human oversight
- Zero-shot accurate performance on novel medical domains
What to watch: Hallucination rates are improving with each model generation. Enterprise deployment infrastructure is maturing. Regulatory frameworks for LLM use in healthcare are emerging. Long-context models (1M+ tokens) enable new document analysis workflows.
Introduction: The ChatGPT Moment
November 30, 2022, 10:00 AM Pacific Time: OpenAI releases ChatGPT to the public. No announcement. No press release. Just a simple blog post and a free web interface.
December 5, 2022 (5 days later): 1 million users.
January 2023 (2 months later): 100 million users, the fastest-growing consumer application in history.
Unlike previous AI breakthroughs (expert systems in the 1980s, deep learning in the 2010s, even GPT-3 in 2020), ChatGPT was immediately accessible to everyone. No coding required. No technical expertise. No API keys. Just type in plain language and receive sophisticated responses.
December 1, 2022, Various Public Health Departments Worldwide:
An epidemiologist types: “Summarize the key evidence on airborne transmission of SARS-CoV-2”
Response arrives in 30 seconds. Structured. Complete. With caveats about evolving evidence.
A health educator types: “Translate this technical CDC guideline to 6th-grade reading level”
Response: Clear, accessible language. Maintains accuracy.
A biostatistician types: “Write Python code to calculate age-standardized mortality rates”
Response: Working code. With explanations.
The realization: AI had crossed a threshold. For the first time, natural language was a sufficient interface for powerful computation. You didn’t need to learn programming, master complex software, or understand algorithms. You could just… ask.
For public health practitioners, the implications were immediate:
Tasks that took hours: - Literature reviews - Report writing - Data interpretation - Health communication translation - Code generation
Now took minutes.
But also new risks: - Hallucinations (confidently stating false information) - Bias (reproducing societal prejudices from training data) - Privacy violations (entering sensitive data into commercial systems) - Over-reliance (using AI for critical decisions without verification) - Equity gaps (differential access to advanced vs. free tools)
The 2023-2025 Explosion
2023-2024 saw an unprecedented wave of releases:
March 2023: GPT-4 (OpenAI, 2023) - Multimodal capabilities, dramatically improved reasoning
July 2023: Claude 2 - 100,000 token context window enabling analysis of entire documents
September 2023: GPT-4V - Vision capabilities for medical images and charts
March 2024: Claude 3 family - Three models (Haiku, Sonnet, Opus) with leading performance
May 2024: GPT-4o - 2x faster, 50% cheaper, native multimodal
June 2024: Claude 3.5 Sonnet - Best reasoning performance to date
September 2024: OpenAI o1 - “Reasoning” model with step-by-step problem solving
December 2024: Gemini 2.0 Flash - Multimodal live interaction
2025 continued the rapid evolution:
January 2025: OpenAI o3-mini - Faster, cheaper reasoning model released to all ChatGPT users
March 2025: Gemini 2.5 Pro - Google’s most intelligent model with thinking capabilities and 1M token context window
April 2025: OpenAI o3 & o4-mini - Advanced reasoning models with agentic tool use across ChatGPT
June 2025: Gemini 2.5 Pro & Flash GA - General availability with Deep Think reasoning mode
July 2025: Grok 4 - xAI’s flagship model with 2M token context and real-time X/web search
August 2025: GPT-5 - OpenAI’s best system yet with unified routing and 94.6% AIME 2025 performance
August 2025: Claude Opus 4.1 - Anthropic’s most capable model in the Claude 4 series
September 2025: Grok 4 Fast - 40% reduction in thinking tokens, 98% cost decrease with frontier performance
September 2025: Claude Sonnet 4.5 - Flagship model with superior reasoning and coding capabilities
September 2025: DeepSeek V3.2-Exp - Sparse Attention architecture for improved efficiency
October 2025: Claude Haiku 4.5 - Fast, efficient model for high-volume multi-agent tasks
November 2025: GPT-5.1 - Adaptive reasoning with faster experiences and lower costs
November 2025: Grok 4.1 - 1483 Elo on LMArena, reduced hallucinations, improved emotional intelligence
November 2025: Gemini 3 Pro - 1501 Elo (LMArena #1), leading reasoning and multimodal understanding
November 2025: GPT-5.1-Codex-Max - First model natively trained for multi-context-window agentic coding
November 2025: Claude Opus 4.5 - Anthropic’s most intelligent model, leading agentic coding
December 2025: GPT-5.2 - 400K token context, 38% fewer errors than GPT-5.1, three tiers (Instant, Thinking, Pro)
The impact on public health:
Positive transformations: - Democratized access to sophisticated analysis tools - Reduced time for routine documentation tasks - Enabled rapid prototyping of automated systems - Lowered barriers to programming and data science - Improved accessibility of technical information
Concerning developments: - Risk of uncritical adoption without understanding limitations - Privacy concerns with sensitive health data - Hallucinations potentially affecting public health decisions - Widening capability gaps between well-resourced and resource-limited settings - Deskilling risks as practitioners rely on AI without developing expertise
LLMs are simultaneously: - Remarkably capable at synthesis, generation, and analysis - Fundamentally limited by hallucinations, biases, and lack of true reasoning
The challenge for public health: How do we harness their power while maintaining rigor, accuracy, and ethical practice?
This chapter addresses that question.
Population Health Information Seeking: The Scale of Consumer AI Usage
OpenAI “AI as a Healthcare Ally” Report (January 2026)
40 million people globally use ChatGPT daily for health questions, representing a fundamental shift in how populations seek health information (OpenAI, January 2026).
Usage Scale: - Over 5% of all ChatGPT messages are health-related - ~230 million people ask health-related questions weekly - 70% of health conversations occur outside traditional clinic hours
What People Ask About: - 55% use AI to understand symptoms - 52% for after-hours health access - 39% for medication guidance - 37% for mental health support
Underserved Communities:
AI is filling healthcare access gaps at the population level:
- Rural communities: ~600,000 healthcare messages weekly
- Hospital deserts (>30 min drive to hospital): ~580,000 messages weekly
- People without regular healthcare access: significant usage (exact figures vary by region)
The Public Health Implication:
This represents a parallel health information infrastructure operating outside traditional public health surveillance and clinical care systems. Population health practitioners must recognize that a significant portion of health information seeking now occurs via AI, with implications for:
- Health literacy interventions (need to address AI-generated information quality)
- Disease surveillance (symptoms discussed with AI may not reach healthcare providers)
- Health equity (differential access to advanced vs. free AI tools)
- Emergency preparedness (after-hours health questions surge during crises)
What Independent Platform Analyses Add
Microsoft Research analyzed over 500,000 de-identified Copilot health conversations from January 2026, providing the first large-scale, multi-platform complement to OpenAI’s self-reported data (Tolmachev, Costa-Gomes, & Sounderajah, 2026, Microsoft Research):
- Nearly 1 in 5 health conversations involve personal symptom assessment and condition management
- 1 in 7 personal health conversations are conducted on behalf of someone else (child, parent, partner)
- Emotional wellbeing queries rise from 3.4% during daytime to 5.2% at night; symptom interpretation also increases in evening hours
- Mobile users ask about symptoms twice as frequently as desktop users; emotional wellbeing queries are 75% more common on phones
Critical Note: Both OpenAI and Microsoft data come from company-disclosed metrics. Independent validation of these figures and their representativeness across different populations is pending. Consumer AI usage for health information is real and growing, but evidence hierarchy matters: company-disclosed platform metrics sit below peer-reviewed research on health outcomes.
When NOT to Use LLMs
Before we explore what LLMs CAN do, you must understand what they should NEVER be used for in public health practice.
Certain tasks are inappropriate for LLMs regardless of model quality, prompt engineering, or organizational safeguards:
NEVER use LLMs for:
1. Final clinical decision-making without human clinician oversight
Risk: Hallucinations could harm patients
Alternative: LLM as research aid, clinician decides
2. Real-time outbreak response decisions
Risk: Delays and errors during critical time-sensitive actions
Alternative: LLM for post-analysis, not emergency response
3. Legal or regulatory submissions without legal review
Risk: Hallucinated citations, incorrect legal interpretation
Alternative: LLM for drafting, lawyer reviews
4. Analyzing identifiable patient data on consumer platforms
Risk: HIPAA violation, privacy breach
Alternative: Enterprise LLM with BAA or complete de-identification
5. High-stakes statistical analysis without validation
Risk: Incorrect methods, calculation errors, misinterpretation
Alternative: LLM suggests approach, statistician implements
6. Automated decision-making without human review
Risk: Bias amplification, unexplainable errors
Alternative: Human-in-the-loop for all consequential decisions
7. Diagnosing medical conditions
Risk: Misdiagnosis, liability, practicing medicine without license
Alternative: Only licensed clinicians diagnose
8. Financial or budget decisions without verification
Risk: Calculation errors, incorrect assumptions
Alternative: LLM drafts, accountant verifies
9. Generating official public health statements without review
Risk: Misinformation, reputational damage
Alternative: LLM drafts, leadership approves
10. Tasks requiring 100% accuracy
Risk: LLMs have 3-27% hallucination rates
Alternative: Traditional methods with verificationThe rule of thumb: If you couldn’t evaluate whether the LLM’s output is correct, don’t use it for that task.
Transition: Now that you understand these critical boundaries, let’s explore how LLMs actually work. Understanding the technology helps you recognize both capabilities and limitations.
How Large Language Models Work
The detailed material is maintained in Large Language Model Foundations for Public Health. This section anchor remains here for continuity.
Choosing the Right LLM: A Decision Framework
The detailed material is maintained in Selecting and Using Large Language Models. This section anchor remains here for continuity.
Validation and Quality Control: Detecting Hallucinations
Static accuracy is not sufficient evidence of reliability. Dynamic health-LLM red-teaming has shown that initially correct answers can become incorrect under adaptive, domain-specific perturbations, while physician studies show that evaluator seniority and practice setting can change model ratings. Automated judges should therefore be tested for abstention, family preference, and agreement across evaluator strata. See the canonical evaluation evidence and controls for the supporting studies and implementation requirements.
The Hallucination Problem
LLMs generate plausible-sounding text without true understanding or fact-checking. They “hallucinate” - confidently state false information - at concerning rates. Studies report hallucination rates of 3-27% across different models and tasks, with medical and scientific queries particularly prone to errors (Ji et al., 2023 on survey of hallucination; Alkaissi & McFarlane, 2023 on medical hallucinations).
The WHO’s 2024 guidance on LMMs identifies a systemic risk beyond individual hallucinations: the potential for LMMs to “undermine human epistemic authority” in healthcare and science (WHO, 2024).
The concern: When LLMs provide authoritative-seeming responses at scale, users may gradually shift from treating LLM outputs as suggestions to treating them as reliable sources of knowledge. This erodes:
- Clinical expertise: Practitioners may defer to LLM outputs rather than applying clinical judgment
- Scientific skepticism: Researchers may accept LLM-generated claims without verification
- Patient autonomy: Patients may trust LLM health advice over qualified professionals
Mitigation: Maintain explicit awareness that LLMs are pattern-completion engines, not knowledge bases. Every LLM output requires human validation before becoming the basis for decisions.
Common Hallucination Types
1. Fabricated citations
LLM output: "A 2023 study in The Lancet (Johnson et al., 2023;401:1847-1854)
found that..."
Reality: No such article exists
Verification: Search PubMed, check journal table of contents2. Incorrect statistics
LLM output: "Measles vaccine effectiveness is 97% after one dose"
Reality: Effectiveness is ~93% after one dose, 97% after two doses
Verification: Check CDC Pink Book, primary studies3. Outdated information presented as current
LLM output: "Current WHO recommendation for malaria treatment is..."
Reality: Recommendation updated 6 months ago (after LLM training cutoff)
Verification: Check current WHO guidelines directly4. Overgeneralization from limited data
LLM output: "Studies show intervention X is effective in all populations"
Reality: Studies primarily in high-income countries; effectiveness unclear elsewhere
Verification: Examine geographic and demographic diversity of evidence base5. Nonsensical outputs that sound plausible
LLM output: "The R0 of this outbreak is 2.3, indicating exponential decay"
Reality: R0 > 1 indicates exponential growth, not decay (logical error)
Verification: Domain expertise recognizes contradictionVerification Strategies
Strategy 1: Citation Checking
Every factual claim should have a verifiable source:
Workflow:
1. LLM provides output with citations
2. For each citation, check:
☐ Does the article/source exist?
☐ Are authors and year correct?
☐ Does the source actually say what's claimed?
☐ Is the source credible (peer-reviewed, authoritative)?
☐ Is the information current and applicable?
Tools:
- PubMed: biomedical literature
- Google Scholar: broad academic search
- DOI lookup: Digital Object Identifier resolution
- Journal websites: verify article details
- Preprint servers: bioRxiv, medRxiv (note: not peer-reviewed)Strategy 3: Logical Consistency Checks
Does the output make sense?
Red flags:
- Internal contradictions (claims A and B cannot both be true)
- Implausible numbers (110% vaccine effectiveness, negative disease incidence)
- Incorrect units (mixing prevalence and incidence terminology)
- Temporal impossibilities (2024 study cited before 2024)
- Methodological nonsense ("double-blind retrospective cohort study")
Example:
LLM: "The outbreak had 50 cases with a case fatality rate of 5%, resulting
in 10 deaths"
Check: 5% of 50 = 2.5, not 10 → Math error, investigate furtherStrategy 4: Code Execution and Testing
For LLM-generated code:
1. Read code carefully before running (malicious code rare but possible)
2. Test on small sample/synthetic data first
3. Verify outputs against manual calculations
4. Check for errors/warnings
5. Review logic (does approach make sense?)
6. Test edge cases (empty data, missing values, outliers)
Example workflow:
LLM generates R code to calculate disease incidence rates
→ Run on 10-row test dataset with known answer
→ Verify output matches expected result
→ Test with edge cases (zero population, missing data)
→ If all tests pass, apply to full dataset
→ Spot-check random samples from full resultsLLMs can generate insecure code. Always: - Review for hardcoded credentials or sensitive data - Check for SQL injection vulnerabilities - Verify file path security - Test input validation - Have security-minded review for production use
Strategy 5: Subject Matter Expert Review
For consequential decisions, always involve domain experts:
LLM Role: Research assistant, draft generator, idea catalyst
Human Expert Role: Verification, interpretation, decision-making
Workflow:
1. LLM generates analysis/recommendations
2. Epidemiologist/SME reviews for:
- Scientific accuracy
- Appropriate methodology
- Contextual appropriateness
- Practical feasibility
- Ethical considerations
3. Expert modifies/approves/rejects output
4. Expert takes responsibility for final decision
NEVER: Use LLM output without expert review for high-stakes decisionsRed Flags Checklist
When reviewing LLM outputs, be suspicious if:
Content red flags:
☐ Very specific statistics without sources
☐ Multiple citations from same year/journal (may be fabricated batch)
☐ Overly confident language ("definitely," "always," "never")
☐ Lack of nuance or caveats (real science has uncertainty)
☐ Too good to be true (perfect solution to complex problem)
☐ Recent developments (post LLM training cutoff) presented as fact
☐ Detailed quotes without clear sources
☐ Consensus where you know controversy exists
Technical red flags:
☐ Statistical tests with exact p-values (p=0.0234) for data you provided
(LLM didn't actually run tests, may hallucinate values)
☐ Code that doesn't run or produces errors
☐ Methodological impossibilities
☐ Mixing of incompatible methods or frameworks
Style red flags:
☐ Repetitive phrasing (may indicate training data patterns)
☐ Sudden topic shifts (attention wandering)
☐ Overly generic descriptions (lacks specific detail)
☐ Inconsistent terminologyTransition: Now that you know how to validate LLM outputs, let’s explore practical workflows for common public health tasks.
Practical Use Cases and Workflows
The WHO’s 2024 LMM guidance provides a structured framework for assessing LMM applications in health, identifying potential benefits alongside risks for each use category (WHO, 2024):
| Use Category | Potential Benefits | Key Risks |
|---|---|---|
| Diagnosis/clinical care | Assist complex cases, reduce documentation burden | Inaccurate responses, automation bias, skills degradation |
| Patient-guided use | Virtual health assistant, improve health literacy | Manipulation, privacy violations, reduced clinician interaction |
| Clerical/administrative | Documentation, translation, EHR completion | Inconsistent responses, errors in records |
| Education | Dynamic learning materials, simulated scenarios | May reinforce errors, undermine critical thinking |
| Research | Literature synthesis, manuscript drafting | Cannot be held accountable, may fabricate citations |
This framework should inform your assessment of any proposed LLM use case in public health practice.
Use Case 1: Literature Review and Evidence Synthesis
Scenario: Summarize evidence on effectiveness of community health worker interventions for maternal health in low-resource settings.
Workflow:
Step 1: Initial Search (Use Perplexity AI or Claude with search)
Prompt: "Find peer-reviewed systematic reviews and meta-analyses on community
health worker interventions for maternal health outcomes in low and middle-income
countries, published 2019-2024. Provide: author, year, journal, key findings,
sample size, and PMID."
Output: List of 10-15 studies with details
Action: Verify each PMID in PubMed
Step 2: Deep Dive on Key Studies (Use Claude for long context)
Prompt: "I'm pasting 5 systematic review abstracts [paste]. For each, extract:
1. Specific interventions evaluated
2. Outcomes measured
3. Effect sizes (with confidence intervals)
4. Quality of evidence (GRADE rating if provided)
5. Applicability to Sub-Saharan Africa
Then synthesize: What interventions show strongest evidence?"
Output: Detailed extraction and synthesis
Action: Spot-check against original papers
Step 3: Gap Analysis
Prompt: "Based on this evidence synthesis, what are the major research gaps?
What populations, interventions, or outcomes have insufficient evidence?
What are methodological limitations across studies?"
Output: Gap analysis
Action: Review for reasonableness
Step 4: Practice Implications
Prompt: "Given this evidence, what are 5 key recommendations for a health
ministry planning to scale community health worker programs? Consider:
strength of evidence, implementation feasibility, cost-effectiveness, equity."
Output: Practice recommendations
Action: Validate with program managers
Time: ~2-3 hours (vs 2-3 days manually)
Quality: Comparable if citations verified; faster iterationUse Case 2: Survey Data Analysis (Qualitative)
Scenario: Analyze 500 open-ended responses about barriers to vaccination.
Workflow:
Step 1: Data Preparation
- De-identify: Remove names, locations, personally identifying details
- Format: Plain text, one response per line or numbered list
- Sampling: If >500 responses, may analyze sample (but note limitation)
Step 2: Initial Thematic Analysis (Use ChatGPT or Claude)
Prompt: "You are analyzing survey responses about vaccination barriers.
TASK: Identify major themes, sub-themes, and frequency.
RESPONSES: [paste de-identified responses]
ANALYSIS:
1. Read all responses
2. Identify 5-8 major themes
3. For each theme:
- Define the theme clearly
- Identify 2-3 sub-themes
- Estimate % of responses mentioning this theme
- Provide 3 representative quotes
4. Note any surprising or unexpected findings
FORMAT: Markdown with clear sections"
Output: Thematic analysis
Action: Review sample of responses manually to validate themes
Step 3: Deeper Analysis of Priority Theme
Prompt: "Focus on the 'Access barriers' theme you identified.
1. What specific access issues did respondents mention?
2. Are there demographic patterns? (if demographic data available)
3. Which barriers are most amenable to intervention?
4. What solutions did respondents suggest (if any)?"
Output: Detailed analysis of one theme
Action: Validate against policy options
Step 4: Visualization and Reporting
Prompt: "Create a summary table:
| Theme | Frequency | Key Sub-themes | Representative Quote | Intervention Opportunity |
Then draft 2 paragraphs summarizing key findings for a report to leadership."
Output: Table and summary
Action: Edit for tone and audience; add context
Time: ~1-2 hours (vs 1-2 days manually)
Quality: Good for initial analysis; human should review subset
Limitation: May miss nuanced cultural meaningsUse Case 3: Code Generation for Data Analysis
Scenario: Create R code to visualize disease trends over time, stratified by demographic groups.
Workflow:
Step 1: Describe Data and Goal
Prompt: "Write R code (using ggplot2) to visualize disease incidence trends.
DATA STRUCTURE:
- CSV file with columns: date, age_group, race_ethnicity, case_count, population
- Date range: 2019-2024
- Age groups: 0-17, 18-44, 45-64, 65+
- Race/ethnicity: White, Black, Hispanic, Asian/PI, Other
- Weekly data
GOAL: Create 2 visualizations:
1. Overall trend: Line plot of incidence rate over time
2. Stratified trends: Small multiples (faceted) by age and race/ethnicity
REQUIREMENTS:
- Calculate incidence rate per 100,000 population
- Use appropriate theme (theme_minimal)
- Clear labels and titles
- Color-blind friendly palette
- Save as high-resolution PNG
Provide complete, runnable code with comments."
Output: R code
Action: Review code for logic, test on sample data
Step 2: Code Execution
# Run the code in R/RStudio on test data first
Step 3: Debugging (if errors)
Prompt: "I'm getting this error: [paste error message]
Here's my data structure: [paste str(data) output]
Please fix the code."
Output: Revised code
Action: Test again
Step 4: Refinement
Prompt: "The plot works but I want to:
1. Add a smooth trend line (LOESS)
2. Highlight pandemic period (2020-2021) with shaded region
3. Adjust y-axis to start at 0
4. Make facet labels more readable
Update the code."
Output: Enhanced code
Action: Test and iterate
Time: ~30-60 minutes (vs 2-3 hours coding from scratch)
Quality: Usually good for standard visualizations; may need debugging
Benefit: Especially valuable for those less comfortable with codingTransition: Individual use of LLMs is one thing, but how should organizations implement these tools at scale? Let’s explore organizational governance.
Organizational Implementation: Policies and Governance
Developing an LLM Usage Policy
Organizations should establish clear policies before widespread LLM adoption. Key components:
1. Scope and Applicability
Define:
- Which tools are approved for use (ChatGPT Enterprise, Claude Team, etc.)
- Which tools are prohibited (consumer versions without BAA)
- Who the policy applies to (all staff, specific roles)
- Which use cases are covered (analysis, writing, research)2. Privacy and Data Protection
Requirements:
- Never upload PHI to non-HIPAA-compliant LLMs
- De-identify data before using consumer LLMs (even then, exercise caution)
- Use enterprise LLMs with BAAs for any health-related data
- No personally identifiable information in prompts
- Obtain approval before uploading organizational proprietary data
- Document what data was shared with which LLM3. Acceptable Use Cases
Approved:
- Literature review and research (with citation verification)
- Drafting communications (with human review)
- Data analysis code generation (with testing)
- Learning and skill development
- Administrative tasks (meeting summaries, scheduling)
Prohibited:
- Final clinical decision-making without human clinician
- Uploading identified patient data to consumer LLMs
- Automated decision-making without human review
- Generating official statements without approval
- Real-time emergency response4. Quality Control and Validation
Requirements:
- Verify all factual claims and citations
- Have subject matter experts review technical content
- Test all generated code before production use
- Document when LLMs were used in work products
- Maintain human accountability for all decisions5. Training Requirements
All staff using LLMs must complete:
- Data privacy and HIPAA compliance training
- Effective prompting techniques
- Hallucination detection and verification
- Appropriate use cases and limitations
- Security awareness (prompt injection, etc.)
Frequency: Initial + annual refresher
Assessment: Quiz or practical exercise6. Accountability and Oversight
Establish:
- Designated LLM governance committee or officer
- Incident reporting process for privacy breaches or errors
- Regular audits of LLM usage
- Feedback mechanism for improving policies
- Clear escalation path for questions or concerns7. System Prompt Governance
System prompts are hidden instructions that define LLM behavior before user interaction begins. Unlike user prompts (which practitioners control), system prompts are configured at the deployment layer and invisible to end users.
Why this matters for public health: A Mount Sinai study tested 20 LLMs across 5 million clinical decisions and found that assigning different “physician personas” (ethical orientation crossed with cognitive style) shifted affirmative action rates from 36.9% to 46.4% under identical clinical evidence. This 9.5 percentage-point swing in treatment recommendations occurred without any change in underlying clinical facts (Klang et al., 2026, preprint).
System Prompt Governance Requirements:
- Document all system prompts used in deployed LLM tools
- Require governance review before modifying production prompts
- Test persona configurations for unintended decision shifts
- Audit system prompt changes as you would policy changes
- Ensure transparency about AI "personality" in clinical contextsFor organizations deploying LLM-based tools (chatbots, decision support), system prompt configuration should be treated as clinical policy, not technical implementation.
Sample Policy Template for Public Health Organizations
Steps to Adapt for Your Organization:
- Replace bracketed placeholders with your organization’s information
- Identify governance committee members from privacy, IT, clinical, legal, and programmatic areas
- Select and procure approved enterprise LLMs with Business Associate Agreements
- Develop training materials based on this chapter’s content
- Create reporting workflows integrated with existing incident response
- Pilot with small group (10-20 staff) for 30 days, gather feedback
- Refine policy based on pilot experience
- Roll out organization-wide with mandatory training
- Monitor compliance through periodic audits
- Update quarterly as technology and best practices evolve
Common Customization Needs:
- State/local health departments: Add state-specific privacy laws, public records requirements
- Clinical settings: Emphasize medical device regulations, clinical decision support standards
- Academic institutions: Address research ethics, IRB considerations, student use
- Small organizations (<50 staff): Simplify governance to single oversight officer
- International organizations: Add GDPR, local data protection laws
Policy Communication:
- All-staff email announcement from leadership
- Mandatory training session (60-90 minutes)
- Quick reference card (1-page summary)
- Regular reminders (quarterly)
- New hire onboarding inclusion
Summary and Key Takeaways
Large language models offer significant potential for public health practice when used responsibly. This chapter emphasized understanding both technical foundations and practical implementation, with a safety-first approach: understanding risks and limitations before using capabilities.
Core Principles
Understand the technology: LLMs use tokenization, embeddings, and transformer architectures. Pre-training is followed by variable post-training pipelines that may combine demonstrations, human or AI preference data, synthetic responses, safety data, and verifiable rewards. These choices create different capabilities and limitations.
Privacy is non-negotiable: Never upload PHI to consumer LLMs. Enterprise use still requires a permitted disclosure, a BAA when the vendor is a business associate, technical safeguards, and organizational approval.
Always verify consequential outputs: Unsupported output rates vary by system and task. Citation checking, authoritative source review, and domain review are essential.
Match tool to task: Different LLMs excel at different tasks. Choose based on requirements (context length, multimodal capabilities, code generation, real-time information access, etc.).
Prompt engineering matters: Prompt structure can change output quality, but effects are model- and task-specific. Use context, clear tasks, constraints, output specifications, and examples, then evaluate the result.
Human expertise remains essential: LLMs are assistants, not replacements. Domain experts must review, interpret, and take responsibility for decisions.
Organizational governance: Establish clear policies on approved tools, data protection, acceptable uses, quality control, and training before widespread adoption.
Recognize when NOT to use LLMs: Clinical decisions, real-time emergency response, tasks requiring 100% accuracy, and sensitive identifiable data are inappropriate for LLM use.
Looking Ahead
As LLM capabilities continue to advance, public health practitioners must maintain vigilance about appropriate use. The tools will become more powerful, but core principles remain:
- Protect privacy above all else
- Verify everything - trust but verify
- Keep humans in the loop for consequential decisions
- Stay informed about evolving best practices
The future of public health practice will increasingly involve AI assistance, but human judgment, ethical reasoning, and professional accountability cannot be delegated to algorithms.
Emerging AI Architectures: Beyond Text-Only LLMs
The detailed material is maintained in Emerging AI Architectures for Public Health. This section anchor remains here for continuity.
Check Your Understanding
The detailed material is maintained in Large Language Model Exercises for Public Health. This section anchor remains here for continuity.
Further Resources
This section organizes resources by user type and learning goal. Choose the category that best matches your needs.
For Getting Started (Beginners)
If you’re new to LLMs and want practical guidance:
- DeepLearning.AI: ChatGPT Prompt Engineering for Developers - Free 1-hour course by Andrew Ng. Excellent starting point, no coding required for most content.
- Learn Prompting - Interactive tutorials on prompting techniques. Start here if you learn by doing.
- OpenAI Platform Documentation - Official docs with clear examples and best practices.
- Claude Prompt Library - Ready-to-use prompts for common tasks that you can adapt.
For Privacy & Compliance (Administrators)
If you need to understand legal and regulatory requirements:
- HHS HIPAA Guidance - Official HIPAA compliance guidance
- HHS AI and HIPAA - Specific guidance on AI and protected health information
- OCR Breach Portal - Database of reported breaches (learn from others’ mistakes)
- GDPR Official Text - EU data protection regulation (for international work)
- WHO Ethics and Governance of AI for Health - Policy framework for responsible AI use
For Technical Deep Dives (Researchers & Developers)
If you want to understand how LLMs work under the hood:
Foundational Papers: - Vaswani et al., 2017: Attention Is All You Need - The original transformer paper that started it all - Brown et al., 2020: Language Models are Few-Shot Learners (GPT-3) - Landmark paper on scaling language models - Ouyang et al., 2022: Training language models to follow instructions (InstructGPT) - How RLHF works - Wei et al., 2022: Chain-of-Thought Prompting - Improving reasoning with step-by-step prompts
Accessible Explanations: - The Illustrated Transformer by Jay Alammar - Visual, intuitive explanation of transformers - LLM Visualization - Interactive visualization of how LLMs generate text
For Public Health Applications (Practitioners)
LLMs in healthcare and public health:
- Singhal et al., 2023: Large language models encode clinical knowledge (Med-PaLM) (Singhal et al., 2023) - Landmark study on medical LLMs
- Thirunavukarasu et al., 2023: Large language models in medicine - Thorough review in Nature Medicine
- Ayers et al., 2023: Comparing Physician and AI Chatbot Responses - Quality comparison study
- CDC AI Strategy - Public health AI initiatives and guidance
- CDC GenAI Considerations - STLT guidance for generative AI adoption
- NAM: AI in Health Care: Hope, Hype, Promise, Peril (2022) - National Academy of Medicine report on AI in health care
- NAM: Generative AI in Health and Medicine (2025) - Opportunities, risks, and implementation readiness for LLMs in clinical and public health settings
- NAM: AI Code of Conduct for Health and Medicine (2025) - Six commitments and ten principles for responsible health AI
Limitations and Risks: - Ji et al., 2023: Survey of Hallucination in NLG - Detailed review of hallucination problem - Alkaissi & McFarlane, 2023: Artificial Hallucinations in ChatGPT - Essential reading on hallucinations in medical contexts - Obermeyer et al., 2019: Dissecting racial bias in an algorithm (Obermeyer et al., 2019) - Landmark study on bias in health algorithms - Weidinger et al., 2021: Ethical and social risks of harm from Language Models - Detailed risk taxonomy
LLM Tools and Platforms
Commercial LLMs (Consumer & Enterprise): - OpenAI ChatGPT - GPT-4o, o1 models. Free and paid tiers. - Anthropic Claude - Claude 3.5 Sonnet. Longer context windows. - Google Gemini - Multimodal capabilities, massive context. - xAI Grok - Real-time X/Twitter data access. - Microsoft Copilot - Integrated with Office 365. - Perplexity AI - Web search integration, citations.
Open-Source LLMs: - Meta Llama - Open weights, can run locally - Mistral AI - European open-source LLM - DeepSeek - Strong coding capabilities - Ollama - Tool for running LLMs locally - Hugging Face - Model hub and inference API
Specialized Medical LLMs: - Med-PaLM 2 - Google Health’s medical LLM - BioBERT - Biomedical text mining - PubMedBERT - Microsoft biomedical model
API Platforms (For Developers)
- OpenAI API - GPT models, well-documented
- Anthropic API - Claude access
- Google Vertex AI - Gemini and other models
- Azure OpenAI Service - Enterprise GPT deployment
- Together AI - Open model hosting
- Hugging Face Inference API - Access to many models
Online Courses & Tutorials
Free Courses: - DeepLearning.AI: ChatGPT Prompt Engineering - 1 hour, by Andrew Ng - Learn Prompting - Interactive, self-paced - Fast.ai: Practical Deep Learning - Technical but accessible
Paid Courses: - Coursera: Generative AI for Everyone - Non-technical overview by Andrew Ng - Coursera: Generative AI Specialization - More technical series
Policy & Governance Resources
For organizations implementing LLM policies:
- WHO: Ethics and Governance of AI for Health - Complete policy framework
- NIH AI Policy - Research guidelines
- UK NHS AI Lab - National health system AI implementation
- FDA AI/ML-Based Software as a Medical Device (SaMD) Action Plan - Regulatory framework
Community & Discussion
- LMSys Chatbot Arena - Compare different LLMs side-by-side, see benchmark rankings
- r/LocalLLaMA - Community for running LLMs locally
- Alignment Forum - AI safety and alignment discussions
- APHA AI Working Group - Public health AI community (check for current status)
Prompt Engineering Tools
- OpenAI Playground - Experiment with prompts, adjust parameters
- Anthropic Workbench - Claude prompt development environment
- PromptPerfect - Automatic prompt optimization
- LangChain - Framework for building LLM applications
Staying Current
LLM development moves fast. Stay informed:
- Import AI Newsletter by Jack Clark - Weekly AI news
- The Batch by DeepLearning.AI - Weekly AI news
- Last Week in AI - Curated AI news
- Papers with Code - Latest research with code implementations
- Hugging Face Daily Papers - Trending ML research
Books
For deeper understanding:
- “The Alignment Problem” by Brian Christian - AI safety and ethics (accessible, narrative)
- “Life 3.0” by Max Tegmark - Future of AI (philosophical, accessible)
- “Artificial Intelligence: A Guide for Thinking Humans” by Melanie Mitchell - AI fundamentals (technical but accessible)
Model System Cards & Documentation
Technical details on specific models:
- GPT-4 System Card - Detailed capabilities and limitations
- Claude Constitutional AI - Safety approach explanation
- Gemini Technical Report - Architecture and capabilities
Health Communication Resources
For translating technical content:
- CDC Clear Communication Index - Health communication best practices
- Plain Language Guidelines - Federal plain language guidance
- Hemingway Editor - Readability checking tool
- Readable - Advanced readability metrics
Week 1: Foundations 1. Watch DeepLearning.AI prompt engineering course (1 hour) 2. Read HHS HIPAA and AI guidance (1 hour) 3. Experiment with free ChatGPT or Claude (2 hours)
Week 2: Practice 1. Try all 6 prompting techniques from this chapter on a real task 2. Complete “Check Your Understanding” questions 3. Draft a use case relevant to your work
Week 3: Deep Dive 1. Read 3 papers from “Public Health Applications” section 2. Review your organization’s data governance policies 3. Identify one workflow where LLMs could assist (with proper safeguards)
Week 4: Implementation 1. Draft an organizational LLM usage policy 2. Conduct training with colleagues 3. Pilot one use case with full verification workflow
This concludes the in-depth chapter on Large Language Models in Public Health: Theory and Practice.