AI Deployment in Healthcare: Why Most Prototypes Fail
Moving an AI prototype into routine service requires more than a validated model. Production systems depend on data pipelines, integration, monitoring, versioning, governance, and incident response. Models can lose performance when populations, workflows, documentation, or outcome relationships change. Workflow fit and organizational readiness therefore deserve the same scrutiny as algorithm performance.
This chapter addresses why AI prototypes fail to reach production. You will learn to:
- Apply MLOps principles (containerization, CI/CD pipelines, automated testing)
- Implement monitoring for data drift and concept drift
- Develop maintenance workflows (retraining, versioning, A/B testing)
- Navigate health IT integration (EHRs, FHIR APIs, HL7 standards)
- Understand regulatory requirements (FDA SaMD, post-market surveillance)
- Execute safe deployment strategies (blue-green, canary releases, rollbacks)
- Assess organizational readiness (training, change management, stakeholder buy-in)
- Recognize why technical excellence alone fails (Epic sepsis vs. retinopathy screening)
- Understand how workflow integration determines AI value vs. risk
Prerequisites: Evaluating AI Systems, Ethics, Bias, and Equity, Privacy, Security, and Governance, AI Safety: Protecting Patients and Populations.
Introduction: The Deployment Gap
Why Most AI Projects Fail
Gartner reported in a 2021 cross-industry survey that 53% of surveyed AI projects moved from prototype to production. The survey does not establish a healthcare or public health deployment rate.
The reality:
“Machine learning is 5% ML algorithms and 95% data engineering, infrastructure, monitoring, and maintenance.” , Common industry observation, validated by Sculley et al., 2015, NIPS
Effort categories for production ML:
- Model development: Data preparation, training, and validation
- Deployment and integration: Interfaces, workflow testing, security, and release controls
- Monitoring and maintenance: Performance surveillance, incident response, updates, and retirement
Estimate each category for the intended environment. There is no transferable percentage allocation across organizations.
Why Deployment Is Hard in Public Health
1. Regulatory Requirements
FDA Software as Medical Device (SaMD) guidance creates significant compliance burden: - Pre-market clearance (510(k), De Novo, PMA) - Clinical validation requirements - Post-market surveillance obligations - Change control for algorithm updates
2. Integration Complexity
Healthcare IT infrastructure is notoriously fragmented: - Heterogeneous EHR products and locally configured implementations - Legacy systems with proprietary interfaces - Multiple data standards (HL7 v2, HL7 v3, FHIR, DICOM, X12) - Inconsistent data quality and completeness
3. Reliability Demands
Healthcare tolerance for downtime approaches zero: - Downtime costs and clinical consequences vary by function, redundancy, duration, and recovery process - Clinical decisions cannot wait for system recovery - Patient safety depends on system availability
4. Audit Requirements
Regulatory and legal requirements mandate complete traceability: - Every prediction must be logged - Audit trails for all data access - Explainability for clinical decisions - Ability to reconstruct historical predictions
5. Model Decay
Population health patterns change over time: - Wong et al., 2021, JAMA Internal Medicine - Epic Sepsis Model performed substantially worse than vendor-reported (AUC 0.63 vs 0.76-0.83) - Davis et al., 2017, JAMIA - Clinical prediction models lose calibration over time (AKI cohort) - Finlayson et al., 2021, NEJM - Dataset shift causes AI models to fail across sites
The Cost of Poor Deployment
Epic Sepsis Model Controversy (2021):
Wong et al., 2021, JAMA Internal Medicine revealed critical failures in one of healthcare’s most widely deployed AI systems. Epic’s Sepsis Model (ESM) achieved only 33% sensitivity (missing 2 in 3 sepsis cases) with a 12% positive predictive value (approximately 7 false alarms per true case).
Key deployment failures: - Inadequate external validation before wide deployment - Poor generalization across different hospital sites - Insufficient monitoring post-deployment - No mechanism to detect performance degradation - Alert burden not considered in deployment planning
Impact: - Clinicians developed alert fatigue, ignoring warnings - Delayed treatment for missed cases (false negatives) - Loss of trust in AI-based clinical decision support - Regulatory and media scrutiny
This case exemplifies a safety failure, not merely a deployment problem. For detailed analysis of what went wrong from a safety engineering perspective, see AI Safety in Healthcare, which uses Epic’s sepsis model as a central case study in hazard analysis and failure mode prevention.
The deployment lesson: Technical validation (Evaluating AI Systems) + ethical review (Ethics, Bias, and Equity) + privacy protection (Privacy and Security) + safety validation (AI Safety) must ALL succeed before deployment proceeds. What follows is the “how” of deploying AI that has passed these prerequisites.
Cost-Effectiveness Analysis: Before You Build
Before committing resources to AI implementation, health department directors face a fundamental question: Is this investment worth it? A systematic review of 19 studies across oncology, cardiology, ophthalmology, and infectious diseases found that AI interventions can improve diagnostic accuracy, enhance quality-adjusted life years, and reduce costs by minimizing unnecessary procedures (El Arab & Al Moosa, 2025, npj Digital Medicine). However, the same review warns that economic benefits may be overstated because indirect costs, infrastructure investments, and equity considerations are often underreported.
Total Cost of Ownership Framework
Organizations that fail to account for comprehensive costs risk budget overruns of 30 to 40 percent within the first year. A complete Total Cost of Ownership (TCO) analysis must include:
Direct Costs:
| Category | Typical Range | Notes |
|---|---|---|
| Software licensing | $50,000 to $500,000 | Ready-made tools; custom development adds 30 to 40 percent |
| Infrastructure (annual) | $20,000 to $60,000 | Cloud computing, data storage, processing |
| Initial implementation | $100,000 to $200,000 | For mid-sized health departments |
| Validation studies | $5,000 to $500,000 | Depends on complexity and regulatory pathway |
Indirect Costs:
| Category | Typical Range | Notes |
|---|---|---|
| Staff training | $10,000 to $25,000 upfront | Plus ongoing education |
| Workflow disruption | 20 to 30 percent productivity loss | During 3 to 6 month integration period |
| Change management | $30,000 to $75,000 | Often underestimated |
| Maintenance (annual) | $30,000 to $50,000 | Updates, monitoring, retraining |
Hidden Costs (frequently omitted):
- Data pipeline optimization (one healthcare system found 63% of expenses here)
- GPU cluster management and cloud computing overages
- Vendor lock-in and license renewals
- Data migration and interoperability challenges
- Reputational risk mitigation if models produce unsafe recommendations
The CHEERS-AI Framework
The CHEERS-AI checklist (Consolidated Health Economic Evaluation Reporting Standards for AI), released in 2024 and endorsed by ISPOR, provides 38 reporting items for economic evaluations of AI interventions (Elvidge et al., 2024, Value in Health). Ten items are AI-specific, covering:
- How the AI affects clinical care (diagnosing, treating, informing management)
- Validation methodology and performance metrics
- How AI learning occurs over time (static vs. adaptive models)
- AI-specific sources of uncertainty
- Implementation requirements
When evaluating vendor proposals or internal business cases, require alignment with CHEERS-AI standards.
ROI Timeline Expectations
Return on investment varies significantly by AI complexity:
| AI Type | Typical ROI Timeline | Example Applications |
|---|---|---|
| Simple automation (RPA, chatbots) | 6 to 12 months | Appointment scheduling, FAQ responses |
| Diagnostic decision support | 12 to 18 months | Image triage, risk scoring |
| Complex predictive models | 18 to 24 months | Outbreak forecasting, resource optimization |
| Generative AI applications | 24+ months | Report generation, clinical documentation |
Cost stabilization typically occurs after 18 to 24 months, when initial implementation costs decline and optimization gains materialize.
The Critical Decision: AI vs. Traditional Approaches
When deciding between AI investment and traditional staffing (for example, $200K for AI vs. $200K for additional epidemiologists), evaluate both options against the same criteria:
| Criterion | AI Solution | Traditional Staffing |
|---|---|---|
| Evidence of effectiveness | Peer-reviewed validation studies? External validation? | Established methods with known performance? |
| Cost per case detected | Include full TCO, not just licensing | Include salary, benefits, training, turnover |
| Scalability | Can handle volume increases without proportional cost? | Linear cost scaling with volume |
| Time to impact | 12 to 24 month implementation timeline | Immediate impact after hiring/onboarding |
| Sustainability | Requires ongoing technical maintenance | Requires ongoing supervision and retention |
| Equity implications | Bias risks across populations? | Human judgment biases? |
| Flexibility | Adapts to changing requirements? | Staff can pivot to new priorities |
Neither option is inherently superior. The right choice depends on specific context: outbreak response may need immediate human expertise; high-volume screening may benefit from AI scalability.
Cost-Effectiveness Calculation Template
For health departments considering AI investments, document these elements:
INVESTMENT CASE: [AI System Name]
===============================
1. PROBLEM STATEMENT
Current process inefficiency: [quantify]
Burden (cases, hours, costs): [baseline metrics]
Unmet need AI would address: [specific gap]
2. COST ANALYSIS (5-YEAR TCO)
Year 1 (Implementation + licensing): $______
Years 2 through 5 (Maintenance + updates): $______ x 4 = $______
Infrastructure (5 years): $______
Training and change management: $______
Locally justified contingency buffer: $______
TOTAL 5-YEAR TCO: $______
3. BENEFIT ANALYSIS
Time savings: ______ hours/year x $____/hour = $______
Error reduction: ______ cases x $____/case = $______
Capacity increase: ______ additional cases x $____/case = $______
TOTAL 5-YEAR BENEFITS: $______
4. COMPARISON METRICS
Net Present Value (5% discount): $______
Return on Investment: ______%
Payback Period: ______ months
Cost per case improvement: $______
5. NON-FINANCIAL CONSIDERATIONS
Equity impact: [positive/neutral/negative + explanation]
Staff acceptance: [high/medium/low + mitigation plan]
Regulatory pathway: [510(k)/De Novo/exempt + timeline]
Vendor lock-in risk: [high/medium/low]
6. ALTERNATIVE COMPARISON
Option B (traditional approach): 5-year cost = $______
Incremental cost-effectiveness ratio: $______/[outcome unit]When NOT to Invest in AI
AI investment is inappropriate when:
- Problem is poorly defined: “Make surveillance better” is not actionable
- Data infrastructure is inadequate: No EHR integration, unreliable data quality
- Staffing cannot support implementation: No data scientists, no IT capacity
- Regulatory pathway is unclear: High-risk application with uncertain FDA requirements
- Existing solutions are adequate: Traditional methods meet performance needs
- Timeline is unrealistic: Expecting 6-month deployment for complex systems
- Budget excludes maintenance: Year 1 costs only, no sustainability plan
The hypothetical bed-allocation case study illustrates how teams can structure an economic model. Its figures are teaching assumptions, not measured Johns Hopkins outcomes. Any real implementation requires locally verified costs, benefits, organizational commitment, data infrastructure, and change management.
For low- and middle-income country (LMIC) contexts, cost-effectiveness analysis requires additional considerations: unreliable electricity, intermittent internet, limited technical capacity, and sustainability beyond donor funding cycles. See Global Health and Equity for infrastructure assessment, offline-first design, and implementation guidance tailored to resource-limited environments.
MLOps Fundamentals
The detailed material is maintained in MLOps for Public Health AI. This section anchor remains here for continuity.
Deployment Strategies
The detailed material is maintained in Deployment and Production Monitoring. This section anchor remains here for continuity.
System Integration
The detailed material is maintained in System Integration and Regulatory Compliance. This section anchor remains here for continuity.
Organizational Readiness and Change Management
Technical excellence does not guarantee deployment success. The Epic sepsis model worked technically but failed organizationally. Retinopathy screening AI succeeded because it fit workflows. The difference? Organizational readiness and change management determine whether clinicians adopt or override AI systems.
Assessing Organizational Readiness
Before deployment, evaluate readiness across five dimensions:
1. Leadership Commitment
| Indicator | Ready | Not Ready |
|---|---|---|
| Executive sponsor | Named, accountable, budget authority | “IT will handle it” |
| Clinical champion | Respected physician leading adoption | No clinical leadership |
| Resource commitment | Dedicated staff, protected time | “Add to existing duties” |
| Timeline expectations | 12 to 24 month horizon | “Deploy in 3 months” |
2. Workflow Integration Potential
| Indicator | Ready | Not Ready |
|---|---|---|
| Workflow documented | Current state mapped, pain points identified | “Everyone knows how we work” |
| Integration point clear | Specific decision moment where AI adds value | “Use it however you want” |
| Time available | 30 seconds to review AI output | Already overwhelmed |
| Fallback process | Clear path when AI unavailable | System dependency assumed |
3. Technical Infrastructure
| Indicator | Ready | Not Ready |
|---|---|---|
| EHR integration | FHIR APIs available, IT support allocated | “We’ll figure it out later” |
| Data quality | Clean, complete, timely data feeds | 40% missing values, weeks-old data |
| Monitoring capability | Can track performance in production | “Trust the validation study” |
| Rollback mechanism | Can disable in minutes | No plan for failure |
4. Staff Capacity
| Indicator | Ready | Not Ready |
|---|---|---|
| Training time | 2 to 4 hours per user allocated | “Send an email” |
| Ongoing support | Help desk, super-users, feedback channels | One-time training only |
| Cognitive load | AI reduces burden or is burden-neutral | Another alert to ignore |
| Trust baseline | Staff believes AI can help | “Another IT initiative” |
5. Governance Structure
| Indicator | Ready | Not Ready |
|---|---|---|
| Decision rights | Clear who approves go-live, pauses, discontinuation | “We’ll cross that bridge” |
| Incident response | Protocol for errors, near-misses, harms | No plan |
| Performance thresholds | Specific metrics triggering review | “We’ll know if it’s bad” |
| Discontinuation criteria | Explicit conditions for stopping | “We invested too much to stop” |
Governance Maturity Assessment
The five readiness dimensions above evaluate whether a specific deployment is ready to proceed. A broader question: does the organization itself have governance structures capable of managing AI across its portfolio?
A systematic review of 35 healthcare AI governance frameworks (2019–2024) found that most assume access to advanced resources: in-house data science teams, enterprise data warehouses, and dedicated AI ethics committees (Hussein et al., 2026). Many public health agencies, community health systems, and regional networks lack these resources, creating a governance gap where AI deployments proceed without adequate oversight or organizations attempt to adopt frameworks they cannot operationalize.
The resulting Healthcare AI Governance Readiness Assessment (HAIRA) proposes five maturity levels across seven governance domains:
Seven Governance Domains:
| Domain | Focus |
|---|---|
| Organizational Structure | Governance body, committee composition, decision authority |
| Problem Formulation | Needs assessment, clinical objectives, ROI evaluation |
| External Algorithm Evaluation | Vendor assessment, independent validation |
| Algorithm Development | Internal vs. vendor development capabilities |
| Model Evaluation and Validation | Testing protocols, bias assessment, prospective studies |
| Deployment and Integration | Workflow integration, change management, shadow deployment |
| Monitoring and Maintenance | Performance tracking, drift detection, incident response |
Five Maturity Levels:
| Level | Target Organization | Key Characteristics |
|---|---|---|
| Level 1: Initial/Ad Hoc | Small practices | No formal governance; relies on vendor claims; reactive monitoring |
| Level 2: Defined | Systems with deployed AI | Basic oversight committee; structured evaluation criteria; regular performance reviews |
| Level 3: Established | Community/regional networks | Multidisciplinary governance committee; internal validation capability; proactive monitoring with intervention thresholds |
| Level 4: Advanced | Major academic medical centers | Executive-level AI officer; substantial internal development; real-time monitoring with automated alerts |
| Level 5: Leading | Top academic health systems | Center of excellence; sets industry standards; predictive analytics for risk management |
HAIRA uses a minimum-domain (“weakest-link”) rule: an organization’s overall level is capped by its lowest-scoring domain. A system with Level 4 capabilities in algorithm development but Level 2 monitoring remains at Level 2 overall. This reflects safety-critical environments, where a single governance gap can undermine otherwise strong capabilities.
Application to public health: Public health agencies vary substantially in staffing, infrastructure, procurement authority, and analytic capacity. HAIRA can be used as a structured self-assessment, but an agency’s maturity level must be measured rather than inferred. Organizations can identify their current capabilities and target incremental advancement across specific domains.
The review also identified several comprehensive frameworks covering six or more governance domains, including the Duke ABCDS framework (Bedoya et al., 2022), which provides a tested governance model currently managing over 50 predictive models in production, and the NIST AI Risk Management Framework (NIST, 2023), which organizes governance around four core functions: Govern, Map, Measure, and Manage. NIST also published an April 7, 2026 concept note for a Trustworthy AI in Critical Infrastructure profile (NIST, 2026). For equity-focused governance, the HEAAL framework provides 37 step-by-step procedures for evaluating AI solutions through a health equity lens (Kim et al., 2024).
For monitoring-specific guidance, NIST’s Center for AI Standards and Innovation (CAISI) cataloged post-deployment monitoring challenges and proposed monitoring categories in NIST AI 800-4 (Rao et al., 2026).
Resistance Patterns and Interventions
Clinician resistance follows predictable patterns. Address them proactively:
Pattern 1: Alert Fatigue (“I just click through everything”)
- Root cause: High false positive rates, interruptions during cognitive tasks
- Intervention: Reduce alert volume (target <5% of encounters), deliver at natural pause points
- Metric: Alert override rate <70%
Pattern 2: Autonomy Threat (“The computer is practicing medicine”)
- Root cause: AI perceived as replacing judgment rather than supporting it
- Intervention: Frame as “decision support” not “decision making,” preserve override authority
- Metric: Qualitative feedback, adoption rates by physician vs. non-physician
Pattern 3: Accountability Ambiguity (“Who’s responsible if it’s wrong?”)
- Root cause: Unclear liability when AI contributes to errors
- Intervention: Explicit institutional policy on AI-assisted decisions, malpractice coverage confirmation
- Metric: Staff comfort survey, documented concerns
Pattern 4: Workflow Disruption (“This slows me down”)
- Root cause: AI adds steps without removing others
- Intervention: Time-motion studies before/after, remove offsetting tasks
- Metric: Task completion time, documentation burden
Pattern 5: Trust Deficit (“I’ve seen these systems fail”)
- Root cause: Previous negative experiences with health IT, awareness of AI failures
- Intervention: Transparent performance data, local validation results, peer testimonials
- Metric: Trust survey scores, voluntary usage rates
The Discontinuation Decision
Organizations rarely plan for stopping AI systems, creating sunk-cost pressure to continue ineffective deployments. Establish explicit discontinuation criteria before go-live:
Automatic Pause Triggers (require 48-hour review):
- Sensitivity drops >15% from baseline
- False positive rate exceeds 90%
3 patient safety events in 30 days
- Staff override rate exceeds 85%
Formal Discontinuation Review (quarterly):
- Net clinical benefit analysis (harms vs. benefits)
- Cost per case detected vs. alternatives
- Staff satisfaction and adoption trends
- Comparison to original success criteria
Discontinuation Criteria (any one sufficient):
- Fails to meet minimum performance thresholds for 2 consecutive quarters
- Clinical champions withdraw support
- Cost per benefit exceeds alternative approaches by >2x
- Patient or staff safety events attributable to system
Discontinuation Process:
- Leadership decision documented with rationale
- Staff communication explaining decision
- Gradual phase-out (not abrupt) to avoid workflow disruption
- Post-implementation review capturing lessons learned
- Archive system for audit purposes
Change Management Timeline
Successful AI deployment follows an organizational adoption curve:
| Phase | Duration | Focus | Success Indicators |
|---|---|---|---|
| Preparation | 3 to 6 months | Stakeholder alignment, workflow analysis, training development | Leadership commitment, clinical champion identified |
| Pilot | 3 to 6 months | Small-scale deployment, intensive support, rapid iteration | >70% adoption, <80% override rate, positive feedback |
| Expansion | 6 to 12 months | Gradual rollout, peer learning, support scaling | Consistent metrics across sites, declining support needs |
| Sustainment | Ongoing | Monitoring, retraining, continuous improvement | Stable performance, integrated into standard workflows |
Most failures occur because organizations skip preparation and pilot phases, attempting full deployment in 3 to 6 months instead of 12 to 24 months.
Case Studies: Learning from Real-World Deployments
Case Study 1: Epic Sepsis Model, Deployment Without Decision-Grade Evidence
At the evaluated threshold, external validation found 33% sensitivity and 12% positive predictive value (Wong et al., 2021). The deployment lesson is that integration into many hospitals does not establish transportability, usable lead time, acceptable alert burden, adoption, or outcome benefit.
The full case and source critique are maintained in The AI Morgue: Epic Sepsis Model. Deployment teams should use the case to require external validation, workflow simulation, prospective silent evaluation, threshold selection tied to capacity, outcome monitoring, and a stop rule before scale. ### Case Study 2: Google Health Diabetic Retinopathy Screening - Success Story
Background:
Gulshan et al., 2016, JAMA - Development and validation of deep learning algorithm for diabetic retinopathy
Krause et al., 2018, Ophthalmology - Grader variability and the importance of reference standards
Google Health developed deep learning model for diabetic retinopathy screening from retinal fundus photographs. Successfully deployed in Thailand and India.
Success Factors:
1. Rigorous multi-site validation: - Validated across 54 sites in US and India - 128,175 images from diverse populations - Multiple graders for ground truth labels - Performance on par with ophthalmologists (AUC 0.991)
2. Appropriate use case: - High unmet need (limited access to ophthalmologists in rural areas) - Clear diagnostic criteria (diabetic retinopathy well-defined) - Screening (not diagnostic) - lower risk than treatment decisions
3. Thoughtful deployment design: - Offline capability (mobile screening units) - Image quality checks before prediction - Clear referral pathways for positive screens - Nurse-operated (no ophthalmologist needed on-site)
4. Co-design with clinicians: - Extensive input from ophthalmologists - User testing with nurses and technicians - Workflow integration carefully planned - Training programs for users
5. Continuous monitoring: - Track real-world performance - Collect feedback from users - Iterative improvements based on field experience
Key Implementation Features:
class ClinicalScreeningSystem:
"""
Retinal screening system design based on Google Health's approach
Key features:
- Image quality assessment
- Offline capability
- Clear referral pathways
- Performance monitoring
"""
def __init__(self, model_path: str):
# Load TensorFlow Lite model for edge deployment
import tensorflow as tf
self.interpreter = tf.lite.Interpreter(model_path=model_path)
self.interpreter.allocate_tensors()
def screen_patient(self, image_path: str) -> Dict:
"""
Complete screening workflow
Steps:
1. Assess image quality
2. If adequate, make prediction
3. Generate clear recommendation
4. Log for quality assurance
"""
# Step 1: Image quality check (CRITICAL)
quality = self._assess_image_quality(image_path)
if quality['score'] < 0.7:
return {
'result': 'Inadequate Image Quality',
'referable': None,
'action': 'RETAKE IMAGE',
'quality_issues': quality['issues'],
'instructions': 'Ensure good lighting, proper focus, and eye is centered'
}
# Step 2: Make prediction
prediction = self._predict(image_path)
dr_severity = self._classify_severity(prediction)
# Step 3: Generate recommendation
referable = dr_severity in ['Moderate', 'Severe', 'Proliferative']
if referable:
action = 'REFER TO OPHTHALMOLOGIST'
urgency = 'Within 1 month' if dr_severity == 'Moderate' else 'Within 1 week'
else:
action = 'No referral needed'
urgency = 'Routine annual screening'
# Step 4: Log result
self._log_screening({
'image_path': image_path,
'dr_severity': dr_severity,
'referable': referable,
'quality_score': quality['score']
})
return {
'result': f'Diabetic Retinopathy: {dr_severity}',
'referable': referable,
'action': action,
'urgency': urgency,
'confidence': prediction['confidence'],
'quality_score': quality['score']
}
def _assess_image_quality(self, image_path: str) -> Dict:
"""
Assess image quality before prediction
Critical for deployment success - prevents predictions on poor images
Checks:
- Adequate illumination
- Proper focus
- Eye centered in frame
- Sufficient field of view
"""
import cv2
img = cv2.imread(image_path)
issues = []
# Check brightness
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
mean_brightness = np.mean(gray)
if mean_brightness < 50:
issues.append('Image too dark')
elif mean_brightness > 200:
issues.append('Image too bright / overexposed')
# Check focus (Laplacian variance)
laplacian_var = cv2.Laplacian(gray, cv2.CV_64F).var()
if laplacian_var < 100:
issues.append('Image out of focus')
# Check if eye is centered (simplified - real implementation more sophisticated)
height, width = img.shape[:2]
center_region = img[height//3:2*height//3, width//3:2*width//3]
if np.mean(center_region) < 30:
issues.append('Eye not properly centered')
# Calculate overall quality score
quality_score = 1.0
quality_score -= len(issues) * 0.2 # Deduct 0.2 per issue
quality_score = max(0, quality_score)
return {
'score': quality_score,
'issues': issues,
'adequate': quality_score >= 0.7
}
def _predict(self, image_path: str) -> Dict:
"""Make DR prediction"""
# Preprocessing
import tensorflow as tf
img = tf.keras.preprocessing.image.load_img(
image_path, target_size=(299, 299)
)
img_array = tf.keras.preprocessing.image.img_to_array(img)
img_array = tf.expand_dims(img_array, 0)
img_array = img_array / 255.0
# Run inference
input_details = self.interpreter.get_input_details()
output_details = self.interpreter.get_output_details()
self.interpreter.set_tensor(input_details[0]['index'], img_array.astype(np.float32))
self.interpreter.invoke()
output = self.interpreter.get_tensor(output_details[0]['index'])[0]
# Output is 5-class: None, Mild, Moderate, Severe, Proliferative
severity_labels = ['None', 'Mild', 'Moderate', 'Severe', 'Proliferative']
predicted_class = np.argmax(output)
confidence = float(output[predicted_class])
return {
'severity_class': predicted_class,
'severity_label': severity_labels[predicted_class],
'confidence': confidence,
'probabilities': output.tolist()
}
def _classify_severity(self, prediction: Dict) -> str:
"""Map prediction to clinical severity"""
return prediction['severity_label']
def _log_screening(self, result: Dict):
"""Log screening for quality assurance and monitoring"""
# In production: Write to database
# Could periodically sample for expert review
logging.info(f"Screening logged: {result}")Deployment lessons:
- Image quality checks are essential - Prevents garbage-in-garbage-out
- Offline capability critical for resource-limited settings
- Clear, actionable recommendations - Not just probability scores
- Integration with care pathways - Screening delivers no value without referral system
- User training and support - Technology alone insufficient
Case Study 3: COVID-19 Deterioration Prediction - Lessons in Adaptation
Background:
Rapid deployment of COVID-19 deterioration models during pandemic revealed challenges of deploying AI in crisis with rapidly evolving disease.
Wynants et al., 2020, BMJ - “Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal”
Challenges:
- Rapidly evolving disease
- Alpha → Delta → Omicron variants with different presentations
- Models trained on one variant failed on next
- Data scarcity early in pandemic
- Limited training data
- High risk of overfitting
- Pressure to deploy despite insufficient validation
- High stakes
- ICU bed allocation
- Ventilator rationing
- Life-or-death decisions
- Changing treatment standards
- Dexamethasone adoption changed outcomes
- Remdesivir approval
- Vaccination effects on disease progression
Successful Adaptations:
class AdaptiveCOVIDPredictor:
"""
COVID model with adaptations for evolving disease
Key features:
- Variant-specific models
- Uncertainty quantification
- Conservative recommendations when uncertain
- Rapid retraining capability
"""
def __init__(self):
# Multiple models for different variants/time periods
self.models = {
'pre_delta': load_model('covid_2020.pkl'),
'delta': load_model('covid_delta.pkl'),
'omicron': load_model('covid_omicron.pkl')
}
# Track which model performs best recently
self.current_variant = 'omicron'
self.model_performance = {}
def predict(self, patient_data: Dict) -> Dict:
"""
Predict with variant-specific model and uncertainty estimate
"""
# Select appropriate model
if 'variant' in patient_data and patient_data['variant'] in self.models:
model = self.models[patient_data['variant']]
model_name = patient_data['variant']
else:
# Use most recent model
model = self.models[self.current_variant]
model_name = self.current_variant
# Make prediction
deterioration_risk = model.predict(patient_data)
# Estimate uncertainty
# For ensemble: use prediction variance across models
all_predictions = [m.predict(patient_data) for m in self.models.values()]
uncertainty = np.std(all_predictions)
# Generate recommendation
recommendation = self._generate_recommendation(
deterioration_risk,
uncertainty,
patient_data
)
return {
'deterioration_risk': float(deterioration_risk),
'uncertainty': float(uncertainty),
'model_used': model_name,
'recommendation': recommendation,
'confidence': 'low' if uncertainty > 0.2 else 'moderate' if uncertainty > 0.1 else 'high'
}
def _generate_recommendation(self, risk: float, uncertainty: float, patient_data: Dict) -> str:
"""
Conservative recommendations when uncertain
During crisis with evolving disease, err on side of caution
"""
# High uncertainty → Conservative recommendation
if uncertainty > 0.2:
return (
"High uncertainty due to limited data for current variant. "
"Recommend close monitoring and low threshold for escalation of care."
)
# Standard risk-based recommendations
if risk > 0.7:
return "High risk of deterioration. Consider ICU monitoring or transfer."
elif risk > 0.4:
return "Moderate risk. Increase monitoring frequency. Ensure oxygen available."
else:
return "Low risk. Continue standard COVID care protocol."
def update_performance(self, predictions: List, outcomes: List):
"""
Track model performance over time
Rapidly detect when models degrade (e.g., new variant)
"""
from sklearn.metrics import roc_auc_score
for model_name, model in self.models.items():
# Get predictions from this model
model_predictions = [model.predict(p) for p in predictions]
# Calculate AUC
try:
auc = roc_auc_score(outcomes, model_predictions)
self.model_performance[model_name] = {
'auc': auc,
'timestamp': datetime.now()
}
except:
pass
# Log performance
print("\nModel Performance Update:")
for name, perf in self.model_performance.items():
print(f" {name}: AUC = {perf['auc']:.3f}")
def trigger_retraining(self):
"""
Rapid retraining when performance degrades
During pandemic, needed ability to retrain quickly
"""
# Check if any model performing poorly
poor_performers = [
name for name, perf in self.model_performance.items()
if perf['auc'] < 0.70
]
if poor_performers:
print(f"[WARNING] Performance degradation detected: {poor_performers}")
print("Triggering emergency retraining...")
# Initiate rapid retraining pipeline
# In production: Automated pipeline with latest dataKey Lessons:
- Build in adaptability from start
- Expect disease/treatment evolution
- Design for rapid retraining
- Multiple models for different scenarios
- Quantify and communicate uncertainty
- Do not hide when model uncertain
- Conservative recommendations when uncertain
- Clearly indicate confidence level
- Continuous validation essential
- Track performance in real-time
- Detect degradation quickly
- Trigger retraining automatically
- Avoid over-promising
- Communicate limitations clearly
- Do not claim certainty that does not exist
- Maintain clinician trust through transparency
Key Takeaways
Deployment is not the end. It’s the beginning - Most effort comes after deployment through monitoring and maintenance.
MLOps practices are non-negotiable - Version control, CI/CD, monitoring are essential for production ML, not optional.
Monitor everything - Model performance, data drift, system health, user behavior. What you do not monitor, you cannot fix.
External validation is critical - Internal validation insufficient. Test across diverse populations, sites, and conditions.
Alert fatigue is real - High false positive rates destroy clinician trust. Optimize thresholds for clinical utility, not just statistical performance.
Build for adaptability - Models degrade. Disease evolves. Treatments change. Design for continuous retraining from the start.
Regulatory compliance is ongoing - FDA clearance is not one-and-done. Post-market surveillance, adverse event reporting, and periodic validation required.
Integration matters as much as accuracy - Model can be perfect but fail if does not integrate with clinical workflow or EHR systems.
Transparency builds trust - Document limitations, report performance honestly, communicate uncertainty clearly.
Learn from failures - Epic sepsis model and other failures teach valuable lessons. Study them, do not repeat them.
Hands-On Exercise: Build a Complete MLOps Pipeline
Objective
Build end-to-end MLOps pipeline including deployment, monitoring, and automated retraining for a clinical prediction model.
Scenario
You’re deploying a hospital readmission prediction model across 3 hospitals. Each hospital has different EHR systems and patient populations. You need to:
- Deploy model to production
- Monitor performance across sites
- Detect drift
- Trigger retraining when needed
- Integrate with hospital IT systems
Provided Materials
readmission_model.pkl- Trained modelhospital_*.csv- Historical data from 3 hospitalstest_patients.csv- New patients for prediction
Tasks
Part 1: Deployment (30 min)
- Create FastAPI service for readmission predictions
- Implement
/predictendpoint with input validation - Add
/healthendpoint for monitoring - Include Prometheus metrics
- Containerize with Docker
- Write Dockerfile
- Include health checks
- Optimize image size
- Deploy to Kubernetes (or Docker Compose if K8s unavailable)
- Write deployment manifests
- Configure health probes
- Set resource limits
Deliverable: Working API that returns predictions
Part 2: Monitoring Dashboard (30 min)
- Set up Prometheus scraping
- Configure scrape interval
- Define retention period
- Create Grafana dashboard with panels for:
- Predictions per minute
- Latency (p50, p95, p99)
- Error rate
- Risk category distribution
- Model performance (if outcomes available)
- Configure alerts for:
- High latency (>1s p95)
- High error rate (>5%)
- Low prediction volume
Deliverable: Grafana dashboard screenshot and alert configuration
Part 3: Drift Detection (20 min)
- Implement drift detector
- Calculate PSI for each feature
- Use KS test for distribution comparison
- Compare weekly data to training baseline
- Create drift report
- Generate HTML report with visualizations
- Highlight features with significant drift
- Recommend action (retrain vs. monitor)
Deliverable: Drift detection script and sample report
Part 4: Automated Retraining (20 min)
- Build retraining pipeline
- Fetch latest data
- Validate data quality
- Train new model
- Compare to production model
- Promote if better
- Create trigger logic
- Trigger on drift detection
- Trigger on performance degradation
- Scheduled weekly check
Deliverable: Working retraining script
Part 5: Integration (20 min)
- Implement HL7/FHIR handler (choose one)
- Parse incoming patient data messages
- Extract features
- Make prediction
- Generate response message
- Add database logging
- Log all predictions
- Track outcomes when available
- Enable performance analysis
Deliverable: Integration code and test cases
Bonus Challenges
- Blue-green deployment: Implement zero-downtime deployment
- Multi-site monitoring: Track performance separately per hospital
- Fairness monitoring: Add subgroup performance tracking
- Cost optimization: Right-size resources based on load
Evaluation Criteria
- Functionality: Does it work?
- Robustness: Error handling, input validation
- Monitoring: Comprehensive metrics and alerts
- Documentation: Clear README, code comments
- Best practices: Following MLOps principles
Check Your Understanding
The detailed material is maintained in Public Health AI Deployment Exercises. This section anchor remains here for continuity.
Discussion Questions
Trade-offs: The Epic sepsis model had a 12% positive predictive value but caught only 33% of sepsis cases. Is this acceptable? How would you balance sensitivity vs. specificity for a life-threatening condition?
Retraining frequency: How often should clinical prediction models be retrained? Should retraining be scheduled (monthly/quarterly) or triggered by performance degradation? What are pros/cons of each approach?
Regulatory burden: Does FDA regulation of AI/ML medical devices help or hurt? Does it protect patients or slow innovation? How can we balance safety with speed?
Deployment strategy: Blue-green deployment requires running two full production environments. For resource-constrained hospitals, is this feasible? What are lower-cost alternatives that maintain safety?
Alert thresholds: Should alert thresholds be set centrally by model developers or customized by each hospital? What if hospital chooses threshold that reduces sensitivity below acceptable level?
Liability: When AI-assisted decision leads to patient harm, who is responsible? The hospital? The clinician? The AI vendor? The data scientists who built it?
Drift detection: If data drift detected but model performance has not degraded yet, should you intervene? When is preemptive retraining justified vs. waiting for actual performance drop?
Further Resources
Essential Books
MLOps: - Reliable Machine Learning by Chen, Murphy, et al. (O’Reilly, 2022) - Google’s approach to production ML - Machine Learning Design Patterns by Lakshmanan, Robinson, Munn (O’Reilly, 2020) - Building Machine Learning Powered Applications by Ameisen (O’Reilly, 2020)
Production Systems: - Designing Data-Intensive Applications by Kleppmann (O’Reilly, 2017) - Distributed systems fundamentals - Site Reliability Engineering by Google (Free online) - Operational excellence
Key Papers
Deployment & Monitoring: - Sculley et al., 2015, NIPS - “Hidden technical debt in machine learning systems” - Breck et al., 2017, IEEE Big Data - “The ML test score: A rubric for ML production readiness” - Sato et al., 2019, IEEE Software - “Continuous delivery for machine learning”
Healthcare AI Deployment: - Wong et al., 2021, JAMA Internal Medicine - External validation of Epic sepsis model - Sendak et al., 2020, JMIR Medical Informatics - Real-world sepsis model integration - Rajkomar et al., 2018, npj Digital Medicine - Scalable and accurate deep learning for EHR
Drift Detection: - Rabanser et al., 2019, NeurIPS - “Failing loudly: An empirical study of methods for detecting dataset shift” - Lu et al., 2019, IEEE Transactions on Knowledge and Data Engineering - “Learning under concept drift: A review” - Gama et al., 2014, ACM Computing Surveys - “A survey on concept drift adaptation”
Regulatory: - Benjamens et al., 2020, npj Digital Medicine - “The state of AI-based FDA-approved medical devices” - FDA, 2021 - “Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan”
Tools & Platforms
MLOps: - MLflow - Model tracking, registry, deployment - Weights & Biases - Experiment tracking, visualization - DVC - Data version control - Kubeflow - ML workflows on Kubernetes
Monitoring: - Prometheus + Grafana - Metrics and dashboards - Evidently AI - ML monitoring, drift detection - Arize - ML observability platform - Fiddler - ML monitoring and explainability
Deployment: - FastAPI - Modern Python API framework - Docker - Containerization - Kubernetes - Container orchestration - Seldon Core - ML deployment on Kubernetes
Healthcare Integration: - HAPI FHIR - Open source FHIR server - FHIR Client - Python FHIR client - HL7apy - Python HL7 library
Courses & Tutorials
- Made With ML - MLOps - Comprehensive MLOps course (Goku Mohandas)
- Full Stack Deep Learning - Production ML (UC Berkeley)
- MLOps Specialization - DeepLearning.AI (Coursera)
- Kubeflow 101 - Official tutorials
Guidelines
FDA Guidance: - FDA Software as a Medical Device (SaMD) - Good Machine Learning Practice for Medical Device Development - Clinical Decision Support Software Guidance
Best Practices: - Google’s ML Engineering Best Practices - Microsoft’s Responsible AI Guidelines - NIST AI Risk Management Framework
Congratulations! You’ve completed the Deployment, Monitoring, and Maintenance chapter. You now have practical knowledge of:
- MLOps principles and lifecycle
- Deployment strategies (blue-green, canary, shadow)
- Comprehensive monitoring (performance, drift, system health)
- Automated retraining pipelines
- EHR integration (FHIR, HL7)
- Regulatory compliance (FDA pathways)
- Real-world case studies and lessons learned