MLOps for Public Health AI

Versioning, reproducibility, testing, and release controls for public health AI systems. The material is maintained separately so each operational question has a stable, focused reference.

Learning Objectives
  • Identify the evidence and controls relevant to this decision area
  • Distinguish technical performance from operational and population impact
  • Apply the included framework without extending claims beyond the cited evidence

Use explicit targets, populations, thresholds, and decision consequences. Require external evidence and local monitoring where deployment can affect people or programs. Preserve uncertainty and document limits.

Introduction

This focused reference is part of the broader MLOps overview. It preserves the detailed methods, examples, and exercises while reducing page size and improving direct navigation.

MLOps Fundamentals

What Is MLOps?

MLOps (Machine Learning Operations) applies DevOps principles to machine learning systems, enabling reliable and efficient deployment, monitoring, and maintenance.

Kreuzberger et al., 2023, IEEE Access define MLOps as:

“A set of practices that aims to deploy and maintain machine learning models in production reliably and efficiently.”

Core principles:

1. Reproducibility - Every model run can be exactly recreated - Version control for code, data, models, environment - Deterministic training procedures

2. Automation - Minimize manual steps (human error) - CI/CD pipelines for testing and deployment - Automated retraining when needed

3. Versioning - Track all artifacts: data, models, code - Enable rollback to previous versions - Compare performance across versions

4. Monitoring - Continuous performance measurement - Detect drift, degradation, anomalies - Alert on issues before they cause harm

5. Collaboration - Bridge data scientists, engineers, clinicians - Shared understanding of system behavior - Clear ownership and accountability

For comprehensive MLOps frameworks, see Treveil et al., 2020, Introducing MLOps and Alla & Adari, 2021, Beginning MLOps with MLflow.

The MLOps Lifecycle

┌─────────────────────────────────────────────────────────────┐
│     MLOps Lifecycle       │
├─────────────────────────────────────────────────────────────┤
│                │
│ ┌────────────┐ ┌────────────┐ ┌────────────┐   │
│ │ Data  │──▶│ Model │──▶│ Deploy │   │
│ │ Pipeline │ │ Training │ │ to Prod │   │
│ └────────────┘ └────────────┘ └────────────┘   │
│  │    │     │     │
│  └─────────────────┴─────────────────┘    │
│       │         │
│     ┌────▼────┐        │
│     │ Monitor │        │
│     │& Alert │        │
│     └────┬────┘        │
│       │         │
│     ┌────▼────┐        │
│     │Retrain │        │
│     │Decision │        │
│     └─────────┘        │
└─────────────────────────────────────────────────────────────┘

Key stages:

1. Data Pipeline - Automated data extraction from EHR - Quality validation and cleaning - Feature engineering - Version-controlled datasets

2. Model Training - Reproducible training environment - Hyperparameter tracking - Performance metrics logging - Model registration

3. Deployment - Automated testing (unit, integration, performance) - Gradual rollout (canary, blue-green) - Rollback capability - Documentation generation

4. Monitoring - Performance tracking by subgroup - Data drift detection - Concept drift detection - System health metrics

5. Retraining Decision - Triggered by performance degradation - Scheduled periodic retraining - New data availability - Regulatory requirements

Version Control for Machine Learning

What to version:

1. Code (Git, GitHub, GitLab)

git commit -m "Add calibration layer to sepsis model"
git tag -a v1.2.0 -m "Production release with improved calibration"
git push origin v1.2.0

2. Data (DVC, LakeFS)

# Track data with DVC
dvc add data/training_set_2024_Q1.csv
git add data/training_set_2024_Q1.csv.dvc
git commit -m "Add Q1 2024 training data"

# Retrieve specific data version
dvc checkout data/training_set_2024_Q1.csv.dvc

3. Models (MLflow, Weights & Biases)

import mlflow
import mlflow.sklearn
from datetime import datetime

# Set experiment
mlflow.set_experiment("sepsis-prediction-v2")

# Start run with automatic logging
with mlflow.start_run(run_name=f"rf_model_{datetime.now().strftime('%Y%m%d_%H%M')}"):

 # Log parameters
 params = {
  'n_estimators': 100,
  'max_depth': 10,
  'min_samples_split': 5,
  'class_weight': 'balanced',
  'random_state': 42
 }
 mlflow.log_params(params)

 # Train model
 from sklearn.ensemble import RandomForestClassifier
 model = RandomForestClassifier(**params)
 model.fit(X_train, y_train)

 # Evaluate and log metrics
 from sklearn.metrics import roc_auc_score, recall_score, precision_score

 train_auc = roc_auc_score(y_train, model.predict_proba(X_train)[:, 1])
 val_auc = roc_auc_score(y_val, model.predict_proba(X_val)[:, 1])
 val_sensitivity = recall_score(y_val, model.predict(X_val))
 val_specificity = recall_score(1-y_val, 1-model.predict(X_val))
 val_ppv = precision_score(y_val, model.predict(X_val))

 mlflow.log_metrics({
  'train_auc': train_auc,
  'val_auc': val_auc,
  'val_sensitivity': val_sensitivity,
  'val_specificity': val_specificity,
  'val_ppv': val_ppv
 })

 # Log model
 mlflow.sklearn.log_model(
  model,
  "model",
  registered_model_name="sepsis_predictor",
  signature=mlflow.models.infer_signature(X_val, model.predict(X_val))
 )

 # Log artifacts (plots, feature importance)
 import matplotlib.pyplot as plt
 from sklearn.metrics import roc_curve, auc

 # ROC curve
 fpr, tpr, _ = roc_curve(y_val, model.predict_proba(X_val)[:, 1])
 plt.figure(figsize=(8, 6))
 plt.plot(fpr, tpr, label=f'AUC = {val_auc:.3f}')
 plt.plot([0, 1], [0, 1], 'k--')
 plt.xlabel('False Positive Rate')
 plt.ylabel('True Positive Rate')
 plt.title('ROC Curve')
 plt.legend()
 plt.savefig('roc_curve.png', dpi=150, bbox_inches='tight')
 mlflow.log_artifact('roc_curve.png')
 plt.close()

 # Feature importance
 feature_names = X_train.columns
 importances = model.feature_importances_
 indices = np.argsort(importances)[::-1][:10]

 plt.figure(figsize=(10, 6))
 plt.barh(range(len(indices)), importances[indices])
 plt.yticks(range(len(indices)), [feature_names[i] for i in indices])
 plt.xlabel('Feature Importance')
 plt.title('Top 10 Most Important Features')
 plt.tight_layout()
 plt.savefig('feature_importance.png', dpi=150, bbox_inches='tight')
 mlflow.log_artifact('feature_importance.png')
 plt.close()

 print(f"[OK] Run ID: {mlflow.active_run().info.run_id}")
 print(f"[OK] Validation AUC: {val_auc:.3f}")

Promote model to production:

from mlflow.tracking import MlflowClient

client = MlflowClient()

# Search for best model in experiment
experiment = client.get_experiment_by_name("sepsis-prediction-v2")
runs = client.search_runs(
 experiment_ids=[experiment.experiment_id],
 filter_string="metrics.val_auc > 0.80", # Minimum acceptable performance
 order_by=["metrics.val_auc DESC"],
 max_results=1
)

if len(runs) == 0:
 print("[ERROR] No models meet minimum performance threshold")
else:
 best_run = runs[0]
 best_run_id = best_run.info.run_id
 best_auc = best_run.data.metrics['val_auc']

 print(f"[OK] Best model - Run ID: {best_run_id}, AUC: {best_auc:.3f}")

 # Get current production model for comparison
 try:
  prod_versions = client.get_latest_versions("sepsis_predictor", stages=["Production"])
  if prod_versions:
   prod_auc = float(prod_versions[0].tags.get('val_auc', 0))
   print(f"Current production AUC: {prod_auc:.3f}")

   # Only promote if better
   if best_auc > prod_auc:
    print("[OK] New model is better, promoting...")
   else:
    print("[WARNING] New model not better than production, not promoting")
    exit(0)
 except:
  print("[INFO] No current production model, will promote")

 # Register model version
 model_uri = f"runs:/{best_run_id}/model"
 model_details = mlflow.register_model(model_uri, "sepsis_predictor")

 # Add tags
 client.set_model_version_tag(
  name="sepsis_predictor",
  version=model_details.version,
  key="val_auc",
  value=str(best_auc)
 )

 # Transition to production
 client.transition_model_version_stage(
  name="sepsis_predictor",
  version=model_details.version,
  stage="Production",
  archive_existing_versions=True
 )

 print(f"[OK] Model version {model_details.version} promoted to Production!")

4. Environment (Docker, Conda)

FROM python:3.9-slim

WORKDIR /app

# Install system dependencies
RUN apt-get update && apt-get install -y \
 gcc \
 g++ \
 && rm -rf /var/lib/apt/lists/*

# Copy requirements
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY api.py .
COPY models/ ./models/

# Create non-root user
RUN useradd -m -u 1000 mluser && chown -R mluser:mluser /app
USER mluser

# Expose port
EXPOSE 8000

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
 CMD python -c "import requests; requests.get('http://localhost:8000/health')"

# Run application
CMD ["uvicorn", "api:app", "--host", "0.0.0.0", "--port", "8000"]

5. Experiments (MLflow, Neptune, Weights & Biases)

For comprehensive model tracking best practices, see Zaharia et al., 2018, IEEE Data Eng. Bull. and the MLflow documentation.

CI/CD for Machine Learning

Continuous Integration/Continuous Deployment adapted for ML workflows.

Sato et al., 2019, IEEE Software - “Continuous Delivery for Machine Learning”

Typical ML CI/CD pipeline:

# .github/workflows/ml-pipeline.yml
name: ML Model CI/CD Pipeline

on:
 push:
 branches: [main, develop]
 pull_request:
 branches: [main]
 schedule:
 - cron: '0 2 * * 0' # Weekly retraining check

jobs:
 data-validation:
 runs-on: ubuntu-latest
 steps:
  - uses: actions/checkout@v3

  - name: Set up Python
  uses: actions/setup-python@v4
  with:
   python-version: '3.9'

  - name: Install dependencies
  run: |
   pip install -r requirements.txt

  - name: Validate Data Quality
  run: |
   python scripts/validate_data.py --input data/latest/training_data.csv

  - name: Check Data Drift
  run: |
   python scripts/check_drift.py \
   --reference data/baseline/training_data.csv \
   --current data/latest/training_data.csv \
   --threshold 0.05

  - name: Upload drift report
  if: always()
  uses: actions/upload-artifact@v3
  with:
   name: drift-report
   path: reports/drift_analysis.html

 model-training:
 needs: data-validation
 runs-on: ubuntu-latest
 steps:
  - uses: actions/checkout@v3

  - name: Set up Python
  uses: actions/setup-python@v4
  with:
   python-version: '3.9'

  - name: Install dependencies
  run: |
   pip install -r requirements.txt

  - name: Train Model
  env:
   MLFLOW_TRACKING_URI: ${{ secrets.MLFLOW_TRACKING_URI }}
  run: |
   python train.py \
   --data data/latest/training_data.csv \
   --experiment-name sepsis-prediction-ci \
   --run-name ci-run-${{ github.run_number }}

  - name: Evaluate Model
  run: |
   python evaluate.py \
   --model-uri runs:/${{ steps.train.outputs.run_id }}/model \
   --test-data data/latest/test_data.csv \
   --output-path reports/evaluation.json

  - name: Compare with Production
  run: |
   python scripts/compare_models.py \
   --candidate-uri runs:/${{ steps.train.outputs.run_id }}/model \
   --production-uri models:/sepsis_predictor/Production

  - name: Upload evaluation report
  uses: actions/upload-artifact@v3
  with:
   name: evaluation-report
   path: reports/

 model-testing:
 needs: model-training
 runs-on: ubuntu-latest
 steps:
  - uses: actions/checkout@v3

  - name: Set up Python
  uses: actions/setup-python@v4
  with:
   python-version: '3.9'

  - name: Install dependencies
  run: |
   pip install -r requirements.txt
   pip install pytest pytest-cov

  - name: Unit Tests
  run: |
   pytest tests/unit/ -v --cov=src --cov-report=xml

  - name: Integration Tests
  run: |
   pytest tests/integration/ -v

  - name: Performance Tests
  run: |
   python tests/performance_test.py \
   --model-uri models:/sepsis_predictor/Staging \
   --target-latency-ms 100 \
   --target-throughput-qps 50

  - name: Upload coverage
  uses: codecov/codecov-action@v3

 fairness-audit:
 needs: model-training
 runs-on: ubuntu-latest
 steps:
  - uses: actions/checkout@v3

  - name: Fairness Assessment
  run: |
   python scripts/fairness_audit.py \
   --model-uri runs:/${{ steps.train.outputs.run_id }}/model \
   --test-data data/latest/test_data.csv \
   --protected-attributes race,gender,age_group

  - name: Upload fairness report
  uses: actions/upload-artifact@v3
  with:
   name: fairness-report
   path: reports/fairness_audit.html

 security-scan:
 needs: model-testing
 runs-on: ubuntu-latest
 steps:
  - uses: actions/checkout@v3

  - name: Scan dependencies for vulnerabilities
  run: |
   pip install safety
   safety check --file requirements.txt

  - name: Docker image security scan
  uses: aquasecurity/trivy-action@master
  with:
   image-ref: 'sepsis-predictor:latest'
   format: 'sarif'
   output: 'trivy-results.sarif'

 deploy-staging:
 needs: [model-testing, fairness-audit, security-scan]
 runs-on: ubuntu-latest
 if: github.ref == 'refs/heads/develop'
 steps:
  - uses: actions/checkout@v3

  - name: Build Docker Image
  run: |
   docker build -t sepsis-predictor:${{ github.sha }} .
   docker tag sepsis-predictor:${{ github.sha }} sepsis-predictor:staging

  - name: Push to Registry
  env:
   DOCKER_REGISTRY: ${{ secrets.DOCKER_REGISTRY }}
   DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
   DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
  run: |
   echo $DOCKER_PASSWORD | docker login $DOCKER_REGISTRY -u $DOCKER_USERNAME --password-stdin
   docker push sepsis-predictor:${{ github.sha }}
   docker push sepsis-predictor:staging

  - name: Deploy to Kubernetes Staging
  env:
   KUBECONFIG: ${{ secrets.KUBECONFIG_STAGING }}
  run: |
   kubectl set image deployment/sepsis-predictor \
   sepsis-predictor=sepsis-predictor:${{ github.sha }} \
   -n staging
   kubectl rollout status deployment/sepsis-predictor -n staging

  - name: Smoke Tests
  run: |
   python tests/smoke_test.py \
   --endpoint https://staging.sepsis-predictor.hospital.org \
   --timeout 300

 deploy-production:
 needs: deploy-staging
 runs-on: ubuntu-latest
 if: github.ref == 'refs/heads/main'
 environment:
  name: production
  url: https://sepsis-predictor.hospital.org
 steps:
  - uses: actions/checkout@v3

  - name: Promote to Production in MLflow
  env:
   MLFLOW_TRACKING_URI: ${{ secrets.MLFLOW_TRACKING_URI }}
  run: |
   python scripts/promote_to_production.py \
   --run-id ${{ steps.train.outputs.run_id }}

  - name: Tag Docker Image
  run: |
   docker tag sepsis-predictor:${{ github.sha }} sepsis-predictor:production
   docker push sepsis-predictor:production

  - name: Blue-Green Deployment to Production
  env:
   KUBECONFIG: ${{ secrets.KUBECONFIG_PRODUCTION }}
  run: |
   # Deploy green version
   kubectl apply -f k8s/production/deployment-green.yaml
   kubectl wait --for=condition=available --timeout=300s \
   deployment/sepsis-predictor-green -n production

   # Run validation tests
   python tests/production_validation.py \
   --endpoint https://green.sepsis-predictor.hospital.org

   # Switch traffic from blue to green
   kubectl patch service sepsis-predictor \
   -p '{"spec":{"selector":{"version":"green"}}}' \
   -n production

   # Monitor for 10 minutes
   echo "Monitoring green deployment for 10 minutes..."
   sleep 600

   # If successful, scale down blue
   kubectl scale deployment sepsis-predictor-blue --replicas=0 -n production

   echo "[OK] Production deployment complete!"

  - name: Notify Team
  if: always()
  uses: 8398a7/action-slack@v3
  with:
   status: ${{ job.status }}
   text: 'Production deployment: ${{ job.status }}'
   webhook_url: ${{ secrets.SLACK_WEBHOOK }}

For comprehensive CI/CD for ML, see Zhou et al., 2020, IEEE ICAICE - “Towards MLOps: A Case Study of ML Pipeline Platform”.