MLOps · Chapter 05

MLOps and Automation

Automate the path from data and code changes to validated, deployable, and maintainable ML systems.

6 lecture hoursSyllabus: 5.1–5.4Focus: pipelines & CI/CD/CT

Chapter outcomes

  1. Design ML pipelines and workflow automation for repeatable execution.
  2. Explain continuous integration, delivery, and training (CI/CD/CT) for ML.
  3. Apply testing and deployment automation strategies.
  4. Describe ML infrastructure and platform management concerns.
5.1

ML Pipelines and Workflow Automation

Codify multi-step ML processes as reliable, observable workflows.

Pipeline building blocks

  • Steps/tasks: data ingestion, validation, feature building, training, evaluation, packaging, deployment.
  • Orchestration: dependencies (DAGs), scheduling, event triggers, parameterization.
  • Artifacts: datasets, models, metrics, reports passed between steps with metadata.
  • Reliability: retries, idempotency, timeouts, alerts, and reproducible environments.
Data Validate Train Evaluate Package Deploy
Figure 5.1 — Automated pipelines connect data changes to validated deployments.
Best practice: Make pipeline steps deterministic where possible and idempotent so reruns are safe.
5.2

Continuous Integration, Delivery and Training

Extend software CI/CD with data- and model-aware automation.

CI, CD, and CT in ML systems

PracticeTriggered byAutomatesOutputs/Gates
CI (Continuous Integration)Code/config changesBuild, lint, unit/integration tests, data/schema checksValidated change ready for pipeline run
CD (Continuous Delivery/Deployment)Approved model/service artifactPackaging, staging deploy, release, rollback setupDeployed service with guardrails
CT (Continuous Training)Schedule, new data, drift/perf signalsRetraining, evaluation, candidate registrationNew candidate model for promotion
ML nuance: A passing code build does not guarantee a good model; CT must include data/model validation and evaluation gates.
5.3

Testing and Deployment Automation

Automate quality checks across data, code, models, and runtime behavior.

Testing layers for ML automation

  • Data tests: schema, ranges, nulls, freshness, distribution expectations.
  • Code tests: unit/integration tests for features, preprocessing, APIs.
  • Model tests: metric thresholds, slice/regression tests, behavioral/invariance tests.
  • System tests: end-to-end pipeline runs, load/latency, canary health checks.
gates:
  data_validation: pass
  unit_tests: pass
  f1_min: 0.82
  p95_latency_ms: 150
  canary_error_rate_max: 0.5%
Deployment automation: use scripted rollouts (canary/blue-green), automated health checks, and one-step rollback. Keep model, config, and infrastructure changes versioned together where practical.
5.4

ML Infrastructure and Platform Management

Platforms enable teams to build and operate ML systems consistently at scale.

Platform capabilities and management

Core capabilities

Compute/storage, experiment tracking, pipeline orchestration, model registry, serving, monitoring.

Developer experience

Templates, standardized environments, reusable components, self-service workflows.

Operations

Resource quotas, scheduling, cost visibility, incident management, reliability practices.

Governance

Access control, audit logs, policy enforcement, artifact lineage and approvals.

Platform goal: reduce duplicated effort and operational risk without blocking experimentation and domain-specific needs.

Checkpoint

Design task

You want to retrain a production model weekly and deploy it automatically if safe. Outline the minimum automated pipeline and gates.

Answer: Scheduled CT trigger → data validation → reproducible training run → evaluation (overall/slices + regression tests) → package/register candidate → staging deploy & tests → canary release with latency/error/quality guardrails → auto-rollback and alerts on failure.

Summary

1

Pipeline first

Automated workflows make ML processes repeatable and observable.

2

CI/CD/CT

ML automation must validate code, data, and model behavior.

3

Test and gate

Automated testing and guarded deployments reduce production risk.

4

Platform enablement

Managed infrastructure and tooling scale MLOps across teams.

References

  1. Google Cloud MLOps levels and CD4ML (Continuous Delivery for Machine Learning).
  2. Chip Huyen. Designing Machine Learning Systems, workflow and operations topics.
  3. Kubeflow/MLflow/Airflow docs for pipeline, tracking, and orchestration concepts.
End of chapter

Questions?

Next: Chapter 06 — Monitoring and Maintenance of ML Systems