Design ML-powered products as production systems: frame the decision, specify the requirements, architect the flow, then study how real systems fail.
5 lecture hoursSyllabus: 1.1–1.4Focus: system thinking5 evidence-based cases
What you should be able to do
Define an ML system, ML system design, a model, and MLOps, and explain why the distinctions matter.
Translate a product objective into a decision, a prediction target, a label, and measurable requirements.
Explain the ML lifecycle and the components and flows of a production reference architecture.
Compare batch, online, and streaming systems on freshness, latency, cost, and failure risk.
Diagnose failures across data, model, system, decision, and organizational layers — and propose controls.
Thread through the chapter: five documented systems — Zillow Offers, Epic Sepsis, a Thailand diabetic-retinopathy deployment, a commercial health-risk algorithm, and Uber ATG Tempe.
Roadmap
1.1
Frame
What decision changes? What does success mean? What constraints are non-negotiable?
1.2
Architect
Which data, model, serving, and control-plane components are required?
1.3
Operate
When must predictions run: on a schedule, on request, or on an event?
1.4
Break
How do systems fail, how do failures compound, and which controls actually help?
Recurring question: “Which part of the system produced this outcome?” rarely has the answer “the model alone.”
1.1
ML System Objectives, Requirements and Design
Start with the decision, not the model. Requirements are the bridge between product intent and architecture.
Definitions that must be precise
Term
Working definition
Why it matters
AI
Umbrella term for systems that perform tasks associated with human intelligence.
Too broad to guide engineering decisions.
Machine learning
A set of methods that learn behaviour from data rather than from explicitly programmed rules.
Shifts the source of correctness from code to data and evaluation.
Model
A learned mapping from inputs to outputs, stored as an artifact with parameters and preprocessing.
One component — not the whole product.
ML system
An integrated software system that combines data, learned models, decision logic, serving infrastructure, feedback, and governance to produce predictions or actions under constraints. [1, 4]
Explains why identical models behave differently in different systems.
MLOps
Practices, culture, and tooling that combine ML, DevOps, and data engineering to build, deploy, and maintain ML systems reliably and efficiently. [4, 15]
Makes the lifecycle repeatable, observable, and auditable.
A model is not a system. Production behaviour is produced by data, code, configuration, infrastructure, interfaces, people, and incentives acting together.
What is ML system design?
ML system design — the engineering discipline of specifying, building, and operating systems that train, serve, and maintain learned models under production constraints. It joins two concerns: machine learning, which optimises data and model quality, and distributed systems, which constrain latency, throughput, reliability, and cost. [21]
Aspect
Traditional software system
ML system
Core logic
Hand-written rules and business logic
Learned parameters plus a decision policy
Data
Transactional and schema-managed
High-volume and variable; distributions and labels change
Probabilistic output, deterministic contract. A prediction may be uncertain; the service level — latency, availability, traceability — is still a hard requirement. [21]
Anatomy of an ML system
Data sources and pipelines — events, transactions, sensors, documents, labels, feedback.
Features and metadata — transformations, definitions, lineage, point-in-time correctness.
Model and artifacts — training code, parameters, preprocessing, evaluation evidence.
Decision layer — thresholds, business rules, human review, automation level.
Operations and governance — monitoring, incident response, access control, audit.
Figure 1.0 — The model is one node in a feedback system.
From business objective to ML task
Design rule: write the decision, the error costs, and the feedback delay before choosing an algorithm. [1, 5, 7]
Is ML the right solution?
Signals that ML may fit
The decision repeats at scale.
The outcome is measurable and feedback is available.
The pattern is stable enough to learn but too complex to hand-code.
Inputs are available at prediction time.
Errors can be bounded, reviewed, or reversed.
Signals that it may not
A simple rule or lookup already meets the objective.
Labels are missing, delayed, or expensive.
The decision is rare or irreversible.
The environment changes faster than the system can learn.
Nobody owns the outcome or the failure.
Baseline first: start with the simplest defensible rule, then require ML to beat it on the real decision metric — not just on a model benchmark. [7]
Requirements are multi-dimensional
Class
Question to answer
Example requirement
Functional
What must the system do?
Return a fraud score for every authorization request.
Quality
How good is “good enough”?
Recall ≥ 0.80 at precision ≥ 0.90 on the frozen evaluation set.
Latency & throughput
How fast, at what load?
p95 ≤ 120 ms; sustain 5,000 requests/s at peak.
Reliability
What happens when it fails?
99.95% monthly availability; fall back to rules in ≤ 50 ms.
Cost
What can we spend?
≤ $0.002 per scored request; training ≤ $5k per run.
Security & privacy
What must be protected?
No raw card data in logs; PII encrypted at rest and in transit.
Governance
What evidence is required?
Decision log with model version; human review above $10k.
Non-functional requirements are architecture drivers. They decide whether you need caching, streaming, redundancy, a feature store, or a human review queue.
Make requirements measurable
Example: card-fraud scoring
SLI: measured p95 latency, precision at the operating threshold, alert rate.
Can we explain a decision and prove subgroup performance?
Lineage, slice metrics, model cards, audit logs
Subgroup gap; explanation coverage
These objectives compete. The architecture is the record of which one was traded away, for which workload, and with whose sign-off. [21]
Design trade-offs are unavoidable
Accuracy vs latency / cost
Larger models and richer features can improve quality, but increase compute, memory, and response time.
Freshness vs stability
Frequent updates improve relevance but raise regression risk and operational load.
Automation vs control
Automated decisions scale, but high-impact errors need review, override, and appeal paths.
Personalization vs privacy
User-level features improve fit but increase data sensitivity and regulatory exposure.
Complexity vs maintainability
Bespoke pipelines can fit today's data but become brittle and expensive to change.
Global vs local models
One model is simpler; per-market models fit local behaviour but multiply validation and operations.
Record the decision: options considered, criteria, chosen trade-off, owner, and review date. Undocumented trade-offs become invisible technical debt. [2]
Design review checklist
What decision changes because of this system, and who owns it?
What is the cost of a false positive versus a false negative?
Which inputs are genuinely available at prediction time?
What is the fallback when the model, features, or a dependency is unavailable?
How will we detect that the system is wrong — not just slow or down?
Who can override a prediction, and how is that recorded?
What is the rollback plan, and how quickly can it execute?
What privacy, security, fairness, and audit constraints apply?
What are the unit economics at expected and peak volume?
What evidence is required before launch and before each model change?
Stop rule: if questions 1, 2, and 5 have no clear answer, do not start modelling yet.
1.2
ML Lifecycle and Production System Architecture
Production ML is a loop, not a pipeline that ends at deployment.
The ML lifecycle is iterative
Each stage has an exit gate. Skipping evaluation or monitoring does not remove the step; it only moves the failure into production. Lifecycle variants exist (CRISP-DM, TDSP, MLOps), but the loop is universal. [1, 4]
Gate design: gates should be evidence-based and automated where possible — not meetings that can be skipped under deadline pressure. [3, 4]
Production reference architecture
Figure 1.1 — A reference architecture, not a vendor stack. Solid arrows: data and model flow. Dashed arrow: requests and feedback. Online features are fetched from the feature store at request time. [4, 15]
Read the diagram by boundaries: each arrow is a contract — schema, version, latency, ownership, and failure behaviour must be defined.
Data plane and control plane
Data plane. The components that actually move and transform data and serve requests. It produces the prediction.
Control plane. The components that configure, schedule, observe, secure, and govern the data plane. It decides how the prediction is produced and whether it is allowed.
Failure pattern: the data plane keeps serving while the control plane has deployed a stale configuration. The system looks healthy and returns wrong answers. [6]
Offline path: training and evaluation
Snapshot the data at a consistent point in time; record the version.
Validate schema, ranges, freshness, and label quality before training.
Build features with the same definitions that will be used in serving.
Evaluate on a clean holdout and on critical slices, not just an aggregate metric.
Register the artifact with metrics, lineage, and approval evidence.
Gate promotion on quality, latency, cost, and safety thresholds.
Reproducibility is a system property: if you cannot rebuild the run from recorded inputs, you cannot debug a regression. [3, 4]
Training at scale: parallelism and checkpoints
Strategy
How work is split
Buys you
Costs you
Data parallelism
Replicate the model; split each batch; exchange gradients.
Throughput beyond one device.
Communication grows; the slowest worker sets step time.
Model parallelism
Split layers (pipeline) or operators (tensor) across devices.
Models larger than one device.
Idle bubbles; cross-device traffic.
Sharded data parallelism
Partition parameters, gradients, and optimiser state.
Large models, still data-parallel.
All-gather traffic; tuning complexity.
Parameter server vs all-reduce
Parameter server: workers push gradients to servers that own the weights; asynchronous updates suffer stale gradients and stragglers.
All-reduce: workers exchange gradients collectively; synchronous and bandwidth-bound, so all wait for the slowest.
Checkpoints are a requirement, not a nicety
Save weights, optimiser state, schedule, RNG, and data position together.
Validate a checkpoint by restoring it and reproducing the next step.
On preemption, resume instead of restarting days of work.
Scale changes the failure mode: a distributed job often fails slowly, as lost throughput or quiet divergence. [21]
Features: fetch precomputed online features and compute request-time features.
Model: run the versioned artifact with its packaged preprocessing.
Post-process: calibrate, threshold, apply business rules, format the response.
Telemetry: log inputs, model version, prediction, latency, and outcome link.
Fallback: if a dependency fails, return a safe default or rule-based decision.
The serving path defines the model's real inputs. If preprocessing differs from training, the artifact is no longer the model you evaluated. [1, 2]
Serving fast, failing soft
Caches
Feature cache: entity features keyed by id with a TTL, avoiding a database round trip.
Prediction cache: memoised scores for repeated requests; key on model and feature version.
Artifact cache: weights and preprocessing held warm in memory; load once, never per request.
Indexes
Vector index (HNSW, IVF-PQ): approximate nearest neighbours over embeddings.
Inverted index: lexical retrieval over tokens; exact, interpretable, good for rare terms.
Hash index: exact key lookup, dedupe, and shard routing in constant time.
Retrieval then ranking
Generate cheap candidates from an index.
Score only the shortlist with the expensive model.
Keeps latency bounded while the model stays large.
Fallback ladder: cached score → smaller model → heuristic rule → safe default or abstain. Each rung has an explicit latency budget and logs why it was taken. [21]
Stale is a correctness bug: a cache that outlives the model or data version serves scores from a model that is gone. [21]
Training–serving skew
Training–serving skew. A difference between the data, features, or preprocessing used in training and those used in serving, causing production performance to diverge from offline estimates.
Common causes
Separate batch and online feature code.
Point-in-time errors: using future information in training.
Different defaults for missing values.
Library or model-version mismatch.
Controls
One feature definition used by both paths.
Log training-time feature values and compare with serving.
Contract tests and shadow scoring.
Package preprocessing with the model.
Feature store, registry, and metadata
Feature store
A system for defining, computing, storing, and serving features consistently to offline training and online inference.
A record of datasets, jobs, parameters, artifacts, and their relationships across the lifecycle.
Value: impact analysis, debugging, compliance.
These are control-plane services. They reduce skew and make rollback and audit possible; they do not improve the model directly. Document intended use and limitations with model cards and datasheets. [4, 18, 19]
Automation: CI, CD, and CT
Practice
Trigger
Automates
Evidence / gate
CI — continuous integration
Code, config, or schema change
Build, lint, unit/integration tests, data validation
Service passes integration, load, and canary checks
CT — continuous training
Schedule, new data, drift or performance signal
Retraining, evaluation, candidate registration
Candidate beats baseline and passes quality gates
A passing build is not a passing model. CI must validate data and code; CT must validate model behaviour and decision impact; CD must validate the running system. [3, 15, 16]
Promotion path
Dev → staging → shadow → canary → production. Each step has a defined owner, evidence, and rollback trigger.
Rollback path
Versioned model + versioned config + feature compatibility + a tested switch. Rollback is a first-class feature, not an afterthought.
Architecture quality attributes and anti-patterns
Quality attributes to design for
Reproducibility: rebuild any run and artifact.
Observability: know what happened and why.
Modifiability: change one component without breaking others.
Scalability: meet peak load without redesign.
Security: least privilege, isolation, audit.
Cost efficiency: unit economics at expected volume.
Recurring anti-patterns [2]
Glue code: bespoke scripts holding the system together.
Pipeline jungles: unmanaged, overlapping data paths.
Dead experimental code: abandoned branches still running.
Configuration debt: hidden settings that change behaviour.
Undeclared consumers: downstream users of outputs nobody tracks.
CACE: changing anything changes everything.
Technical debt in ML is systemic. It accumulates at data and configuration boundaries, where correctness is hard to see and ownership is often unclear.
1.3
Batch, Online and Real-Time ML Systems
The operating mode is a product and architecture decision, not a default.
Online is not real-time. Online means “on request.” Real-time means “within a bounded freshness relative to events.” A system can be online but use stale features, or streaming but not user-facing. [1, 6]
Batch architecture
Properties
Throughput-oriented: process large partitions efficiently.
Simple operations: retries, backfills, and replays are straightforward.
Staleness: predictions reflect the last run, not the current moment.
Blast radius: a bad run can write wrong scores for every entity at once.
Controls: watermarks, idempotent sinks, dead-letter queues, lag alerts.
Use when: the value depends on reacting within seconds and the event stream is the natural source of truth. [6]
Hybrid architectures are the norm
Pattern
Training
Inference
Typical use
Offline scoring
Batch
Batch
Churn, risk tiers, reporting
Classic online serving
Batch
Online
Fraud, ranking, recommendations
Fresh-feature serving
Batch + streaming features
Online
Personalization, velocity checks, pricing
Streaming decisions
Batch or online
Streaming
Anomaly detection, live monitoring, triggers
Continual learning
Incremental / online
Online or streaming
Rare, high-risk; needs strong guardrails
Choose per component, not per company. A batch-trained model can serve online; a streaming feature pipeline can feed an online model; a batch scorer can feed a dashboard.
Do not conflate training freshness with serving freshness. Retraining cadence and feature freshness are independent design choices.
Choosing a mode: a worked example
Questions to settle
How fresh must the decision be: seconds, minutes, hours, or days?
What is the end-to-end latency budget?
What is the peak volume and cost per decision?
Can the product tolerate stale or missing predictions?
What is the operational cost of stateful streaming?
What is the cost of a wrong decision versus a late one?
Card fraud
Batch only: nightly analysis; too late to block a transaction.
Online: score at authorization; <100 ms; fetch recent velocity features.
Streaming: update velocity and merchant-risk features as events arrive; alert on bursts.
Decision: hybrid online inference + streaming features + batch retraining.
Rule of thumb: start with the simplest mode that meets the freshness requirement; add streaming only where the value of event-time reaction is measurable. [1, 5]
Failure modes by operating mode
Mode
Characteristic failure
Early signal
Control
Batch
Job silently fails or writes partial/stale results
Freshness SLI, row counts, partition completeness
Data contracts, idempotent writes, output validation, alerting
Observability must match the mode. A batch job needs freshness and completeness checks; an online service needs latency and dependency health; a stream needs lag, ordering, and state integrity. [6, 20]
Batch question
“Is the output complete, current, and valid?”
Online question
“Is the response fast, available, and correct for this request?”
Stream question
“Is the state consistent with the event history?”
1.4
ML System Challenges, Trade-offs and Failure Modes
Failures are usually system failures wearing a model costume.
A failure taxonomy
Layer
Typical failures
Root cause
Control
Data
Missing/stale data, schema change, label leakage, sampling bias, distribution shift
Upstream change, no contracts, unrepresentative collection
Features or preprocessing differ between the two paths.
Batch imputes zero, online imputes the mean; library versions diverge.
Training-serving parity checks on logged requests.
Degradation is the observable, drift is one explanation. A metric drop can also come from a broken dependency, a stale feature, or a decision-policy change. Diagnose before retraining. [21]
Retraining is a treatment, not a diagnosis. If the cause is skew or a pipeline bug, retraining hides the symptom and ships the bug again. [15, 21]
How failures compound
Silent failures are the dangerous ones. A wrong prediction delivered on time looks healthy on every system dashboard; only decision- and outcome-level signals reveal it. The Google Flu parable is the canonical case: strong correlations, weak causal mechanism, and no correction loop. [5, 8, 14]
Case 1 · Zillow Offers (2021)
What happened
Zillow Offers made automated cash offers for homes, relying on forecasts of home prices 3–6 months ahead. [9]
In Q3 2021 it purchased 9,680 homes and ended the quarter with 9,790 in inventory.
It recorded a $304 million inventory write-down after buying above its own estimates of future selling prices.
The Homes segment lost $422 million before taxes; average gross loss was about $80,771 per home sold.
On 2 Nov 2021 it announced a wind-down, a ~25% workforce reduction, and expected a further $240–265 million in Q4 losses.
System view
The system converted a price forecast into an irreversible capital commitment. Forecast error, inventory exposure, and slow feedback compounded: the model was a point estimate, but the business needed a risk-aware decision policy.
Missing controls
Uncertainty-aware offer limits; exposure caps by market; faster mark-to-market feedback; circuit breaker; independent risk review of the decision rule.
Lesson: a point prediction is not a decision policy. When actions are expensive or irreversible, model risk becomes financial risk.
Case 2 · Epic Sepsis Model (2021)
What happened
A proprietary sepsis prediction model implemented at hundreds of US hospitals. [10]
External validation at Michigan Medicine: 27,697 patients, 38,455 hospitalizations.
Hospitalization-level AUC was 0.63 (95% CI 0.62–0.64) — poor discrimination.
Sepsis occurred in 2,552 (7%); the model missed 1,709 (67%) of them.
It alerted for 6,971 of 38,455 hospitalizations (18%), creating a large alert burden.
System view
Vendor-reported performance did not hold in a new population. A low-yield alert at a high threshold trained clinicians to ignore the system. The failure sits at the model–workflow boundary, not only in the model.
Missing controls
External validation; calibration checks; alert-burden and positive-predictive-value analysis; local threshold governance; a plan for clinician trust and escalation.
Lesson: validation is local. Alert fatigue is a system failure, not a user failure.
Case 3 · Thailand diabetic-retinopathy screening (2020)
What happened
A deep-learning system for diabetic eye disease was studied across 11 clinics in Pathum Thani and Chiang Mai over 8 months. [11]
Lighting, room layout, camera handling, and patient mobility produced variable image quality.
“Ungradable” images created tension between the model's quality thresholds and the nurses' workflow.
Nurses had to manage retakes, referrals, and patient expectations; the model's output did not fit the existing process.
The study protocol was changed so specialists reviewed ungradable images with records instead of automatic referral.
System view
The input distribution includes the clinic environment and the people operating the camera. Lab accuracy did not imply clinical utility. Data quality is a socio-technical property, not only an image property.
Missing controls
Human-centred evaluation before scale; image-quality feedback; flexible thresholds; workflow co-design; a clear pathway for uncertain results.
Lesson: deployment context is part of the input distribution. Design the workflow, not just the model.
Case 4 · Commercial health-risk algorithm (2019)
What happened
A widely used commercial algorithm affected millions of patients by predicting health-care costs to identify those needing extra care. [12]
At the same risk score, Black patients were considerably sicker than White patients.
The bias arose because the algorithm predicted cost, not illness; unequal access meant less was spent on Black patients.
Remedying the disparity would increase the share of Black patients receiving additional help from 17.7% to 46.5%.
System view
The label was a convenient proxy for the true target. Standard accuracy metrics can look good while the objective is wrong. The failure is in problem framing and measurement, not in the optimizer.
Missing controls
Outcome-oriented labels; subgroup impact analysis; fairness review; validation of the proxy against the intended construct; ongoing monitoring of allocation outcomes.
Lesson: predicting a convenient proxy can optimize the wrong thing. Bias can be invisible to aggregate accuracy.
Case 5 · Uber ATG, Tempe (2018)
What happened
An automated test vehicle moving at 45 mph struck and killed a pedestrian crossing outside a crosswalk at night. [13]
The ADS detected the pedestrian 5.6 seconds before impact but never accurately classified her as a pedestrian or predicted her path.
By the time a collision was judged imminent, the situation exceeded the braking response specification.
System design precluded emergency braking for collision mitigation; it relied on the operator.
The operator was visually distracted by a phone and looked up about 1 second before impact.
System view
Perception, classification, prediction, planning, and actuation form one chain. Detection alone was not enough. The human “backup” was not engaged, and the machine had no independent braking authority.
Missing controls
Safety envelope and fallback manoeuvre; independent emergency braking; driver monitoring; operator oversight; a safety culture that treats automation complacency as a design risk.
Lesson: a system is only as safe as its weakest handoff. Human backup is not a control if the human is not engaged.
Pattern: in every case, the model was not the only thing that failed. The missing control was usually at a boundary — data, workflow, decision, or human handoff.
Threshold by transaction amount and review capacity
Sepsis alert
Alert fatigue; unnecessary tests
Missed deterioration
High sensitivity may be justified, but burden must be measured
DR screening
Unnecessary referral, travel, anxiety
Missed sight-threatening disease
Uncertain cases need a review pathway, not automatic referral
Credit / hiring
Wrongful denial or rejection
Risky approval or missed talent
Legal and fairness constraints dominate; explainability required
Do not use 0.5 by default. The operating threshold should come from error costs, capacity, and policy — and it should be revisited when the environment changes. [1, 5]
A dashboard is not a control. A control changes behaviour when a threshold is crossed: it blocks a release, fails a pipeline, routes to a fallback, or stops the action. [3, 17]
Incident response and learning
Detect: an SLI, quality signal, or user report crosses a threshold.
Triage: scope, severity, affected users, and immediate risk.
Mitigate: rollback, fallback, disable the feature, or reduce automation.
Diagnose: data, model, system, decision, or organizational layer?
Learn: blame-free postmortem with a timeline and contributing factors.
Change: assign owners and verify that the control actually works.
Artifacts to keep
Decision logs with model and feature versions.
Dashboards for system, data, model, and outcome signals.
Runbooks with rollback and escalation steps.
Incident timeline and postmortem.
Action items with owners and due dates.
Updated risk register and test cases.
Measure what matters: time to detect, time to mitigate, percentage of incidents caught by monitoring rather than users, and repeat-incident rate. [20]
Self-check
Work through these before moving on
Define an ML system and explain why a model alone is not a system.
Turn “reduce customer churn” into a decision, a prediction target, a label, and two non-functional requirements.
Compare batch, online, and streaming for a card-fraud system: freshness, latency, and failure risk.
Name three causes of training–serving skew and one control for each.
Choose one case study. Which layer failed, what looked healthy, and which control was missing?
Why can a model with high AUC still fail in production? Give two distinct reasons.
Pick three of the five objectives. For a fraud scorer, name the trade-off between two of them and the measure you would report.
Classify each as data drift, concept drift, label drift, or skew: (a) mobile traffic doubles; (b) fraudsters copy the detector; (c) online imputes the mean but batch imputes zero; (d) annotations were redefined.
Hint: answer in terms of decisions, data, architecture, and controls — not only model metrics.
Summary
1
Frame the decision
Objectives, error costs, and feedback delay determine whether ML is appropriate and what “good” means.
2
Requirements become architecture
Latency, reliability, cost, privacy, and governance drive components, contracts, and controls.
3
The lifecycle is a loop
Data, model, deployment, monitoring, and retraining are connected by evidence gates.
4
Mode follows the decision
Batch, online, and streaming trade freshness against latency, complexity, and cost.
5
Failures are systemic
Data, model, system, decision, and organizational layers fail together; controls must be layered.
6
Evidence over intuition
External validation, real-world evaluation, and postmortems are how the system learns.
If you remember one thing: production ML is a system, not a model. Design the system around the decision it changes.
References
Huyen, C. Designing Machine Learning Systems. O’Reilly Media, 2022.
Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.-F., Dennison, D. “Hidden Technical Debt in Machine Learning Systems.” Advances in Neural Information Processing Systems (NeurIPS), 2015, pp. 2503–2511.
Breck, E., Cai, S., Nielsen, E., Salib, M., Sculley, D. “The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction.” IEEE International Conference on Big Data, 2017, pp. 1123–1132. DOI: 10.1109/BigData.2017.8258038.
Kreuzberger, D., Kühl, N., Hirschl, S. “Machine Learning Operations (MLOps): Overview, Definition, and Architecture.” IEEE Access, vol. 11, 2023, pp. 31866–31879. DOI: 10.1109/ACCESS.2023.3262138.
Paleyes, A., Urma, R.-G., Lawrence, N. D. “Challenges in Deploying Machine Learning: A Survey of Case Studies.” ACM Computing Surveys, vol. 55, no. 6, 2023, pp. 1–29. DOI: 10.1145/3533378.
Kleppmann, M. Designing Data-Intensive Applications. O’Reilly Media, 2017.
Zinkevich, M. “Rules of Machine Learning: Best Practices for ML Engineering.” Google, 2017. https://developers.google.com/machine-learning/guides/rules-of-ml
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., Bouchachia, A. “A Survey on Concept Drift Adaptation.” ACM Computing Surveys, vol. 46, no. 4, 2014, Article 44. DOI: 10.1145/2523813.
Zillow Group, Inc. “Q3 2021 Shareholder Letter.” 2 November 2021. SEC Exhibit 99.3. https://www.sec.gov/Archives/edgar/data/1617640/000161764021000085/exhibit993.htm
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., Ghous, M., Singh, K. “External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients.” JAMA Internal Medicine, vol. 181, no. 8, 2021, pp. 1065–1070. DOI: 10.1001/jamainternmed.2021.2626.
Beede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., Vardoulakis, L. M. “A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy.” Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, pp. 1–12. DOI: 10.1145/3313831.3376718.
Obermeyer, Z., Powers, B., Vogeli, C., Mullainathan, S. “Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations.” Science, vol. 366, no. 6464, 2019, pp. 447–453. DOI: 10.1126/science.aax2342.
National Transportation Safety Board. Collision Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian, Tempe, Arizona, March 18, 2018. Highway Accident Report NTSB/HAR-19/03, 2019. https://www.ntsb.gov/investigations/Pages/HWY18MH010.aspx
Lazer, D., Kennedy, R., King, G., Vespignani, A. “The Parable of Google Flu: Traps in Big Data Analysis.” Science, vol. 343, no. 6176, 2014, pp. 1203–1205. DOI: 10.1126/science.1248506.
Google Cloud. “MLOps: Continuous Delivery and Automation Pipelines in Machine Learning.” Architecture Center. https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning
Sato, D., Wider, A., Windheuser, C. “Continuous Delivery for Machine Learning (CD4ML).” Martin Fowler, 2019. https://martinfowler.com/articles/cd4ml.html
Tabassi, E. Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1, National Institute of Standards and Technology, 2023. DOI: 10.6028/NIST.AI.100-1.
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., Gebru, T. “Model Cards for Model Reporting.” Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*), 2019, pp. 220–229. DOI: 10.1145/3287560.3287596.
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H., Crawford, K. “Datasheets for Datasets.” Communications of the ACM, vol. 64, no. 12, 2021, pp. 86–92. DOI: 10.1145/3458723.
Beyer, B., Jones, C., Petoff, J., Murphy, N. R. Site Reliability Engineering: How Google Runs Production Systems. O’Reilly Media, 2016.
Ul Haq, F. “ML System Design: A Complete Guide (2026).” System Design Handbook. https://www.systemdesignhandbook.com/guides/ml-system-design/
DOI links resolve at https://doi.org/<DOI>. Web sources were accessed in November 2026.
End of chapter
Questions?
Next: Chapter 02 — Data Engineering for ML Systems