Data / ML · MLOps

ML Drift Detection — A Field Guide

Drift is why models decay. Detecting it means distinguishing a change in the inputs from a change in the underlying relationship — two different problems with two different fixes.

John Kihiu12 min read

Drift is the reason a good model degrades over time, and detecting it is central to keeping production ML healthy. But "drift" covers two genuinely different phenomena that people conflate, and they call for different detection and different responses. Getting the distinction right is what makes drift detection actionable rather than just an alarm that something changed.

Data drift versus concept drift

TypeWhat changedExample
Data driftThe input distribution — the model sees different inputsA new customer demographic your model never trained on
Concept driftThe input-output relationship itselfFraud patterns evolve, so the same features now mean something different

Data drift means the inputs moved but the underlying rules still hold; the model may cope or may be extrapolating badly. Concept drift means the very relationship the model learned has changed, so even unchanged inputs now map to different outcomes — the more dangerous kind, because the model's learned logic is now wrong. They need different responses, which is why naming which one you have matters.

Detecting each

Data drift is detectable without labels: compare the distribution of recent input features against the training distribution using statistical tests or distance measures, and flag when they diverge. Because it needs no ground truth, it is your fast, always-available early warning. Concept drift is harder — it ultimately requires ground-truth labels to confirm, because you are detecting a change in the input-output relationship. Where labels are delayed, a sustained drop in a proxy or a rise in prediction drift can hint at concept drift before labels confirm it.

Thresholds and action

Detection is only useful with sensible thresholds and a defined response. Set thresholds that distinguish meaningful drift from noise — too sensitive and you drown in false alarms and start ignoring them; too loose and you miss real decay. And decide in advance what a drift alert triggers: investigate, retrain on recent data, or escalate. Drift detection wired to no response is just a dashboard nobody acts on.

Data drift warns; concept drift confirms decay

Use data drift as the early, label-free warning that conditions have shifted, and treat it as a prompt to investigate rather than proof the model is wrong. Reserve the conclusion that the model has actually decayed for when quality metrics or concept-drift signals confirm it. Confusing 'the inputs changed' with 'the model is broken' leads to needless retraining; keeping them distinct keeps your response proportionate.

Drift detection separates data drift — a shift in inputs, detectable without labels and useful as an early warning — from concept drift, a change in the input-output relationship that ultimately needs labels to confirm and signals genuine model decay. Detect each with appropriate methods, set thresholds that cut noise without hiding real drift, and wire every alert to a defined action, so drift detection actively protects model quality instead of merely reporting that the world moved.

John Kihiu
Acumatica ERP Developer · Laravel Engineer

Independent software engineer in Nairobi specialising in Acumatica customisations, Laravel backends, and tax fiscalisation integrations across East and Southern Africa.