Drift is the reason a good model degrades over time, and detecting it is central to keeping production ML healthy. But "drift" covers two genuinely different phenomena that people conflate, and they call for different detection and different responses. Getting the distinction right is what makes drift detection actionable rather than just an alarm that something changed.
Data drift versus concept drift
| Type | What changed | Example |
|---|---|---|
| Data drift | The input distribution — the model sees different inputs | A new customer demographic your model never trained on |
| Concept drift | The input-output relationship itself | Fraud patterns evolve, so the same features now mean something different |
Data drift means the inputs moved but the underlying rules still hold; the model may cope or may be extrapolating badly. Concept drift means the very relationship the model learned has changed, so even unchanged inputs now map to different outcomes — the more dangerous kind, because the model's learned logic is now wrong. They need different responses, which is why naming which one you have matters.
Detecting each
Data drift is detectable without labels: compare the distribution of recent input features against the training distribution using statistical tests or distance measures, and flag when they diverge. Because it needs no ground truth, it is your fast, always-available early warning. Concept drift is harder — it ultimately requires ground-truth labels to confirm, because you are detecting a change in the input-output relationship. Where labels are delayed, a sustained drop in a proxy or a rise in prediction drift can hint at concept drift before labels confirm it.
Thresholds and action
Detection is only useful with sensible thresholds and a defined response. Set thresholds that distinguish meaningful drift from noise — too sensitive and you drown in false alarms and start ignoring them; too loose and you miss real decay. And decide in advance what a drift alert triggers: investigate, retrain on recent data, or escalate. Drift detection wired to no response is just a dashboard nobody acts on.
Use data drift as the early, label-free warning that conditions have shifted, and treat it as a prompt to investigate rather than proof the model is wrong. Reserve the conclusion that the model has actually decayed for when quality metrics or concept-drift signals confirm it. Confusing 'the inputs changed' with 'the model is broken' leads to needless retraining; keeping them distinct keeps your response proportionate.
Drift detection separates data drift — a shift in inputs, detectable without labels and useful as an early warning — from concept drift, a change in the input-output relationship that ultimately needs labels to confirm and signals genuine model decay. Detect each with appropriate methods, set thresholds that cut noise without hiding real drift, and wire every alert to a defined action, so drift detection actively protects model quality instead of merely reporting that the world moved.
Independent software engineer in Nairobi specialising in Acumatica customisations, Laravel backends, and tax fiscalisation integrations across East and Southern Africa.