Course contentsShow
← Machine Learning and Deep Learning
Lesson 3042 of 3,53866. Production Monitoring and Data DriftFree lesson

Label Drift Fundamentals

How the distribution of target labels P(Y) shifts over time and its impact on model performance and calibration.

Label Drift Fundamentals

What you'll learn: How the distribution of target labels shifts over time and threatens model performance even when input features remain stable.

What Is Label Drift?

Label drift occurs when the distribution of your target variable P(Y) changes over time, independent of changes in your input features. Unlike concept drift (where the relationship between X and Y changes), label drift means the frequency of different classes or target value ranges shifts.

Think of a fraud detection model: During normal times, fraud might represent 0.5% of transactions. During a holiday shopping season or after a data breach, fraud spikes to 3%. Your model still receives similar-looking transactions (features), but the underlying prevalence of the positive class has fundamentally shifted.

Why Label Drift Matters

Performance degradation: Models trained on one class distribution may perform poorly when deployed against another. A model optimized for 0.5% fraud rate will make different decision boundaries than one expecting 3%.

Calibration breakdown: Probability calibration suffers most. If your model was trained to output well-calibrated probabilities for the original distribution, those probabilities become misleading. A 0.8 predicted probability might actually correspond to 0.95 in the new distribution.

Metric misinterpretation: Precision, recall, and F1 scores shift purely due to base rate changes, making performance comparisons across time periods misleading without adjustment.

Detection Strategies

Monitor the distribution of predicted labels (since true labels arrive late) and compare to training-time distributions. Statistical tests on label proportions or class balance can reveal drift before ground truth becomes available.

Key Takeaway: Label drift changes what your model should predict (the distribution P(Y)), causing miscalibration and performance degradation even when feature distributions remain stable—requiring retraining or recalibration to restore alignment with production reality.