Course contentsShow
← AI Engineering
Lesson 1423 of 1,88634. Data Flywheels and Continuous ImprovementFree lesson

Scheduling and Triggering Retraining

Deciding between time-based, data-volume-based, and performance-based triggers for model updates.

Scheduling and Triggering Retraining

What you'll learn: How to decide when your model needs an update by choosing the right trigger mechanism for automated retraining.

The Core Decision

Once you've built a continuous improvement pipeline, you face a critical question: when should retraining actually happen? There are three primary strategies, each with distinct trade-offs:

Time-Based Triggers

Schedule retraining at fixed intervals (daily, weekly, monthly). This is predictable and easy to manage—like changing your car's oil every 3,000 miles regardless of driving conditions. The downside? You might retrain too often when nothing has changed, or too rarely when your model is rapidly degrading.

Data-Volume-Based Triggers

Retrain after collecting N new examples (e.g., 10,000 user interactions). This approach scales naturally with system usage. A high-traffic period automatically triggers more frequent updates. However, data volume doesn't guarantee quality or distribution shift—you might accumulate lots of redundant examples without meaningful signal.

Performance-Based Triggers

Monitor your production KPIs continuously and trigger retraining when metrics degrade beyond a threshold (e.g., accuracy drops 2% or user satisfaction falls below 4.0). This is the most adaptive approach, responding to actual model degradation rather than arbitrary schedules. The challenge is defining reliable thresholds and avoiding false alarms from natural metric fluctuation.

Hybrid Approaches

In practice, many systems combine triggers: "Retrain weekly or when we collect 50,000 new examples or if F1-score drops below 0.85." This balances predictability with responsiveness.

Key Takeaway: Choose time-based triggers for stability and predictability, data-volume triggers to scale with usage, and performance-based triggers to respond to actual degradation—or combine all three for a robust retraining strategy.