Canary and Blue-Green Deployments
What you'll learn: How to safely roll out new ML models to production by testing them gradually (canary) or switching instantly between stable versions (blue-green).
The Core Idea
When you deploy a new model version, you risk breaking production if something goes wrong. Two strategies minimize this risk:
Canary deployments route a small percentage of traffic (say 5%) to the new model while the rest hits the old version. You monitor performance, accuracy, latency, and errors. If metrics look good, you gradually increase traffic to 25%, 50%, 75%, then 100%. If something breaks, you roll back immediately.
Blue-green deployments run two identical production environments. "Blue" serves all traffic with the current model. You deploy the new model to "Green" and test it thoroughly while it handles zero real users. When ready, you flip a switch—routing 100% of traffic to Green instantly. Blue becomes your instant rollback target.
When to Use Each
Use canary when you want gradual validation with real traffic and can tolerate some users seeing the new model's behavior. This pairs perfectly with A/B testing (lesson 1614) and model shadows where you compare outputs side-by-side.
Use blue-green when you need atomic cutover—all users see consistent behavior, and you have strong confidence from pre-deployment testing. It's simpler but more binary: the new model works or it doesn't.
Practical Connection
Both strategies rely on your serving framework (lesson 1607) supporting traffic routing and your model registry (lesson 1610) tracking multiple deployed versions simultaneously. Your monitoring must track version-specific metrics to catch degradation early.
Key Takeaway: Canary deployments reduce risk through gradual rollout with real traffic monitoring, while blue-green deployments enable instant, all-or-nothing switches between fully tested environments.