Rolling Deployment
What you'll learn: How to incrementally replace old model instances with new ones across servers, ensuring zero downtime during live model updates.
The Core Idea
Rolling deployment is a gradual transition strategy where you update your ML model servers one at a time (or in small batches) rather than all at once. Imagine replacing workers on an assembly line during their shift changes—the line never stops, but over time, everyone on the floor is new.
When you have multiple instances of your model running behind a load balancer, rolling deployment replaces them sequentially: update instance 1, wait for health checks, then instance 2, and so on. Traffic continues flowing to healthy instances throughout the process.
How It Works
- Select a subset of model servers (e.g., 1 of 10)
- Remove from load balancer pool temporarily
- Deploy new model version to that instance
- Run health checks to verify the new model serves correctly
- Add back to pool and repeat with next instance
- Monitor metrics continuously during rollout
Unlike canary deployment (which splits traffic between versions), rolling deployment is purely sequential replacement. Unlike blue-green (which maintains two full environments), it uses the same infrastructure, updating in-place.
When to Use It
Rolling deployment works well when:
- You need zero downtime guarantees
- Your model change is low-risk (already tested via shadow/canary)
- You want to limit infrastructure costs (no duplicate environment needed)
- You can tolerate temporary version coexistence during the rollout window
Key Takeaway: Rolling deployment provides zero-downtime model updates by incrementally replacing instances one at a time, balancing safety with resource efficiency through controlled, sequential transitions.