Multi-Window and Multi-Burn-Rate Alerting
You'll learn: How to create smart alerts that catch both sudden incidents and slow degradations before your error budget runs out.
The Problem with Single-Window Alerts
Traditional alerts fire when a threshold is crossed in one fixed time window (e.g., "error rate > 5% for 5 minutes"). This approach misses two critical scenarios:
- Fast burns: A catastrophic failure might consume 10% of your monthly error budget in 10 minutes
- Slow burns: A subtle regression might eat 2% per day, unnoticed until it's too late
Single-window alerts either fire too often (alert fatigue) or too late (budget exhausted).
How Multi-Window, Multi-Burn-Rate Works
This approach monitors error budget consumption at multiple time scales simultaneously:
- Short window + high burn rate: Catches fast-moving incidents (1 hour at 14.4x burn rate)
- Long window + moderate burn rate: Detects slow degradations (6 hours at 6x burn rate)
Think of it like home security: motion sensors catch intruders (fast), while a water leak detector catches slow flooding (slow). You need both.
Each alert layer combines:
- A short window for immediate signal
- A longer window for trend confirmation (reduces false positives)
Example Configuration Logic
For a 99.9% monthly SLO (43.2 minutes of allowed downtime):
- Page immediately: 2% budget consumed in 1 hour (14.4x burn) + confirmed over 5 minutes
- Page during business hours: 5% budget consumed in 6 hours (6x burn) + confirmed over 30 minutes
- Ticket for investigation: 10% consumed in 3 days (2x burn) + confirmed over 6 hours
This layered approach gives you early warning for slow problems while still reacting instantly to outages.
Key Takeaway: Multi-window, multi-burn-rate alerting protects your error budget from both sudden incidents and gradual degradations by monitoring consumption at multiple time scales, dramatically reducing both false positives and delayed detection.