Threshold Selection: Static vs. Dynamic Baselines
You'll learn: How to choose between fixed thresholds and adaptive baselines when configuring alerts that actually reflect real-world traffic patterns.
The Core Dilemma
When you set up alerts based on metrics (like response time or error rate), you need to decide what value triggers the alarm. You have two main approaches:
Static thresholds are fixed values that never change. "Alert if CPU > 80%" or "Alert if error rate > 5%." Simple, predictable, but blind to context.
Dynamic baselines adapt to normal patterns over time. They learn that your API gets 1000 req/sec at noon but only 50 req/sec at 3am, and adjust what "normal" means accordingly.
When to Use Each
Static Thresholds Work Best For:
- Hard limits: disk space at 95%, memory exhausted
- SLO boundaries: you promised 99.9% uptime, that's your line
- Simple systems: small apps with predictable, consistent traffic
- Safety thresholds: some values are always dangerous regardless of patterns
Dynamic Baselines Work Best For:
- Seasonality: e-commerce sites with weekend spikes, B2B apps quiet at night
- Growing systems: traffic doubles yearly; yesterday's "high" is today's normal
- Anomaly detection: spotting unusual patterns that static rules miss
- Rate-of-change: sudden drops (100→10 req/sec) matter even if both are "low"
The Analogy
Static thresholds are like a home thermostat set to 72°F year-round. Dynamic baselines are like your body temperature—98.6°F is normal, but 99.5°F signals something's wrong even though it's "not that high."
Key Takeaway: Use static thresholds for absolute limits and contractual obligations; use dynamic baselines when normal behavior varies predictably or when you need to detect anomalies in evolving systems.