Failure Thresholds and Detection
What you'll learn: How to configure the triggers that cause a circuit breaker to transition from closed to open state.
A circuit breaker doesn't flip open after a single failure—that would be too sensitive. Instead, you configure failure thresholds that define when enough errors have occurred to warrant opening the circuit. Think of it like a smoke detector: it doesn't trigger from a single particle, but from sustained detection above a certain concentration.
Core Detection Mechanisms
Failure count threshold is the simplest approach: open the circuit after N consecutive failures (e.g., "open after 5 failed requests in a row"). This works well for low-traffic services but can be too slow for high-volume systems.
Failure rate threshold is more sophisticated: open when the error rate exceeds a percentage over a time window (e.g., "open when >50% of requests fail over 10 seconds"). This adapts better to varying traffic loads.
Error Classification
Not all errors should trigger the circuit breaker. You typically distinguish:
- Transient errors (timeouts, 503s, network failures) → count toward threshold
- Client errors (400, 401, 404) → ignore, these aren't service health issues
- Server errors (500, 502, 503) → count toward threshold
Sampling Windows
The detection window matters enormously. A rolling window continuously evaluates the last N requests or M seconds. A tumbling window resets at fixed intervals. Rolling windows react faster but require more memory to track request history.
Example Configuration
failure_threshold: 50%
minimum_requests: 20
sampling_window: 10 seconds
error_types: [timeout, 500, 502, 503]
This says: "Open the circuit if ≥50% of requests fail, but only after seeing at least 20 requests in the last 10 seconds."
Key Takeaway: Circuit breakers need carefully tuned thresholds and error classification to avoid false positives (opening unnecessarily) while still protecting your system from cascading failures.