Course contentsShow
← System Design
Lesson 1048 of 1,91927. Circuit Breakers and BulkheadsFree lesson

Failure Thresholds and Detection

Configuring failure rate thresholds and error types that trigger circuit breaker state transitions.

Failure Thresholds and Detection

What you'll learn: How to configure the triggers that cause a circuit breaker to transition from closed to open state.

A circuit breaker doesn't flip open after a single failure—that would be too sensitive. Instead, you configure failure thresholds that define when enough errors have occurred to warrant opening the circuit. Think of it like a smoke detector: it doesn't trigger from a single particle, but from sustained detection above a certain concentration.

Core Detection Mechanisms

Failure count threshold is the simplest approach: open the circuit after N consecutive failures (e.g., "open after 5 failed requests in a row"). This works well for low-traffic services but can be too slow for high-volume systems.

Failure rate threshold is more sophisticated: open when the error rate exceeds a percentage over a time window (e.g., "open when >50% of requests fail over 10 seconds"). This adapts better to varying traffic loads.

Error Classification

Not all errors should trigger the circuit breaker. You typically distinguish:

  • Transient errors (timeouts, 503s, network failures) → count toward threshold
  • Client errors (400, 401, 404) → ignore, these aren't service health issues
  • Server errors (500, 502, 503) → count toward threshold

Sampling Windows

The detection window matters enormously. A rolling window continuously evaluates the last N requests or M seconds. A tumbling window resets at fixed intervals. Rolling windows react faster but require more memory to track request history.

Example Configuration

failure_threshold: 50%
minimum_requests: 20
sampling_window: 10 seconds
error_types: [timeout, 500, 502, 503]

This says: "Open the circuit if ≥50% of requests fail, but only after seeing at least 20 requests in the last 10 seconds."

Key Takeaway: Circuit breakers need carefully tuned thresholds and error classification to avoid false positives (opening unnecessarily) while still protecting your system from cascading failures.