Load Shedding Under Cascading Failure
What you'll learn: How to intentionally reject requests early to prevent complete system collapse during cascading failures.
The Core Idea
When your system detects cascading failure conditions—rising latency, growing queues, circuit breakers tripping—the counterintuitive solution is to deliberately drop requests before they enter the system. This prevents saturation and preserves capacity for requests that can succeed.
Think of it like a lifeboat with limited capacity. When the ship is sinking and everyone rushes to board, accepting everyone means the lifeboat sinks too. By limiting who boards (load shedding), you save those you can actually support.
How Load Shedding Works
Instead of accepting all incoming traffic and letting it overwhelm your degraded system:
- Monitor health signals: queue depth, response times, error rates, circuit breaker states
- Define rejection criteria: thresholds that indicate your system can't handle current load
- Reject early at the edge: return errors (like HTTP 503) immediately, before consuming resources
- Preserve capacity: for critical operations or users that can still be served
Strategic Rejection
Load shedding isn't random—prioritize what to shed:
- Drop non-critical requests first (analytics, recommendations)
- Preserve critical path operations (checkout, authentication)
- Reject low-priority users before premium users
- Fail fast with clear error messages so clients can retry later
The Paradox
Counterintuitively, serving fewer requests during a cascading failure often means serving more successfully than attempting to handle everything and failing completely. It's about maximizing throughput under constraints, not capacity.
Key Takeaway: Load shedding intentionally rejects requests at system edges when cascading failure is detected, preventing complete saturation and preserving capacity for requests the degraded system can actually handle—sacrificing some traffic to save the rest.