Course contentsShow
← System Design
Lesson 1078 of 1,91927. Circuit Breakers and BulkheadsFree lesson

Cascading Failure Propagation Mechanics

How failures spread: timeouts accumulate, thread pools exhaust, queues back up, and resources deplete across service boundaries.

Cascading Failure Propagation Mechanics

What You'll Learn

You'll learn exactly how failures spread through distributed systems—the mechanical chain reaction that turns a small problem into a system-wide outage.

The Propagation Chain

When a service slows down or fails, the damage doesn't stay contained. Here's the precise mechanism:

1. Timeouts Accumulate

Service A calls Service B (which is struggling). Service A waits for its timeout—say 5 seconds. Now Service A is slower. Service C calls Service A and also waits. Timeouts stack: C waits for A's timeout + B's timeout. A 5-second delay becomes 10+ seconds upstream.

2. Thread Pools Exhaust

While Service A waits for B, its threads are blocked. New requests arrive but no threads are available to handle them. The thread pool fills up completely. Service A stops accepting requests—it's now effectively down too.

3. Queues Back Up

Requests that can't get threads sit in queues. Memory fills. Queue depths explode from 10 to 10,000. Eventually the queue is full, and new requests are rejected outright.

4. Resources Deplete Across Boundaries

Service A holds database connections while waiting for B. Those connections never close. Connection pools exhaust. Now other services using the same database can't get connections either—failure crosses service boundaries.

The domino effect: One slow dependency → blocking calls → exhausted resources → more blocking → upstream services fail → their dependencies fail → total system collapse.

This is why we need circuit breakers (stop calling failing services) and bulkheads (isolate resources so one failure can't consume everything).

Key Takeaway:
Cascading failures propagate mechanically through timeout accumulation, resource exhaustion, and queue saturation—turning localized issues into system-wide outages without proper isolation patterns.