Request Hedging and Redundancy
You'll learn: How sending duplicate requests to multiple backends can reduce tail latency when some service instances respond slowly.
The Core Idea
Imagine you order a ride-share and the app sends your request to three nearby drivers simultaneously—whoever accepts first gets the job. Request hedging applies this strategy to distributed systems: when you need a response quickly, you send the same request to multiple backend instances and use whichever responds first, canceling the others.
This pattern addresses a common partial failure scenario: not complete unavailability, but unpredictable slowness. One server might be experiencing garbage collection pauses, disk contention, or momentary CPU spikes. Rather than waiting for that slow instance, hedging lets you benefit from the fastest available responder.
How It Works
You send the original request to one backend. If it doesn't respond within a threshold (say, 50ms—your P95 latency), you immediately send duplicate requests to one or more other instances. The first successful response wins; you cancel or ignore the rest.
Trade-offs:
- Reduces tail latency dramatically—users experience fewer slow requests
- Increases load on your backends (you're doing extra work)
- Requires idempotent operations (safe to execute multiple times)
- Works best when failures are uncorrelated (instances fail independently)
When to Use It
Hedging shines for read operations with tight latency requirements where backend capacity exists. It's particularly effective when your P99 latency is much worse than P50—hedging "fills in" those slow outliers. This complements your bulkhead pattern by better utilizing available resources across isolated backend pools.
Key Takeaway: Request hedging sends duplicate requests to multiple backends and accepts the first response, trading increased backend load for dramatically reduced tail latency—especially valuable for user-facing read operations.