Course contentsShow
← System Design
Lesson 1120 of 1,91928. Timeouts and DeadlinesFree lesson

Hedged Requests for Tail Latency

Issue duplicate requests after a timeout threshold to reduce tail latency without full retry cost.

Hedged Requests for Tail Latency

What you'll learn: How to issue duplicate requests after a timeout threshold to reduce tail latency without paying the full cost of complete retries.

The Core Idea

Hedged requests are a tail-latency optimization where you send a duplicate request to a different server after a small timeout (typically P95 or P99 latency), without waiting for the original request to fail completely. Whichever response comes back first wins, and you cancel the other.

Unlike full retries that wait for failure or timeout, hedging proactively launches a "backup" request when the first one seems slow. This dramatically reduces high-percentile latency (P99, P99.9) with minimal extra load.

How It Works

You set a hedging threshold based on historical latency percentiles—say, 100ms at P95. If the first request hasn't returned after 100ms, you immediately issue a duplicate to another replica. The moment either request completes, you use that response and cancel the other.

The timing matters: hedge too early and you double traffic; hedge too late and you gain nothing. The sweet spot is usually around P95-P99 of normal latency.

The Benefit

Hedging targets the worst cases. Most requests (95%+) complete before the hedge fires, adding zero overhead. Only slow requests trigger the hedge, and even a small percentage of hedges can cut P99 latency in half because the second request often hits a faster server or avoids whatever caused the first delay (GC pause, network hiccup, etc.).

This is cheaper than aggressive retries because you're not waiting for failure—you're racing two requests.

Key Takeaway: Hedged requests reduce tail latency by issuing duplicate requests after a percentile-based timeout threshold, allowing you to cancel whichever is slower—minimal cost for dramatic P99 improvement.