Course contentsShow
← System Design
Lesson 1095 of 1,91928. Timeouts and DeadlinesFree lesson

Timeout Configuration and Tuning

Strategies for setting initial timeout values, measuring actual latencies, and iteratively tuning over time.

Timeout Configuration and Tuning

What you'll learn: How to systematically set, measure, and refine timeout values to match your system's actual performance characteristics.

The Art of Setting the Right Timeout

Setting timeouts isn't a one-and-done task—it's an iterative process. Too aggressive, and you'll kill healthy requests during normal latency spikes. Too generous, and you'll waste resources waiting for dead connections while cascading failures spread.

Think of it like setting the thermostat in a house you've never lived in before: you make an educated guess, observe how it feels over time, and adjust based on real experience.

Starting with Reasonable Defaults

Begin with conservative estimates based on expected behavior:

  • Network calls to nearby services: 1-3 seconds
  • Database queries: 500ms-2 seconds
  • External third-party APIs: 5-10 seconds

These aren't magic numbers—they're starting points that give you breathing room while you gather data.

Measure, Don't Guess

Once your system runs under real traffic, instrument everything. Track:

  • P50, P95, P99, and P99.9 latencies for each operation
  • Distribution of response times throughout the day
  • Error rates correlated with timeout values

Your actual P99 latency should inform your timeout floor. If requests complete in 800ms at P99, a 500ms timeout guarantees 1% failure rate from timeouts alone.

Iterative Tuning

Set your timeout to something like P99 + buffer (often 1.5-2× P99). Monitor for:

  • Timeout-related errors (are you timing out too early?)
  • Resource exhaustion (are you waiting too long?)
  • User experience degradation

Gradually tighten timeouts during stable periods, then loosen if you see elevated timeout errors during traffic spikes or degraded downstream performance.

Key Takeaway: Timeout tuning is a continuous feedback loop—start conservative, measure real-world latencies obsessively, and adjust incrementally based on P99 data and error rates, not gut feeling.