Course contentsShow
← Software Engineering
Lesson 1143 of 1,93334. Logging and ObservabilityFree lesson

Percentiles and Aggregation

Using p50, p95, p99 to understand distribution, and why averages hide critical latency outliers.

Percentiles and Aggregation

What you'll learn: Why percentiles like p50, p95, and p99 reveal the true user experience that averages conceal.

The Problem with Averages

When measuring response times, averages lie. Imagine a web service where 95% of requests complete in 100ms, but 5% take 5 seconds. The average might be around 350ms—which sounds fine—but 1 in 20 users is waiting five full seconds, experiencing a broken service.

Averages smooth out the pain. They give you a single number that hides the worst experiences your users are having.

Enter Percentiles

Percentiles slice your data distribution to show what percentage of requests fall below a certain threshold:

  • p50 (median): Half of all requests are faster than this value
  • p95: 95% of requests are faster; only 5% are slower
  • p99: 99% are faster; this captures the "worst" 1% of experiences

If your p50 is 100ms but your p99 is 5 seconds, you know that most users are fine, but your worst-off users are suffering badly.

Why This Matters

Real users don't experience "average latency"—they experience their request. Percentiles tell you:

  • Who's hurting: High p95/p99 values reveal outliers that averages mask
  • Where capacity issues hide: Slow tail latencies often indicate resource contention, retries, or downstream failures
  • What SLAs should target: Service level objectives often use p95 or p99 ("99% of requests under 200ms")

Analogy: If the average depth of a river is 3 feet, that's safe—unless one section is 10 feet deep. You need to know the distribution, not just the mean.

Key Takeaway: Always monitor p95 and p99 percentiles alongside averages; they reveal the critical latency outliers that destroy user experience even when your average looks healthy.