Course contentsShow
← Software Engineering
Lesson 1205 of 1,93336. Performance EngineeringFree lesson

Percentiles and the Long Tail

Why p50, p95, and p99 percentiles reveal more than averages, and how outliers impact user experience.

Percentiles and the Long Tail

You'll learn why averages hide critical performance problems and how percentiles reveal the true user experience.

Why Averages Lie

Imagine a restaurant where 95 customers wait 2 minutes for food, but 5 customers wait 60 minutes. The average wait time is about 5 minutes—sounds reasonable! But those 5 unlucky customers had a terrible experience, and your average completely hid that reality.

This is the "long tail" problem: a small percentage of requests take dramatically longer than the rest, dragging out into a "tail" on the right side of a distribution graph.

Understanding Percentiles

Percentiles tell you what percentage of requests completed at or below a certain latency:

  • p50 (median): 50% of requests were faster than this
  • p95: 95% of requests were faster than this (5% were slower)
  • p99: 99% of requests were faster than this (1% were slower)

If your API's p50 latency is 20ms but p99 is 5 seconds, you have a serious long-tail problem. One in every 100 requests is painfully slow.

Why This Matters

Users don't experience averages—they experience individual requests. If 1% of your users hit 5-second delays, and you have millions of users, that's thousands of frustrated people daily. Heavy users who make many requests are more likely to hit these outliers.

The long tail often exposes real issues: garbage collection pauses, cache misses, retry storms, or resource contention that averages completely mask.

Key Takeaway: Always measure p95 and p99 latencies alongside averages—they reveal the outliers that significantly degrade user experience for your most active users.