Course contentsShow
← Cloud and Infrastructure
Lesson 1640 of 2,98933. Service Mesh FeaturesFree lesson

Performance Metrics and Histograms

Analyze request duration histograms and percentile metrics for SLO monitoring.

Performance Metrics and Histograms

What you'll learn: How to use request duration histograms and percentile metrics to monitor and enforce SLOs in a service mesh.

Understanding Histograms vs Simple Averages

When monitoring service performance, average response time tells only part of the story. Imagine a coffee shop where most orders take 2 minutes, but 5% take 20 minutes—the average might look decent, but those outliers destroy user experience. Histograms capture the full distribution of request durations, grouping measurements into buckets (e.g., 0-10ms, 10-50ms, 50-100ms, 100-500ms, etc.).

Percentiles for SLO Monitoring

Service meshes automatically collect histogram data for request durations. This enables you to calculate percentile metrics:

  • p50 (median): 50% of requests complete faster than this value
  • p95: 95% of requests complete faster than this value
  • p99: 99% of requests complete faster than this value

These percentiles are critical for SLOs (Service Level Objectives). For example, you might target "p99 latency under 200ms" rather than "average latency under 100ms"—the former protects against tail latencies that impact real users.

Why Histograms Matter in Service Mesh

Your service mesh data plane collects timing data for every request transparently. Instead of manually instrumenting code, you get histogram metrics automatically via Prometheus or similar systems. These histograms let you:

  • Detect performance regressions before they violate SLOs
  • Identify which services contribute to tail latency
  • Make informed decisions during canary deployments (compare p99 latencies between versions)
  • Alert on degradation that simple averages would hide

Key Takeaway: Histograms and percentile metrics reveal the true performance distribution, making them essential for meaningful SLO monitoring and catching outliers that averages obscure.