Course contentsShow
← Software Engineering
Lesson 1150 of 1,93334. Logging and ObservabilityFree lesson

Sampling Strategies

Understanding why sampling is necessary at scale and the trade-offs between head-based, tail-based, and adaptive sampling.

Sampling Strategies

You'll learn why tracing every single request is impractical at scale and how to choose which traces to keep.

When your distributed system handles millions of requests per hour, collecting complete trace data for every single transaction becomes prohibitively expensive. Storage costs explode, network bandwidth suffers, and analysis tools slow to a crawl. This is where sampling strategies come in—the art of intelligently deciding which traces to keep and which to discard.

Why Sampling Matters

Think of sampling like taking photographs at a concert. You don't need a photo every millisecond to understand what happened—a few well-chosen shots tell the story while keeping your phone's storage manageable.

Head-Based Sampling

This approach makes the sampling decision at the start of a request, before any spans are created. You might decide "keep 1% of all traces" and apply this uniformly.

Pros: Simple, predictable overhead, no buffering needed Cons: You might drop the exact error trace you needed to debug, because the decision was made before knowing the outcome

Tail-Based Sampling

This strategy waits until the trace completes, then decides whether to keep it based on what actually happened—duration, errors, specific services involved.

Pros: Keep all errors and slow requests while sampling normal traffic Cons: Requires buffering complete traces, adds latency and complexity, harder to implement across distributed collectors

Adaptive Sampling

Dynamically adjusts sampling rates based on traffic volume, error rates, or system load—essentially combining aspects of both approaches with real-time intelligence.

Pros: Balances coverage with cost automatically Cons: Most complex to implement and tune correctly

Key Takeaway: Sampling is essential for production observability at scale; head-based sampling is simple but blind to outcomes, tail-based sampling is smarter but more expensive, and adaptive sampling tries to get the best of both worlds.