Course contentsShow
← System Design
Lesson 1270 of 1,91932. Monitoring and AlertingFree lesson

Monitoring Resolution and Retention Tradeoffs

Balancing metric collection frequency and storage duration against cost, balancing granularity with long-term trend analysis.

Monitoring Resolution and Retention Tradeoffs

You'll learn how to balance the frequency of metric collection and storage duration against operational costs and analytical needs.

The Tradeoff Explained

Think of monitoring like recording a video: high resolution (frames per second) and long retention (storage time) both increase quality and utility, but they also dramatically increase storage costs. In monitoring, you face the same dilemma with metrics.

Resolution is how often you collect data points (every second? every minute?). Higher resolution catches short-lived spikes and anomalies but generates more data.

Retention is how long you store data. Longer retention enables year-over-year comparisons and trend analysis but multiplies storage requirements.

Finding the Balance

Most systems use tiered strategies:

  • High resolution, short retention: 1-second intervals kept for 24 hours—perfect for debugging live incidents
  • Medium resolution, medium retention: 1-minute intervals kept for 30 days—balances detail with historical analysis
  • Low resolution, long retention: 5-minute or hourly aggregates kept for 1+ years—captures long-term trends cheaply

This mirrors what you learned about Aggregation and Roll-Ups (lesson 1179) and Metrics Federation and Long-Term Storage (lesson 1206).

Practical Considerations

Your resolution needs depend on your system's behavior. A payment API might need second-level granularity to catch fraud patterns, while daily active users can be sampled every 5 minutes without information loss.

Storage costs scale linearly with both factors: doubling resolution OR retention doubles your storage bill. A metric collected every second for a year costs 3,600x more than the same metric collected hourly.

Key Takeaway: Design monitoring resolution and retention policies based on actual analytical needs—collect frequently and keep briefly for operational debugging, then downsample and aggregate for long-term trend analysis to control costs while preserving critical visibility.