Metric Retention Policies
What you'll learn: How to balance the cost of storing time-series metrics with your observability needs using tiered retention and downsampling.
The Core Challenge
Metrics are extremely valuable—but storing millions of data points per second forever is prohibitively expensive. A single counter metric scraped every 15 seconds generates 5,760 data points per day. Multiply that by thousands of metrics and months of retention, and your storage costs explode.
Metric retention policies define how long you keep data at different resolutions. The key insight: you don't need the same detail level forever. Last hour's CPU spike matters at 15-second granularity. Last year's trend? One-hour averages suffice.
Tiered Retention Strategy
Think of metric retention like photo storage: keep recent vacation photos at full resolution, but compress older albums to save space.
Common retention tiers:
- Raw/high-resolution (1-15 seconds): Keep 1-7 days for debugging active incidents
- Medium-resolution (1-5 minutes): Keep 30-90 days for recent trend analysis
- Low-resolution (1 hour): Keep 1-2 years for long-term capacity planning
Downsampling
Downsampling reduces data resolution by aggregating raw points. Instead of 240 fifteen-second samples per hour, store four values: min, max, avg, and maybe p95.
This preserves critical statistical properties while dramatically reducing storage. A counter like request rate can be safely averaged; histograms retain percentile buckets at coarser granularity.
Prometheus's default is 2 weeks raw, then external systems like Thanos or Cortex downsample for long-term storage.
Key Takeaway: Define retention policies that match your observability needs—keep high-resolution data for recent troubleshooting, downsample aggressively for historical trends, and delete what you'll never query to control costs.