Query Optimization and Best Practices
You'll learn how to write efficient PromQL queries that don't overwhelm your Prometheus server or slow down dashboards.
When you query Prometheus, you're asking it to sift through potentially millions of time-series data points. Poorly crafted queries can consume excessive memory, CPU, and time—like asking a librarian to check every book in the building instead of using the card catalog.
Core Optimization Principles
Reduce Cardinality: High cardinality (many unique label combinations) creates exponentially more time series. Avoid queries that select metrics with wildcard label matching on high-cardinality labels like user_id or request_id. Instead, aggregate or filter to specific values:
# Avoid: selecting all user metrics
rate(requests_total[5m])
# Better: aggregate first
sum(rate(requests_total[5m])) by (endpoint)
Limit Time Ranges: Longer range vectors require more data processing. Use the shortest range that meets your analysis needs. A [1h] range vector processes 12x more data than [5m].
Avoid Expensive Operations: Some functions are computationally heavy. Use rate() instead of increase() when possible for counters. Avoid nested subqueries unless necessary—they multiply processing costs.
Pre-aggregate with Recording Rules: If you repeatedly run complex queries, use recording rules to pre-compute them. This shifts the work from query-time to scrape-time, making dashboards snappier.
Practical Tips
Filter early using label matchers: {job="api", status="500"} is faster than selecting everything then filtering. Keep aggregations simple and meaningful—every by() clause dimension adds complexity.
Key Takeaway: Efficient PromQL queries reduce cardinality, use minimal time ranges, leverage recording rules for repeated calculations, and filter early with specific label matchers to minimize computational overhead.