Cardinality Management
What you'll learn: How to prevent metrics systems from collapsing under the weight of too many unique label combinations.
The Cardinality Problem
Cardinality in metrics refers to the number of unique time series your system tracks. Each unique combination of metric name and label values creates a new time series. For example, an HTTP request counter with labels method, path, status, and user_id creates one time series per unique combination:
http_requests_total{method="GET", path="/api/users", status="200", user_id="12345"}
http_requests_total{method="GET", path="/api/users", status="200", user_id="67890"}
If you have 10 methods, 100 paths, 10 status codes, and 1 million users, you potentially have 10 × 100 × 10 × 1,000,000 = 10 billion time series. Time-series databases must store, index, and query all of these—causing memory exhaustion, slow queries, and system crashes.
Managing Cardinality
Limit high-cardinality dimensions: Never use unbounded labels like user_id, session_id, or ip_address. These create exponential growth. Use them in logs or traces instead.
Aggregate at the edge: Group similar values. Instead of tracking every URL path, use patterns: /api/users/:id becomes /api/users/{id}.
Sample when necessary: For debugging scenarios, sample high-cardinality data (e.g., track only 1% of unique identifiers).
Monitor your cardinality: Track the number of active time series. Prometheus exposes prometheus_tsdb_symbol_table_size_bytes and similar metrics to alert you before disaster strikes.
Key Takeaway: Cardinality management is about choosing labels carefully—use low-cardinality dimensions (service, region, status code) in metrics, and save high-cardinality identifiers (user IDs, trace IDs) for logs and traces where they belong.