Metrics vs Logs vs Traces
You'll learn how metrics, logs, and traces differ in their structure, cardinality, and when to use each observability tool.
The Three Pillars of Observability
Think of debugging a distributed system like investigating a crime scene. Metrics are the security camera snapshots showing "someone was here at 3 PM." Logs are the detailed witness statements with full context. Traces are the complete timeline tracking one suspect's journey through multiple locations.
Dimensionality and Cardinality
Metrics are numerical measurements aggregated over time windows—counters, gauges, histograms. They have low cardinality: a few dimensions like service name, endpoint, and status code. Example: http_requests_total{service="api", status="200"} can efficiently represent millions of requests as a single number.
Logs are discrete, timestamped text records with high cardinality. Each request might generate a unique log entry with user IDs, error messages, and stack traces. They provide rich context but are expensive to store and query at scale.
Traces track individual requests across services with medium-to-high cardinality. Each trace has a unique ID and shows the complete path through your system with timing information. They're sampled (not every request) to manage volume.
When to Use Each
Use metrics for dashboards, alerts, and trend analysis ("Is error rate increasing?"). Use logs for debugging specific failures ("Why did user X's request fail?"). Use traces for understanding latency and dependencies ("Which service is slow in this transaction?").
Metrics answer "how many/how much," logs answer "what happened," and traces answer "where did time go."
Key Takeaway: Metrics offer low-cardinality aggregated numbers for alerting, logs provide high-cardinality detailed context for debugging, and traces show request flows across services—use all three together for complete observability.