Backend Storage Considerations
You'll learn: How different storage backends (Cassandra, Elasticsearch, S3) handle trace data retention and the trade-offs between them.
The Challenge of Storing Traces
When you've instrumented your services with OpenTelemetry and chosen a tracing backend like Jaeger, Tempo, or Zipkin, those systems need somewhere to persist trace data. This isn't like storing logs—traces have unique access patterns: you query by trace ID, time ranges, and service tags. Your storage choice dramatically impacts performance, cost, and retention capabilities.
Storage Backend Options
Cassandra is a distributed NoSQL database that excels at write-heavy workloads. It's a natural fit for traces because spans arrive constantly and need fast writes. Cassandra scales horizontally and handles high cardinality well, making it popular for Jaeger deployments. However, it requires operational expertise and dedicated infrastructure.
Elasticsearch offers powerful full-text search and querying capabilities. While it can store traces, it's more resource-intensive than Cassandra for write operations. Elasticsearch shines when you need complex queries across trace attributes, but at the cost of higher memory and CPU usage.
S3 (and object storage) provides incredibly cheap, durable storage. Tempo by Grafana uses this approach—traces are stored as objects, drastically reducing costs. The trade-off? Queries are slower since you're reading from object storage rather than a database optimized for real-time lookups.
Retention Strategy Trade-offs
Short retention (days to weeks) keeps costs manageable but limits historical analysis. Longer retention (months) enables trend analysis but exponentially increases storage costs. Many teams implement tiered retention: keep recent traces in fast storage (Cassandra/Elasticsearch), then age out to cheaper object storage, and eventually delete after a compliance window.
Key Takeaway: Choose Cassandra for high-throughput operational tracing, Elasticsearch for rich querying needs, or S3-based solutions like Tempo for cost-effective long-term storage—your retention strategy should match your actual query patterns and budget constraints.