Storage Capacity Estimation
You'll learn how to calculate the total storage your URL shortener will need over years of operation.
Core Idea
Storage capacity estimation combines your traffic projections with data sizes to predict disk requirements. For a URL shortener, you're storing short codes, original URLs, metadata (creation time, user ID, expiration), and possibly analytics. Unlike in-memory caching decisions, this tells you how many terabytes of persistent storage to provision.
The Calculation Framework
Start with your daily write rate from traffic estimation. If you expect 100 million new URLs per day, multiply by retention period. For a 10-year retention:
- Total URLs: 100M/day × 365 days × 10 years = 365 billion URLs
Now estimate per-record size:
- Short code: 7 bytes (e.g., "aB3xK9z")
- Original URL: ~500 bytes average
- Metadata (timestamps, user ID, etc.): ~100 bytes
- Total per URL: ~607 bytes, round to 1 KB for indexes and overhead
Raw storage: 365 billion × 1 KB = 365 TB
Growth Buffer and Replication
Add 20-30% buffer for unexpected growth and database overhead. With 3x replication for availability:
- Base: 365 TB
- With 30% buffer: 475 TB
- With 3x replication: 1,425 TB (~1.4 PB)
The Business Lens
This number drives hardware budgets, cloud storage costs (AWS S3, EBS pricing), and architectural choices (can you afford geographic distribution?). It also reveals if you need partitioning—1.4 PB won't fit on one machine.
Key Takeaway:
Storage estimation multiplies daily writes by retention period and per-record size, then factors in replication and growth buffers to predict multi-year capacity needs.