Cache Memory Requirements
What you'll learn: You'll learn how to calculate the cache size needed to keep the most popular shortened URLs in memory for blazing-fast redirects.
The 80-20 Rule in URL Shorteners
Not all URLs are created equal. In a URL shortener, a small percentage of URLs receive the vast majority of traffic. This follows the Pareto Principle or 80-20 rule: roughly 80% of your requests hit only 20% of your URLs.
Think of it like a coffee shop: most customers order the same few popular drinks (latte, cappuccino, Americano), while hundreds of obscure menu items sit idle. You wouldn't stock ingredients for every drink equally—you'd keep the popular ones front and center.
Calculating Cache Size
Let's work through a concrete example:
Given:
- 200 million total unique URLs stored
- 100 million daily requests
- Average URL metadata: ~500 bytes (short URL, long URL, metadata)
Step 1: Identify hot URLs
20% of 200 million = 40 million URLs generate 80% of traffic
Step 2: Calculate cache memory
40 million URLs × 500 bytes = 20 GB
Step 3: Add overhead
Account for cache metadata (headers, TTL, etc.), typically 20–30% overhead:
20 GB × 1.25 = ~25 GB cache requirement
Why This Matters
With 25 GB of RAM caching hot URLs, you'll serve 80% of redirects directly from memory (sub-millisecond latency) instead of hitting the database. The remaining 20% of requests for less popular URLs still go to disk, but that's acceptable—they're infrequent enough not to overwhelm your system.
Key Takeaway: Using the 80-20 rule, cache the top 20% of URLs to handle 80% of traffic with minimal memory—usually tens of gigabytes for systems handling billions of URLs.