Course contentsShow
← System Design
Lesson 27 of 1,9191. System Design Mindset and First PrinciplesFree lesson

Memory Requirements for Caching

Calculate RAM needed for caching based on cache hit ratio, hot data size, and access patterns.

Memory Requirements for Caching

What you'll learn: How to calculate the RAM you need to effectively cache data in your system.

Why Cache Memory Matters

When you estimated bandwidth and storage in previous lessons, you focused on total data. Caching is different—you're keeping a subset of your data in ultra-fast RAM so users get instant responses. But RAM is expensive, so you need to calculate exactly how much you need.

The Core Calculation

Three factors determine your cache memory requirements:

1. Hot Data Size

Not all data is accessed equally. If you have 1TB of total data but only 100GB gets accessed frequently (the "hot" data), you only need to cache that 100GB.

2. Cache Hit Ratio

Your cache hit ratio is the percentage of requests served from cache. If you want a 90% hit ratio, you need enough RAM to store the most popular 90% of accessed data. Higher ratios mean more RAM but faster responses.

3. Access Patterns

If 20% of your content generates 80% of traffic (a common pattern), you can achieve an 80% hit ratio by caching just that 20% of data.

Example Calculation

Imagine a video platform:

  • Total video library: 500TB
  • 5% of videos are watched daily (hot data): 25TB
  • You want an 80% cache hit ratio
  • You need RAM for: 25TB × 0.80 = 20TB of cache

That's spread across many servers—if each has 256GB RAM, you need ~80 cache servers.

The Tradeoff

More cache = higher hit ratio = better performance, but RAM costs money. Balance user experience against infrastructure costs based on your requirements from earlier lessons.

Key Takeaway: Calculate cache memory by identifying your hot data size, target cache hit ratio, and access patterns—typically caching 10-30% of hot data achieves 70-90% hit ratios in real systems.