Cache Memory Hierarchies
You'll learn how caching layers stack from ultra-fast CPU caches down to slower application caches, each trading speed for capacity.
The Tower of Speed
Modern systems use multiple levels of caching, arranged like a pyramid: smaller and faster at the top, larger and slower at the bottom. Each level serves as a backup for the one above it, creating a seamless memory hierarchy that balances performance against cost.
From Silicon to Software
CPU Caches (L1/L2/L3) sit closest to the processor cores. L1 cache is tiny (32-64 KB) but blazingly fast (~1 nanosecond access). L2 is slightly larger (256 KB-1 MB, ~3-10 ns), and L3 is shared across cores (4-32 MB, ~10-40 ns). You rarely control these directly, but understanding them helps explain why cache-friendly data structures matter.
RAM serves as the next tier (~100 ns access time). Your application's in-memory caches live here—Redis, Memcached, or simple hash maps. These are orders of magnitude slower than L1 but vastly larger (gigabytes).
Disk-Based Caches (SSDs or HDDs) provide even more space at the cost of milliseconds per access. CDNs, browser caches, and database buffer pools often operate at this level.
The Trade-Off Principle
Each level is roughly 10-100x slower but 10-100x larger than the one above. The art of caching is keeping your "hottest" data (most frequently accessed) in the fastest tier possible. A cache miss at L1 falls through to L2, then L3, then RAM—each step adding latency.
Think of it like storing frequently-used tools: your pocket (L1) holds one screwdriver, your toolbelt (L2) holds a few more, your toolbox (RAM) holds dozens, and your garage (disk) holds everything.
Key Takeaway: Cache hierarchies create layers of progressively slower but larger storage, and effective caching strategies place hot data in the fastest accessible tier to minimize average access time.