CPU and Disk Caching Layers
What you'll learn: How hardware-level caching in CPUs and operating systems forms the invisible foundation beneath all software caching strategies.
The Hardware Foundation
Before you write a single line of code, multiple layers of caching are already working to speed up your applications. These hardware and OS-level caches are so fundamental that every program benefits from them automatically.
CPU caches (L1/L2/L3) are tiny, ultra-fast memory banks built directly into your processor:
- L1 cache (~32-64 KB): Fastest, closest to CPU cores, accessed in ~1 nanosecond
- L2 cache (~256 KB-1 MB): Slightly slower, often per-core or shared between pairs
- L3 cache (~8-32 MB): Slowest CPU cache, shared across all cores, still 10-100x faster than RAM
Think of these like having sticky notes (L1) on your desk, a drawer (L2) beside you, and a filing cabinet (L3) in your office—each progressively slower but larger than the last.
Operating System Page Cache
The OS page cache sits between your application and the disk. When you read a file, the operating system keeps recently accessed disk blocks in RAM. Future reads of the same data come from memory instead of disk—a difference of microseconds versus milliseconds.
This is why running a database query twice often feels faster the second time, even without application-level caching.
Why This Matters for System Design
Understanding these invisible layers helps you recognize that:
- Data locality matters: Accessing nearby memory addresses is dramatically faster
- Some "database performance" comes from the OS, not your query optimization
- Your application cache builds on top of these foundation layers
Key Takeaway: CPU and OS caches automatically accelerate all software, forming the invisible bottom layers of the caching hierarchy that higher-level application caches build upon.