Course contentsShow
← System Design
Lesson 127 of 1,9194. Caching StrategiesFree lesson

CPU and Disk Caching Layers

Understanding L1/L2/L3 CPU caches and OS page cache as foundational hardware and system-level caching.

CPU and Disk Caching Layers

What you'll learn: How hardware-level caching in CPUs and operating systems forms the invisible foundation beneath all software caching strategies.

The Hardware Foundation

Before you write a single line of code, multiple layers of caching are already working to speed up your applications. These hardware and OS-level caches are so fundamental that every program benefits from them automatically.

CPU caches (L1/L2/L3) are tiny, ultra-fast memory banks built directly into your processor:

  • L1 cache (~32-64 KB): Fastest, closest to CPU cores, accessed in ~1 nanosecond
  • L2 cache (~256 KB-1 MB): Slightly slower, often per-core or shared between pairs
  • L3 cache (~8-32 MB): Slowest CPU cache, shared across all cores, still 10-100x faster than RAM

Think of these like having sticky notes (L1) on your desk, a drawer (L2) beside you, and a filing cabinet (L3) in your office—each progressively slower but larger than the last.

Operating System Page Cache

The OS page cache sits between your application and the disk. When you read a file, the operating system keeps recently accessed disk blocks in RAM. Future reads of the same data come from memory instead of disk—a difference of microseconds versus milliseconds.

This is why running a database query twice often feels faster the second time, even without application-level caching.

Why This Matters for System Design

Understanding these invisible layers helps you recognize that:

  • Data locality matters: Accessing nearby memory addresses is dramatically faster
  • Some "database performance" comes from the OS, not your query optimization
  • Your application cache builds on top of these foundation layers

Key Takeaway: CPU and OS caches automatically accelerate all software, forming the invisible bottom layers of the caching hierarchy that higher-level application caches build upon.