Access Time and Latency
What you'll learn: You'll understand the dramatic differences in how long it takes to retrieve data from different levels of the memory hierarchy.
What Are Access Time and Latency?
Access time (or latency) is the delay between requesting data from a storage device and receiving it. Think of it like asking someone a question: if they're standing right next to you (registers), they answer in nanoseconds. If they're across town (hard drive), it takes milliseconds—a million times longer!
Typical Access Times Across the Hierarchy
Here's how long it takes to access data at each level:
- CPU Registers: ~0.25 nanoseconds (instantaneous)
- L1 Cache: ~1 nanosecond
- L2 Cache: ~4 nanoseconds
- L3 Cache: ~10-20 nanoseconds
- Main Memory (RAM): ~100 nanoseconds
- Solid-State Drive (SSD): ~100,000 nanoseconds (0.1 milliseconds)
- Hard Disk Drive (HDD): ~5-10 milliseconds
Why This Matters
Notice the enormous gaps: RAM is 100× slower than L1 cache, and a hard drive is 50,000 times slower than RAM! This is why the memory hierarchy exists—to hide these delays by keeping frequently-used data in the fastest locations.
The CPU would sit idle waiting for data if we only used slow storage. By organizing memory in layers (which you learned about in previous lessons on locality of reference and the storage pyramid), we minimize the average access time the processor experiences.
Key Takeaway: Access times vary dramatically across the memory hierarchy—from nanoseconds in registers and caches to milliseconds on disk—making it critical to keep frequently-accessed data in the fastest storage levels.