Multi-Level Cache Performance Analysis
What you'll learn: How to calculate the combined performance impact when your CPU uses multiple cache levels (L1, L2, L3) working together.
The Hierarchical Cache Chain
Modern CPUs don't have just one cache—they have several levels working together. When the CPU needs data, it checks L1 first (fastest, smallest), then L2 if L1 misses, then L3 if L2 misses, and finally main memory if all caches miss.
Think of it like looking for your keys: first you check your pocket (L1), then your desk (L2), then the living room (L3), and finally the whole house (main memory). Each level takes longer but is more likely to have what you need.
Hierarchical AMAT Formula
The Average Memory Access Time (AMAT) for multi-level caches builds on itself:
Single-level: AMAT = HitTime_L1 + MissRate_L1 × MissPenalty_L1
Two-level: The L1 miss penalty is accessing L2:
AMAT = HitTime_L1 + MissRate_L1 × (HitTime_L2 + MissRate_L2 × MissPenalty_L2)
Three-level: L2's miss penalty becomes accessing L3:
AMAT = HitTime_L1 + MissRate_L1 × (HitTime_L2 + MissRate_L2 × (HitTime_L3 + MissRate_L3 × MemoryTime))
Quick Example
- L1: 1 cycle hit, 5% miss rate
- L2: 10 cycles hit, 20% miss rate
- Memory: 100 cycles
AMAT = 1 + 0.05 × (10 + 0.20 × 100)
= 1 + 0.05 × 30
= 1 + 1.5
= 2.5 cycles average
Notice how L2's 20% miss rate only affects the 5% of accesses that already missed L1 (5% × 20% = 1% reach memory).
Key Takeaway: Multi-level caches use hierarchical AMAT formulas where each level's miss penalty includes the access time of the next level, dramatically reducing the effective memory access time by filtering most requests at faster levels.