Measuring Cache Effectiveness
You'll learn how to quantify how well a cache is performing using three key metrics.
Why Measure Cache Performance?
Just knowing you have a cache isn't enough—you need to measure whether it's actually helping. Think of a cache like a notepad where you jot down frequently used phone numbers. To know if it's useful, you'd track how often you find the number there versus having to look it up in the full directory.
The Three Key Metrics
Hit Rate
The hit rate is the percentage of memory accesses that find the data in cache. If your program makes 100 memory requests and 85 are found in cache, your hit rate is 85%.
Hit Rate = (Cache Hits / Total Accesses) × 100%
Miss Rate
The miss rate is simply the opposite—the percentage of accesses that don't find data in cache, forcing a trip to slower main memory.
Miss Rate = (Cache Misses / Total Accesses) × 100%
Miss Rate = 100% - Hit Rate
Average Memory Access Time (AMAT)
This combines everything into one number: the average time to complete a memory access, accounting for both fast hits and slow misses.
AMAT = Hit Time + (Miss Rate × Miss Penalty)
Where hit time is how long a cache access takes, and miss penalty is the extra time when you must fetch from main memory.
Example
If cache access takes 1 cycle, main memory takes 100 cycles, and you have a 90% hit rate:
AMAT = 1 + (0.10 × 100) = 1 + 10 = 11 cycles average
Key Takeaway: Hit rate, miss rate, and average memory access time let you quantify cache performance—higher hit rates and lower AMAT mean your cache is working effectively.