Course contentsShow
← Computer Science
Lesson 1114 of 2,87226. Memory Hierarchy and CachingFree lesson

Hit Rate and Miss Penalty

Fundamental metrics for measuring memory hierarchy performance and their impact on overall speed.

Hit Rate and Miss Penalty

You'll learn the two fundamental metrics that determine how fast your computer's memory hierarchy actually performs.

When your CPU needs data, it first checks faster, smaller memory levels (like cache) before going to slower, larger storage. Two metrics tell us how well this works:

Hit Rate: Your Success Percentage

The hit rate is the percentage of times the CPU finds what it needs in the faster memory level without having to go further down the hierarchy. If your cache has a 95% hit rate, that means 95 out of 100 memory requests are satisfied immediately from cache.

The miss rate is simply the opposite: miss rate = 1 - hit rate. A 95% hit rate means a 5% miss rate.

Miss Penalty: The Cost of Failure

When you don't find data in the fast memory (a "miss"), you pay a miss penalty—the extra time required to fetch that data from the next, slower level. This penalty includes the access time of the slower memory plus the time to move the data up to the faster level.

Think of it like a library analogy: if the book you need is on your desk (hit!), you grab it instantly. If not (miss!), you must walk to the bookshelf across the room—that walking time is your miss penalty.

Why Both Matter

Even a small miss rate hurts if the miss penalty is huge. A 5% miss rate sounds great, but if fetching from main memory takes 100× longer than cache, those 5% of requests will dominate your actual performance.

The formula for average memory access time combines both: Average Access Time = Hit Time + (Miss Rate × Miss Penalty)

Key Takeaway: Hit rate measures how often fast memory succeeds, while miss penalty measures the time cost when it fails—together they determine real-world memory performance.