BiteSizedChunks.comLearn one small thing at a time.
Course contentsShow
← Embedded Systems & Computer Architecture
Lesson 136 of 9362. Instruction Set Architecture and CPU MicroarchitectureFree lesson

Calculating Effective Memory Access Time

Derive the mathematical formula for average memory access time by incorporating hit rates, miss rates, and individual latency penalties.

In this lesson, you will learn how to mathematically derive the average speed at which your CPU retrieves data from the memory hierarchy, accounting for cache hits and misses.

The Memory Hierarchy Dilemma

In our previous lessons, we learned that while caches provide fast data access, they aren't infinite. When the CPU looks for data, it first checks the cache. If the data is there, we have a hit and the latency is very low. If it’s not there, we have a miss, and the processor must incur a significant miss penalty to fetch the data from the much slower main memory (DRAM).

Defining the Variables

To calculate the Average Memory Access Time (AMAT), we need three pieces of information:

  • Hit Time: The time to access the cache.
  • Miss Rate: The percentage of accesses that result in a miss.
  • Miss Penalty: The extra time required to fetch data from main memory when a miss occurs.

The AMAT Formula

Think of the total time as a weighted average. You always pay the Hit Time because every request starts at the cache. Then, you add the "penalty portion," which is the frequency of misses multiplied by the cost of those misses:

AMAT = Hit Time + (Miss Rate × Miss Penalty)

Practical Example

Imagine an embedded system where your cache Hit Time is 1 clock cycle. When you have a miss, it takes 50 clock cycles to pull the data from memory (Miss Penalty = 50). If your cache has a Miss Rate of 10% (0.10), your calculation looks like this:

AMAT = 1 + (0.10 × 50)
AMAT = 1 + 5
AMAT = 6 cycles

Even though our cache is lightning fast, the 10% miss rate drags our effective performance down to an average of 6 cycles per access. This formula is vital for embedded systems engineers; it shows that even small improvements in the hit rate can have a massive impact on the overall performance of your processor pipeline, which we have been optimizing throughout this module.

Key Takeaway: The Effective Memory Access Time is the sum of the base hit latency and the product of the miss rate and miss penalty, representing the "true" cost of data retrieval in a layered memory system.