Course contentsShow
← Computer Science
Lesson 1113 of 2,87226. Memory Hierarchy and CachingFree lesson

Memory Bandwidth Considerations

Data transfer rates and bandwidth limitations at each level of the hierarchy.

Memory Bandwidth Considerations

What you'll learn: You'll understand how the speed at which data can be transferred between memory levels affects system performance, independent of access time alone.

Beyond Access Time

You've already learned about access time and latency—how long it takes to start reading or writing data. But there's another critical factor: bandwidth, which is the amount of data that can flow through the memory system per second, typically measured in gigabytes per second (GB/s).

Think of it like a highway: access time is how long it takes to get onto the highway, but bandwidth is how many cars can travel down the highway at once. A wide highway (high bandwidth) can move more traffic than a narrow one, even if both have the same on-ramp wait time.

Bandwidth at Each Level

Each level of the memory hierarchy has different bandwidth characteristics:

  • CPU Registers: Extremely high bandwidth (hundreds of GB/s or more), matching the processor's internal data paths
  • Cache (L1/L2/L3): High bandwidth (tens to hundreds of GB/s), but decreases as you move to outer cache levels
  • Main Memory (RAM): Moderate bandwidth (20-50 GB/s for typical systems)
  • Secondary Storage (SSD/HDD): Much lower bandwidth (0.5-3 GB/s for SSDs, even less for HDDs)

Why Bandwidth Matters

Bandwidth becomes the bottleneck when you're moving large blocks of data. Even if your RAM has low latency, if your program needs to process large arrays or stream video, insufficient bandwidth will slow everything down. Modern systems carefully balance bandwidth at each level to prevent data starvation—the CPU waiting idle because data can't arrive fast enough.

Key Takeaway: Memory bandwidth determines how much data can flow per second at each hierarchy level; systems must balance bandwidth across levels to prevent bottlenecks when processing large amounts of data.