Course contentsShow
← Computer Science
Lesson 1108 of 2,87226. Memory Hierarchy and CachingFree lesson

Speed vs Capacity Tradeoff

Understanding the inverse relationship between memory speed and capacity at each hierarchy level.

Speed vs Capacity Tradeoff

You'll learn: Why faster memory is always smaller, and slower memory can be much larger—a fundamental constraint in computer architecture.

The Fundamental Tradeoff

In the memory hierarchy you just learned about, there's an unavoidable inverse relationship: the faster a memory level can operate, the less storage capacity it can economically provide. Conversely, the more capacity you want, the slower the access speed becomes.

This isn't a design flaw—it's a physical and economic reality driven by:

  • Cost per bit: Fast memory (like CPU registers or cache) uses expensive materials and complex circuitry
  • Physical distance: Data must travel shorter distances for faster access; you can't fit as much storage physically close to the CPU
  • Power consumption: Faster memory requires more energy, limiting how much you can deploy
  • Technology constraints: SRAM (fast) uses 6+ transistors per bit, while DRAM (slower) uses 1 transistor + capacitor

The Pyramid in Action

Think of a restaurant's prep workflow:

  • Chef's hands (registers): Instantly accessible, but can only hold a few items
  • Cutting board (cache): Very quick to reach, limited workspace
  • Refrigerator (RAM): A few steps away, holds much more
  • Storage room (disk): Requires walking, but can store inventory for months

Each level increases capacity dramatically while sacrificing speed.

Why This Matters

Computer architects exploit this tradeoff by placing small amounts of extremely fast memory close to the CPU for frequently used data, backed by progressively larger but slower storage tiers. Without this hierarchy, you'd either have tiny, expensive systems or unbearably slow computers.

Key Takeaway: Memory speed and capacity have an inverse relationship—faster memory is necessarily smaller and more expensive, while larger storage is slower but more affordable per bit.