Impact of Block Size on Performance
What You'll Learn
You'll discover why making cache blocks bigger isn't always better—there's a sweet spot between capturing nearby data and the cost of fetching it.
The Block Size Dilemma
When a cache miss occurs, the system fetches an entire block (cache line) from slower memory, not just the single byte you requested. The question is: how big should each block be?
Larger Blocks: The Benefits
Spatial locality rewards larger blocks. If your program accesses memory sequentially (like reading array elements), a bigger block brings more useful neighbors in one trip. This reduces future compulsory misses—you've already got the data you'll need next.
Think of it like grocery shopping: buying a 12-pack of soda means fewer trips to the store if you drink one can per day.
Larger Blocks: The Costs
But there's a catch:
-
Increased miss penalty: Fetching a 128-byte block takes longer than fetching 16 bytes. Every miss now stalls your CPU for more cycles.
-
Fewer total blocks: With fixed cache size, larger blocks mean fewer blocks fit in cache. This can increase conflict and capacity misses because unrelated data now competes for fewer slots.
-
Wasted bandwidth: If your program has poor spatial locality (jumping around memory randomly), you're fetching lots of data you'll never use.
It's like buying that 12-pack when you only drink soda occasionally—you've spent more money and fridge space for mostly unused cans.
The Tradeoff
Small blocks minimize miss penalty and maximize block count, but miss spatial locality opportunities. Large blocks exploit spatial locality but amplify miss costs and reduce cache flexibility. Real systems typically use 64-byte blocks as a practical middle ground.
Key Takeaway: Block size is a balancing act—larger blocks improve hit rates through spatial locality but increase miss penalties and reduce the number of cacheable blocks, requiring careful tuning based on workload access patterns.