In this lesson, you will learn how computers use two specific patterns in memory access to make your programs run significantly faster by predicting what data to keep ready in the cache.
When a processor fetches data from main memory (RAM), it is slow. To combat this, we use a small, ultra-fast memory called cache. But how does the cache know what to store? It relies on the Principle of Locality, which states that programs rarely access memory at random; instead, they follow predictable patterns.
1. Temporal Locality (Time)
Temporal locality is the concept that if you access a specific memory address, you are likely to access that same address again very soon. Think of this like a bookmark in a book; if you just read a page, you’ll probably look back at that same page in a few moments.
// Example of high Temporal Locality
for (int i = 0; i < 100; i++) {
sum += total; // 'total' is accessed repeatedly in a short time
}
Because the CPU knows total is used in every loop iteration, it keeps that value in the cache to avoid going back to slow RAM.
2. Spatial Locality (Space)
Spatial locality is the concept that if you access a memory address, you are likely to access the neighboring addresses soon after. This is like reading a sentence; if you read the first word, you will naturally move to the word immediately to its right.
// Example of high Spatial Locality
int list[100];
for (int i = 0; i < 100; i++) {
sum += list[i]; // The CPU fetches the next element in the array
}
When the cache fetches list[0], it doesn't just grab that single integer. It grabs a "cache line" containing list[0], list[1], list[2], and so on. Since the program is almost certainly going to need the next item in the array, the cache has already "pre-loaded" that data, turning a potential memory stall into a high-speed cache hit.
By exploiting these two patterns, cache controllers keep the most relevant data physically close to the processor, effectively bridging the performance gap we discussed in earlier lessons.
Key Takeaway: Temporal locality is the tendency to reuse the same data, while spatial locality is the tendency to access data stored near previously accessed memory.