In this lesson, you will learn how to reorganize your processing logic to ensure that frequently accessed data remains resident in the CPU cache, minimizing costly trips to main memory.
The Principle of Temporal Locality
Temporal locality is the observation that if a piece of data is accessed now, it is likely to be accessed again in the near future. In high-performance Rust systems, we don't just care about where data lives (spatial locality); we care about when we touch it. If your processing loop touches a memory address, then performs a long series of unrelated operations before touching that same address again, the CPU may have already evicted that data from the L1 or L2 cache to make room for the unrelated operations.
Reorganizing for Temporal Locality
To optimize for temporal locality, you must group operations so that they consume "hot" data in one concentrated pass. Instead of iterating over a massive collection of objects multiple times to perform different sub-tasks, restructure your logic to perform all necessary operations on a single element before moving on to the next.
Consider this inefficient pattern:
// Inefficient: High latency between repeated access
for item in &mut items { item.prepare(); }
// ... a long sequence of unrelated processing code ...
for item in &mut items { item.finalize(); }
In the example above, by the time the second loop begins, the data from the first loop has likely been evicted from the cache. If items is large, the CPU must fetch the memory back from the L3 cache or RAM for the second pass.
The "Fusion" Optimization
By fusing these operations into a single loop, you keep the item in the CPU's registers or L1 cache for the duration of its lifecycle in that function:
// Efficient: Keeping data "hot"
for item in &mut items {
item.prepare();
// Perform intermediate steps while the item is already loaded
item.finalize();
}
By keeping the time delta between repeated accesses as small as possible, you maximize the efficiency of your CPU's cache hierarchy. This is especially vital when processing large buffers or high-frequency stream data where memory throughput becomes the primary bottleneck.
Key Takeaway: To improve temporal locality, fuse disparate processing stages into a single execution pass to ensure that frequently accessed data remains "hot" in the cache registers.