Memory Layout for Performance

In this lesson, you will learn how organizing your data structures to be "cache-friendly" can drastically speed up your parallel processing tasks.

The Problem: The CPU Cache Gap

Modern CPUs are incredibly fast, but fetching data from main RAM is relatively slow. To bridge this gap, CPUs use a "cache"—a tiny, lightning-fast storage area. When the CPU needs data, it fetches a "cache line" (typically 64 bytes) from RAM. If your data is laid out sequentially in memory, the CPU can grab multiple related items in one single fetch. If your data is scattered, the CPU misses the cache and is forced to wait for the slow RAM, creating a "cache miss."

Structure Data for Locality

In a parallel pipeline, especially when iterating over large collections, you want your data to be "packed." This is why an array or List<T> is often better for performance than a List<LinkedNode<T>>.

Consider the difference in memory layout:

// Poor layout: Objects are scattered across the heap (pointer chasing)
var nodes = new List<Node>(); 

// Good layout: Data is stored contiguously in memory
var points = new Point[10000]; 

When processing an array of structs, the CPU fetches a chunk of memory that contains several elements at once. If you use a class, the array only stores references (pointers) to objects scattered elsewhere in memory. Following these pointers causes cache misses, which can negate the speed gains you worked so hard to achieve using Parallel.For or dataflow pipelines.

Practical Tips for Pipelines

1. Prefer Structs for Small Data: For small, immutable chunks of data passing through your pipeline, use struct. This keeps the data "in-line" rather than creating a new object on the heap that the garbage collector must track.
2. Minimize Padding: Be mindful that the C# compiler adds padding to align data. Keep related, frequently accessed fields together at the top of your structs.
3. Array of Structs vs. Struct of Arrays: For high-performance processing, sometimes it is faster to use a "Struct of Arrays" (e.g., one array for X coordinates, one for Y) if your parallel logic only needs to access specific fields, as it allows the CPU to load only the relevant data into the cache.

Key Takeaway: By keeping data contiguous in memory through structs and arrays, you maximize cache hits and allow your parallel processing tasks to operate at the true speed of your CPU.