CPU Affinity and Cache Locality

In this lesson, you will learn how to pin threads to specific physical CPU cores to reduce the performance penalties associated with cache misses and context switching.

The Problem: Thread Migration

By default, the operating system's scheduler is free to move your threads across different CPU cores. While this is great for general multitasking, it is detrimental to high-performance systems. When a thread moves from Core A to Core B, the "warm" data sitting in Core A’s L1/L2 cache becomes useless. The thread must then wait for the CPU to fetch that same data from main memory into Core B’s cache, causing a massive latency spike known as a cache miss.

The Solution: CPU Affinity

CPU Affinity (or "pinning") acts as a constraint: you tell the OS, "this thread is only allowed to run on Core X." By keeping a thread on a specific core, you ensure that the data it accesses stays resident in that core’s local cache. This improves cache locality, significantly speeding up tight loops or data-intensive workloads.

In Rust, while the standard library provides basic threading primitives, pinning requires platform-specific calls (like pthread_setaffinity_np on Linux via the libc crate). A common pattern looks like this:

use std::thread;
use libc::{cpu_set_t, CPU_SET, CPU_ZERO, sched_setaffinity};

fn pin_to_core(core_id: usize) {
    unsafe {
        let mut cpuset: cpu_set_t = std::mem::zeroed();
        CPU_ZERO(&mut cpuset);
        CPU_SET(core_id, &mut cpuset);
        
        let thread = libc::pthread_self();
        sched_setaffinity(thread, std::mem::size_of::<cpu_set_t>(), &cpuset);
    }
}

fn main() {
    thread::spawn(|| {
        pin_to_core(0); // Pin this thread to the first core
        // Heavy computation here benefits from local cache
    }).join().unwrap();
}

When to Use It

You should only use affinity for latency-sensitive tasks where the cost of a cache miss outweighs the cost of the OS scheduler's flexibility. If you pin too many threads to a single core, you create contention, forcing the OS to perform context switches anyway, which defeats the purpose. Use this tool sparingly, typically in dedicated worker pools or high-frequency trading applications.

Key Takeaway: CPU affinity minimizes cache misses by preventing thread migration, ensuring your application’s data stays "hot" in the cache of a specific physical core.