Thread Affinity and Pinning

In this lesson, you will learn how to reduce performance jitter by binding specific heavy-computation threads to dedicated CPU cores, minimizing the cost of cache migration.

The Problem: Cache Locality and Migrations

In a standard multi-threaded environment, the OS scheduler frequently moves threads between different CPU cores to balance load. While this is great for general fairness, it is detrimental to performance-critical code. When a thread moves to a new core, its "warm" L1/L2 cache—which holds the data it was actively processing—remains on the previous core. The new core must then fetch that data from higher-level, slower cache (L3) or main memory, resulting in a significant latency penalty known as a cache miss or migration cost.

Pinning Threads

"Thread affinity" (or pinning) is the practice of instructing the operating system to restrict a thread's execution to a specific subset of physical CPU cores. By pinning a heavy computational thread to a specific core, you ensure that the thread’s working set remains local to that core’s caches, maximizing throughput and ensuring consistent timing.

In Rust, while the standard library doesn't expose cross-platform pinning, libraries like core_affinity allow you to bind a thread to a specific core ID:

use core_affinity;

fn spawn_pinned_thread(core_id: usize) {
    std::thread::spawn(move || {
        let core_ids = core_affinity::get_core_ids().unwrap();
        // Pin the thread to the specified core
        core_affinity::set_for_current(core_ids[core_id]);
        
        // Execute heavy computation here...
        loop { /* crunching numbers */ }
    });
}

When to Use This

Pinning is an architectural choice for high-performance systems where task latency is more important than overall system fairness. It is particularly effective for:

1. Real-time processing: Where jitter from cache misses is unacceptable.
2. High-throughput pipelines: Where threads operate on large, hot data structures.
3. NUMA systems: Ensuring threads reside on the same physical socket as the memory they access.

Be warned: over-pinning can lead to CPU starvation if you bind too many threads to a single core, or waste resources if a core sits idle while other tasks are queued. Always profile using your existing knowledge of throughput metrics to ensure pinning is actually improving performance rather than creating bottlenecks.

Key Takeaway: Pinning threads to specific CPU cores prevents costly cache migrations, stabilizing execution time for high-performance tasks by keeping data local to the core’s cache hierarchy.