NUMA Awareness for Multi-Socket Systems

In this lesson, you will learn how to optimize performance in multi-socket systems by pinning threads to specific cores and anchoring memory allocations to local NUMA nodes.

The Cost of Cross-Socket Communication

On multi-socket servers, the system is divided into Non-Uniform Memory Access (NUMA) nodes. Each CPU socket has its own local bank of memory. While any core can access any memory address, accessing a remote socket’s memory requires traversing an interconnect (like Intel’s UPI or AMD’s Infinity Fabric). This incurs significantly higher latency and lower bandwidth compared to local access. To achieve peak throughput, you must minimize "remote" memory traffic.

Pinning and Locality

The goal is to maintain a "NUMA-local" state: threads should process data that resides in the memory bank attached to their current socket.

1. Thread Affinity: You bind a worker thread to a specific core using system-level affinity masks. This ensures the OS scheduler doesn't migrate your hot threads across sockets.
2. Memory Policy: You ensure that the heap allocations for a specific task occur on the node where the task is running.

In Rust, while standard allocation uses the system allocator, you can manage this by partitioning your data structures and assigning them to thread-local pools. If a thread is pinned to Node 0, it should only pull memory from a pre-allocated pool associated with Node 0.

// Conceptual design for NUMA-aware processing
struct NodeLocalPool {
    // A pre-allocated slab specific to this NUMA node
    data: Vec<MyObject>, 
}

fn worker_thread(node_id: usize, pool: &NodeLocalPool) {
    // 1. Set CPU affinity to a core in node_id
    // 2. Process data only from this node's local pool
    for item in &pool.data {
        process(item); // Cache-friendly, local-memory access
    }
}

By keeping your "hot" working set strictly on the local socket, you avoid the heavy performance penalties of interconnect contention. This approach works in tandem with Spatial Locality and Memory Pooling we discussed previously, effectively localizing both cache lines and physical RAM access.

Key Takeaway: By binding threads to specific cores and restricting memory allocations to the corresponding local NUMA node, you minimize interconnect bottlenecks and achieve consistent, predictable latency.