In this lesson, you will learn how to minimize memory latency by pinning containerized processes to specific Non-Uniform Memory Access (NUMA) nodes using cgroup configurations.
The NUMA Challenge
In modern multi-socket server architectures, memory is "local" to a specific CPU socket. Accessing memory attached to a different CPU socket across the interconnect (like Intel’s UPI or AMD’s Infinity Fabric) introduces significant latency and consumes bandwidth. When a container's threads migrate between CPUs or allocate memory remotely, performance degrades—an issue exacerbated in containerized environments where the scheduler may be unaware of hardware topography.
Pinning with Cgroups
To optimize performance, we use cgroups v2 to enforce topology-aware scheduling. By utilizing the cpuset controller, we bind a container’s process tree to a specific set of physical CPU cores and their associated memory nodes. This ensures that the kernel’s memory allocator prefers memory pages local to the assigned NUMA node, keeping the "data path" as short as possible.
You can configure this by manipulating the cgroup.procs and specific cpuset files within the controller's hierarchy:
# Set the CPU cores to use (e.g., cores 0-3 on NUMA node 0)
echo "0-3" > /sys/fs/cgroup/my_container/cpuset.cpus
# Set the memory node affinity (e.g., node 0)
echo "0" > /sys/fs/cgroup/my_container/cpuset.mems
Why This Matters
When you explicitly define cpuset.mems, the kernel forces the process to allocate memory from the specified NUMA node. If the node runs out of memory, the process will fail to allocate rather than "spilling over" to a remote node, preserving performance predictability. This is critical for high-throughput applications like databases or packet processing engines, where cache locality determines the difference between microsecond and millisecond response times.
By aligning your container's CPU affinity with its memory node affinity, you effectively bypass the overhead of cross-socket memory traffic. This "Topology-Aware" approach turns the kernel into a strict gatekeeper, ensuring that your container's memory footprint always matches its physical processor locality.
Key Takeaway: By binding container processes to specific NUMA nodes via cpuset.mems and cpuset.cpus, you enforce hardware-level locality that eliminates cross-socket memory latency.