In this lesson, you will learn how multi-queue network hardware distributes traffic processing across multiple CPU cores to eliminate bottlenecks in high-throughput environments.
The Bottleneck Problem
In a standard single-queue network interface, all incoming packets trigger interrupts on a single CPU core. As traffic volume grows, this "serialized" processing causes that specific core to hit 100% utilization while other cores sit idle. This creates a performance ceiling, as the single core cannot keep up with the incoming stream, leading to packet drops at the hardware level.
Multi-Queue Architecture
Modern Network Interface Cards (NICs) support Multi-Queue (MQ), which allows the hardware to distribute traffic into several independent queues. Each queue is assigned its own interrupt vector, allowing the Linux kernel to map different queues to different CPU cores.
By spreading the load across multiple cores, we achieve:
- Parallelism: Multiple cores process packet headers and protocols simultaneously.
- Cache Locality: By using
Receive Side Scaling(RSS), hardware hashes incoming traffic based on the IP address and port, ensuring that packets belonging to the same flow are processed by the same core, which keeps the CPU cache warm.
Inspecting Your Queues
You can identify how your NIC distributes traffic by inspecting the interrupt mapping in /proc/interrupts. Look for interface names followed by queue numbers:
# View interrupt distribution for network queues
grep eth0 /proc/interrupts
If you see multiple lines for your device (e.g., eth0-TxRx-0, eth0-TxRx-1), your system is utilizing multiple hardware queues.
Configuration Basics
To ensure your hardware queues are optimally utilized, you must ensure that the CPU affinity (irqbalance) is correctly mapped. In high-performance scenarios, administrators often disable the irqbalance daemon and manually pin hardware interrupts to specific CPU cores using the smp_affinity file for each interrupt in /proc/irq/. This prevents the kernel from "migrating" interrupt handling between cores, which would otherwise invalidate your CPU caches and degrade performance.
Key Takeaway: Multi-queue support prevents CPU bottlenecks by allowing the Linux kernel to process incoming network traffic in parallel across multiple CPU cores via hardware-level flow distribution.