In this lesson, you will learn how to scale Linux connection tracking (conntrack) to handle millions of concurrent flows by optimizing kernel hash tables.
The Challenge of Scale
By default, the Linux kernel tracks connection state—essential for stateful firewalls and NAT—in a global hash table. As your connection count climbs into the millions, two problems emerge: hash collisions significantly degrade performance, and the memory footprint for these tracking entries (conntracks) can exhaust system resources.
Optimizing the Hash Table
To support massive concurrency, you must tune the connection tracking hash table size. The table size is determined by the nf_conntrack_buckets parameter. If this value is too low, the chain length of each hash bucket grows, forcing the CPU to traverse long linked lists for every packet, which destroys lookup speed.
To handle, for example, 2 million connections, you should aim for a bucket size that minimizes collisions. You can adjust this via sysctl:
# Set the hash table size (must be done at boot or via module parameters)
# Calculate based on: hashsize = total_conntrack / 4
sysctl -w net.netfilter.nf_conntrack_buckets=524288
sysctl -w net.netfilter.nf_conntrack_max=2097152
Strategic Tuning
Increasing nf_conntrack_max without adjusting nf_conntrack_buckets is a common mistake that leads to "conntrack table full" errors and performance bottlenecks. Beyond sizing, you must also aggressively reduce the "timeout" values for entries. By shortening the time the kernel keeps idle connections in the hash table, you reclaim memory and CPU cycles that would otherwise be wasted on tracking dead flows.
For instance, lowering the TCP established timeout:
# Reduce stale state retention
sysctl -w net.netfilter.nf_conntrack_tcp_timeout_established=86400
By balancing a large enough hash table to reduce collision chains with aggressive timeout policies to keep the table clean, you allow the kernel to maintain high-speed packet inspection even under extreme load.
Key Takeaway: Managing millions of concurrent connections requires matching your hash bucket count to your maximum connection capacity to prevent lookup latency while aggressively tuning entry timeouts to preserve kernel memory.