In this lesson, you will learn how to distribute outgoing network traffic across multiple gateways simultaneously using Equal-Cost Multi-Path (ECMP) routing.
Understanding ECMP
Normally, when your routing table has multiple paths to the same destination with the same metric, the kernel picks one and sticks with it. ECMP changes this behavior by allowing the Linux kernel to distribute packets across all available "equal-cost" paths. This is not just for redundancy; it is a powerful way to increase total outbound bandwidth by load-balancing traffic across different upstream providers or physical links.
Configuring Multipath Routes
In Linux, you configure ECMP using the ip route command by providing multiple nexthop declarations within a single route entry. Instead of specifying one gateway, you weight them to dictate how traffic is distributed.
For example, if you have two ISP gateways at 192.168.1.1 and 192.168.1.2, you can create a multipath default route like this:
ip route add default scope global \
nexthop via 192.168.1.1 dev eth0 weight 1 \
nexthop via 192.168.1.2 dev eth0 weight 1
In this setup, the kernel uses a hashing algorithm (typically based on the source IP, destination IP, and ports) to determine which nexthop a specific flow should take. By using the weight parameter, you can favor one link over another if they have different capacities—for instance, assigning weight 2 to a faster fiber line and weight 1 to a slower backup line.
Flow Consistency
It is important to note that ECMP usually performs per-flow load balancing rather than per-packet. If the kernel load-balanced every individual packet across different paths, packets would frequently arrive out of order, which wreaks havoc on TCP stacks and causes severe performance degradation. By hashing the connection details (the "5-tuple"), the kernel ensures that all packets belonging to a single TCP stream consistently follow the same path, keeping the connection stable while distributing the overall load of many concurrent connections across your available gateways.
Key Takeaway: ECMP enables aggregate bandwidth utilization by hashing traffic flows across multiple gateway paths of equal cost, ensuring both load distribution and TCP packet ordering.