In this lesson, you will learn how to resolve silent packet loss and throughput bottlenecks in encapsulated networks by precisely tuning Maximum Transmission Unit (MTU) and Maximum Segment Size (MSS) values.
The Problem: Fragmentation and Overhead
When you use encapsulation protocols like VXLAN, Geneve, or WireGuard, the kernel adds extra headers to your original network packets. If your physical network interface (the "underlay") has a standard MTU of 1500 bytes, but your virtual tunnels add 50+ bytes of overhead, your packets become too large.
This triggers one of two performance killers:
- Fragmentation: The kernel splits packets into smaller chunks, increasing CPU overhead and latency.
- Silent Drops: If the "Don't Fragment" (DF) bit is set, the packet is simply discarded, causing connection hangs that are notoriously difficult to debug.
MSS Clamping as the Solution
While adjusting the interface MTU is the best practice, it isn't always possible in complex, multi-tenant virtual networks. MSS Clamping intervenes at the TCP handshake level. By modifying the MSS field in the TCP SYN packet, we instruct the communicating hosts to negotiate smaller individual segments before the data begins flowing. This ensures the total packet size (header + payload) never exceeds the path's bottleneck.
In Linux, you can manage this via iptables or nftables at the mangle stage:
# Clamp MSS to 1360 bytes for outgoing packets on the tunnel interface
iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1360
Why BPF Matters Here
In advanced environments where you need to perform this clamping dynamically based on per-tunnel state (rather than global rules), BPF shines. By using an XDP or TC program, you can inspect the TCP options in real-time, calculate the overhead of your specific encapsulation, and rewrite the MSS value on-the-fly. This prevents the "black hole" effect where packets are dropped mid-transit because the endpoint wasn't aware of the specific tunnel overhead ahead of it.
By proactively shrinking the segment size, you avoid the heavy lifting of kernel-level fragmentation and ensure stable throughput across your virtual overlay.
Key Takeaway: MSS Clamping prevents packet loss in encapsulated networks by forcing TCP hosts to limit their segment size, effectively compensating for the header overhead added by tunnels.