Connection Pooling and Keep-Alive
What you'll learn: How load balancers optimize performance by reusing connections to backend servers instead of constantly opening and closing new ones.
The Problem with Ephemeral Connections
In earlier lessons, you learned that load balancers distribute requests across backend servers. The naive approach creates a new TCP connection for every single request, then immediately closes it. This is like hanging up and redialing the phone after every sentence in a conversation—wasteful and slow.
Opening a TCP connection requires a three-way handshake (SYN, SYN-ACK, ACK), which adds latency. Add TLS/SSL negotiation on top, and you're looking at several round trips before the actual HTTP request even begins. For high-traffic systems, this overhead multiplies into a serious performance bottleneck.
Connection Pooling
Connection pooling means maintaining a collection of already-established connections to backend servers that can be reused across multiple requests. Instead of closing a connection after handling one request, the load balancer returns it to the pool, ready for the next request.
Think of it like a taxi stand: taxis wait ready to go, rather than every passenger having to summon a new taxi from across the city each time.
HTTP Keep-Alive
Keep-Alive (also called persistent connections) is the HTTP-level mechanism that signals "don't close this connection yet." The Connection: keep-alive header tells both sides to keep the TCP connection open for multiple request-response cycles.
Modern load balancers typically maintain two connection pools: one between clients and the load balancer, and another between the load balancer and backend servers. The backend pool is where the biggest performance gains live.
Configuration Considerations
Key settings include pool size (how many connections to maintain), idle timeout (when to close unused connections), and max lifetime (refreshing connections periodically to avoid stale state).
Key Takeaway: Connection pooling and Keep-Alive eliminate the overhead of repeatedly opening/closing connections, dramatically improving load balancer throughput and reducing latency by reusing established TCP connections.