Course contentsShow
← Software Engineering
Lesson 1395 of 1,93341. Scalability and Distributed SystemsFree lesson

Connection Pooling and Keep-Alive

Optimizing load balancer performance through connection reuse and persistent connections to backend servers.

Connection Pooling and Keep-Alive

What you'll learn: How load balancers optimize performance by reusing connections to backend servers instead of constantly opening and closing new ones.


The Problem with Ephemeral Connections

In earlier lessons, you learned that load balancers distribute requests across backend servers. The naive approach creates a new TCP connection for every single request, then immediately closes it. This is like hanging up and redialing the phone after every sentence in a conversation—wasteful and slow.

Opening a TCP connection requires a three-way handshake (SYN, SYN-ACK, ACK), which adds latency. Add TLS/SSL negotiation on top, and you're looking at several round trips before the actual HTTP request even begins. For high-traffic systems, this overhead multiplies into a serious performance bottleneck.

Connection Pooling

Connection pooling means maintaining a collection of already-established connections to backend servers that can be reused across multiple requests. Instead of closing a connection after handling one request, the load balancer returns it to the pool, ready for the next request.

Think of it like a taxi stand: taxis wait ready to go, rather than every passenger having to summon a new taxi from across the city each time.

HTTP Keep-Alive

Keep-Alive (also called persistent connections) is the HTTP-level mechanism that signals "don't close this connection yet." The Connection: keep-alive header tells both sides to keep the TCP connection open for multiple request-response cycles.

Modern load balancers typically maintain two connection pools: one between clients and the load balancer, and another between the load balancer and backend servers. The backend pool is where the biggest performance gains live.

Configuration Considerations

Key settings include pool size (how many connections to maintain), idle timeout (when to close unused connections), and max lifetime (refreshing connections periodically to avoid stale state).

Key Takeaway: Connection pooling and Keep-Alive eliminate the overhead of repeatedly opening/closing connections, dramatically improving load balancer throughput and reducing latency by reusing established TCP connections.