Load Balancing WebSocket Clusters

In this lesson, you will learn how to configure Nginx to maintain "sticky sessions," which are essential for keeping WebSocket connections stable when scaling your application across multiple Node.js instances.

The Challenge of Stateless Balancing

When you scale your application to multiple processes or servers, a standard load balancer typically rotates incoming requests across the cluster. However, WebSockets start with an HTTP handshake that must upgrade to a persistent connection. If a client sends the handshake to Server A, but the subsequent upgrade request is routed to Server B, the connection will fail. Furthermore, if you are using a library like Socket.io, it often relies on long-polling as a fallback, which requires the client to talk to the same server for every request in a sequence.

Implementing Sticky Sessions (Session Affinity)

To solve this, we use "sticky sessions." This instructs the load balancer to look for a specific cookie or IP address and ensure that a client stays tethered to the same Node.js process for the duration of their session.

In Nginx, we use the ip_hash directive or the sticky module to bind a client to an upstream server. Here is how a standard Nginx configuration looks:

http {
    upstream websocket_cluster {
        # 'ip_hash' ensures the same client IP always hits the same server
        ip_hash;
        
        server 127.0.0.1:3001;
        server 127.0.0.1:3002;
    }

    server {
        location /socket.io/ {
            proxy_pass http://websocket_cluster;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "Upgrade";
        }
    }
}

Why This Works

The ip_hash directive generates a hash based on the client's IP address, mapping them to one of your Node.js servers. As long as the client remains on that network path, Nginx will route them to that specific process, allowing the initial handshake and the subsequent persistent WebSocket upgrade to complete successfully on the same instance.

Remember, while sticky sessions solve the routing problem, you must still ensure your application layer can handle cross-server communication (like Pub/Sub) if you need to broadcast events to clients connected to different nodes.

Key Takeaway: Sticky sessions are non-negotiable for WebSocket clusters; you must use load balancer directives like ip_hash to ensure the handshake and the persistent connection reach the same server instance.