In this lesson, you will learn why standard L4 load balancers fail to distribute persistent TCP or gRPC connections effectively and how to architect your C# services to achieve true horizontal balance.
The Problem: The "Sticky" Connection Trap
In many of our previous discussions on event-driven architectures and service discovery, we assumed that requests are ephemeral—one request in, one response out. However, when using gRPC streaming or long-lived TCP connections to a message hub, the connection is established once and stays open indefinitely.
If you place a standard Round-Robin load balancer in front of these services, it will distribute the initial connection correctly. But once that connection is established, the load balancer effectively "forgets" it. If one service instance handles ten high-volume streaming clients while another handles only one, the load remains permanently unbalanced. Unlike stateless HTTP requests, you cannot simply "re-balance" a persistent connection without forcefully disconnecting a client.
Solving with Client-Side Coordination
To handle this, we move the load balancing logic from the network layer to the application layer. Instead of a single static connection string, our C# clients should use a "Service Discovery" mechanism to periodically query the current topology and re-establish connections to underutilized nodes.
Here is how you might implement a simple client-side load-balancing strategy:
public async Task<Channel> GetBalancedChannelAsync(IServiceDiscovery discovery)
{
var instances = await discovery.GetInstancesAsync("message-hub");
// Select the instance with the fewest active persistent connections
var target = instances.OrderBy(i => i.CurrentConnectionCount).First();
return GrpcChannel.ForAddress(target.Address);
}
By tracking connection counts—either via heartbeat metadata or a shared registry—clients can actively choose to connect to the least-burdened hub. When a hub reaches a capacity threshold, it can send a "GoAway" frame (in gRPC) to connected clients, gracefully signaling them to reconnect to a different instance. This shifts the architecture from passive infrastructure balancing to Active Connection Rebalancing, ensuring that your persistent streams remain evenly distributed across your infrastructure even as workers scale horizontally.
Key Takeaway: Because persistent connections ignore traditional load-balancing cycles, you must implement client-aware discovery or protocol-level signaling to redistribute traffic dynamically.