Course contentsShow
← System Design
Lesson 849 of 1,91922. Service MeshFree lesson

Load Balancing Strategies in Service Mesh

Service mesh load balancing algorithms: least request, round robin, random, and locality-aware routing.

Load Balancing Strategies in Service Mesh

What You'll Learn

You'll discover the different algorithms a service mesh uses to distribute traffic intelligently across multiple service instances.

Why Load Balancing Matters in a Mesh

In a service mesh, the sidecar proxy (like Envoy) handles load balancing decisions for each outbound request. Unlike traditional load balancers that sit at the edge, the mesh distributes this intelligence to every service, allowing fine-grained control over how traffic flows between microservices.

Core Load Balancing Algorithms

Round Robin

The proxy sends requests to each service instance in rotation—first to server A, then B, then C, then back to A. Think of it like dealing cards evenly around a table.

Best for: Uniform workloads where all instances have similar capacity.

Least Request (Least Connection)

The proxy tracks active requests at each instance and sends new requests to the instance with the fewest outstanding requests.

Best for: Workloads with variable request duration, preventing overload on slower instances.

Random

Each request goes to a randomly selected instance. Surprisingly effective at scale—randomness tends to distribute evenly across many requests.

Best for: Simple scenarios or when instances are identical and failures are handled separately.

Locality-Aware Routing

The proxy prefers instances in the same availability zone, region, or datacenter to minimize latency and cross-zone costs. It only routes outside the local zone if local instances are unavailable or overloaded.

Best for: Reducing latency and cloud egress costs in multi-region deployments.

Why This Matters

The service mesh's control plane configures these algorithms, but the data plane (sidecar proxies) executes them in real-time for every request—no centralized bottleneck. You can even use different strategies for different services based on their specific needs.

Key Takeaway: Service mesh load balancing happens at the sidecar proxy level, offering algorithms like round robin for simplicity, least request for variable loads, random for stateless uniformity, and locality-aware routing for latency optimization.