Course contentsShow
← System Design
Lesson 880 of 1,91923. API Gateway PatternFree lesson

Request Routing and Load Balancing

How API gateways route requests to backend services and distribute load across service instances.

Request Routing and Load Balancing

What you'll learn: How API gateways intelligently direct incoming requests to the right backend services and distribute load across multiple instances.

The Core Idea

An API gateway sits at the entry point to your microservices system and must solve two critical problems: where to send each request (routing) and which instance to send it to (load balancing).

Request routing examines incoming requests—typically analyzing the URL path, HTTP method, headers, or query parameters—and maps them to the appropriate backend service. For example, /users/* routes to the User Service, while /orders/* routes to the Order Service. The gateway maintains a routing table that defines these mappings.

Load balancing distributes requests across multiple instances of the same service. If your User Service runs five replicas, the gateway must choose which instance handles each request. Common strategies include:

  • Round-robin: Cycle through instances sequentially
  • Least connections: Send to the instance with fewest active requests
  • Weighted: Prefer instances with more resources
  • Health-aware: Skip unhealthy instances entirely

Think of the gateway as a postal sorting center. Routing is like reading the address to determine which delivery truck (service) should handle it. Load balancing is like deciding which specific driver on that truck's team should take the package, based on their current workload.

The gateway typically integrates with service discovery to maintain an up-to-date list of available service instances and their health status, automatically adding or removing instances as they scale.

Key Takeaway: API gateways route requests to the correct service based on request characteristics, then load balance across healthy instances to ensure even distribution and optimal resource utilization.