Async Request Batching

In this lesson, you will learn how to reduce network overhead by grouping multiple individual asynchronous requests into a single, efficient bulk operation.

The Problem: The "Chatty" Interface

Imagine your application needs to fetch user details for ten different IDs. If you send ten separate HTTP requests, your server spends most of its time waiting for the network "handshake" to complete for each individual request. This is often called a "chatty" interface. Even if your server is fast, the latency adds up quickly.

The Solution: Request Batching

Instead of firing off requests immediately, Async Request Batching involves collecting those individual calls over a tiny window of time—perhaps just a few milliseconds—and bundling them into one "bulk" request. You send one request to your database or API, receive a single combined response, and then dispatch the individual results back to the parts of your code that asked for them.

Think of it like a bus: instead of ten people taking ten separate taxis to the same destination, they all wait a minute at the bus stop and ride one bus together.

Putting it into Practice

In Node.js, you can implement this using a small buffer and setTimeout. Here is a simplified concept:

const pendingRequests = [];

function fetchUser(userId) {
  return new Promise((resolve) => {
    // Add request to the queue
    pendingRequests.push({ userId, resolve });

    // If this is the first item, start the timer
    if (pendingRequests.length === 1) {
      setTimeout(async () => {
        const batch = pendingRequests.splice(0); // Take everything
        const ids = batch.map(r => r.userId);
        
        // One bulk network call
        const users = await db.getUsersByIds(ids);
        
        // Distribute results back to the original callers
        batch.forEach((req, index) => req.resolve(users[index]));
      }, 50); // Wait 50ms to gather requests
    }
  });
}

By waiting briefly, you trade a tiny bit of latency for a massive improvement in network efficiency and reduced load on your downstream services.

Key Takeaway: Batching minimizes network round trips by grouping disparate tasks into one high-efficiency operation, drastically improving your application's throughput.