Async Iterator Composition

You will design pipes that transform the output of one async iterator and feed it into the next.

In this lesson, you will learn how to build modular data pipelines by chaining async iterators together so that each step transforms data before passing it to the next.

The Concept of Composition

In asynchronous data processing, we often need to perform a series of operations—like fetching a list of user IDs, transforming them into user objects, and finally filtering for "active" users. Instead of nesting all this logic into one massive function, Async Iterator Composition allows us to treat each step as a distinct, reusable generator that "pipes" data forward.

Think of it like a factory assembly line. Each machine (generator) takes an input, modifies it, and places it back on the belt for the next machine to grab.

Building a Pipeline

We can connect these generators using a simple for await...of loop. In this pattern, the output of one generator becomes the input for the next.

// 1. A source generator
async function* getNumbers() {
  for (let i = 1; i <= 3; i++) yield i;
}

// 2. A transformer generator
async function* square(source) {
  for await (const num of source) {
    yield num * num;
  }
}

// 3. The consumer (The Pipeline)
async function run() {
  const source = getNumbers();
  const squared = square(source);

  for await (const val of squared) {
    console.log(val); // Output: 1, 4, 9
  }
}

Why Compose?

The power of this approach is laziness and memory efficiency. Because we use yield, the numbers are not all processed at once. The "square" function only requests the next number from "getNumbers" when it is ready to work. If you had a stream of thousands of items, this composition ensures you only ever keep one item in memory at a time, rather than creating an enormous array of data.

You can chain as many of these as you like. By passing an iterator into a generator function, you create a "pipe" that flows data through your transformation logic step-by-step. This keeps your code clean, testable, and highly readable as your processing requirements grow.

Key Takeaway: Async iterator composition allows you to build memory-efficient, step-by-step data pipelines by passing the output of one generator as the input to another.