Course contentsShow
← Python
Lesson 684 of 1,06011. Standard Library EssentialsFree lesson

Combining Multiple Iterables into Single Streams

Use itertools.chain to flatten nested data structures or merge disparate collections into a unified iteration process without creating intermediate lists.

In this lesson, you will learn how to efficiently merge multiple collections into a single, continuous stream of data using itertools.chain.

The Problem with Intermediate Lists

When you have several distinct collections—such as a list of names, a set of identifiers, and a tuple of configuration settings—you might be tempted to combine them using the + operator: all_data = list1 + list2 + list3. While this works, it forces Python to create a brand-new list in memory, copying every element from your existing collections. If those collections are large, this approach creates unnecessary memory overhead.

The itertools.chain Solution

Instead of creating intermediate objects, itertools.chain treats multiple iterables as a single, unified sequence. It returns an iterator that yields the first element from the first iterable, moves to the next, and continues until all provided iterables are exhausted. Crucially, it does this "lazily," meaning it only keeps track of which iterable it is currently reading, requiring virtually no extra memory.

Consider this example:

from itertools import chain

users = ["Alice", "Bob"]
admins = ("Charlie", "Dana")
guests = {"Eve", "Frank"}

# Combine them into one stream without copying
all_attendees = chain(users, admins, guests)

for person in all_attendees:
    print(person)

Flattening Nested Structures

itertools.chain is also highly effective for flattening nested structures. If you have a list of lists and want to process every individual item without manually writing nested for loops, you can use the from_iterable class method:

nested_data = [[1, 2], [3, 4], [5, 6]]
flat_stream = chain.from_iterable(nested_data)

# This will print 1, 2, 3, 4, 5, 6 sequentially
for number in flat_stream:
    print(number)

By using chain, you transform disparate data sources into a seamless pipeline, keeping your code memory-efficient and highly readable regardless of the volume of data.

Key Takeaway: Use itertools.chain to iterate over multiple collections as a single stream without the memory cost of creating temporary intermediate lists.