In this lesson, you will learn how to transform complex, nested structures into a single-dimensional stream of elements using memory-efficient generator expressions.
The Challenge of Nested Data
When dealing with multidimensional collections—like a list of lists—you often find yourself writing nested for loops to access the inner elements. While this works, it can become cumbersome and, more importantly, it often forces you to load the entire flattened collection into memory at once. If your dataset is massive, this can lead to performance bottlenecks or memory exhaustion.
Streamlining with Generators
A generator expression allows you to "flatten" these collections on the fly. Instead of creating a new, massive list, Python produces elements one by one as you iterate over them.
Consider this nested list:
matrix = [[1, 2, 3], [4, 5], [6, 7, 8, 9]]
To flatten this without creating an intermediate list, you can use a nested generator expression inside a function or a loop:
# Create a generator that yields elements one at a time
flattened = (item for sublist in matrix for item in sublist)
# Consume the generator
for num in flattened:
print(num, end=" ")
# Output: 1 2 3 4 5 6 7 8 9
How It Works
The syntax (item for sublist in matrix for item in sublist) mirrors the structure of a nested for loop. The first part, for sublist in matrix, extracts each inner collection. The second part, for item in sublist, extracts the individual elements from those inner collections.
Because we use parentheses instead of square brackets, Python treats this as a generator. It keeps track of the "state" of the iteration internally, only calculating the next value when you request it. This is highly memory-efficient because it never stores the full flattened sequence in your RAM; it simply provides a clean, single-dimensional stream of data for your processing logic to consume.
Key Takeaway: Generator expressions provide a memory-efficient way to flatten nested structures by yielding elements one at a time rather than allocating space for a new, monolithic collection.