In this lesson, you will learn how to write memory-efficient generator expressions that produce values lazily instead of creating full collections in memory.
The Concept of Lazy Evaluation
When you use a list comprehension like [x**2 for x in range(1000000)], Python immediately calculates every single square and stores them all in your computer's RAM. If the dataset is large, this can slow down or crash your program.
A generator expression solves this by using parentheses instead of square brackets. It doesn't store the results; it remembers the "recipe" for creating the values and produces them one at a time, only when requested. This is known as lazy evaluation.
Syntax Comparison
Notice the difference in syntax:
# List comprehension: Stores all million values in memory
squares_list = [x**2 for x in range(1000000)]
# Generator expression: Stores only the rule for creating values
squares_gen = (x**2 for x in range(1000000))
When you print squares_list, you see a massive list. When you print squares_gen, you see something like <generator object <genexpr> at 0x...>, which tells you that the generator is waiting for instructions to provide the next item.
How to Evaluate Them
Since generators don't hold their data, you access them using iteration. You can loop over them or pass them to a function that consumes iterables, such as sum() or list():
# Iterating over the generator
for val in squares_gen:
# Logic happens here, one value at a time
if val > 100:
break
Because the generator generates values on-the-fly, you never hold more than one square in memory at any given moment. Once a value is yielded and processed, it can be discarded by the system, making your program significantly more scalable.
Key Takeaway: Generator expressions use parentheses to create memory-efficient, lazy iterators that produce values on demand instead of storing an entire collection in your system's memory.