Course contentsShow
← Python
Lesson 629 of 1,06010. Advanced Language FeaturesFree lesson

Optimizing Memory Usage with Generator Functions

You will replace traditional list-returning functions with generators using the yield keyword to produce values lazily and reduce memory overhead.

Optimizing Memory Usage with Generator Functions

In this lesson, you will learn how to replace memory-intensive list-returning functions with generators to produce values lazily, significantly reducing your application's memory footprint.

When you write a function that collects thousands of items into a list and returns it, Python must allocate enough memory to hold every single one of those items at once. If your dataset grows large enough, this can lead to memory exhaustion. A generator solves this by using the yield keyword instead of return.

From Lists to Generators

Consider a function that generates a large sequence of numbers:

# Memory-intensive: creates the entire list in RAM
def get_numbers(n):
    result = []
    for i in range(n):
        result.append(i * 2)
    return result

# Memory-efficient: produces one value at a time
def yield_numbers(n):
    for i in range(n):
        yield i * 2

When you call get_numbers(1_000_000), Python constructs a list object containing one million integers. However, when you call yield_numbers(1_000_000), the function returns a generator object immediately without executing the code inside. The code only runs when you iterate over it (e.g., in a for loop), calculating each value "on the fly" and discarding it once it has been processed.

Why This Matters

Because generators maintain their internal state between iterations, they keep track of where they are in the sequence without needing to store the entire collection in memory. This "lazy evaluation" is the cornerstone of writing scalable Python code. Whenever you find yourself building a list just to pass it into a loop or a comprehension, consider swapping it for a generator. You preserve the logic of your function while ensuring your application stays lean, even when processing massive data streams.

Key Takeaway: Use the yield keyword to create generators that produce data lazily, preventing the need to store entire collections in memory at once.