Course contentsShow
← Python
Lesson 320 of 1,0605. Advanced Iteration and ComprehensionsFree lesson

Architecting Event-Driven Systems with Generator Patterns

You will design event-driven architectures that utilize generators to handle asynchronous updates and stream processing in complex Python applications.

In this lesson, you will learn how to use generator coroutines to build decoupled, event-driven architectures that react to incoming data streams.

At its heart, an event-driven system is a collection of components that wait for messages and perform actions only when those messages arrive. By using generators as coroutines—utilizing the .send() method—we can turn these components into "event listeners" that maintain their own internal state without needing complex class structures.

The Event-Driven Generator Pattern

In this pattern, a "dispatcher" sends data to a generator coroutine. The generator pauses at a yield statement, processes the event, updates its internal state, and waits for the next update. This keeps your system memory-efficient because you aren't loading all events into a list; you are reacting to them as they flow through the pipeline.

Consider a simple event logger that tracks how many error events it has received:

def error_tracker():
    count = 0
    while True:
        event = yield
        if event == "ERROR":
            count += 1
            print(f"Logged {count} errors so far.")

# Initialize the event listener
logger = error_tracker()
next(logger)  # Prime the generator

# Simulate an event stream
events = ["INFO", "ERROR", "DEBUG", "ERROR"]
for e in events:
    logger.send(e)

Why This Architecture Works

  1. Separation of Concerns: The code generating the events (the loop) doesn't need to know how the events are handled; it simply sends data into the sink.
  2. State Encapsulation: The count variable is "trapped" inside the generator, preventing global variable clutter while allowing the component to remember its history across multiple send() calls.
  3. Lazy Execution: The logic inside the generator only triggers when send() is invoked, making this ideal for handling high-frequency streams where you only want to process data on demand.

By chaining these coroutines using yield from or simple composition, you can build complex, reactive systems that process data flow step-by-step, ensuring your architecture remains modular and highly responsive to input.

Key Takeaway: Generator coroutines allow you to build stateful, event-reactive components that process data streams lazily without the overhead of managing complex objects or global state.