In this lesson, you will learn how to orchestrate multiple asynchronous operations to run concurrently and aggregate their results using asyncio.gather.
The Power of Aggregation
When working with asynchronous coroutines, running them one after another using await forces your program to wait for the first task to finish before starting the second. This creates a bottleneck. asyncio.gather() solves this by allowing you to schedule multiple awaitable objects—such as coroutines or Task objects—to run concurrently on the event loop. It acts as a synchronization point, pausing execution only until all specified tasks have completed, at which point it returns a list containing the results of every task in the order they were provided.
Practical Implementation
Imagine you need to fetch data from three different APIs. Instead of awaiting them sequentially, you group them:
import asyncio
async def fetch_data(id):
await asyncio.sleep(1) # Simulate network I/O
return f"Data from {id}"
async def main():
# Schedule all three tasks to run concurrently
results = await asyncio.gather(
fetch_data(1),
fetch_data(2),
fetch_data(3)
)
# results will be ['Data from 1', 'Data from 2', 'Data from 3']
print(results)
asyncio.run(main())
Why Use Gather?
- Efficiency: You reduce the total execution time from the sum of all tasks to the duration of the longest single task.
- Ordered Results: Unlike manually managing individual tasks,
gatherguarantees that the resulting list maps exactly to the order of the arguments passed, making it easy to track which output belongs to which input. - Simplicity: It provides a clean, high-level interface to handle concurrency without needing to manually track
Taskobjects or manage complex callbacks.
If one of the coroutines raised an exception, gather would immediately raise that exception, while the other tasks would continue to run in the background. This makes it an excellent tool for batch processing where you need all components to succeed as a single unit of work.
Key Takeaway: asyncio.gather transforms sequential bottlenecks into parallel operations by wrapping multiple coroutines into a single awaitable that returns a collected list of results.