Fan-Out/Fan-In Pattern
You'll learn how to split a single task into many parallel function invocations and recombine the results.
What Is Fan-Out/Fan-In?
The Fan-Out/Fan-In pattern divides a large workload into smaller chunks, processes them concurrently across multiple serverless function instances, then aggregates the results back into a single response. Think of it like a restaurant kitchen: one order ticket (fan-out) gets distributed to multiple stations (grill, salad, dessert) working in parallel, then the expeditor combines everything (fan-in) onto one plate for delivery.
Core Mechanism
Fan-Out occurs when a coordinator function or event triggers many parallel invocations. Each worker function handles a subset of data independently—no coordination needed between workers.
Fan-In collects results from all parallel workers. This typically involves:
- Writing worker results to temporary storage (object store, database)
- A final aggregator function that reads all results once complete
- Using completion triggers (like "last worker finished" events)
Common Use Cases
- Batch processing: Split a large dataset across 100 functions, each processing 1% of records
- Map-reduce operations: Parallel computation followed by result merging
- Parallel API calls: Query multiple external services simultaneously
- Image/video processing: Different functions handle different transformations
Coordination Challenges
The fan-in stage needs to know when all workers complete. Solutions include tracking worker count in a database, using coordination services, or leveraging event-driven completion signals combined with the Event-Driven Invocation Pattern you've learned.
Key Takeaway: Fan-Out/Fan-In transforms single-threaded workloads into massively parallel operations by distributing work across many function instances and intelligently aggregating results, dramatically reducing total processing time.