Fan-Out/Fan-In Pattern
What you'll learn: How to scale your CI/CD pipeline by executing multiple independent tasks in parallel, then consolidating their results before moving to the next stage.
The Core Concept
The Fan-Out/Fan-In Pattern is a pipeline architecture where a single stage splits (fans out) into multiple parallel tasks, which then converge (fan in) at a synchronization point. Think of it like a river splitting into multiple streams, each flowing independently, then merging back into one river downstream.
This pattern combines the Parallel Execution Pattern you've already learned with an explicit merge or consolidation step. While parallel execution runs tasks simultaneously, fan-out/fan-in adds structure: you deliberately branch your workflow, then wait for all branches to complete before proceeding.
When to Use It
Imagine your CI/CD pipeline needs to:
- Build for multiple platforms (Linux, Windows, macOS)
- Run different test suites (unit, integration, E2E)
- Deploy to multiple regions simultaneously
- Generate various artifacts (binaries, containers, documentation)
You fan out these tasks to run in parallel, saving time. Once all complete, you fan in to a single point that might:
- Collect all build artifacts into one release package
- Aggregate test results into a unified report
- Verify all regional deployments succeeded before marking the release complete
The Synchronization Point
The "fan-in" stage is critical—it's a gate that ensures all parallel tasks finish successfully before the pipeline continues. If any branch fails, the entire fan-in stage typically fails, preventing broken builds from progressing.
Key Takeaway: The Fan-Out/Fan-In Pattern maximizes pipeline efficiency by parallelizing independent work, then enforces a quality gate that waits for complete results before advancing to subsequent stages.