Pipeline DAG and Needs
What you'll learn: How to use GitLab's needs keyword to create a Directed Acyclic Graph (DAG) that runs jobs in parallel as soon as their dependencies are met, instead of waiting for entire stages to complete.
Understanding Pipeline DAG
By default, GitLab pipelines execute stages sequentially—stage 2 waits for all jobs in stage 1 to finish. But what if a deployment job only needs the build artifact, not the test results? That's where DAG (Directed Acyclic Graph) execution shines.
Think of it like cooking: you don't need to finish chopping all vegetables before you start boiling water. Some tasks can run in parallel once their specific prerequisites are done.
The needs Keyword
The needs keyword lets you specify exact job dependencies, breaking free from strict stage ordering:
stages:
- build
- test
- deploy
build:
stage: build
script: echo "Building..."
unit_test:
stage: test
needs: [build]
script: echo "Testing..."
integration_test:
stage: test
needs: [build]
script: echo "Integration testing..."
deploy:
stage: deploy
needs: [build] # Skips waiting for tests!
script: echo "Deploying..."
In this example, deploy starts as soon as build completes—it doesn't wait for unit_test or integration_test. Both test jobs also run in parallel since they only need build.
Benefits
- Faster pipelines: Jobs run immediately when dependencies are satisfied
- Better parallelization: Multiple jobs execute simultaneously
- Clear dependencies: Explicit relationships between jobs
- Optimized resource usage: No idle time waiting for unrelated jobs
Key Takeaway: Use needs to define precise job dependencies and create DAG pipelines that maximize parallelization and minimize execution time by running jobs as soon as their required predecessors complete.