Course contentsShow
← Cloud and Infrastructure
Lesson 1760 of 2,98936. GitLab CI/CDFree lesson

Pipeline DAG and Needs

Define directed acyclic graphs with needs keyword to optimize pipeline execution and parallelization.

Pipeline DAG and Needs

What you'll learn: How to use GitLab's needs keyword to create a Directed Acyclic Graph (DAG) that runs jobs in parallel as soon as their dependencies are met, instead of waiting for entire stages to complete.

Understanding Pipeline DAG

By default, GitLab pipelines execute stages sequentially—stage 2 waits for all jobs in stage 1 to finish. But what if a deployment job only needs the build artifact, not the test results? That's where DAG (Directed Acyclic Graph) execution shines.

Think of it like cooking: you don't need to finish chopping all vegetables before you start boiling water. Some tasks can run in parallel once their specific prerequisites are done.

The needs Keyword

The needs keyword lets you specify exact job dependencies, breaking free from strict stage ordering:

stages:
  - build
  - test
  - deploy

build:
  stage: build
  script: echo "Building..."

unit_test:
  stage: test
  needs: [build]
  script: echo "Testing..."

integration_test:
  stage: test
  needs: [build]
  script: echo "Integration testing..."

deploy:
  stage: deploy
  needs: [build]  # Skips waiting for tests!
  script: echo "Deploying..."

In this example, deploy starts as soon as build completes—it doesn't wait for unit_test or integration_test. Both test jobs also run in parallel since they only need build.

Benefits

  • Faster pipelines: Jobs run immediately when dependencies are satisfied
  • Better parallelization: Multiple jobs execute simultaneously
  • Clear dependencies: Explicit relationships between jobs
  • Optimized resource usage: No idle time waiting for unrelated jobs

Key Takeaway: Use needs to define precise job dependencies and create DAG pipelines that maximize parallelization and minimize execution time by running jobs as soon as their required predecessors complete.