Course contentsShow
← Data Engineering
Lesson 349 of 8255. Batch ETL/ELT Pipelines with dbtFree lesson

Enforcing Documentation Coverage with CI Checks

Integrate automated checks into your CI pipeline to ensure all new models and columns meet mandatory documentation standards before deployment.

In this lesson, you will learn how to automate your documentation standards so that your data team never pushes a model to production without proper descriptions.

Why Documentation Fails Without Enforcement

We have previously covered how to add description fields to your schema.yml files to improve discoverability. However, as pipelines grow, it is easy for developers to accidentally add new models or columns without providing definitions. Without automated checks, your "data catalog" quickly becomes filled with mystery columns, defeating the purpose of your documentation efforts.

The CI Gatekeeper

To solve this, we integrate a "documentation coverage check" into our CI/CD pipeline. Instead of manually auditing files, we use the dbt-checkpoint framework or native dbt commands to scan the project during a Pull Request.

If a developer submits a new model or adds a new column to an existing model without a corresponding description, the CI check will fail, blocking the merge. This treats documentation as a first-class citizen, just like unit tests or data quality checks.

You can trigger this check in your CI pipeline by running:

dbt test --select source_status --vars '{"dbt_docs_check": true}'

More commonly, we use a command like dbt-checkpoint or a pre-commit hook that specifically validates that every column defined in your models/ directory has a non-empty description field:

# Example snippet for a pre-commit configuration
- repo: https://github.com/dbt-checkpoint/dbt-checkpoint
  rev: v1.0.0
  hooks:
    - id: check-column-desc-are-not-empty
    - id: check-model-has-all-columns-described

What Happens in CI?

  1. Developer submits a PR: They add a price column to the orders model but forget to update the schema.yml.
  2. CI Pipeline triggers: The pre-commit or CI task parses the manifest.json file.
  3. Failure: The pipeline detects that the price column lacks a description and returns a non-zero exit code.
  4. Correction: The PR is blocked. The developer adds the description, commits the change, and the pipeline passes.

By shifting this responsibility to the CI runner, you ensure that your documentation remains accurate and helpful, allowing downstream analysts to trust the metadata without having to read raw SQL.

Key Takeaway: Automating documentation coverage in CI transforms metadata from an optional "nice-to-have" into a mandatory requirement for every deployment.