In this lesson, you will learn how to distinguish between critical data failures that should stop your pipeline and minor anomalies that only require a notification.
Why Severity Matters
When you run dbt test, the default behavior is for any failed test to trigger an "error" status. In a production pipeline, this usually halts the entire process. However, in real-world data engineering, not all data issues are created equal.
If a primary key is duplicated, that is a critical failure—you must stop the pipeline to prevent broken downstream dashboards. But what if a column that should contain "mostly" positive numbers has a few negatives? That might be an anomaly you want to investigate without stopping the delivery of your reports. This is where severity levels come in.
Configuring Severity
You can control this behavior in your schema.yml file by adding a severity configuration to your tests. dbt supports two levels: error (default) and warn.
models:
- name: stg_orders
columns:
- name: order_id
tests:
- unique:
severity: error # This will halt the pipeline
- name: order_amount
tests:
- accepted_values:
values: [10, 20, 30]
severity: warn # Pipeline continues, but alerts you
How Thresholds Work
When you set severity: warn, dbt will still execute the test and report the failure in your logs, but it won’t mark the test suite as "failed" in a way that causes the command to exit with a non-zero code. This allows your CI/CD pipeline or orchestration tool (like Airflow) to proceed with the next steps of the transformation, while you receive a warning message in your console or logs.
Using warn is a powerful way to implement "soft" data quality checks. It allows you to monitor data drift or minor inconsistencies over time without causing unnecessary downtime for your stakeholders. Reserve error for business-critical invariants and warn for patterns you want to observe before taking action.
Key Takeaway: Use severity: error to block data pipelines when core business logic is compromised, and severity: warn to track non-critical anomalies without interrupting the flow of your data.