Data Governance and Quality
You'll learn how data warehouses enforce quality and lineage versus how data lakes handle raw data sprawl—and why governance matters for both.
Core Idea
Data warehouses and data lakes take fundamentally different approaches to data quality and governance. Warehouses enforce structure and quality before data enters (schema-on-write), while lakes accept anything first and worry about quality later (schema-on-read).
Warehouses: Strict Control
In a warehouse, data governance is built into the pipeline:
- Schema enforcement: Only data matching the defined schema gets in
- Quality checks: ETL processes validate, clean, and transform data upfront
- Lineage tracking: Clear documentation of where data came from and how it was transformed
- Access controls: Role-based permissions on tables and columns
- Compliance: GDPR, HIPAA rules enforced at ingestion
Think of it like a library catalog system—every book is classified, indexed, and shelved properly before anyone can access it.
Lakes: Flexible Sprawl
Data lakes accept raw data first, ask questions later:
- No upfront validation: Any format, any source, dumped into storage
- Data sprawl: Multiple copies, outdated versions, unknown lineage
- Quality issues: Garbage in, garbage out—discovered only when queried
- Governance challenges: Hard to track what's sensitive, who owns it, or if it's compliant
- Post-hoc governance: Tools like data catalogs and metadata layers added later to organize the chaos
The lake is like a warehouse full of unlabeled boxes—everything's there, but finding trustworthy data requires detective work.
The Tradeoff
Warehouses trade flexibility for trust. Lakes trade governance for speed and cost. Modern solutions try to split the difference with "data lakehouse" patterns that layer warehouse-style governance onto lake storage.
Key Takeaway:
Warehouses enforce data quality and lineage upfront through schema and ETL validation; lakes accept raw data freely but face governance challenges from unstructured sprawl.