Course contentsShow
← Data Science
Lesson 1821 of 2,14543. Data Pipelines and OrchestrationFree lesson

Hybrid Approaches and Modern Data Stacks

Explore how modern pipelines often combine ETL and ELT, using tools like dbt for transformation after loading.

Hybrid Approaches and Modern Data Stacks

What You'll Learn

You'll discover how modern data pipelines strategically combine ETL and ELT patterns, leveraging tools like dbt to transform data after loading into cloud warehouses.

The Evolution Beyond Pure ETL or ELT

In practice, most organizations don't use only ETL or only ELT—they blend both approaches based on specific needs. Modern data stacks have emerged that capitalize on the strengths of each pattern while mitigating their weaknesses.

How Hybrid Pipelines Work

Think of it like meal prep: some ingredients you wash and chop at the market (ETL), while others you bring home whole and prepare in your kitchen (ELT). Similarly, you might:

  • Extract and lightly transform sensitive data (masking PII) before loading to comply with regulations
  • Load raw data into your cloud warehouse for most sources
  • Transform within the warehouse using SQL-based tools like dbt (data build tool)

This hybrid pattern is sometimes called "EtLT"—minimal transformation before loading, then heavy transformation in the warehouse.

The Modern Data Stack Pattern

A typical modern stack combines:

  1. Extraction tools (Fivetran, Airbyte) that load raw data with minimal transformation
  2. Cloud data warehouses (Snowflake, BigQuery, Redshift) providing scalable compute
  3. Transformation layers like dbt that version-control SQL transformations, run tests, and document data models
  4. Orchestration to schedule and monitor the entire pipeline

dbt has become particularly popular because it lets analysts write modular, tested SQL transformations that run directly in the warehouse—leveraging the warehouse's power while maintaining software engineering best practices.

Why This Approach Works

You get raw data quickly (auditability), transform with the warehouse's compute power (cost-efficiency), and maintain clear transformation logic in version control (reliability).

Key Takeaway: Modern data pipelines typically combine ETL and ELT strategies, using tools like dbt to perform SQL-based transformations after loading raw data into cloud warehouses—balancing speed, flexibility, and governance.