Data Engineering
Build data platforms: modeling, ETL/ELT, lakehouses, warehouses, dbt, Airflow, Spark, Kafka, streaming and data quality.
825 lessons·All lessons free
- 1Mapping the Data Engineering Lifecycle Stages
- 2Defining Data Engineering Responsibilities and Scope
- 3Evaluating Source Systems and Data Ingestion Patterns
- 4Selecting Storage Architectures for Analytical Workloads
- 5Implementing Data Transformation and Processing Logic
- 6Orchestrating Complex Data Pipeline Workflows
- 7Ensuring Data Quality and Observability Standards
- 8Securing Data Pipelines and Managing Access Controls
- 9Identifying Limitations of Monolithic Data Architectures
- 10Principles of Distributed Data Systems
- 11Comparing Batch Processing and Streaming Architectures
- 12Decoupling Storage and Compute in Modern Pipelines
- 13Understanding the Traditional ETL Workflow
- 14Analyzing the Shift Toward ELT Paradigms
- 15Leveraging Cloud Data Warehouses for Transformation
- 16Optimizing Data Ingestion for ELT Pipelines
- 17Implementing Schema-on-Read Strategies
- 18Managing Data Gravity and Movement Costs
- 19Evaluating Architectural Trade-offs in Distributed Systems
- 20Selecting Architectures Based on Business Requirements
- 21Distinguishing Between OLTP and OLAP Workloads
- 22Evaluating Row-Based Storage for Transactional Efficiency
- 23Understanding Columnar Storage for Analytical Queries
- 24Optimizing Data Compression via Columnar Layouts
- 25Analyzing Disk I/O Patterns in Storage Architectures
- 26Mapping Storage Formats to Query Requirements
- 27Implementing Vectorized Execution in Columnar Engines
- 28Balancing Write Latency and Read Performance
- 29Architecting Hybrid Transactional and Analytical Processing Systems
- 30Evaluating Storage Trade-offs in Distributed Environments
- 31Evolution of Cloud Data Architectures
- 32Decoupling Compute and Storage Layers
- 33Managed Services versus Self-Hosted Infrastructure
- 34The Role of Cloud Data Warehouses
- 35Implementing Modern Data Ingestion Pipelines
- 36Selecting Cloud Object Storage Solutions
- 37Orchestrating Workflows in Distributed Environments
- 38Data Transformation Using ELT Paradigms
- 39Integration of Cloud-Native Data Catalogs
- 40Securing Pipelines in the Cloud
- 41Monitoring Performance of Cloud Services
- 42Managing Data Quality at Scale
- 43Optimizing Costs for Cloud Data Stacks
- 44Facilitating Data Democratization via Managed BI
- 45Adapting to Emerging Data Stack Trends
- 46Distinguishing OLTP and OLAP Data Workloads
- 47Fundamentals of Entity Relationship Modeling
- 48Applying Normalization to Reduce Data Redundancy
- 49Transitioning from Normalized Schemas to Dimensional Models
- 50Designing Fact Tables for Analytical Metrics
- 51Implementing Dimension Tables for Contextual Analysis
- 52Utilizing Surrogate Keys in Data Warehousing
- 53Handling Slowly Changing Dimensions with Type Techniques
- 54Denormalization Strategies for Query Performance
- 55Managing Granularity and Aggregation Levels
- 56Implementing Snowflake Schemas for Data Integrity
- 57Modeling Data Vault for Scalable Architectures
- 58Optimizing Data Models for Columnar Storage
- 59Evaluating Trade-offs in Data Modeling Approaches
- 60Version Control Systems for Data Infrastructure
- 61Declarative Infrastructure as Code Principles
- 62Automated Provisioning with Infrastructure Tools
- 63Continuous Integration for Data Pipelines
- 64Continuous Deployment Strategies for Data Engineering
- 65Managing Environment Parity with Configuration Management
- 66Immutable Infrastructure Patterns in Data Platforms
- 67Automated Testing for Infrastructure Security
- 68Observability and Monitoring for Automated Deployments
- 69Defining Data Governance Frameworks and Policies
- 70Implementing Principle of Least Privilege Access
- 71Applying Role-Based Access Control Models
- 72Enforcing Data Encryption at Rest and Transit
- 73Anonymizing and Masking Sensitive Data Fields
- 74Establishing Comprehensive Data Audit Logging
- 75Automating Compliance and Security Policy Testing
- 76Managing Secrets and Sensitive Configuration Credentials
- 77Implementing Data Lineage for Regulatory Compliance
- 78Structuring Data Retention and Deletion Workflows