Course contentsShow
← Machine Learning and Deep Learning
Lesson 2840 of 3,53861. Experiment Tracking and Model ManagementFree lesson

DVC: Data Version Control Fundamentals

Introduction to DVC for tracking datasets with Git-like commands while storing actual data remotely.

DVC: Data Version Control Fundamentals

What you'll learn: How DVC enables Git-like versioning of datasets while keeping large files in remote storage instead of your repository.

The Problem DVC Solves

Git excels at versioning code but struggles with large datasets. Storing gigabytes of training data directly in Git bloats repositories and slows operations. You need code and data versioned together, but stored separately.

How DVC Works

DVC acts as a lightweight layer on top of Git. Think of it like this: Git tracks a "receipt" (metadata file), while DVC handles the actual "groceries" (data) in a separate warehouse (remote storage).

When you add a dataset with DVC, it:

  1. Computes a content hash of your data (using content-addressable storage, which you learned in the previous lesson)
  2. Creates a .dvc file containing metadata and the hash—this small file goes into Git
  3. Moves actual data to a cache directory (.dvc/cache)
  4. Can push/pull data to/from remote storage (S3, GCS, Azure, SSH, etc.)

Git-Like Commands

DVC mirrors Git's intuitive interface:

  • dvc add data/train.csv — track a dataset (like git add)
  • dvc push — upload data to remote storage
  • dvc pull — download data from remote storage
  • dvc checkout — sync data to match the current Git commit

Your Git repository only contains the lightweight .dvc files, while teammates can reconstruct the exact dataset by pulling from shared remote storage.

The Workflow

You commit both code changes (via Git) and data changes (via DVC) together. When switching Git branches or commits, dvc checkout ensures your data matches that point in history—reproducibility without repository bloat.

Key Takeaway: DVC provides Git-like version control for datasets by storing metadata in Git while keeping actual data in efficient remote storage, enabling reproducible ML pipelines without bloating repositories.