Course contentsShow
← Data Science
Lesson 1873 of 2,14543. Data Pipelines and OrchestrationFree lesson

DVC (Data Version Control) Basics

Learn DVC fundamentals: tracking datasets with Git, storing data remotely, and maintaining lightweight repositories.

DVC (Data Version Control) Basics

You'll learn how to track large datasets with Git-like commands while keeping your repository lightweight and data stored separately.

The Core Problem DVC Solves

Git is perfect for code but chokes on large datasets. Committing a 5GB CSV file directly makes your repository bloated, slow, and impractical for collaboration. DVC bridges this gap by letting you version control data references in Git while storing actual data files elsewhere.

How DVC Works

Think of DVC like a library catalog system. Git stores the catalog cards (small .dvc files containing metadata and checksums), while the actual books (your datasets) live in a remote warehouse (cloud storage like S3, Google Drive, or Azure Blob).

When you track a dataset with dvc add data/train.csv, DVC:

  1. Computes a hash of your file
  2. Creates a tiny data/train.csv.dvc file with the hash
  3. Adds the actual data file to .gitignore

You commit the .dvc file to Git (it's just a few bytes), not the data itself.

Remote Storage Connection

DVC connects to remote storage using dvc remote add. Your team can then:

  • dvc push to upload tracked data to remote storage
  • dvc pull to download data referenced by .dvc files
  • dvc checkout to switch between data versions

This keeps your Git repository fast while maintaining full version control over datasets. Each Git commit points to specific data versions through the .dvc files.

Real Benefit

Your repository stays under 100MB even with terabytes of training data. Team members clone lightweight repos and selectively pull only needed datasets.

Key Takeaway: DVC separates data storage from version tracking—Git tracks lightweight .dvc metadata files while actual datasets live in remote storage, giving you Git-like versioning for data without bloating repositories.