Course contentsShow
← Data Science
Lesson 2052 of 2,14547. Reproducible ResearchFree lesson

Conda Environments and Dependencies

Managing dependencies with conda through environment.yml files, including non-Python packages and system libraries.

Conda Environments and Dependencies

You'll learn how to use conda's environment.yml files to manage both Python packages and system-level dependencies for truly reproducible projects.

The Complete Environment Solution

While requirements.txt handles Python packages, conda solves a broader problem: it manages non-Python dependencies like C libraries, R packages, and system tools. Think of conda as a universal package manager that can install anything your data science project needs—from NumPy (which relies on BLAS/LAPACK libraries) to specialized geospatial tools.

The environment.yml File

Instead of separate requirements files, conda uses environment.yml to declare your complete environment:

name: myproject
channels:
  - conda-forge
  - defaults
dependencies:
  - python=3.10
  - numpy=1.24.0
  - pandas>=1.5
  - gdal  # geospatial C++ library
  - r-base=4.2  # R language
  - pip:
    - custom-package==1.0.0  # pip-only packages

This single file specifies:

  • The environment name
  • Channels (repositories) to search
  • Conda packages with versions
  • System libraries that Python packages need
  • Even pip packages as a fallback

Creating and Sharing

Export your current environment:

conda env export > environment.yml

Recreate it anywhere:

conda env create -f environment.yml

The power: someone on Windows can recreate your Linux environment's behavior because conda handles platform-specific builds of system libraries automatically.

Why This Matters

Many data science packages (scikit-learn, TensorFlow, PyTorch) depend on optimized C/Fortran libraries. Conda installs compatible versions of everything—Python, packages, and underlying libraries—preventing the "works on my machine" problem.

Key Takeaway: Conda's environment.yml provides complete reproducibility by managing Python packages, system libraries, and non-Python dependencies in a single, shareable specification file.