Conda Environments and Dependencies
You'll learn how to use conda's environment.yml files to manage both Python packages and system-level dependencies for truly reproducible projects.
The Complete Environment Solution
While requirements.txt handles Python packages, conda solves a broader problem: it manages non-Python dependencies like C libraries, R packages, and system tools. Think of conda as a universal package manager that can install anything your data science project needs—from NumPy (which relies on BLAS/LAPACK libraries) to specialized geospatial tools.
The environment.yml File
Instead of separate requirements files, conda uses environment.yml to declare your complete environment:
name: myproject
channels:
- conda-forge
- defaults
dependencies:
- python=3.10
- numpy=1.24.0
- pandas>=1.5
- gdal # geospatial C++ library
- r-base=4.2 # R language
- pip:
- custom-package==1.0.0 # pip-only packages
This single file specifies:
- The environment name
- Channels (repositories) to search
- Conda packages with versions
- System libraries that Python packages need
- Even pip packages as a fallback
Creating and Sharing
Export your current environment:
conda env export > environment.yml
Recreate it anywhere:
conda env create -f environment.yml
The power: someone on Windows can recreate your Linux environment's behavior because conda handles platform-specific builds of system libraries automatically.
Why This Matters
Many data science packages (scikit-learn, TensorFlow, PyTorch) depend on optimized C/Fortran libraries. Conda installs compatible versions of everything—Python, packages, and underlying libraries—preventing the "works on my machine" problem.
Key Takeaway: Conda's environment.yml provides complete reproducibility by managing Python packages, system libraries, and non-Python dependencies in a single, shareable specification file.