Course contentsShow
← Machine Learning and Deep Learning
Lesson 2849 of 3,53861. Experiment Tracking and Model ManagementFree lesson

Setting Random Seeds Correctly

Learn to set seeds for Python, NumPy, PyTorch, and CUDA to control stochastic processes in training.

Setting Random Seeds Correctly

You'll learn how to properly set random seeds across Python, NumPy, PyTorch, and CUDA to ensure reproducible training runs.

The Core Idea

Machine learning training involves many stochastic (random) processes: weight initialization, data shuffling, dropout masks, and more. To reproduce experimental results, you must control all sources of randomness by setting seeds for every library that generates random numbers.

Think of random seeds like planting identical gardens: if you use the same seed packet (number) and growing conditions (code), you'll get the same flowers (results) every time.

Setting Seeds in Practice

You need to set seeds for four distinct systems:

1. Python's Built-in Random Module

import random
random.seed(42)

2. NumPy

import numpy as np
np.random.seed(42)

3. PyTorch CPU Operations

import torch
torch.manual_seed(42)

4. PyTorch CUDA Operations

torch.cuda.manual_seed(42)
torch.cuda.manual_seed_all(42)  # For multi-GPU setups

Complete Setup Function

def set_all_seeds(seed=42):
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)
    torch.cuda.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)

Call this before creating models, dataloaders, or any random operations.

Important Notes

  • Set seeds at the start of your script, before any imports that might trigger random operations
  • Each library maintains its own random state—setting one doesn't affect others
  • In distributed training, you may want different seeds per process (covered in earlier lessons on distributed training)

Key Takeaway: Reproducible ML experiments require setting random seeds for Python, NumPy, PyTorch CPU, and PyTorch CUDA operations—missing even one source can break reproducibility.