Reproducibility: Random Seeds
What You'll Learn
You'll learn how to set random seeds in PyTorch, NumPy, and Python to make your training runs produce identical results every time.
Why Randomness Matters in Deep Learning
Deep learning involves many sources of randomness: weight initialization, data shuffling, dropout masks, and data augmentation. While this randomness helps models generalize, it also means running the same code twice can produce different results—different accuracies, different loss curves, even different best models.
Think of it like shuffling a deck of cards. Every shuffle produces a different order. But what if you need to show someone exactly the same hand you drew yesterday? You'd need to control the shuffle.
Setting Random Seeds
Random number generators aren't truly random—they're pseudo-random, starting from a "seed" value. The same seed always produces the same sequence of "random" numbers.
To make your PyTorch training reproducible, set seeds for all three libraries that introduce randomness:
import torch
import numpy as np
import random
def set_seed(seed=42):
"""Set seeds for reproducibility"""
random.seed(seed) # Python's random module
np.random.seed(seed) # NumPy operations
torch.manual_seed(seed) # PyTorch CPU operations
if torch.cuda.is_available():
torch.cuda.manual_seed(seed) # Current GPU
torch.cuda.manual_seed_all(seed) # All GPUs
# For completely deterministic behavior (may reduce performance)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
# Call at the start of your training script
set_seed(42)
Now when you train, checkpoint, compare experiments, or debug, you'll get exactly the same results each run—same validation accuracy, same weight values, everything.
Key Takeaway: Setting random seeds for Python, NumPy, and PyTorch ensures your training results are reproducible, making debugging easier and experiments comparable. Call your seed-setting function at the very start of your training script.