Course contentsShow
← Machine Learning and Deep Learning
Lesson 810 of 3,53819. PyTorch FundamentalsFree lesson

Training vs Evaluation Mode: model.train() and model.eval()

Learn how train() and eval() modes affect layers like Dropout and BatchNorm during training and inference.

Training vs Evaluation Mode: model.train() and model.eval()

What You'll Learn

You'll learn how PyTorch models behave differently during training versus inference, and why you must explicitly tell your model which mode it's in.

The Two Faces of a Neural Network

Some layers in your network act differently depending on whether you're training or making predictions. The two most important examples are Dropout and Batch Normalization:

  • Dropout randomly drops neurons during training to prevent overfitting, but during evaluation you want to use the full network capacity—no dropping!
  • Batch Normalization computes statistics from the current batch during training, but uses running averages collected during training when in evaluation mode.

Think of it like a student: during practice (training), they try experimental techniques and make mistakes. During the exam (evaluation), they use their best, most reliable approach.

Switching Modes in PyTorch

PyTorch requires you to explicitly set the mode:

# Before training loop
model.train()  # Enables dropout, updates BatchNorm stats
# ... training code ...

# Before evaluation/inference
model.eval()  # Disables dropout, uses fixed BatchNorm stats
# ... validation or prediction code ...

These methods don't change the parameters themselves—they set a flag that Dropout and BatchNorm layers check internally. If you forget to call model.eval() before testing, your Dropout layers will keep randomly dropping neurons, giving you inconsistent and worse results!

When to Use Each

  • model.train(): Before every training loop
  • model.eval(): Before validation, testing, or inference on new data

Key Takeaway: Always call model.train() before training and model.eval() before evaluation—layers like Dropout and BatchNorm behave fundamentally differently in each mode, and forgetting this switch leads to incorrect results and poor performance.