Loading and Resuming Training
What You'll Learn
You'll learn how to load a saved checkpoint and properly restore your model, optimizer, and scheduler states so training continues exactly where it left off.
Why Loading Matters
In the previous lesson, you learned to save checkpoints. Now you need to bring them back to life! Loading isn't just about the model weights—you must restore the entire training state to resume seamlessly.
Think of it like pausing a video game and saving your progress. When you load that save file, you don't just want your character's stats—you want your exact location, inventory, quest progress, everything exactly as it was.
How to Load Checkpoints
Use torch.load() to read the checkpoint dictionary from disk, then restore each component:
# Load the checkpoint dictionary
checkpoint = torch.load('checkpoint.pth')
# Restore model weights
model.load_state_dict(checkpoint['model_state_dict'])
# Restore optimizer state (momentum buffers, etc.)
optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
# Restore scheduler state if you saved one
if 'scheduler_state_dict' in checkpoint:
scheduler.load_state_dict(checkpoint['scheduler_state_dict'])
# Retrieve training metadata
start_epoch = checkpoint['epoch'] + 1 # Resume from next epoch
best_loss = checkpoint['loss']
print(f"Resuming from epoch {start_epoch}")
Now your training loop starts from start_epoch instead of 0, and everything—learning rates, momentum, Adam's moving averages—picks up precisely where it stopped.
Common Pattern
After loading, set the model to the appropriate mode:
model.train() # If resuming training
# or
model.eval() # If evaluating/inference only
Key Takeaway: Use torch.load() to retrieve the checkpoint dictionary, then use .load_state_dict() on your model, optimizer, and scheduler to restore the complete training state—enabling seamless resumption from any saved point.