PyTorch State Dicts and Checkpoints
What you'll learn: How to save and load PyTorch models using state dictionaries and full checkpoints, and when to use each approach.
The Core Concept
PyTorch offers two primary ways to persist trained models: state dictionaries and full checkpoints. Think of a state dict as saving just the "settings" of your model (the learned weights and biases), while a checkpoint is like saving your entire workspace—model architecture, optimizer state, training epoch, and more.
State Dictionaries
A state dict is a Python dictionary that maps each layer to its parameter tensor. It's the recommended way to save models because it's lightweight and portable. You save only the learned parameters, not the model class definition itself.
import torch
# Save state dict only
torch.save(model.state_dict(), 'model_weights.pth')
# Load requires you to instantiate the model first
model = MyModel()
model.load_state_dict(torch.load('model_weights.pth'))
Full Checkpoints
A checkpoint saves everything: the model, optimizer state, epoch number, loss history—useful for resuming training exactly where you left off.
# Save full checkpoint
checkpoint = {
'model_state_dict': model.state_dict(),
'optimizer_state_dict': optimizer.state_dict(),
'epoch': epoch,
'loss': loss
}
torch.save(checkpoint, 'checkpoint.pth')
# Load and resume
checkpoint = torch.load('checkpoint.pth')
model.load_state_dict(checkpoint['model_state_dict'])
optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
When to Use Which
Use state dicts for deployment (inference only). Use checkpoints during training to resume interrupted experiments or implement early stopping with rollback.
Key Takeaway: State dicts save just the weights (best for deployment), while checkpoints save the complete training state (best for resuming training). Always prefer state dicts for production inference.