Course contentsShow
← Machine Learning and Deep Learning
Lesson 804 of 3,53819. PyTorch FundamentalsFree lesson

Automatic Parameter Registration

Learn how nn.Module automatically tracks parameters when you assign nn.Parameter or other modules as attributes.

Automatic Parameter Registration

What you'll learn: How PyTorch's nn.Module automatically tracks learnable parameters when you assign them as attributes.

The Magic Behind PyTorch Modules

When you create a custom neural network by subclassing nn.Module, something powerful happens behind the scenes: PyTorch automatically discovers and registers all learnable parameters you assign as instance attributes. You don't need to manually maintain a list of parameters—the module does it for you.

How It Works

When you assign an nn.Parameter or another nn.Module (like nn.Linear) as an attribute in your __init__ method, PyTorch's metaclass machinery detects it and adds it to the module's internal parameter registry. This happens through Python's descriptor protocol working with nn.Module's special __setattr__ method.

Think of it like this: Imagine you're organizing a team project. Instead of manually tracking every team member, you have a smart attendance system that automatically registers anyone who walks through the door (gets assigned as an attribute). That's what nn.Module does with parameters.

What Gets Registered

  • nn.Parameter objects: Raw tensors wrapped as learnable parameters
  • Other nn.Module instances: Sub-modules like nn.Linear, nn.Conv2d, whose parameters get recursively registered
  • nn.ModuleList and nn.ModuleDict: Special containers that preserve registration

Why This Matters

Automatic registration means:

  • .parameters() returns all learnable weights without manual bookkeeping
  • Optimizers can access everything that needs gradients
  • Moving models between devices (.to(device)) affects all parameters automatically
  • Saving and loading state works seamlessly

Key Takeaway: By simply assigning nn.Parameter or sub-modules as attributes in your nn.Module.__init__, PyTorch automatically tracks them—no manual registration needed, making model building intuitive and error-resistant.