Automatic Parameter Registration
What you'll learn: How PyTorch's nn.Module automatically tracks learnable parameters when you assign them as attributes.
The Magic Behind PyTorch Modules
When you create a custom neural network by subclassing nn.Module, something powerful happens behind the scenes: PyTorch automatically discovers and registers all learnable parameters you assign as instance attributes. You don't need to manually maintain a list of parameters—the module does it for you.
How It Works
When you assign an nn.Parameter or another nn.Module (like nn.Linear) as an attribute in your __init__ method, PyTorch's metaclass machinery detects it and adds it to the module's internal parameter registry. This happens through Python's descriptor protocol working with nn.Module's special __setattr__ method.
Think of it like this: Imagine you're organizing a team project. Instead of manually tracking every team member, you have a smart attendance system that automatically registers anyone who walks through the door (gets assigned as an attribute). That's what nn.Module does with parameters.
What Gets Registered
nn.Parameterobjects: Raw tensors wrapped as learnable parameters- Other
nn.Moduleinstances: Sub-modules likenn.Linear,nn.Conv2d, whose parameters get recursively registered nn.ModuleListandnn.ModuleDict: Special containers that preserve registration
Why This Matters
Automatic registration means:
.parameters()returns all learnable weights without manual bookkeeping- Optimizers can access everything that needs gradients
- Moving models between devices (
.to(device)) affects all parameters automatically - Saving and loading state works seamlessly
Key Takeaway: By simply assigning nn.Parameter or sub-modules as attributes in your nn.Module.__init__, PyTorch automatically tracks them—no manual registration needed, making model building intuitive and error-resistant.