Course contentsShow
← Machine Learning and Deep Learning
Lesson 807 of 3,53819. PyTorch FundamentalsFree lesson

ModuleList and ModuleDict for Dynamic Architectures

Learn when and how to use nn.ModuleList and nn.ModuleDict for models with variable or named layer collections.

ModuleList and ModuleDict for Dynamic Architectures

What You'll Learn

You'll learn how to use nn.ModuleList and nn.ModuleDict to build models with variable numbers of layers or named layer collections that PyTorch can properly track.

Why We Need Them

When you create a custom nn.Module, PyTorch automatically registers submodules you assign as attributes. But what if you don't know how many layers you need in advance, or want to store them in a collection? Using a regular Python list or dict won't work—PyTorch won't register those layers' parameters!

Think of it like this: PyTorch needs to know about all your model's layers to track their parameters for training. Regular Python containers are "invisible" to PyTorch, but ModuleList and ModuleDict are "visible."

Using ModuleList

Use nn.ModuleList when you need a variable number of similar layers:

import torch.nn as nn

class DynamicNet(nn.Module):
    def __init__(self, num_layers, hidden_size):
        super().__init__()
        self.layers = nn.ModuleList([
            nn.Linear(hidden_size, hidden_size) 
            for _ in range(num_layers)
        ])
    
    def forward(self, x):
        for layer in self.layers:
            x = layer(x)
        return x

Using ModuleDict

Use nn.ModuleDict when you need named access to different layer options:

class MultiTaskNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.shared = nn.Linear(128, 64)
        self.heads = nn.ModuleDict({
            'classification': nn.Linear(64, 10),
            'regression': nn.Linear(64, 1)
        })
    
    def forward(self, x, task):
        x = self.shared(x)
        return self.heads[task](x)

Key Takeaway: Use nn.ModuleList for variable-length layer sequences and nn.ModuleDict for named layer collections—both ensure PyTorch properly registers all parameters for training.