Common Layer Types: Linear, Conv2d, and More
What you'll learn: You'll discover PyTorch's pre-built layer types that serve as the building blocks for standard neural network architectures.
The Core Idea
Instead of manually defining weights and biases for every layer in your custom nn.Module, PyTorch provides ready-made layer classes that handle parameter initialization and forward computation automatically. These are the Lego bricks of deep learning.
Essential Built-In Layers
nn.Linear — The fully connected layer. Applies a linear transformation: y = xW^T + b. Perfect for standard feedforward networks and the final classification layer.
import torch.nn as nn
linear = nn.Linear(in_features=128, out_features=10)
# Automatically creates weight matrix (10, 128) and bias vector (10)
nn.Conv2d — The 2D convolutional layer. Slides filters across images to detect spatial patterns. Essential for computer vision tasks.
conv = nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, padding=1)
# Creates 64 filters, each 3x3, operating on 3-channel input (RGB)
nn.BatchNorm2d — Batch normalization for 2D data. Normalizes activations to stabilize and accelerate training.
bn = nn.BatchNorm2d(num_features=64)
# Normalizes 64-channel feature maps
Why Use Built-In Layers?
These layers automatically register their parameters (weights, biases) with your module, so optimizers can find and update them. They also handle proper initialization using proven schemes—no more guessing appropriate weight ranges.
Think of them as appliances in your kitchen: you could build a microwave from scratch, but using a pre-built one is more reliable and saves tremendous effort.
Key Takeaway: PyTorch's built-in layers like nn.Linear, nn.Conv2d, and nn.BatchNorm2d are pre-configured, parameter-managing components that simplify building standard architectures while ensuring proper initialization and gradient flow.