Implementing a Basic Autoencoder in PyTorch
You'll learn how to code a simple fully-connected autoencoder for MNIST images using PyTorch's sequential modules.
Building the Architecture
Now that you understand autoencoder concepts, let's build one! A basic autoencoder for MNIST (28×28 grayscale images) consists of two parts: an encoder that compresses 784 pixels down to a smaller latent representation, and a decoder that reconstructs the original image.
The Encoder Module
The encoder takes the flattened image and progressively reduces dimensionality through fully-connected layers:
import torch.nn as nn
encoder = nn.Sequential(
nn.Flatten(), # 28x28 → 784
nn.Linear(784, 256), # First compression
nn.ReLU(),
nn.Linear(256, 64), # Second compression
nn.ReLU(),
nn.Linear(64, 16) # Bottleneck: latent space
)
The Decoder Module
The decoder mirrors the encoder, expanding from the bottleneck back to the original dimensions:
decoder = nn.Sequential(
nn.Linear(16, 64), # Expand from bottleneck
nn.ReLU(),
nn.Linear(64, 256), # Continue expanding
nn.ReLU(),
nn.Linear(256, 784), # Back to original size
nn.Sigmoid(), # Output pixel values [0,1]
nn.Unflatten(1, (28, 28)) # Reshape to image
)
Combining Into an Autoencoder
class Autoencoder(nn.Module):
def __init__(self):
super().__init__()
self.encoder = encoder
self.decoder = decoder
def forward(self, x):
latent = self.encoder(x)
reconstructed = self.decoder(latent)
return reconstructed
Think of this like a funnel and reverse funnel: information flows down through progressively smaller layers, then expands back out symmetrically.
Key Takeaway: A basic PyTorch autoencoder uses sequential fully-connected layers in the encoder to compress data into a bottleneck, then mirrors that structure in the decoder to reconstruct the original input.