Course contentsShow
← Machine Learning and Deep Learning
Lesson 1434 of 3,53832. Generative Models: Autoencoders and VAEsFree lesson

Implementing a Basic Autoencoder in PyTorch

Building a simple fully-connected autoencoder for MNIST, defining encoder and decoder as sequential modules.

Implementing a Basic Autoencoder in PyTorch

You'll learn how to code a simple fully-connected autoencoder for MNIST images using PyTorch's sequential modules.

Building the Architecture

Now that you understand autoencoder concepts, let's build one! A basic autoencoder for MNIST (28×28 grayscale images) consists of two parts: an encoder that compresses 784 pixels down to a smaller latent representation, and a decoder that reconstructs the original image.

The Encoder Module

The encoder takes the flattened image and progressively reduces dimensionality through fully-connected layers:

import torch.nn as nn

encoder = nn.Sequential(
    nn.Flatten(),              # 28x28 → 784
    nn.Linear(784, 256),       # First compression
    nn.ReLU(),
    nn.Linear(256, 64),        # Second compression
    nn.ReLU(),
    nn.Linear(64, 16)          # Bottleneck: latent space
)

The Decoder Module

The decoder mirrors the encoder, expanding from the bottleneck back to the original dimensions:

decoder = nn.Sequential(
    nn.Linear(16, 64),         # Expand from bottleneck
    nn.ReLU(),
    nn.Linear(64, 256),        # Continue expanding
    nn.ReLU(),
    nn.Linear(256, 784),       # Back to original size
    nn.Sigmoid(),              # Output pixel values [0,1]
    nn.Unflatten(1, (28, 28))  # Reshape to image
)

Combining Into an Autoencoder

class Autoencoder(nn.Module):
    def __init__(self):
        super().__init__()
        self.encoder = encoder
        self.decoder = decoder
    
    def forward(self, x):
        latent = self.encoder(x)
        reconstructed = self.decoder(latent)
        return reconstructed

Think of this like a funnel and reverse funnel: information flows down through progressively smaller layers, then expands back out symmetrically.

Key Takeaway: A basic PyTorch autoencoder uses sequential fully-connected layers in the encoder to compress data into a bottleneck, then mirrors that structure in the decoder to reconstruct the original input.