Implementing LSTM in PyTorch
You'll learn how to build LSTM layers in PyTorch, handle their unique input shapes, and manage the hidden and cell states that give LSTMs their memory.
The Core Idea
PyTorch provides nn.LSTM as a ready-to-use module that encapsulates all the gates and equations you've learned. Unlike simple layers, LSTMs require you to manage two states: the hidden state (h) and the cell state (c). Understanding input shapes is crucial because LSTMs expect data in a specific sequence format.
Building an LSTM Layer
import torch
import torch.nn as nn
# Create an LSTM layer
# input_size: number of features per timestep
# hidden_size: number of LSTM units
# num_layers: how many LSTM layers to stack
lstm = nn.LSTM(input_size=10, hidden_size=20, num_layers=2, batch_first=True)
Setting batch_first=True means your input shape is (batch_size, sequence_length, input_size) — more intuitive than the default.
Managing States
# Input: (batch_size=3, sequence_length=5, input_size=10)
x = torch.randn(3, 5, 10)
# Initialize hidden and cell states (num_layers, batch_size, hidden_size)
h0 = torch.zeros(2, 3, 20)
c0 = torch.zeros(2, 3, 20)
# Forward pass
output, (hn, cn) = lstm(x, (h0, c0))
# output: (3, 5, 20) — hidden state at each timestep
# hn: (2, 3, 20) — final hidden state for each layer
# cn: (2, 3, 20) — final cell state for each layer
If you don't provide initial states, PyTorch initializes them to zeros automatically. The returned (hn, cn) can be fed into the next sequence for continuity.
Key Takeaway: PyTorch's nn.LSTM handles all gate operations internally; you focus on input shapes (batch, sequence, features) and propagating the hidden/cell state tuple (h, c) between sequences when needed.