Course contentsShow
← Machine Learning and Deep Learning
Lesson 1018 of 3,53823. Recurrent Neural NetworksFree lesson

LSTM Forward Pass Equations

The complete mathematical formulation of LSTM forward propagation through all four computations: forget, input, cell, and output.

LSTM Forward Pass Equations

What You'll Learn

You'll learn the complete mathematical formulation that ties together all four LSTM computations into a single forward pass through time.

The Complete LSTM Picture

You've learned each gate individually—now let's see how they work together in one unified forward pass. An LSTM cell takes an input x_t and previous hidden state h_{t-1}, then produces a new hidden state h_t and cell state c_t through four coordinated steps.

The Four Equations in Order

Think of these equations as a recipe executed in sequence at each time step:

1. Forget Gate

f_t = σ(W_f · [h_{t-1}, x_t] + b_f)

Decides what to keep from the old cell state.

2. Input Gate and Candidate

i_t = σ(W_i · [h_{t-1}, x_t] + b_i)
c̃_t = tanh(W_c · [h_{t-1}, x_t] + b_c)

Determines what new information to add.

3. Cell State Update

c_t = f_t ⊙ c_{t-1} + i_t ⊙ c̃_t

Combines old memories (filtered by forget) with new candidates (filtered by input).

4. Output Gate and Hidden State

o_t = σ(W_o · [h_{t-1}, x_t] + b_o)
h_t = o_t ⊙ tanh(c_t)

Produces the final output for this time step.

The Flow

These equations execute sequentially: each gate reads the same inputs [h_{t-1}, x_t] but uses different weight matrices. The ⊙ symbol means element-wise multiplication. The σ is the sigmoid function (outputs 0-1 for gating), while tanh outputs -1 to 1 for actual values.

Key Takeaway: The LSTM forward pass combines four parameterized transformations—forget gate, input gate with candidate, cell update, and output gate—to create a memory cell that can learn long-term dependencies while avoiding vanishing gradients.