Course contentsShow
← Machine Learning and Deep Learning
Lesson 1000 of 3,53823. Recurrent Neural NetworksFree lesson

The Basic RNN Cell

Understanding the recurrent computation: hidden state update, weight sharing across time, and the feedback loop.

The Basic RNN Cell

What you'll learn: How a single RNN cell processes sequential data by maintaining and updating a hidden state over time.

The Core Idea

Unlike feedforward networks that process each input independently, a Recurrent Neural Network (RNN) cell has memory. It takes two inputs at each time step: the current data point and a hidden state from the previous step. This hidden state acts like the network's "memory" of what it has seen so far.

Think of reading a sentence word-by-word. When you read "The cat sat on the...", your brain doesn't process "the" in isolation—it remembers the previous words to understand context. An RNN cell works similarly: it combines new information with its memory of the past.

The Recurrent Computation

At each time step t, the RNN cell performs this update:

h_t = f(W_h · h_{t-1} + W_x · x_t + b)

Where:

  • h_t is the new hidden state (the updated memory)
  • h_{t-1} is the previous hidden state
  • x_t is the current input
  • W_h and W_x are weight matrices
  • f is an activation function (typically tanh)

Weight Sharing Across Time

Crucially, the same weights (W_h and W_x) are used at every time step. This is called weight sharing—the RNN doesn't learn different rules for position 1 vs position 10. It learns one set of rules that apply throughout the sequence.

The Feedback Loop

The magic happens because h_t becomes the input for the next step. This creates a feedback loop: output feeds back as input, allowing information to persist and influence future predictions.

Key Takeaway: An RNN cell maintains a hidden state that gets updated at each time step using the same shared weights, creating a feedback loop that allows the network to remember and use information from earlier in the sequence.