Course contentsShow
← Machine Learning and Deep Learning
Lesson 1021 of 3,53823. Recurrent Neural NetworksFree lesson

GRU Reset and Update Gates

How the reset gate controls access to previous hidden state and the update gate balances old versus new information.

GRU Reset and Update Gates

What you'll learn: How GRU gates control which information flows from the past and how much new information to accept.

The Two-Gate Design

While LSTMs use three gates and a separate cell state, GRUs simplify things with just two gates: the reset gate and the update gate. These two mechanisms work together to manage memory without the complexity of LSTM's triple-gate system.

The Reset Gate: Forgetting the Past

The reset gate determines how much of the previous hidden state should be "forgotten" when computing the new candidate hidden state. Think of it like deciding whether to start fresh or build on your recent memory.

  • When the reset gate outputs values close to 0, it effectively ignores most of the previous hidden state
  • When it outputs values close to 1, it fully incorporates the past information
  • This lets the GRU drop irrelevant context when encountering something new

Analogy: Like selectively forgetting irrelevant details from a previous conversation when a new topic begins.

The Update Gate: Balancing Old vs New

The update gate decides the balance between keeping the old hidden state versus accepting new candidate information. It's a single gate that performs what would require both LSTM's forget and input gates.

  • Values close to 0 → mostly accept new information (forget the old)
  • Values close to 1 → mostly keep old information (ignore the new)
  • It creates a weighted blend: h_new = (1 - z) * h_old + z * h_candidate

Analogy: Like a DJ's crossfader, smoothly blending between the old track (previous state) and the new track (candidate state).

How They Work Together

The reset gate filters the past before computing what's new, while the update gate blends old and new after computing the candidate. This two-step process provides flexible memory control with fewer parameters than LSTM.

Key Takeaway: The reset gate controls access to previous hidden state when forming new memories, while the update gate balances how much old versus new information to retain in the final hidden state.