Course contentsShow
← Machine Learning and Deep Learning
Lesson 1016 of 3,53823. Recurrent Neural NetworksFree lesson

LSTM Input Gate and Candidate Values

How the input gate selects which new information to add, combining with tanh-generated candidate values to update the cell state.

LSTM Input Gate and Candidate Values

What you'll learn: How LSTMs decide what new information to add to memory and combine it with freshly generated candidate values.

The Input Gate's Role

After the forget gate decides what to discard from memory, the LSTM needs to determine what new information to add. This is where the input gate comes in. Think of it like a bouncer at a club—not every piece of new information deserves to enter your long-term memory. The input gate produces values between 0 and 1 that act as filters for new content.

Candidate Values: The Raw Update

Separately, the LSTM creates candidate values using a tanh activation. These candidates represent the potential new information that could be added to the cell state. The tanh produces values between -1 and 1, allowing the network to propose both positive and negative updates to memory.

Think of candidate values as rough draft notes—they're what could be remembered, but haven't been vetted yet.

Combining Gate and Candidates

The magic happens when you multiply the input gate's output with the candidate values. The input gate acts as a selective filter:

  • If the input gate outputs ~1 for a dimension, that candidate value passes through fully
  • If it outputs ~0, that candidate is blocked
  • Values in between allow partial updates

This filtered result is then added to the cell state (after the forget gate has done its job), updating the LSTM's memory with relevant new information.

Analogy

Imagine taking notes during a lecture. The candidate values are everything the professor says. The input gate is your judgment about what's important enough to write down. You only add to your notebook what passes both criteria: said AND deemed important.

Key Takeaway: The input gate selectively filters candidate values (generated by tanh) to determine exactly what new information gets added to the LSTM's cell state memory.