Course contentsShow
← Machine Learning and Deep Learning
Lesson 1026 of 3,53823. Recurrent Neural NetworksFree lesson

Encoding Variable-Length Sequences

How the encoder RNN processes input sequences of different lengths into fixed-size context vectors.

Encoding Variable-Length Sequences

What you'll learn: How encoder RNNs convert input sequences of any length into a fixed-size context vector that captures the essential information.

The Challenge of Variable Lengths

In real-world applications, input sequences rarely come in uniform sizes. Sentences have different word counts, time series vary in duration, and video clips differ in length. Yet machine learning models often need a consistent representation to work with downstream tasks.

How Encoders Handle This

The encoder RNN (whether vanilla RNN, LSTM, or GRU) processes input sequences step-by-step, regardless of their length. As it reads each element:

  1. It updates its hidden state at every timestep
  2. The hidden state accumulates information from all previous inputs
  3. After processing the final element, the last hidden state serves as a fixed-size context vector

Think of it like reading a book summary: whether the book is 200 or 800 pages, your brain distills it into a fixed mental representation that captures the core meaning.

Why This Works

The encoder's final hidden state acts as a compressed representation because:

  • LSTMs and GRUs use gating mechanisms to selectively remember important information and forget irrelevant details
  • The cell state (in LSTMs) carries long-term dependencies through the entire sequence
  • The hidden state dimensions remain constant (e.g., 256 or 512 units), regardless of input length

For example, encoding the sentence "Hello" (1 word) and "The quick brown fox jumps" (5 words) both produce hidden states of the same shape—perhaps [512]—ready for the decoder or classifier.

Key Takeaway: Encoder RNNs transform variable-length input sequences into fixed-size context vectors by processing inputs sequentially and using the final hidden state as a compressed representation of the entire sequence.