Course contentsShow
← Machine Learning and Deep Learning
Lesson 1025 of 3,53823. Recurrent Neural NetworksFree lesson

Encoder-Decoder Architecture Fundamentals

Understanding the two-component structure that maps variable-length input sequences to variable-length output sequences.

Encoder-Decoder Architecture Fundamentals

What you'll learn: How the encoder-decoder structure enables RNNs to transform variable-length input sequences into variable-length output sequences.

The Two-Component Structure

The encoder-decoder architecture is a fundamental design pattern that solves a key challenge: how do we map an input sequence of one length to an output sequence of a potentially different length? This is the backbone of tasks like machine translation (English sentence → French sentence) or text summarization (long article → short summary).

How It Works

The architecture consists of two main components:

  1. Encoder: An RNN (typically LSTM or GRU) that processes the entire input sequence step-by-step and compresses all the information into a fixed-size context vector (also called a "thought vector"). This context vector is usually the final hidden state of the encoder RNN.

  2. Decoder: Another RNN that takes the context vector and generates the output sequence one element at a time. The decoder is initialized with the encoder's context vector and produces outputs sequentially until a stop signal is reached.

The Translation Analogy

Think of it like a translator who first reads an entire English paragraph (encoding it into understanding), then writes out the French translation word-by-word based on that understanding (decoding). The translator doesn't translate word-by-word simultaneously—they first comprehend the full meaning, then produce the translation.

Key Characteristics

  • Variable-length handling: Input can be 5 words, output can be 8 words
  • Information bottleneck: All input information must flow through the context vector
  • Sequential generation: Decoder produces outputs one step at a time
  • Many-to-many architecture: Combines the encoder (many-to-one) with decoder (one-to-many)

Key Takeaway: The encoder-decoder architecture uses two RNNs—one to compress a variable-length input into a fixed context vector, and another to expand that context into a variable-length output sequence.