Course contentsShow
← Machine Learning and Deep Learning
Lesson 1028 of 3,53823. Recurrent Neural NetworksFree lesson

Decoder Architecture and Conditional Generation

How the decoder RNN generates output sequences conditioned on the encoder's context vector step-by-step.

Decoder Architecture and Conditional Generation

What you'll learn: How a decoder RNN takes the encoder's context vector and generates an output sequence one step at a time.

The Decoder's Job

You've seen how the encoder compresses an input sequence into a fixed-size context vector. Now the decoder's job is to unpack that information and generate the target sequence. Think of it like a translator who listened to a complete sentence in one language (the encoder) and now must speak it out in another language, word by word.

Step-by-Step Generation

The decoder is itself an RNN (typically an LSTM or GRU) that generates output conditionally — meaning each output depends on:

  1. The context vector from the encoder (usually provided as the initial hidden state)
  2. The previous output token (what it just generated)
  3. Its own hidden state (memory of what it's generated so far)

At each time step, the decoder:

  • Takes the previous word it predicted (or a start token <SOS> initially)
  • Combines it with its hidden state
  • Produces a probability distribution over all possible next words
  • Samples or selects the most likely word
  • Updates its hidden state for the next step
  • Continues until it produces an end token <EOS>

Conditional Generation

This is "conditional" because the entire output sequence is conditioned on the encoder's context. Same decoder architecture, different context vector → different output sequence. For translation, the same decoder network can translate any source sentence because the context vector tells it what to translate.

Key Takeaway: The decoder RNN generates target sequences autoregressively (one token at a time), conditioned on the encoder's context vector, using each previous output to inform the next prediction.