Course contentsShow
← Machine Learning and Deep Learning
Lesson 1095 of 3,53824. The Transformer ArchitectureFree lesson

The Decoder Stack

Decoder layers with masked self-attention, cross-attention, and feed-forward components.

The Decoder Stack

What you'll learn: How the Transformer decoder combines three key components—masked self-attention, cross-attention, and feed-forward layers—to generate output sequences one token at a time.

The Decoder's Role

While the encoder processes the entire input sequence in parallel (as you learned previously), the decoder generates output sequences autoregressively—one token at a time, using both the encoder's output and previously generated tokens. The decoder stack achieves this through multiple identical layers, each containing three distinct sublayers.

Three Components Per Decoder Layer

Each decoder layer has:

  1. Masked Self-Attention: The decoder attends to previously generated output tokens. The mask (upper triangular) prevents the decoder from "peeking" at future tokens during training—essential for autoregressive generation.

  2. Cross-Attention: This is where encoder and decoder communicate! The decoder's queries attend to the encoder's keys and values, allowing the decoder to focus on relevant parts of the input sequence when generating each output token.

  3. Feed-Forward Network: Just like in the encoder, each position passes through a position-wise feed-forward network independently.

The Flow

Think of it like writing a translation while consulting the original text:

  • Masked self-attention: "What have I written so far?"
  • Cross-attention: "What parts of the source sentence should I focus on now?"
  • Feed-forward: "Process this information to decide the next word"

Each sublayer has residual connections and layer normalization, just like the encoder. Multiple decoder layers stack on top of each other (typically 6-12 layers), with the final layer's output feeding into a linear projection and softmax for token prediction.

Key Takeaway: The decoder stack uses masked self-attention for its own outputs, cross-attention to query the encoder, and feed-forward networks—stacking these layers enables sophisticated sequence generation while attending to input context.