Course contentsShow
← Machine Learning and Deep Learning
Lesson 1605 of 3,53835. Modern Large Language Models: ArchitectureFree lesson

Why Decoder-Only: From Encoder-Decoder to GPT

Understand the architectural shift from BERT/T5 to decoder-only models and why autoregressive generation favors this design.

Why Decoder-Only: From Encoder-Decoder to GPT

What you'll learn: Why modern large language models like GPT abandoned the encoder-decoder architecture in favor of a simpler decoder-only design.

The Architectural Evolution

Early transformer models like BERT used encoder-only architectures (for understanding text), while T5 used encoder-decoder architectures (encoding input, then decoding output). These worked well for specific tasks, but autoregressive language generation revealed a key insight: you don't always need both components.

Why Decoder-Only Wins for Generation

Think of it like writing a story. An encoder-decoder model is like having one person read and understand a prompt (encoder), then hand notes to another person who writes the continuation (decoder). A decoder-only model is like one person who reads and writes simultaneously, treating everything—prompt and response—as a single continuous sequence.

Key Advantages

  1. Simplicity: One unified component instead of two separate stacks
  2. Autoregressive by nature: Decoders naturally predict the next token given previous tokens—perfect for text generation
  3. Training efficiency: The entire sequence (input + output) becomes training data, with each position learning to predict the next
  4. Scaling: Simpler architecture scales more predictably to billions of parameters

The Self-Attention Insight

Decoder-only models use causal masking in self-attention—each token can only attend to previous tokens. This means the prompt tokens naturally inform generation tokens without needing a separate encoder to "process" the input first.

Key Takeaway: Decoder-only architectures like GPT emerged as the dominant design for large language models because autoregressive generation doesn't require separate encoding—a single causal decoder handles both understanding the prompt and generating responses efficiently.