Course contentsShow
← AI Engineering
Lesson 58 of 1,8862. Working with Pre-trained ModelsFree lesson

Working with Different Model Types

Distinguish between encoder-only (BERT), decoder-only (GPT), and encoder-decoder (T5) architectures.

Working with Different Model Types

What you'll learn: How to recognize and choose between the three major transformer architectures based on what they're designed to do.

The Three Core Architectures

Not all transformer models are built the same way. Think of them like different tools in a toolbox—each optimized for specific jobs.

Encoder-Only (BERT)

These models read and understand text. They process your entire input at once, looking in all directions (bidirectional). Perfect for:

  • Classification (sentiment, topic)
  • Named entity recognition
  • Question answering (when you have context)

Analogy: Like a careful reader who examines a complete document before answering questions about it.

Decoder-Only (GPT)

These models generate text. They read left-to-right only (unidirectional), predicting what comes next. Perfect for:

  • Text generation
  • Completing prompts
  • Creative writing tasks

Analogy: Like an author writing a story, only able to see what they've already written, not what comes later.

Encoder-Decoder (T5)

These models transform text from one form to another. They have both components: the encoder understands your input, the decoder produces new output. Perfect for:

  • Translation
  • Summarization
  • Text-to-text tasks

Analogy: Like a translator who first fully comprehends a sentence in one language (encoder) before writing it out in another (decoder).

Why This Matters

When you load models using the AutoClasses you learned earlier, understanding the architecture helps you pick the right tool. A BERT model won't generate creative text well, and GPT isn't optimized for classification tasks—even though technically you could force them to try.

Key Takeaway: Match the architecture to your task—encoders for understanding, decoders for generation, encoder-decoders for transformation.