Course contentsShow
← Machine Learning and Deep Learning
Lesson 1216 of 3,53827. Pretrained Language Models: GPT Family and BeyondFree lesson

T5: Text-to-Text Framework Fundamentals

Understand T5's unified text-to-text approach where all NLP tasks are cast as text generation problems.

T5: Text-to-Text Framework Fundamentals

What you'll learn: How T5 unifies all NLP tasks by converting them into a single text-to-text format.

The Core Innovation

T5 (Text-to-Text Transfer Transformer) takes a radically simple approach: every NLP task becomes a text generation problem. Whether you're translating languages, answering questions, classifying sentiment, or summarizing documents, T5 treats them all the same way—it reads text input and generates text output.

The Text-to-Text Philosophy

Instead of designing specialized model heads for different tasks (like you would with BERT), T5 uses task-specific text prefixes to signal what operation to perform:

  • Translation: Input: "translate English to German: Hello" → Output: "Hallo"
  • Classification: Input: "sentiment: This movie was great!" → Output: "positive"
  • Summarization: Input: "summarize: [long article]" → Output: "[short summary]"
  • Question Answering: Input: "question: What is AI? context: [text]" → Output: "Artificial Intelligence..."

Why This Matters

This unified framework means one model architecture can handle everything. You don't need separate classification heads, span-prediction layers, or task-specific modifications. T5 uses an encoder-decoder architecture (unlike GPT's decoder-only design), which gives it bidirectional context from the encoder while maintaining the flexibility of autoregressive generation from the decoder.

Think of T5 as a universal translator between text formats—it learned during pretraining that the prefix tells it how to transform the input text.

Connection to What You Know

You've seen encoder-only (BERT) and decoder-only (GPT) models. T5 brings back the full encoder-decoder Transformer, but with a twist: instead of task-specific outputs, everything is tokenized text generation.

Key Takeaway: T5 unifies all NLP tasks into text-to-text transformations, using task prefixes to guide a single encoder-decoder model across translation, classification, summarization, and beyond.