T5: Text-to-Text Framework Fundamentals
What you'll learn: How T5 unifies all NLP tasks by converting them into a single text-to-text format.
The Core Innovation
T5 (Text-to-Text Transfer Transformer) takes a radically simple approach: every NLP task becomes a text generation problem. Whether you're translating languages, answering questions, classifying sentiment, or summarizing documents, T5 treats them all the same way—it reads text input and generates text output.
The Text-to-Text Philosophy
Instead of designing specialized model heads for different tasks (like you would with BERT), T5 uses task-specific text prefixes to signal what operation to perform:
- Translation: Input:
"translate English to German: Hello"→ Output:"Hallo" - Classification: Input:
"sentiment: This movie was great!"→ Output:"positive" - Summarization: Input:
"summarize: [long article]"→ Output:"[short summary]" - Question Answering: Input:
"question: What is AI? context: [text]"→ Output:"Artificial Intelligence..."
Why This Matters
This unified framework means one model architecture can handle everything. You don't need separate classification heads, span-prediction layers, or task-specific modifications. T5 uses an encoder-decoder architecture (unlike GPT's decoder-only design), which gives it bidirectional context from the encoder while maintaining the flexibility of autoregressive generation from the decoder.
Think of T5 as a universal translator between text formats—it learned during pretraining that the prefix tells it how to transform the input text.
Connection to What You Know
You've seen encoder-only (BERT) and decoder-only (GPT) models. T5 brings back the full encoder-decoder Transformer, but with a twist: instead of task-specific outputs, everything is tokenized text generation.
Key Takeaway: T5 unifies all NLP tasks into text-to-text transformations, using task prefixes to guide a single encoder-decoder model across translation, classification, summarization, and beyond.