Course contentsShow
← Machine Learning and Deep Learning
Lesson 1317 of 3,53829. Core NLP TasksFree lesson

Machine Translation with Transformers

Understand how encoder-decoder transformers perform neural machine translation and the role of parallel corpora in training.

Machine Translation with Transformers

What you'll learn: How transformer encoder-decoder models translate text between languages using parallel training data.

The Core Idea

Machine Translation (MT) is the task of automatically converting text from one language (the source) to another (the target). With transformers, this becomes a sequence-to-sequence problem: the encoder processes the source sentence into contextual representations, and the decoder generates the translation token-by-token while attending to both the encoded source and previously generated target tokens.

Think of it like a bilingual interpreter who first listens carefully to understand the full meaning in one language (encoder), then speaks it fluently in another language (decoder), constantly referring back to what was originally said to stay accurate.

Parallel Corpora: The Foundation

Training a translation model requires parallel corpora—datasets containing sentences paired across languages (e.g., English-French sentence pairs). The model learns alignment patterns: which source words/phrases correspond to which target words/phrases, along with grammatical transformations needed between languages.

During training, the encoder sees the source sentence, and the decoder is trained to predict the target sentence using teacher forcing—it sees the correct previous target tokens and learns to predict the next one.

How It Works

  1. Encode: Source sentence → Encoder → Rich contextual representations
  2. Decode: Decoder generates target tokens autoregressively, using cross-attention to the encoder's output
  3. Cross-Attention: This is the magic—the decoder "looks at" relevant parts of the source while generating each target word

The same T5 or BART models you've seen can be fine-tuned for translation by training them on parallel sentence pairs.

Key Takeaway: Transformer encoder-decoder models perform neural machine translation by encoding source sentences and decoding target translations with cross-attention, trained on parallel corpora containing aligned sentence pairs across languages.