Cross-Attention Mechanism
What You'll Learn
You'll learn how the decoder attends to encoder outputs using queries from the decoder and keys/values from the encoder.
The Core Idea
In an encoder-decoder Transformer, cross-attention is where the magic of connecting input and output sequences happens. Unlike self-attention (where a sequence attends to itself), cross-attention allows the decoder to "look at" the encoder's outputs while generating each token.
Here's how it works:
The Three Vectors Come From Different Places
In cross-attention:
- Queries (Q): Come from the decoder's current hidden states (what we're trying to generate)
- Keys (K): Come from the encoder's outputs (the input we're translating/processing from)
- Values (V): Also come from the encoder's outputs
An Analogy
Think of translating English to French:
- The decoder (currently writing French) asks queries like "What part of the English sentence is relevant now?"
- The encoder provides keys (pointers to English words) and values (their meanings)
- The decoder matches its query against all keys to find which English words to focus on
- It then takes a weighted sum of those values to inform the current French word
The Attention Flow
- Decoder generates queries from its current state
- These queries compute attention scores with encoder keys
- Softmax normalizes these scores into weights
- Weighted sum of encoder values creates the context
- This context helps the decoder generate the next token
Key Takeaway: Cross-attention bridges encoder and decoder by letting decoder queries attend to encoder keys and values, enabling the model to selectively focus on relevant input information when generating each output token.