Visualizing Self-Attention in Transformers
What you'll learn: How to extract and display attention patterns from transformer layers to see which tokens the model focuses on when processing each word.
The Core Idea
In the previous lesson, you learned that attention weights can serve as interpretability tools. Now we take that further: transformers compute self-attention matrices that show how strongly each token attends to every other token in the sequence. By extracting and visualizing these matrices, you can literally see which words the model considers relevant when processing each position.
Think of it like highlighting a sentence: when you read the word "it," your brain automatically looks back to find what "it" refers to. Self-attention does the same thing computationally, and we can visualize those connections.
How It Works
Each transformer layer produces attention weights—typically a matrix of shape [num_heads, sequence_length, sequence_length]. For a given query token at position i, the values in row i show how much attention it pays to all other positions (including itself).
Common visualization approaches:
- Heatmaps: Display the attention matrix as a color-coded grid where darker colors = stronger attention
- Attention graphs: Draw arrows between tokens weighted by attention strength
- Head-specific views: Plot each attention head separately to see different learned patterns (some heads track syntax, others semantics)
Multi-head attention means you get multiple attention patterns per layer, revealing that different heads specialize in different linguistic relationships (e.g., one head might track subject-verb agreement, another might link pronouns to antecedents).
Why It Matters
These visualizations help you debug model behavior, understand failure modes, and verify whether the model is using sensible reasoning paths—especially valuable for diagnosing issues in language models or domain-specific transformers.
Key Takeaway: Visualizing self-attention matrices reveals token-to-token relationships learned by transformers, offering direct insight into which context each position considers most relevant during processing.