Course contentsShow
← Machine Learning and Deep Learning
Lesson 3273 of 3,53871. Interpretability: Neural Network MethodsFree lesson

Attention Head Analysis in Transformers

Investigating what different attention heads learn, from simple copying to complex syntactic and semantic patterns.

Attention Head Analysis in Transformers

You'll learn how researchers investigate what individual attention heads in transformers actually learn, from simple token copying to sophisticated linguistic patterns.

The Core Idea

Transformers use multi-head attention, where each head can theoretically learn different patterns. But what do they learn? Attention head analysis systematically examines individual heads across layers to understand their specialized roles. Think of it like discovering that different members of a team have naturally specialized: one always takes notes (copying), another tracks who's speaking to whom (syntax), and another understands context (semantics).

What Different Heads Learn

Researchers have discovered distinct behavior patterns:

Simple patterns: Some heads perform nearly direct copying—attending strongly to the previous token or a specific positional offset. These act like "induction heads" that complete repetitive patterns.

Syntactic patterns: Certain heads track grammatical relationships, like subject-verb agreement or dependency parsing. A head might consistently connect verbs to their subjects, even across long distances.

Semantic patterns: More sophisticated heads capture meaning-based relationships, attending to semantically related words regardless of position or syntax (e.g., linking "king" to "crown").

Analyzing Heads in Practice

You examine attention weight matrices for specific heads across many examples, looking for consistent patterns. A head that always attends token i to token i-1 shows positional copying. One that connects question words to their answers demonstrates semantic understanding.

Layer depth matters: early layers often learn positional and syntactic patterns, while deeper layers capture more abstract semantic relationships.

Key Takeaway: Individual attention heads in transformers specialize in distinct computational patterns ranging from simple token copying to complex syntactic and semantic relationships, revealing the hierarchical nature of learned representations.