Attention Rollout and Flow
What you'll learn: How to aggregate attention weights across multiple transformer layers to trace the complete path of information from input tokens to final predictions.
The Challenge of Multi-Layer Attention
When you visualized attention in a single layer (as in previous lessons), you saw which tokens attended to which others at that layer. But transformers have many stacked layers—so how does information actually flow from the input all the way through to the output? A single layer's attention doesn't tell you the full story.
What is Attention Rollout?
Attention rollout is a technique that combines attention weights across all layers to create a single attention map showing how input tokens influence the final representation. Think of it like tracing a river from source to sea: water doesn't jump directly there—it flows through tributaries, lakes, and streams. Similarly, information flows through intermediate layers.
The key insight: if token A attends to token B in layer 1, and token B attends to token C in layer 2, then token A indirectly attends to token C. Attention rollout multiplies these attention matrices together (with a residual connection added) to compute this transitive relationship.
The Rollout Algorithm
At each layer, you:
- Take the attention matrix (averaged across heads)
- Add the identity matrix (modeling the residual connection)
- Normalize the result
- Multiply with the previous layer's rolled-out attention
This gives you a final attention map from input → output that respects the full depth of the network.
Key Takeaway: Attention rollout aggregates layer-wise attention matrices through matrix multiplication to reveal end-to-end information flow, helping you understand which input tokens most influenced a specific prediction across the entire network depth.