Polysemantic Neurons and Superposition
What you'll learn: How neural networks represent more features than they have neurons by using superposition, making interpretability significantly more challenging.
The Overcrowding Problem
In the previous lesson, we analyzed individual neurons to understand what features they detect. But here's a surprising puzzle: neural networks often need to represent more features than they have neurons. How is this possible?
The answer is superposition — neurons don't represent just one feature each. Instead, multiple features are packed into the same set of neurons, like trying to fit ten items into five boxes by carefully stacking them.
Polysemantic Neurons
A polysemantic neuron activates for multiple, seemingly unrelated features. For example, a single neuron might fire for:
- Cat ears
- Car wheels
- Circular patterns in text
This isn't a bug — it's an efficient compression strategy the network learns during training.
Why Superposition Happens
Networks use superposition because:
- High-dimensional spaces allow sparse encoding — features can be nearly orthogonal even when there are more features than dimensions
- Not all features appear simultaneously — if "cat ears" and "car wheels" rarely occur in the same input, the network can reuse representational capacity
- Sparsity enables packing — when most features are inactive most of the time, interference between features is manageable
Think of it like a shared parking lot: if people work different shifts, you need fewer parking spaces than total employees.
The Interpretability Challenge
Superposition makes mechanistic interpretability extremely difficult. When you identify that a neuron responds to feature X, it might also respond to features Y and Z in other contexts. Simple neuron-by-neuron analysis breaks down.
Key Takeaway: Neural networks use superposition to represent more features than neurons by encoding multiple features in overlapping ways, creating polysemantic neurons that respond to different features in different contexts — a major obstacle for interpretability research.