The Linearity Hypothesis
You'll learn why neural networks' surprising linear behavior in high dimensions makes them vulnerable to adversarial attacks.
The Core Idea
Ian Goodfellow proposed the linearity hypothesis to explain a paradox: deep neural networks use nonlinear activations (ReLU, tanh, etc.), yet adversarial examples exploit their locally linear behavior.
In high-dimensional spaces, even tiny perturbations can accumulate. Consider a linear classifier with weights w and input x. The prediction is wᵀx. If you add a small perturbation δ to every input dimension, the change in output is wᵀδ.
The key insight: Even if each element of δ is tiny (say, ε = 0.001), if you have 1000 dimensions, the total effect can be 1000 × 0.001 = 1. The perturbation compounds across dimensions!
Why This Matters for Deep Networks
Deep networks, despite their nonlinear activations, often behave piecewise linearly in local regions around data points. ReLU networks are especially prone to this—they're literally linear within each activation region.
This means:
- Small, carefully chosen perturbations can accumulate across thousands or millions of dimensions
- The network's output shifts substantially even though the input looks unchanged to humans
- Adversarial examples aren't "bugs" in the model—they're exploiting the fundamental linear geometry
Think of it like walking on a very slight incline: one step barely changes your height, but 10,000 steps in the same direction takes you up a mountain.
Key Takeaway: The linearity hypothesis explains that adversarial vulnerability stems from networks' high-dimensional linear behavior—small perturbations accumulate across many dimensions, causing large output changes despite imperceptible input changes.