Limitations of Basic Gradient Methods
What You'll Learn
You'll understand why vanilla gradients and saliency maps often fail to provide reliable explanations for neural network predictions.
The Core Problems
Gradient-based attribution methods seem elegant in theory—just follow the derivative to see what matters. But in practice, they suffer from three fundamental weaknesses that undermine their reliability:
1. Saturation
When neurons saturate (reach flat regions of activation functions like sigmoid or tanh), gradients approach zero even though those features may be critically important. A feature that's already maximally activated shows zero gradient—not because it's unimportant, but because you can't push it further. It's like saying your engine doesn't matter because pressing the gas pedal harder when you're already at top speed changes nothing.
2. Noise and Instability
Vanilla gradients are notoriously noisy. Tiny, imperceptible changes to the input can produce wildly different attribution maps. This instability means two nearly identical images might show completely different "important" regions—hardly a trustworthy explanation.
3. Lack of Axiomatic Foundation
Basic gradient methods violate key properties we'd want from good attributions. They don't satisfy sensitivity (if changing a feature changes the output, it should be marked important) or implementation invariance (two functionally identical networks should give the same attributions). Without these axioms, we have no guarantees the explanations are meaningful.
Why This Matters
These limitations explain why methods like SmoothGrad, Integrated Gradients, and SHAP were developed—they attempt to address these core failures through averaging, integration paths, or game-theoretic foundations.
Key Takeaway: Basic gradient methods fail due to saturation hiding important features, noise creating unstable explanations, and absence of principled axioms guaranteeing meaningful attributions—motivating more sophisticated approaches.