Gradient × Input Method
You'll learn: How multiplying gradients by input values creates sharper, more focused saliency maps that highlight features that are both important and actively present.
The Core Problem with Raw Gradients
When you compute vanilla gradients for saliency maps, you're asking: "How much would the output change if I nudged this input feature?" But here's the catch: a feature with a large gradient might not actually contribute much to the current prediction if its input value is near zero. You're measuring potential influence, not actual contribution.
The Gradient × Input Solution
The Gradient × Input method addresses this by elementwise multiplication:
Saliency = ∇f(x) ⊙ x
Where ∇f(x) is the gradient of the output with respect to the input, and x is the input itself.
Why This Works Better
Think of it like a spotlight analogy:
- Gradient alone: Shows where the stage is sensitive to light changes
- Input alone: Shows where light is currently shining
- Gradient × Input: Shows which lit areas actually matter to what the audience sees
By multiplying these together, you suppress features that either:
- Have large gradients but small input values (high sensitivity, low presence)
- Have large input values but small gradients (high presence, low importance)
This creates cleaner, more interpretable saliency maps that focus on features genuinely driving the prediction—features that are both "turned on" in the input and influential to the model's decision.
Key Takeaway: Gradient × Input improves saliency interpretability by filtering for features that are both important (high gradient) and present (non-zero input), revealing the actual contributors to a prediction rather than just potential influencers.