Course contentsShow
← Machine Learning and Deep Learning
Lesson 3232 of 3,53871. Interpretability: Neural Network MethodsFree lesson

The Vanilla Gradient Method

Computing saliency by taking the gradient of the output with respect to the input, revealing which pixels most affect the prediction when changed.

The Vanilla Gradient Method

You'll learn how to compute saliency maps by calculating gradients of a model's output with respect to its input pixels.

The Core Idea

The Vanilla Gradient method is the simplest approach to creating saliency maps for neural networks. It answers the question: "Which input pixels, if changed slightly, would most affect the model's prediction?"

Here's how it works: You take a trained neural network, feed it an input (like an image), and compute the gradient of the output class score with respect to each input pixel. The gradient tells you how much a tiny change in that pixel would change the model's confidence in its prediction.

Think of it like sensitivity analysis. If you're hiking up a mountain (your prediction score), the gradient tells you which direction is steepest at each point. Pixels with large gradient magnitudes are "steep"—changing them would significantly impact the output.

Mathematical Intuition

For a neural network function f that maps input x to output score, the vanilla gradient is simply:

Saliency(x) = |∂f/∂x|

The absolute value highlights pixels that matter, regardless of whether increasing or decreasing them helps the prediction. Pixels with near-zero gradients are essentially irrelevant to the current prediction.

Practical Application

This method reveals which parts of an image the network "looks at" when making decisions. For example, if classifying a dog image, high-gradient pixels might cluster around the ears, nose, and eyes—features the network relies on.

Key Takeaway: The Vanilla Gradient method computes saliency by taking the gradient of the output with respect to the input, revealing pixel-level sensitivity and providing the foundation for understanding what drives neural network predictions.