Implementing Gradient-Based Saliency in PyTorch
You'll learn how to write PyTorch code that computes and visualizes which input pixels most influence a neural network's classification decision using gradient information.
The Core Idea
Gradient-based saliency reveals what your model "looks at" by computing how much each input pixel affects the output. When you calculate the gradient of the predicted class score with respect to the input image, you get a map showing which pixels, if changed slightly, would most alter the prediction. Large gradient magnitudes highlight important regions.
Think of it like adjusting a photograph's brightness: gradients tell you which pixels, when brightened or darkened, would most change whether the model says "cat" versus "dog."
Implementation Steps
Using PyTorch's autograd system, you:
- Load your image and ensure it requires gradients:
image.requires_grad = True - Forward pass through your model to get predictions
- Select the target class score (usually the predicted class)
- Compute gradients using
.backward()on that score - Extract gradients from
image.grad— these are your saliency values - Visualize by taking absolute values or using other aggregation methods
# Example structure (conceptual)
image.requires_grad = True
output = model(image)
target_score = output[0, predicted_class]
target_score.backward()
saliency = image.grad.abs()
The resulting saliency tensor has the same shape as your input, with high values indicating pixels that strongly influence the classification.
Visualization
You typically convert the multi-channel gradient to a single heatmap (using max across color channels) and overlay it on the original image to see which regions matter most.
Key Takeaway: PyTorch's autograd makes gradient-based saliency straightforward—enable gradients on inputs, run a forward-backward pass, and extract image.grad to reveal pixel-level importance for any prediction.