Guided Backpropagation
What you'll learn: How to modify the backpropagation process to produce cleaner, more interpretable saliency maps by filtering out negative gradients at ReLU layers.
The Core Idea
When you compute saliency maps using vanilla gradients (lesson 3232), you often get noisy, hard-to-interpret results. Guided Backpropagation addresses this by changing how gradients flow backward through ReLU activation functions.
Standard backpropagation through ReLU: During forward pass, ReLU blocks negative values. During backward pass, gradients flow through only where the forward activation was positive.
Guided backpropagation: Goes one step further—it also blocks negative gradients during the backward pass, even if the forward activation was positive. Only positive gradients are allowed to flow backward.
Why This Works
Think of it like a noise filter for your gradient signal. Negative gradients often correspond to features that decrease the class score, which can clutter the visualization. By keeping only positive gradients, you highlight the pixels that most strongly support the prediction, creating sharper, more focused saliency maps.
The intuition: we want to see which input features the network is "looking for" to increase the target class score. Negative gradients represent suppressions rather than activations, so filtering them out reveals the positive evidence the network uses.
Practical Considerations
Guided backpropagation modifies the ReLU gradient computation during a special backward pass used only for visualization—your trained model remains unchanged. The resulting saliency maps typically show much clearer object boundaries and fine-grained details compared to vanilla gradients or Gradient × Input methods.
Key Takeaway: Guided Backpropagation creates sharper, more interpretable saliency maps by passing only positive gradients through ReLU layers during backpropagation, filtering out negative gradient noise that obscures the visualization.