SmoothGrad: Averaging Noisy Gradients
What You'll Learn
You'll learn how SmoothGrad reduces noise in gradient-based saliency maps by averaging gradients computed on multiple noisy versions of the same input.
The Core Problem with Raw Gradients
As you learned in the previous lesson, raw gradients are inherently noisy—they produce saliency maps with scattered, visually distracting artifacts that make interpretation difficult. The gradient at a single point is sensitive to local variations in the neural network's decision boundary.
How SmoothGrad Works
SmoothGrad takes a beautifully simple approach: add small random noise to your input image multiple times, compute the gradient for each noisy version, then average all those gradients together.
Think of it like taking multiple slightly blurry photographs of the same scene and averaging them—the true signal remains while random noise cancels out.
The Algorithm
- Start with your original input x
- Generate n noisy samples: x + noise₁, x + noise₂, ..., x + noiseₙ
- Compute the gradient (saliency) for each noisy sample
- Average all n gradients to get the final SmoothGrad saliency map
The noise is typically Gaussian with small standard deviation (e.g., σ = 0.15 × (max - min) of pixel values). Common values for n range from 20 to 100 samples.
Why It Works
The random perturbations explore slightly different regions of the input space around your original point. Noise-driven artifacts tend to vary randomly across samples and average to near-zero, while genuine signal—the features the model truly relies on—remains consistent and reinforces through averaging.
Key Takeaway: SmoothGrad transforms noisy vanilla gradients into cleaner, more interpretable saliency maps by averaging gradients computed on multiple random perturbations of the input, effectively separating signal from noise.