Course contentsShow
← Machine Learning and Deep Learning
Lesson 3235 of 3,53871. Interpretability: Neural Network MethodsFree lesson

SmoothGrad: Averaging Noisy Gradients

Learning how SmoothGrad reduces noise by averaging gradients computed on multiple noisy versions of the same input.

SmoothGrad: Averaging Noisy Gradients

What You'll Learn

You'll learn how SmoothGrad reduces noise in gradient-based saliency maps by averaging gradients computed on multiple noisy versions of the same input.

The Core Problem with Raw Gradients

As you learned in the previous lesson, raw gradients are inherently noisy—they produce saliency maps with scattered, visually distracting artifacts that make interpretation difficult. The gradient at a single point is sensitive to local variations in the neural network's decision boundary.

How SmoothGrad Works

SmoothGrad takes a beautifully simple approach: add small random noise to your input image multiple times, compute the gradient for each noisy version, then average all those gradients together.

Think of it like taking multiple slightly blurry photographs of the same scene and averaging them—the true signal remains while random noise cancels out.

The Algorithm

  1. Start with your original input x
  2. Generate n noisy samples: x + noise₁, x + noise₂, ..., x + noiseₙ
  3. Compute the gradient (saliency) for each noisy sample
  4. Average all n gradients to get the final SmoothGrad saliency map

The noise is typically Gaussian with small standard deviation (e.g., σ = 0.15 × (max - min) of pixel values). Common values for n range from 20 to 100 samples.

Why It Works

The random perturbations explore slightly different regions of the input space around your original point. Noise-driven artifacts tend to vary randomly across samples and average to near-zero, while genuine signal—the features the model truly relies on—remains consistent and reinforces through averaging.

Key Takeaway: SmoothGrad transforms noisy vanilla gradients into cleaner, more interpretable saliency maps by averaging gradients computed on multiple random perturbations of the input, effectively separating signal from noise.