Course contentsShow
← Machine Learning and Deep Learning
Lesson 3242 of 3,53871. Interpretability: Neural Network MethodsFree lesson

Evaluating Saliency Map Quality

Learning metrics and techniques to assess whether saliency maps truly reflect important features: deletion/insertion curves and sanity checks.

Evaluating Saliency Map Quality

You'll learn how to determine whether a saliency map actually highlights the features your model truly relies on, using quantitative metrics and sanity checks.

The Challenge of Trust

You've generated beautiful saliency maps that highlight parts of your input image or data. But how do you know they're meaningful rather than just visually appealing noise? After all, a saliency map could look plausible while being completely disconnected from what the model actually uses for prediction.

Deletion and Insertion Curves

Deletion curves measure how quickly model performance drops as you progressively remove the most important pixels (according to the saliency map). If the saliency map is accurate, deleting highlighted regions first should cause rapid performance degradation.

Insertion curves work inversely: start with a blank image and progressively add back pixels in order of their saliency scores. A good saliency map should restore performance quickly by adding the most critical features first.

These curves give you quantitative evidence—not just visual intuition—about whether your explanation method identifies truly important features.

Sanity Checks

These are simple tests that saliency methods should pass but sometimes fail:

  • Model parameter randomization: Does the saliency map change if you randomize the trained weights? It should! If it looks the same, the method might just be responding to input patterns, not what the model learned.
  • Data randomization: Train on random labels. Good saliency methods should produce less coherent maps for poorly trained models.

Think of sanity checks as "smoke tests" for explanations—they catch methods that appear to work but are fundamentally broken.

Key Takeaway: Always validate saliency maps using deletion/insertion curves to measure true feature importance and sanity checks to ensure the method responds to actual model behavior, not just input structure.