Course contentsShow
← Machine Learning and Deep Learning
Lesson 3245 of 3,53871. Interpretability: Neural Network MethodsFree lesson

The Two Fundamental Axioms

Sensitivity and Implementation Invariance: what any attribution method should satisfy and why.

The Two Fundamental Axioms

You'll learn the two essential properties that any attribution method must satisfy to be considered valid and trustworthy.

What Makes an Attribution Method "Good"?

After exploring gradient-based saliency maps and path-based attribution intuition, we need formal criteria to judge whether an attribution method is doing its job correctly. Think of these axioms as a "quality seal" — like how a thermometer must give the same reading regardless of whether it displays Celsius or Fahrenheit (just different scales).

The two fundamental axioms are:

1. Sensitivity (a.k.a. Completeness)

If a feature's change causes the model's output to change, the attribution method MUST assign non-zero importance to that feature.

Imagine you're explaining why a spam filter flagged an email. If changing the word "FREE" to "free" causes the prediction to flip from spam to not-spam, then "FREE" must receive some attribution. A method violating this axiom is fundamentally broken — it's ignoring features that actually matter.

2. Implementation Invariance

Two functionally identical networks must produce identical attributions, regardless of internal architecture.

Suppose you have two networks that always produce the exact same outputs for any input, but one uses ReLU activations and the other uses different layer structures. A valid attribution method shouldn't care about these internal implementation details — the explanations should be identical because the function being explained is identical.

This prevents attributions from being artifacts of arbitrary architectural choices rather than true reflections of feature importance.

Key Takeaway: Sensitivity ensures attributions reflect actual model behavior changes, while Implementation Invariance ensures attributions explain the function learned, not the specific neural network implementation. Any attribution method worth using must satisfy both axioms.