Course contentsShow
← Machine Learning and Deep Learning
Lesson 3252 of 3,53871. Interpretability: Neural Network MethodsFree lesson

Sanity Checks and Completeness

Verifying IG correctness: checking that attributions sum to output difference and debugging common issues.

Sanity Checks and Completeness

You'll learn how to verify that your Integrated Gradients attributions are mathematically correct and identify common implementation bugs.

Why Verification Matters

When you compute Integrated Gradients, you're essentially claiming that the sum of all feature attributions explains the difference between your baseline and actual prediction. But implementation bugs are easy to introduce—wrong baseline shape, incorrect integration steps, or improper gradient accumulation can all produce plausible-looking but wrong attributions.

The Completeness Axiom Check

The most important sanity check for IG is completeness: the sum of all attributions should equal the difference between your model's output at the input and at the baseline.

Mathematically:

sum(attributions) ≈ f(x) - f(baseline)

Think of it like balancing a ledger: if you're explaining a prediction change of +0.8, but your attributions sum to +0.5, you've "lost" 0.3 units of explanation somewhere.

Common Issues to Debug

Approximation error: With too few integration steps (small m in Riemann approximation), the sum won't match perfectly. Try increasing steps from 50 to 300+.

Baseline mismatch: If your baseline has the wrong shape or isn't properly broadcast, gradients will be meaningless.

Gradient computation bugs: Forgetting to accumulate gradients properly or using the wrong differentiation target produces invalid attributions.

Numerical precision: For very small or large outputs, floating-point errors accumulate. Check relative error, not just absolute difference.

Practical Verification

Always compute actual_diff = model(input) - model(baseline) and attribution_sum = attributions.sum(), then verify abs(actual_diff - attribution_sum) < tolerance. A tolerance of 0.01–0.05 is typical with sufficient integration steps.

Key Takeaway: Always verify completeness by checking that attribution sums match output differences—this single sanity check catches most IG implementation bugs and ensures your explanations are mathematically grounded.