Sanity Checks and Completeness
You'll learn how to verify that your Integrated Gradients attributions are mathematically correct and identify common implementation bugs.
Why Verification Matters
When you compute Integrated Gradients, you're essentially claiming that the sum of all feature attributions explains the difference between your baseline and actual prediction. But implementation bugs are easy to introduce—wrong baseline shape, incorrect integration steps, or improper gradient accumulation can all produce plausible-looking but wrong attributions.
The Completeness Axiom Check
The most important sanity check for IG is completeness: the sum of all attributions should equal the difference between your model's output at the input and at the baseline.
Mathematically:
sum(attributions) ≈ f(x) - f(baseline)
Think of it like balancing a ledger: if you're explaining a prediction change of +0.8, but your attributions sum to +0.5, you've "lost" 0.3 units of explanation somewhere.
Common Issues to Debug
Approximation error: With too few integration steps (small m in Riemann approximation), the sum won't match perfectly. Try increasing steps from 50 to 300+.
Baseline mismatch: If your baseline has the wrong shape or isn't properly broadcast, gradients will be meaningless.
Gradient computation bugs: Forgetting to accumulate gradients properly or using the wrong differentiation target produces invalid attributions.
Numerical precision: For very small or large outputs, floating-point errors accumulate. Check relative error, not just absolute difference.
Practical Verification
Always compute actual_diff = model(input) - model(baseline) and attribution_sum = attributions.sum(), then verify abs(actual_diff - attribution_sum) < tolerance. A tolerance of 0.01–0.05 is typical with sufficient integration steps.
Key Takeaway: Always verify completeness by checking that attribution sums match output differences—this single sanity check catches most IG implementation bugs and ensures your explanations are mathematically grounded.