Computing IG for Text Models
You'll learn: How to apply Integrated Gradients to NLP models by interpolating word embeddings and attributing importance to individual tokens.
The Core Challenge with Text
Unlike images where you interpolate pixel values directly, text models work with discrete tokens that are first converted to continuous embeddings. You can't simply "blend" the word "cat" halfway to "dog" in token space—but you can interpolate their embedding vectors.
Interpolation Path for Embeddings
The key insight: construct your baseline-to-input path in embedding space, not token space.
- Choose a baseline: typically a zero vector, padding token embedding, or special
[PAD]token - Get embeddings: convert your input tokens to vectors (e.g., from a learned embedding layer)
- Interpolate: create a straight-line path from baseline embedding to input embedding for each token position
- Compute gradients: at each interpolation step, compute ∂(output)/∂(embedding) with respect to the embedding vectors
- Attribute to tokens: the integral approximation gives you an importance score per embedding dimension; typically you sum/norm to get one score per token
Practical Considerations
Since embeddings are high-dimensional (e.g., 768 dimensions), the gradient computation happens in that space. You end up with a scalar importance score for each token position, telling you which words contributed most to the model's prediction.
Common baseline choices: [PAD] embeddings preserve the input length structure, while zero vectors represent "absence of meaning."
Key Takeaway: IG for text models operates on embedding vectors rather than discrete tokens, interpolating in continuous embedding space to attribute prediction importance to each word or subword in the input sequence.