Course contentsShow
← Machine Learning and Deep Learning
Lesson 3250 of 3,53871. Interpretability: Neural Network MethodsFree lesson

Computing IG for Text Models

Applying IG to embeddings: interpolating word vectors and attributing importance to tokens in NLP tasks.

Computing IG for Text Models

You'll learn: How to apply Integrated Gradients to NLP models by interpolating word embeddings and attributing importance to individual tokens.

The Core Challenge with Text

Unlike images where you interpolate pixel values directly, text models work with discrete tokens that are first converted to continuous embeddings. You can't simply "blend" the word "cat" halfway to "dog" in token space—but you can interpolate their embedding vectors.

Interpolation Path for Embeddings

The key insight: construct your baseline-to-input path in embedding space, not token space.

  1. Choose a baseline: typically a zero vector, padding token embedding, or special [PAD] token
  2. Get embeddings: convert your input tokens to vectors (e.g., from a learned embedding layer)
  3. Interpolate: create a straight-line path from baseline embedding to input embedding for each token position
  4. Compute gradients: at each interpolation step, compute ∂(output)/∂(embedding) with respect to the embedding vectors
  5. Attribute to tokens: the integral approximation gives you an importance score per embedding dimension; typically you sum/norm to get one score per token

Practical Considerations

Since embeddings are high-dimensional (e.g., 768 dimensions), the gradient computation happens in that space. You end up with a scalar importance score for each token position, telling you which words contributed most to the model's prediction.

Common baseline choices: [PAD] embeddings preserve the input length structure, while zero vectors represent "absence of meaning."

Key Takeaway: IG for text models operates on embedding vectors rather than discrete tokens, interpolating in continuous embedding space to attribute prediction importance to each word or subword in the input sequence.