Course contentsShow
← Machine Learning and Deep Learning
Lesson 1286 of 3,53829. Core NLP TasksFree lesson

Interpretability in Text Classification

Using attention weights, LIME, and SHAP to understand which words drive model predictions for transparency.

Interpretability in Text Classification

You'll learn how to see inside your text classifier to understand which words actually drove each prediction.

Why It Matters

A classifier might correctly label an email as "spam" or a review as "negative," but why did it decide that? Without interpretability, your model is a black box—accurate but mysterious. For debugging, building trust, or meeting regulations, you need to know which words mattered most.

Three Core Techniques

Attention Weights

Transformer models like BERT use attention mechanisms. By examining these weights, you can see which tokens the model focused on. High attention to words like "terrible" or "amazing" in a sentiment task reveals what influenced the decision.

LIME (Local Interpretable Model-Agnostic Explanations)

LIME perturbs your input text—removing or changing words—then observes how predictions shift. If removing "refund" flips a review from negative to positive, that word was crucial. It builds a simple, local approximation around your prediction.

SHAP (SHapley Additive exPlanations)

SHAP borrows from game theory to assign each word a "contribution score." It answers: "How much did this word push the prediction toward 'positive' vs 'negative'?" SHAP values are consistent and additive, making them reliable for feature importance.

Practical Impact

These methods help you:

  • Debug: Find if your model relies on spurious correlations (like dataset artifacts)
  • Trust: Show stakeholders why a decision was made
  • Improve: Identify weak features or biases in training data

Key Takeaway: Attention weights, LIME, and SHAP transform opaque text classifiers into transparent systems where you can trace predictions back to specific words, enabling debugging, trust, and compliance.