Interpretable Representations
What you'll learn: How LIME converts complex, high-dimensional input data into simplified features that humans can actually understand and reason about.
The Translation Challenge
When LIME explains a model's prediction, it faces a fundamental problem: raw input features are often incomprehensible to humans. Imagine explaining why a model classified an image by listing millions of pixel values, or explaining a text prediction by referencing thousands of word embedding dimensions. These interpretable representations solve this by transforming complex inputs into meaningful, human-friendly features.
From Raw to Readable
Think of interpretable representations as translation layers between machine perception and human understanding:
- Images: Instead of raw pixels, use superpixels—groups of similar, connected pixels that form recognizable image regions (like "the dog's ear" or "sky area")
- Text: Instead of word embeddings or token IDs, use presence/absence of actual words or phrases
- Tabular data: Often the features are already interpretable (age, income), but may need discretization (converting continuous values into bins like "high income" vs "low income")
Why This Matters for LIME
When LIME perturbs inputs in its weighted sampling for local neighborhoods, it operates in this interpretable space. For an image, it toggles superpixels on/off. For text, it removes or keeps individual words. This serves two purposes:
- Perturbations are semantically meaningful — turning off "the word 'excellent'" makes sense; perturbing embedding dimension 247 doesn't
- Explanations reference concepts humans recognize — "the model predicted positive because of the word 'excellent'" vs. "because embedding[247]=0.83"
Key Takeaway: Interpretable representations are the bridge between a model's complex internal view and explanations humans can understand—they transform raw features into semantically meaningful units that support both perturbation sampling and comprehensible explanations.