Perturbation-Based Explanation Generation
What You'll Learn
You'll understand how LIME creates perturbed samples around an instance to explore what the model "sees" when making predictions.
The Core Idea
LIME doesn't peek inside your model—instead, it performs a clever experiment. To explain why a model made a particular prediction for one instance, LIME generates perturbed versions of that instance by slightly modifying its features. Think of it like a scientist testing "what if" scenarios: "What if this pixel were darker? What if this word were removed?"
For each perturbed sample, LIME records what the model predicts. By observing how predictions change as features are tweaked, LIME learns which features matter most for that specific instance.
How It Works
-
Sample generation: LIME creates synthetic neighbors around your instance by randomly perturbing features (e.g., turning pixels on/off, removing words, changing numerical values)
-
Weighted by proximity: Samples closer to the original instance get higher weights—we care more about nearby behavior than distant examples
-
Model querying: Each perturbed sample is fed through your black-box model to get predictions
-
Local surrogate fitting: LIME fits a simple, interpretable model (like linear regression) on these perturbed samples, weighted by proximity
The simple model approximates the complex model's behavior locally—just around your instance of interest. The coefficients of this simple model become your explanation.
Analogy
Imagine explaining why a chef chose certain spices for a dish. You couldn't read the chef's mind, but you could prepare variations—one without cumin, one with extra paprika—and taste each. The dishes that taste most different reveal which ingredients mattered most.
Key Takeaway: LIME generates perturbed samples around an instance and observes model responses to build a local, interpretable approximation that explains the prediction.