Course contentsShow
← Machine Learning and Deep Learning
Lesson 3224 of 3,53870. Interpretability: Feature-Based MethodsFree lesson

Fitting the Surrogate Linear Model

Training a sparse linear model on perturbed samples weighted by proximity to explain local behavior.

Fitting the Surrogate Linear Model

What you'll learn: How LIME trains a simple linear model on perturbed samples to explain what a complex model is doing locally around a specific prediction.

The Core Idea

After generating perturbed samples around the instance you want to explain (from lesson 3221) and weighting them by proximity (lesson 3222), LIME's final step is to fit a sparse linear model on these weighted samples. This surrogate model doesn't try to mimic the entire complex model—just its behavior in the local neighborhood.

Think of it like this: you're trying to understand why a mysterious chef chose specific ingredients for a dish. Instead of learning their entire culinary philosophy, you watch what happens when you slightly modify just that dish's ingredients, paying more attention to variations closer to the original. From those local experiments, you build a simple rule explaining this particular decision.

How It Works

The surrogate model is trained with:

  • Input: Perturbed samples in the interpretable representation (e.g., binary "word present/absent" vectors)
  • Output: The complex model's predictions on those perturbed samples
  • Weights: Proximity weights that emphasize samples closer to the instance being explained
  • Regularization: L1 (Lasso) penalty to encourage sparsity—keeping only the most important features

This weighted regression finds coefficients that explain which features matter most locally. The sparsity constraint ensures you get a small, human-readable set of important features rather than a dense, confusing list.

Why Weighted + Sparse?

The proximity weighting ensures the surrogate model is faithful to the complex model's local behavior, not its global behavior. The sparsity makes the explanation interpretable—you get "this prediction happened because of these 3-5 key features" rather than a wall of numbers.

Key Takeaway: LIME fits a sparse, weighted linear regression on perturbed samples to create a simple, locally-faithful approximation that reveals which features drove a specific prediction.