Course contentsShow
← Machine Learning and Deep Learning
Lesson 3207 of 3,53870. Interpretability: Feature-Based MethodsFree lesson

SHAP Values: Mathematical Definition

Master the mathematical formula for computing SHAP values and understand the marginalization over feature coalitions.

SHAP Values: Mathematical Definition

What you'll learn: The precise mathematical formula that defines SHAP values and how it marginalizes over all possible feature coalitions.

The Core Formula

SHAP values build on Shapley values from cooperative game theory. For a prediction model f, the SHAP value φᵢ for feature i is defined as:

φᵢ = Σ_{S ⊆ F \ {i}} [|S|! (|F| - |S| - 1)! / |F|!] × [f(S ∪ {i}) - f(S)]

Where:

  • F is the set of all features
  • S is a subset (coalition) of features not including feature i
  • |S| is the size of subset S
  • f(S) is the model's expected output when only features in S are "present"

Plain English Interpretation

Imagine you're measuring how much feature i contributes to a prediction. You consider every possible combination of other features that could be present or absent. For each combination S:

  1. Compute the prediction with S alone: f(S)
  2. Add feature i and compute again: f(S ∪ {i})
  3. The difference is i's marginal contribution to that coalition
  4. Weight this by how likely that specific coalition size is (the factorial term)

The factorial weighting ensures smaller and larger coalitions are fairly balanced—this is the "fair distribution" principle from Shapley values.

Why Marginalization Matters

The formula marginalizes over all 2^(|F|-1) possible feature coalitions. This exhaustive averaging is what makes SHAP values satisfy desirable properties like consistency and local accuracy, but it's also why exact computation is exponentially expensive for many features.

Key Takeaway: SHAP values are the weighted average of a feature's marginal contribution across all possible subsets of other features, with weights determined by coalition size probabilities—ensuring a theoretically fair attribution.