Leverage: Distance in X-Space
You'll learn how leverage quantifies how far an observation's predictor values sit from the center of your data.
What Is Leverage?
Leverage measures how unusual an observation is in terms of its predictor values (the X variables), completely independent of the response variable (Y). Think of it as asking: "How far out on the edge of our X-space does this data point sit?"
Imagine you're studying the relationship between study hours and exam scores. Most students study between 5–10 hours. If one student studied 25 hours, their observation has high leverage—not because their score is unusual, but because their study time is far from the typical range.
The Hat Matrix Connection
Leverage is formally measured using the hat matrix (often denoted as H). The diagonal elements of this matrix, called hᵢ, give each observation's leverage value. The hat matrix essentially tells us how much weight each observation has in determining its own fitted value.
Key properties:
- Leverage values range from 0 to 1
- The average leverage is (p+1)/n, where p is the number of predictors and n is sample size
- High leverage means the observation is far from the mean of the predictors
Why It Matters
An observation with high leverage has the potential to strongly influence the regression line—like a fulcrum point on a lever. Whether it actually does influence the fit depends on its Y-value (which we'll explore separately as "influence"). But leverage alone flags: "This X-value is in uncharted territory."
Key Takeaway: Leverage measures how extreme an observation's predictor values are relative to the rest of the data, using the hat matrix diagonal to quantify distance in X-space.