Course contentsShow
← Data Science
Lesson 572 of 2,14515. Regression DiagnosticsFree lesson

Leverage: Distance in X-Space

Measuring how far an observation's predictor values are from the mean of all predictors using the hat matrix.

Leverage: Distance in X-Space

You'll learn how leverage quantifies how far an observation's predictor values sit from the center of your data.

What Is Leverage?

Leverage measures how unusual an observation is in terms of its predictor values (the X variables), completely independent of the response variable (Y). Think of it as asking: "How far out on the edge of our X-space does this data point sit?"

Imagine you're studying the relationship between study hours and exam scores. Most students study between 5–10 hours. If one student studied 25 hours, their observation has high leverage—not because their score is unusual, but because their study time is far from the typical range.

The Hat Matrix Connection

Leverage is formally measured using the hat matrix (often denoted as H). The diagonal elements of this matrix, called hᵢ, give each observation's leverage value. The hat matrix essentially tells us how much weight each observation has in determining its own fitted value.

Key properties:

  • Leverage values range from 0 to 1
  • The average leverage is (p+1)/n, where p is the number of predictors and n is sample size
  • High leverage means the observation is far from the mean of the predictors

Why It Matters

An observation with high leverage has the potential to strongly influence the regression line—like a fulcrum point on a lever. Whether it actually does influence the fit depends on its Y-value (which we'll explore separately as "influence"). But leverage alone flags: "This X-value is in uncharted territory."

Key Takeaway: Leverage measures how extreme an observation's predictor values are relative to the rest of the data, using the hat matrix diagonal to quantify distance in X-space.