Identifying Outliers in Regression Context
You'll learn: How to distinguish outliers from high-leverage points and use residual plots to spot observations with unusual Y values.
What Makes an Outlier in Regression?
An outlier in regression is an observation with an unusual Y value given its X value—it doesn't follow the pattern of the other data points. This is different from a high-leverage point, which has an unusual X value but may fit the pattern perfectly.
Think of it this way: If your regression line predicts that someone who studies 10 hours should score 85% on a test, but one student studied 10 hours and scored 40%, that's an outlier. If another student studied 30 hours (when everyone else studied 5-15 hours), that's high leverage—their X is unusual, not necessarily their Y.
How to Identify Outliers
The best tool is a residuals vs fitted values plot. Outliers show up as points with large residuals—they sit far above or below the horizontal zero line. You've already learned about standardized and studentized residuals; outliers typically have absolute values greater than 2 or 3.
You can also use:
- Index plots of residuals to spot specific observation numbers
- Q-Q plots where outliers appear as points far from the diagonal line
- Cook's Distance and other influence measures (though these capture both outlier and leverage effects combined)
The key distinction: outliers are about vertical distance from the regression line, while leverage is about horizontal distance in the X-space.
What to Do
Investigate outliers carefully. They might be data entry errors, special cases worth understanding, or legitimate extreme values that teach you something important about your model's limitations.
Key Takeaway: Outliers have unusual Y values (large residuals) and are best identified through residual plots; they differ from high-leverage points, which have unusual X values.