Cautions and Limitations
You'll learn to recognize the most common pitfalls when applying linear regression: extrapolation, influential points, and confusing association with causation.
The Traps of Regression
Regression is powerful, but it comes with serious caveats. Even when all assumptions hold and R² looks impressive, misinterpreting or misapplying your model can lead to wildly wrong conclusions.
Extrapolation: Don't Predict Beyond Your Data
Extrapolation means using your regression line to predict values outside the range of your original data. This is dangerous because the linear relationship you found may not hold beyond what you observed.
Analogy: If you measure a child's height from ages 3 to 10 and fit a line, extrapolating to age 30 would predict someone 12 feet tall! The linear pattern doesn't continue forever.
Influential Points: One Observation Can Dominate
An influential point is a data point that dramatically changes the regression line if removed. Often these are outliers in the x-direction (high leverage) combined with unusual y-values. A single influential point can make a weak relationship appear strong or hide a real pattern.
Watch out: Always examine scatter plots and check what happens if you remove suspicious points.
Association ≠ Causation
Perhaps the most critical warning: correlation or a significant regression slope does NOT prove causation. Ice cream sales and drowning deaths are correlated (both increase in summer), but ice cream doesn't cause drowning—there's a lurking variable (warm weather).
Establishing causation requires controlled experiments, not just observational data and regression.
Key Takeaway: Always stay within your data's range, check for influential points that distort your model, and never claim causation from regression alone—correlation can reflect confounding, coincidence, or reverse causation.