R² and Adjusted R² in Multiple Regression
What you'll learn: How to measure model fit in multiple regression and why adjusted R² is essential when comparing models with different numbers of predictors.
Understanding R² in Multiple Regression
You've already seen R² (the coefficient of determination) in simple linear regression—it tells you the proportion of variance in the outcome variable that your model explains. In multiple regression, R² works the same way: it ranges from 0 to 1, where higher values mean your predictors collectively explain more of the variation in Y.
The formula remains conceptually the same:
R² = 1 - (SS_residual / SS_total)
Here's the catch: R² always increases when you add more predictors, even if those predictors are irrelevant or just noise. Add ten random variables to your model? R² goes up, even though your model isn't actually better.
The Problem and the Solution: Adjusted R²
Adjusted R² penalizes you for adding predictors that don't pull their weight. It adjusts R² based on the number of predictors (p) and sample size (n):
Adjusted R² = 1 - [(1 - R²) × (n - 1) / (n - p - 1)]
Analogy: Think of R² as measuring how much luggage fits in your car, while adjusted R² asks whether each suitcase was worth bringing—it can actually decrease if you add useless predictors.
Use adjusted R² when comparing models with different numbers of predictors. A model with fewer predictors but higher adjusted R² is often preferable—it's more parsimonious and less likely to overfit.
Key Takeaway: R² measures explained variance but always increases with more predictors; adjusted R² accounts for model complexity, making it the better metric for comparing multiple regression models with different numbers of variables.