Evaluating Multiple Regression: R² and Adjusted R²
What you'll learn: How to measure multiple regression model quality and why adjusted R² prevents feature bloat.
Understanding R² in Multiple Regression
You already know R² (coefficient of determination) from simple linear regression—it measures what proportion of variance in your target variable the model explains. In multiple regression, R² works the same way: it ranges from 0 to 1, where higher values mean better fit.
But there's a catch: R² always increases when you add more features, even if those features are useless! Add a random column of noise? R² goes up (or stays the same, never decreases). This makes R² misleading when comparing models with different numbers of features.
Why We Need Adjusted R²
Adjusted R² solves this problem by penalizing unnecessary features. It adjusts the R² value based on:
- Number of features in your model
- Number of data points you have
The formula essentially asks: "Does this feature improve the model enough to justify its inclusion?" If a feature barely helps, adjusted R² may actually decrease when you add it—signaling that simpler is better.
Think of it like packing for a trip: R² says "more items = better prepared," but adjusted R² says "only pack items that meaningfully help, or you're just carrying dead weight."
When to Use Each
- R²: Evaluating a single model's goodness of fit
- Adjusted R²: Comparing models with different numbers of features
Adjusted R² is lower than R² (or equal), and the gap widens as you add unhelpful features. This makes it your guardian against overfitting through feature bloat.
Key Takeaway: R² always rewards adding features; adjusted R² penalizes features that don't pull their weight, making it essential for comparing multiple regression models with different feature counts.