Course contentsShow
← Data Science
Lesson 632 of 2,14516. Multiple Linear RegressionFree lesson

Parsimony and Occam's Razor

Apply the principle of parsimony to prefer simpler models when predictive performance is similar.

Parsimony and Occam's Razor

You'll learn why simpler models are often better, even when complex models fit the data slightly better.

The Core Principle

Occam's Razor is a philosophical principle that says: when two explanations work equally well, choose the simpler one. In multiple linear regression, parsimony means preferring models with fewer predictors when they perform comparably to more complex alternatives.

Why does this matter? Adding more predictors always improves raw fit metrics like R-squared, but these complex models often:

  • Overfit to your specific dataset's noise
  • Perform worse on new data
  • Are harder to interpret and maintain
  • Risk including spurious relationships

Applying Parsimony in Practice

You've already learned tools that help balance complexity and performance:

  • Adjusted R-squared penalizes unnecessary predictors
  • AIC and BIC explicitly trade off fit quality against model size
  • Cross-validation reveals whether added complexity helps or hurts generalization

When comparing models, ask: Does adding this predictor meaningfully improve predictive performance, or am I just fitting noise?

An Analogy

Imagine explaining why it's raining. You could say "atmospheric pressure dropped" (simple) or "atmospheric pressure dropped, Jupiter aligned with Mars, and my neighbor wore a blue shirt" (complex). Both might correlate with today's rain, but only the first explains future rain reliably.

Key Takeaway: Parsimony guides you to prefer the simplest model that adequately explains your data—fewer predictors mean better generalization, easier interpretation, and more robust predictions.