Course contentsShow
← Data Science
Lesson 630 of 2,14516. Multiple Linear RegressionFree lesson

Bayesian Information Criterion (BIC)

Apply BIC for model comparison, which penalizes complexity more strongly than AIC for larger sample sizes.

Bayesian Information Criterion (BIC)

What you'll learn: How to use BIC to compare multiple regression models while favoring simpler models, especially with larger datasets.

What Is BIC?

The Bayesian Information Criterion (BIC) is a model selection tool that helps you choose between competing regression models. Like AIC (which you just learned), BIC balances model fit against complexity—but with a twist: BIC penalizes extra predictors more harshly, especially as your sample size grows.

The BIC Formula

BIC = n × ln(RSS/n) + k × ln(n)

Where:

  • n = number of observations
  • RSS = residual sum of squares (lower means better fit)
  • k = number of parameters (including intercept)
  • ln() = natural logarithm

Lower BIC values indicate better models.

BIC vs AIC: The Key Difference

Notice that BIC multiplies k by ln(n), whereas AIC uses a constant penalty of 2. When n > 7, ln(n) > 2, so BIC penalizes complexity more strongly than AIC. With 100 observations, ln(100) ≈ 4.6—more than twice AIC's penalty!

When to Use BIC

Use BIC when:

  • You have larger sample sizes (where overfitting is a real concern)
  • You prefer simpler, more parsimonious models
  • You want to avoid including marginally useful predictors

Practical Interpretation

If Model A has 3 predictors and BIC = 520, and Model B has 5 predictors and BIC = 518, choose Model B (lower is better). But if Model B's BIC were 522, the simpler Model A would win—those 2 extra predictors don't justify their complexity cost.

Key Takeaway: BIC helps you select models by penalizing complexity more aggressively than AIC, making it particularly effective for larger datasets where you want to avoid unnecessary predictors.