Bayesian Information Criterion (BIC)
What you'll learn: How to use BIC to compare multiple regression models while favoring simpler models, especially with larger datasets.
What Is BIC?
The Bayesian Information Criterion (BIC) is a model selection tool that helps you choose between competing regression models. Like AIC (which you just learned), BIC balances model fit against complexity—but with a twist: BIC penalizes extra predictors more harshly, especially as your sample size grows.
The BIC Formula
BIC = n × ln(RSS/n) + k × ln(n)
Where:
n= number of observationsRSS= residual sum of squares (lower means better fit)k= number of parameters (including intercept)ln()= natural logarithm
Lower BIC values indicate better models.
BIC vs AIC: The Key Difference
Notice that BIC multiplies k by ln(n), whereas AIC uses a constant penalty of 2. When n > 7, ln(n) > 2, so BIC penalizes complexity more strongly than AIC. With 100 observations, ln(100) ≈ 4.6—more than twice AIC's penalty!
When to Use BIC
Use BIC when:
- You have larger sample sizes (where overfitting is a real concern)
- You prefer simpler, more parsimonious models
- You want to avoid including marginally useful predictors
Practical Interpretation
If Model A has 3 predictors and BIC = 520, and Model B has 5 predictors and BIC = 518, choose Model B (lower is better). But if Model B's BIC were 522, the simpler Model A would win—those 2 extra predictors don't justify their complexity cost.
Key Takeaway: BIC helps you select models by penalizing complexity more aggressively than AIC, making it particularly effective for larger datasets where you want to avoid unnecessary predictors.