Information Criteria: AIC and BIC
What You'll Learn
You'll understand how AIC and BIC help you choose the best time series model by balancing goodness-of-fit against complexity.
The Model Selection Dilemma
When building time series models (like ARIMA), you face a tradeoff: more parameters (higher p, d, or q values) can fit your data better, but too many parameters lead to overfitting—your model memorizes noise instead of learning patterns.
AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion) solve this by scoring models on both how well they fit AND how simple they are.
How They Work
Both criteria penalize complexity:
AIC = -2 * log(likelihood) + 2 * k
BIC = -2 * log(likelihood) + k * log(n)
Where:
- log(likelihood) measures fit quality (higher = better fit)
- k = number of parameters in your model
- n = number of observations
Lower values are better—they indicate a superior balance of fit and simplicity.
Key Differences
- AIC applies a fixed penalty (2 * k) per parameter
- BIC applies a stronger penalty when you have more data (k * log(n))
- BIC tends to favor simpler models than AIC, especially with larger datasets
- Use both together: if they agree on the best model, you have strong evidence
Practical Example
Suppose you're comparing ARIMA(1,1,1) with 3 parameters vs ARIMA(2,1,2) with 5 parameters. Even if the complex model fits slightly better, its AIC/BIC might be higher (worse) due to the penalty for extra parameters.
Key Takeaway: AIC and BIC are your compass for model selection—choose the model with the lowest AIC or BIC to avoid overfitting while maintaining good predictive power.