Course contentsShow
← Machine Learning and Deep Learning
Lesson 370 of 3,5388. Unsupervised Learning: ClusteringFree lesson

Model Selection: Choosing Number of Components

Using BIC, AIC, or cross-validation to determine the optimal number of Gaussian components.

Model Selection: Choosing Number of Components

What You'll Learn

You'll learn how to determine the optimal number of Gaussian components in a Gaussian Mixture Model using statistical criteria like BIC, AIC, and cross-validation.

The Challenge

After learning how Expectation-Maximization fits a GMM, you face a crucial question: How many components should you use? Too few, and you underfit the data's complexity. Too many, and you overfit, capturing noise as if it were meaningful structure.

Three Statistical Approaches

BIC (Bayesian Information Criterion)

BIC penalizes model complexity more heavily. It balances likelihood (how well the model fits) against the number of parameters. Lower BIC is better. Think of it as rewarding accuracy but charging a steep price for each extra component.

AIC (Akaike Information Criterion)

AIC also balances fit and complexity but penalizes extra parameters less aggressively than BIC. This means AIC might favor slightly more complex models. Lower AIC is better.

Cross-Validation

Split your data into folds. Train GMMs with different component counts on training folds and evaluate likelihood on validation folds. This directly tests generalization ability.

The Analogy

Imagine describing a crowd's height distribution. One bell curve (component) might work for a random sample. But a mixed-gender crowd needs two. Three components would be overkill unless children are present. BIC/AIC help you avoid both oversimplification and unnecessary complexity.

Practical Strategy

Fit GMMs with k = 1, 2, 3, ... components, compute BIC/AIC for each, and pick the k where the criterion is minimized. Often you'll see a clear "elbow" where adding more components stops helping.

Key Takeaway: BIC, AIC, and cross-validation provide principled ways to choose the number of Gaussian components by balancing model fit against complexity, preventing both underfitting and overfitting in mixture models.