Mixture Model Definition
What you'll learn: You'll understand how Gaussian Mixture Models (GMMs) combine multiple Gaussian distributions using weighted sums to represent complex data patterns.
The Core Idea
A mixture model treats your data as if it comes from multiple overlapping subpopulations, each represented by a probability distribution. Instead of forcing all data into one rigid cluster (like K-Means), a mixture model says: "This data point could belong to cluster A or cluster B, with different probabilities."
In a Gaussian Mixture Model (GMM), each subpopulation is modeled as a Gaussian distribution. The complete model is a weighted sum of these Gaussian components.
How It Works
Think of a GMM like a recipe that blends ingredients:
- Components: Each Gaussian distribution (you learned this in "Gaussian Distribution as Cluster Model") represents one "ingredient"
- Mixing coefficients (often written as π₁, π₂, ..., πₖ): These are weights that determine how much each Gaussian contributes to the final mixture
- The crucial constraint: All mixing coefficients must be non-negative and sum to exactly 1.0
For example, if you have 3 Gaussian components with mixing coefficients [0.3, 0.5, 0.2], it means:
- 30% of your data comes from cluster 1
- 50% from cluster 2
- 20% from cluster 3
The probability of any data point is calculated by combining all three Gaussians according to these weights.
Why This Matters
Unlike K-Means which gives hard assignments ("you ARE in cluster 2"), GMMs provide soft, probabilistic assignments ("you're 70% likely cluster 2, 30% likely cluster 1").
Key Takeaway: A Gaussian Mixture Model represents data as a weighted combination of multiple Gaussian distributions, where mixing coefficients act as probabilities that must sum to one.