Course contentsShow
← Machine Learning and Deep Learning
Lesson 365 of 3,5388. Unsupervised Learning: ClusteringFree lesson

Mixture Model Definition

Defining GMMs as weighted sums of Gaussian components with mixing coefficients that sum to one.

Mixture Model Definition

What you'll learn: You'll understand how Gaussian Mixture Models (GMMs) combine multiple Gaussian distributions using weighted sums to represent complex data patterns.

The Core Idea

A mixture model treats your data as if it comes from multiple overlapping subpopulations, each represented by a probability distribution. Instead of forcing all data into one rigid cluster (like K-Means), a mixture model says: "This data point could belong to cluster A or cluster B, with different probabilities."

In a Gaussian Mixture Model (GMM), each subpopulation is modeled as a Gaussian distribution. The complete model is a weighted sum of these Gaussian components.

How It Works

Think of a GMM like a recipe that blends ingredients:

  • Components: Each Gaussian distribution (you learned this in "Gaussian Distribution as Cluster Model") represents one "ingredient"
  • Mixing coefficients (often written as π₁, π₂, ..., πₖ): These are weights that determine how much each Gaussian contributes to the final mixture
  • The crucial constraint: All mixing coefficients must be non-negative and sum to exactly 1.0

For example, if you have 3 Gaussian components with mixing coefficients [0.3, 0.5, 0.2], it means:

  • 30% of your data comes from cluster 1
  • 50% from cluster 2
  • 20% from cluster 3

The probability of any data point is calculated by combining all three Gaussians according to these weights.

Why This Matters

Unlike K-Means which gives hard assignments ("you ARE in cluster 2"), GMMs provide soft, probabilistic assignments ("you're 70% likely cluster 2, 30% likely cluster 1").

Key Takeaway: A Gaussian Mixture Model represents data as a weighted combination of multiple Gaussian distributions, where mixing coefficients act as probabilities that must sum to one.