Gaussian Distribution as Cluster Model
What you'll learn: How a multivariate Gaussian distribution can mathematically represent a cluster with specific shape and position parameters.
The Core Idea
In probabilistic clustering, instead of assigning each point to a single cluster (like K-Means does), we model each cluster as a probability distribution. The most common choice is the Gaussian (normal) distribution.
Think of a cluster as a cloud of points. A Gaussian distribution lets you describe this cloud mathematically using just two parameters:
- Mean (μ): The center point of the cluster
- Covariance (Σ): The shape and spread of the cluster
Why Gaussian Distributions?
A Gaussian distribution naturally captures how data points "belong" to a cluster with varying degrees of confidence. Points near the center have high probability, while points far away have low probability.
Univariate vs Multivariate
- Univariate Gaussian: Models one-dimensional data (single feature)
- Multivariate Gaussian: Models multi-dimensional data (multiple features working together)
The multivariate version is crucial because real datasets have multiple features. The covariance matrix tells us not just how spread out each feature is, but also how features correlate with each other.
The Mathematical Model
Each cluster becomes a Gaussian "bell curve" (in multiple dimensions). The parameters define:
- μ positions the cluster in feature space
- Σ determines if the cluster is round, elongated, or tilted
This is fundamentally different from K-Means, which assumes all clusters are spherical with equal size.
Key Takeaway: A Gaussian distribution models a cluster as a probability cloud defined by its mean (center) and covariance (shape), enabling soft, probabilistic cluster assignments instead of hard boundaries.