t-SNE: Probabilistic Neighbor Embedding
What You'll Learn
You'll understand how t-SNE preserves local structure by matching probability distributions over neighbors in high and low dimensions.
The Core Idea
t-SNE (t-Distributed Stochastic Neighbor Embedding) solves a key problem: how do we visualize high-dimensional data while keeping similar points close together?
Here's the clever approach: t-SNE converts distances into probabilities.
In High Dimensions
For each point, t-SNE computes the probability that it would "pick" each of its neighbors based on distance. Nearby points get high probabilities; distant points get low probabilities. This creates a probability distribution that captures the local neighborhood structure around every point.
Think of it like this: imagine standing in a crowded room. The probability you'd bump into someone depends on how close they are to you. t-SNE mathematically describes this "bumping probability" for every pair of points.
In Low Dimensions (Usually 2D or 3D)
t-SNE then places points in the lower-dimensional space and computes a second set of probabilities using the same logic. Now it asks: "Do the neighborhood probabilities in 2D match those from the original high-dimensional space?"
The Matching Process
t-SNE iteratively moves points around in the low-dimensional space to make these two probability distributions as similar as possible. When high-dimensional neighbors have high probability and they also have high probability in 2D, the structure is preserved.
This is fundamentally different from PCA, which preserves global variance but may lose local relationships.
Key Takeaway: t-SNE preserves local structure by ensuring that the probability of being neighbors remains consistent between high-dimensional and low-dimensional spaces—keeping similar points clustered together in visualizations.