t-SNE: Perplexity and Hyperparameter Tuning
What you'll learn: How the perplexity parameter controls the balance between preserving local versus global structure in t-SNE visualizations.
Understanding Perplexity
Perplexity is t-SNE's most important hyperparameter, and it essentially controls how many neighbors each point considers when forming its local neighborhood. Think of it like adjusting your social circle: a low perplexity means you only pay attention to your closest 2-3 friends, while a high perplexity means you're aware of dozens of acquaintances.
Mathematically, perplexity is related to the variance of the Gaussian distribution used to compute pairwise similarities in high-dimensional space. A perplexity of 30 roughly means each point treats about 30 other points as meaningful neighbors.
The Local-Global Tradeoff
Low perplexity (5-15): t-SNE focuses intensely on very local structure. You'll see tight, detailed clusters, but the overall arrangement and distances between clusters become less meaningful. It's like zooming in with a microscope—great detail up close, but you lose the big picture.
High perplexity (50-100): t-SNE considers broader neighborhoods, capturing more global structure. Clusters are less tightly defined but their relative positions become more informative. However, too high a value can blur important local distinctions.
Typical range: Most practitioners use perplexity between 5 and 50, with 30 being a common default for moderate-sized datasets.
Tuning Guidelines
- Larger datasets can handle (and often benefit from) higher perplexity values
- Try multiple perplexity values—t-SNE results can vary significantly
- If clusters overlap unexpectedly, try lower perplexity
- If everything looks scattered, try higher perplexity
Key Takeaway: Perplexity controls the effective number of neighbors t-SNE considers, creating a tradeoff between preserving fine-grained local structure (low perplexity) and broader global relationships (high perplexity)—always experiment with multiple values.