Course contentsShow
← Machine Learning and Deep Learning
Lesson 405 of 3,5389. Dimensionality ReductionFree lesson

Locally Linear Embedding (LLE)

Reconstructing each point from neighbors and preserving these local linear relationships in the low-dimensional space.

Locally Linear Embedding (LLE)

What You'll Learn

You'll discover how LLE reduces dimensionality by assuming your high-dimensional data lies on a locally linear manifold, preserving the way each point relates to its nearby neighbors.

The Core Idea

Imagine you're mapping a crumpled sheet of paper. While the whole sheet is curved and folded, if you zoom in on any small patch, it looks flat—locally linear. LLE exploits this intuition.

The algorithm works in two steps:

Step 1: Reconstruct Each Point from Neighbors

For every data point, LLE finds its nearest neighbors and figures out the optimal weights to reconstruct that point as a weighted combination of those neighbors. Think of it like saying: "I can describe my position as 30% of neighbor A, 50% of neighbor B, and 20% of neighbor C."

These weights capture the local geometry—the intrinsic relationships between nearby points that shouldn't change even when we move to lower dimensions.

Step 2: Preserve Relationships in Low Dimensions

Once we have these reconstruction weights, LLE finds new coordinates in the lower-dimensional space that preserve these same relationships. The weights stay fixed, but we solve for new positions where each point is still reconstructed by its neighbors using those same weights.

This is different from PCA (which preserves global variance) or Isomap (which preserves geodesic distances). LLE focuses purely on maintaining local linear structure.

Why It Matters

LLE assumes your data lies on a smooth manifold. By preserving how neighbors linearly relate to each other, it can "unfold" complex curved surfaces into simpler lower-dimensional representations while maintaining the essential local structure.

Key Takeaway: LLE reduces dimensions by first learning how to reconstruct each point from its neighbors, then finding a low-dimensional embedding that preserves these exact same local linear relationships.