Course contentsShow
← Machine Learning and Deep Learning
Lesson 387 of 3,5389. Dimensionality ReductionFree lesson

Eigendecomposition for PCA

How eigenvectors of the covariance matrix define principal components and eigenvalues measure variance captured.

Eigendecomposition for PCA

What you'll learn: How eigenvectors and eigenvalues of the covariance matrix unlock the principal components in PCA.

The Core Idea

In PCA, you've already constructed the covariance matrix that captures how features vary together. Now comes the magic: eigendecomposition breaks that covariance matrix into special directions (eigenvectors) and their importance (eigenvalues).

Think of your data as a cloud of points in space. Eigenvectors are like finding the "natural axes" along which this cloud stretches most. Each eigenvector points in a direction, and its corresponding eigenvalue tells you how much variance (information) exists along that direction.

How It Works

When you perform eigendecomposition on the covariance matrix:

  1. Eigenvectors become principal components: Each eigenvector defines a new axis in your feature space. The first eigenvector points in the direction of maximum variance, the second points in the next-highest variance direction (perpendicular to the first), and so on.

  2. Eigenvalues measure captured variance: A large eigenvalue means its eigenvector's direction contains lots of information. A tiny eigenvalue means that direction contains mostly noise or redundancy.

By sorting eigenvectors by their eigenvalues (largest to smallest), you rank the principal components by importance. You can then keep only the top few components that capture most of your data's variance—achieving dimensionality reduction while preserving information.

Why This Matters

Eigendecomposition transforms the abstract goal of "find directions of maximum variance" into a concrete mathematical operation. The eigenvectors literally are your new coordinate system.

Key Takeaway: Eigenvectors of the covariance matrix define the principal component directions, while eigenvalues quantify how much variance each component captures—enabling you to rank and select the most informative dimensions.