Course contentsShow
← Machine Learning and Deep Learning
Lesson 389 of 3,5389. Dimensionality ReductionFree lesson

Choosing Number of Components

Methods for selecting k: explained variance ratio, scree plots, and cumulative variance thresholds.

Choosing Number of Components

What you'll learn: How to decide how many principal components to keep after running PCA, balancing dimensionality reduction with information preservation.

The Challenge

After performing PCA, you have principal components ranked by the variance they capture. But how many should you actually keep? Too few and you lose important information; too many and you defeat the purpose of dimensionality reduction.

Three Methods for Selection

1. Explained Variance Ratio

Each principal component has an explained variance ratio—the fraction of total variance it captures. If PC1 explains 50% of variance and PC2 explains 30%, together they account for 80% of your data's variability.

2. Cumulative Variance Threshold

Set a target (commonly 90%, 95%, or 99%) and keep components until their cumulative variance reaches that threshold. This is like saying "I want to preserve at least 95% of my original information."

Example: If components capture [0.45, 0.30, 0.15, 0.07, 0.03] of variance:

  • PC1 + PC2 = 75% (not enough)
  • PC1 + PC2 + PC3 = 90% ✓ (keep 3 components)

3. Scree Plot

A visual method: plot component number vs. explained variance. Look for the "elbow"—where the curve flattens out. Components before the elbow carry significant information; those after add little value. Think of it like climbing a hill: you stop when the effort (adding components) no longer gives meaningful elevation gain (variance captured).

Practical Guidance

Start with 95% cumulative variance as a reasonable default. Adjust based on your application: simpler models for visualization (2-3 components), more components for machine learning tasks where accuracy matters.

Key Takeaway: Choose the number of principal components by examining explained variance ratios, using cumulative variance thresholds (often 95%), or identifying the elbow in a scree plot where additional components yield diminishing returns.