Course contentsShow
← Machine Learning and Deep Learning
Lesson 495 of 3,53812. Model Validation and TuningFree lesson

Leave-One-Out Cross-Validation (LOOCV)

Understanding LOOCV as an extreme case where k equals the dataset size and its computational trade-offs.

Leave-One-Out Cross-Validation (LOOCV)

You'll learn what happens when you push K-Fold cross-validation to its mathematical extreme and why that matters for your model evaluation strategy.

The Extreme Case of K-Fold

Remember K-Fold Cross-Validation where you split your data into k chunks? LOOCV is what happens when you set k equal to the total number of samples in your dataset. If you have 100 data points, you create 100 folds—each containing exactly one observation.

How LOOCV Works

In each iteration, LOOCV uses exactly one sample as the validation set and all remaining samples for training. This process repeats until every single data point has served as the validation set exactly once. Your final performance estimate is the average across all these iterations.

Think of it like a classroom where each student gets a turn being the "teacher's assistant" while everyone else stays in their seats. Every student experiences both roles, giving you the most complete picture possible.

The Trade-Offs

Advantages:

  • Maximum use of training data (n-1 samples every time)
  • Nearly unbiased performance estimate
  • Deterministic—no randomness in the splits

Disadvantages:

  • Computationally expensive: training n separate models for n data points
  • High variance in performance estimates (each fold is very similar)
  • Impractical for large datasets or complex models

LOOCV shines with small datasets where every data point matters, but becomes prohibitively slow as your dataset grows.

Key Takeaway: LOOCV is K-Fold CV taken to the extreme where k=n, providing maximum data utilization but at the cost of computational efficiency—best reserved for small datasets where thorough validation justifies the expense.