Course contentsShow
← Computer Science
Lesson 2802 of 2,87262. Machine Learning FundamentalsFree lesson

Leave-One-Out Cross-Validation

An extreme form of cross-validation useful for very small datasets, where k equals the dataset size.

Leave-One-Out Cross-Validation

What You'll Learn

You'll learn about an extreme form of cross-validation that trains on all but one data point, ideal for tiny datasets but computationally expensive.

The Core Idea

Leave-One-Out Cross-Validation (LOOCV) is k-fold cross-validation pushed to its logical limit: where k equals the total number of examples in your dataset. If you have 50 data points, you perform 50 training rounds—each time holding out exactly one example as the test set and training on the remaining 49.

Think of it like a cooking class where every student takes turns being the judge. Each student sits out once while everyone else cooks, then tastes and rates the dish. By the end, every student has been the judge exactly once, giving you the most thorough evaluation possible.

Why Use LOOCV?

LOOCV shines when your dataset is very small—perhaps only 20-30 examples. In these scenarios:

  • Traditional k-fold splits might create training sets too small to learn from
  • A single train-test split wastes valuable training data
  • You want maximum training data utilization per iteration

LOOCV maximizes your training set size (n-1 examples) while still providing unbiased performance estimates.

The Tradeoff

The cost is computational: training n models for n data points becomes prohibitively expensive on large datasets. With 10,000 examples, you'd need 10,000 training runs! Additionally, LOOCV can have high variance in its estimates because each training set differs by only one example.

Key Takeaway: Leave-One-Out Cross-Validation trains n models on n-1 examples each, providing maximum data utilization for tiny datasets at the cost of significant computational expense—it's k-fold cross-validation where k = n.