K-Fold Cross-Validation
What you'll learn: How to get more reliable performance estimates by systematically rotating through different train/test splits of your data.
The Problem with a Single Split
In previous lessons, you learned about splitting data into training, validation, and test sets. But what if your single validation set happens to be particularly easy or hard? Your performance estimate might not reflect how your model really performs.
The Core Idea
K-Fold Cross-Validation divides your entire training dataset into k equal-sized "folds" (commonly k=5 or k=10). Then you:
- Train on k-1 folds
- Validate on the remaining 1 fold
- Repeat k times, rotating which fold serves as validation
- Average the k validation scores
Think of it like a basketball player practicing shots from different court positions instead of just one spot — you get a more complete picture of their real ability.
Why It Matters
This approach gives you a more stable and reliable estimate of your model's true performance. Instead of a single validation score that might be lucky or unlucky, you get k different scores that average out random variation.
Each data point appears in the validation set exactly once and in the training set k-1 times. This is especially valuable when you have limited data — you're squeezing maximum information from what you have without needing to set aside a large permanent validation set.
The Workflow
After using k-fold cross-validation to tune hyperparameters or select between models, you typically retrain on all available training data before final evaluation on your held-out test set.
Key Takeaway: K-fold cross-validation rotates through k different train/validation splits to produce more robust performance estimates, reducing the risk of making decisions based on a single lucky or unlucky data split.