Course contentsShow
← Computer Science
Lesson 2797 of 2,87262. Machine Learning FundamentalsFree lesson

The Validation Set

Using a third validation set for hyperparameter tuning without contaminating the test set.

The Validation Set

What you'll learn: How to use a validation set to tune your model's hyperparameters without accidentally cheating on your final test.

The Three-Way Split

You've already learned about splitting data into training and test sets. But there's a problem: if you repeatedly evaluate different model configurations on your test set to pick the best one, you're inadvertently "learning" from the test data. Your final performance estimate becomes overly optimistic—you've contaminated your test set.

The solution? Introduce a validation set.

How It Works

Split your data three ways:

  • Training set (~60-70%): Fit your model's parameters
  • Validation set (~15-20%): Compare different hyperparameters and model choices
  • Test set (~15-20%): Final, one-time evaluation of your chosen model

Think of it like studying for an exam. The training set is your textbook. The validation set is practice problems you use to decide how to study (how many hours? which chapters to focus on?). The test set is the actual exam—you only see it once, and it tells you how well you really learned.

Why This Matters

Without a validation set, every time you adjust hyperparameters based on test set performance, you're effectively training on the test set. You might try 50 different learning rates, pick the best one based on test accuracy, and report that number—but it's not a fair measure of future performance.

The validation set lets you experiment freely. Only after you've finalized all decisions do you touch the test set once to get an honest performance estimate.

Key Takeaway: The validation set protects the integrity of your test set by providing a separate dataset for hyperparameter tuning and model selection decisions.