Training Set Size Effects
What you'll learn: How the amount of training data you use directly influences your model's learning curve and when adding more data can improve performance.
The Core Idea
Imagine learning a language from a phrase book with only 10 sentences versus one with 10,000. More examples give you better understanding — the same applies to machine learning models.
Training set size effects describe how your model's performance changes as you increase or decrease the number of training examples. This relationship appears directly on learning curves, where you plot training and validation scores against the number of training samples.
What Happens as Training Size Grows
With small training sets:
- Training error is artificially low (easy to memorize few examples)
- Validation error is high (poor generalization)
- The gap between them is large
With larger training sets:
- Training error gradually increases (harder to fit all examples perfectly)
- Validation error typically decreases (better generalization)
- The gap narrows
When More Data Helps
Adding more training data is most beneficial when you have high variance (the validation curve is far below training curve). More examples help the model learn true patterns instead of noise.
However, if you have high bias (both curves plateau at poor performance), more data won't help much — your model is too simple to capture the underlying patterns regardless of data volume.
Key Takeaway: Learning curves show whether collecting more training data will improve your model: if validation error is still decreasing as training size increases, more data likely helps; if both curves have plateaued, you need a more complex model or better features instead.