Semi-Supervised Learning
What you'll learn: How to leverage vast amounts of unlabeled data alongside a small labeled dataset to train better machine learning models.
The Core Problem
In supervised learning (lesson 2786), you need lots of labeled examples—data where the correct answer is known. But labeling is expensive: imagine hiring doctors to label thousands of X-rays, or paying annotators to tag millions of images. Meanwhile, unlabeled data is everywhere and costs almost nothing to collect.
Semi-supervised learning bridges this gap by combining a small labeled dataset with a much larger unlabeled dataset to improve model performance beyond what either dataset could achieve alone.
How It Works
The key insight: unlabeled data still contains valuable information about the structure of your problem space.
Think of learning to identify dog breeds. With 100 labeled photos, you learn basic features. But if you also have 10,000 unlabeled animal photos, the model can discover:
- Natural clusters (different breeds tend to group together visually)
- The overall distribution of features in the animal world
- Patterns that help separate dogs from non-dogs
Common approaches:
- Self-training: Train on labeled data, predict labels for unlabeled data, add high-confidence predictions to training set, repeat
- Co-training: Use multiple "views" of data (e.g., image + text description) where each helps label the other
- Cluster assumptions: Assume decision boundaries should pass through low-density regions, not cluster centers
The Sweet Spot
Semi-supervised learning shines when:
- You have abundant unlabeled data but labeling is expensive
- The unlabeled data shares the same distribution as labeled data
- The underlying structure (clusters, manifolds) is meaningful for your task
Key Takeaway: Semi-supervised learning amplifies limited labeled data by exploiting patterns in abundant unlabeled data, often achieving performance closer to fully supervised models at a fraction of the labeling cost.