One-vs-One (OvO) Strategy
What You'll Learn
You'll learn how the One-vs-One approach extends binary logistic regression to multiclass problems by training a classifier for every pair of classes.
The Core Idea
Imagine you're teaching a student to distinguish between cats, dogs, and birds. Instead of comparing each animal against "everything else," you could teach them three specific comparisons: cats vs. dogs, cats vs. birds, and dogs vs. birds.
One-vs-One (OvO) does exactly this: for a problem with N classes, it trains N×(N-1)/2 binary classifiers—one for every unique pair of classes. When predicting, each classifier "votes" for its preferred class, and the class with the most votes wins.
How It Works
For 3 classes (A, B, C), OvO creates:
- Classifier 1: A vs. B
- Classifier 2: A vs. C
- Classifier 3: B vs. C
When you want to classify a new sample:
- All three classifiers make predictions
- Each classifier votes for one of its two classes
- You tally the votes (e.g., A gets 2 votes, B gets 1, C gets 0)
- The class with the most votes becomes your final prediction
OvO vs. OvR
Unlike One-vs-Rest (which trains N classifiers), OvO trains more classifiers but each one works with a smaller, simpler subset of data—just two classes at a time. This can be beneficial when individual classifiers train quickly, though it requires more overall training.
Key Takeaway: One-vs-One trains a binary classifier for every pair of classes and uses majority voting to predict the final class, making it especially useful when binary classifiers are fast to train.