Course contentsShow
← Machine Learning and Deep Learning
Lesson 308 of 3,5386. Tree-Based MethodsFree lesson

AdaBoost: Adaptive Boosting Algorithm

Learning how AdaBoost reweights training samples to focus on misclassified instances in each iteration.

AdaBoost: Adaptive Boosting Algorithm

What You'll Learn

You'll discover how AdaBoost strategically reweights training samples after each iteration, forcing subsequent weak learners to concentrate on the mistakes of their predecessors.

The Core Mechanism: Learning from Mistakes

AdaBoost (Adaptive Boosting) trains a sequence of weak learners, but here's the clever twist: it doesn't treat all training samples equally.

Initially, every training example gets the same weight (importance). After the first weak learner makes predictions, AdaBoost examines which samples were misclassified. It then increases the weights of those misclassified samples and decreases the weights of correctly classified ones.

When the next weak learner trains, it sees these adjusted weights. It naturally pays more attention to the high-weight (previously misclassified) samples because getting them right now matters more to the overall loss function.

The Reweighting Dance

Think of it like a teacher reviewing homework:

  • Students who got problems wrong get extra attention (higher weight)
  • Students who got them right need less focus (lower weight)
  • The next practice session emphasizes the troublesome concepts

This continues iteratively. Each new weak learner inherits the weight distribution from the previous round, trains on that weighted dataset, and then AdaBoost updates the weights again based on the new errors.

Why This Works

By forcing each successive learner to focus on the "hard" examples that previous learners struggled with, AdaBoost builds a diverse ensemble. Early learners handle easy patterns; later learners specialize in edge cases. Together, they form a strong classifier.

Key Takeaway: AdaBoost adaptively reweights training samples after each iteration, increasing weights for misclassified instances so the next weak learner focuses on correcting previous mistakes—creating a powerful ensemble through sequential error correction.