Course contentsShow
← Computer Science
Lesson 2850 of 2,87263. Classical ML AlgorithmsFree lesson

Pruning: Pre-pruning vs Post-pruning

Comparing early stopping during tree construction with pruning back a fully-grown tree to prevent overfitting.

Pruning: Pre-pruning vs Post-pruning

What you'll learn: How to prevent decision tree overfitting by stopping growth early (pre-pruning) or cutting back a fully-grown tree (post-pruning).

The Core Idea

You've seen how decision trees can overfit by growing too deep and memorizing training data. Pruning is the solution—but you have two strategic options for when to apply it.

Pre-pruning (also called "early stopping") means you stop the recursive partitioning process before the tree is fully grown. You set stopping criteria—like maximum depth, minimum samples per node, or minimum information gain—and halt splitting when any criterion is met. Think of it like a gardener deciding not to let branches grow in the first place.

Post-pruning takes the opposite approach: let the tree grow to full depth, then systematically remove branches that don't improve performance on validation data. You start with a complete (likely overfit) tree, then prune it back by collapsing nodes whose removal reduces validation error or increases a cost-complexity measure. This is like sculpting—you start with excess material and carve away what's unnecessary.

Trade-offs

Pre-pruning is computationally efficient (less tree to build) but risks underfitting—you might stop too early and miss important patterns deeper in the tree.

Post-pruning is more computationally expensive but often produces better results because it makes decisions with full information about the tree's structure. You can see which branches actually hurt generalization rather than guessing in advance.

Many practitioners prefer post-pruning for better accuracy, while pre-pruning is favored when computational resources are limited or when trees are used in ensemble methods where individual tree quality matters less.

Key Takeaway: Pre-pruning prevents overgrowth by setting early stopping rules, while post-pruning grows the full tree first then cuts back—trading computational cost for typically better generalization.