Stopping Criteria and Tree Depth
You'll learn how decision trees decide when to stop growing and why limiting depth prevents overfitting.
The Problem with Endless Growth
In theory, a decision tree could continue splitting until every training example sits in its own leaf node. This would memorize the training data perfectly but catastrophically overfit—imagine a map so detailed it's 1:1 scale with reality, completely useless for navigation.
Core Stopping Conditions
Decision trees use termination criteria to halt the recursive partitioning algorithm. Common stopping rules include:
Maximum Depth
Limits how many splits can occur from root to leaf. A max_depth=3 tree can only ask three sequential questions. This is your primary defense against overfitting—deeper trees capture more training noise.
Minimum Samples per Leaf
Refuses to create a leaf with fewer than min_samples_leaf examples. If a split would create a node with only 2 samples when you require 5, the algorithm stops splitting that branch.
Minimum Samples to Split
Won't attempt splitting a node unless it contains at least min_samples_split examples. This prevents tiny nodes from being divided further.
Minimum Impurity Decrease
Only accepts a split if it reduces impurity (entropy or Gini) by at least a threshold amount. Weak splits that barely improve classification get rejected.
Why This Matters
Without stopping criteria, trees grow until they perfectly classify training data (zero impurity at every leaf), learning irrelevant patterns. With proper limits, trees generalize better to unseen test data—balancing the bias-variance tradeoff you learned earlier.
Key Takeaway: Stopping criteria like maximum depth, minimum samples per leaf, and minimum impurity decrease control tree complexity, preventing overfitting by forcing the algorithm to stop before memorizing training data noise.