Hyperparameters: Max Depth, Min Samples Split, Min Samples Leaf
What You'll Learn
You'll discover how three critical hyperparameters act as "brakes" to prevent decision trees from growing too complex and overfitting your data.
Why Control Tree Growth?
Remember from earlier lessons that unconstrained trees can overfit—they memorize training data by growing branches for every tiny pattern. These three hyperparameters give you precise control over when to stop splitting nodes.
The Three Key Constraints
Max Depth
This limits how many levels deep your tree can grow. Think of it as setting a maximum number of questions you can ask before making a final decision.
max_depth=3means the tree stops after 3 levels of splits- Shallow trees (small depth) = simpler models, less overfitting
- Deep trees = more complex patterns captured, higher overfitting risk
Min Samples Split
This sets the minimum number of samples required to consider splitting a node further.
min_samples_split=20means a node needs at least 20 samples to be eligible for splitting- Higher values prevent splits on small subsets, creating simpler trees
- Analogy: Don't make business decisions based on just 2 customer opinions
Min Samples Leaf
This enforces the minimum number of samples that must end up in each leaf node after a split.
min_samples_leaf=10means every final decision must represent at least 10 samples- Prevents creating leaves with just 1-2 samples (classic overfitting sign)
- Ensures your predictions are based on sufficient evidence
How They Work Together
These hyperparameters often interact. Even if max_depth allows more growth, min_samples_split might stop it earlier. You'll tune them together using validation data to find the sweet spot between underfitting (too simple) and overfitting (too complex).
Key Takeaway: Max depth, min samples split, and min samples leaf are complementary "stopping rules" that constrain tree complexity, helping you balance model expressiveness with generalization ability.