Course contentsShow
← Machine Learning and Deep Learning
Lesson 291 of 3,5386. Tree-Based MethodsFree lesson

Hyperparameters: Max Depth, Min Samples Split, Min Samples Leaf

Controlling tree complexity through depth limits, minimum samples for splits, and leaf size constraints.

Hyperparameters: Max Depth, Min Samples Split, Min Samples Leaf

What You'll Learn

You'll discover how three critical hyperparameters act as "brakes" to prevent decision trees from growing too complex and overfitting your data.

Why Control Tree Growth?

Remember from earlier lessons that unconstrained trees can overfit—they memorize training data by growing branches for every tiny pattern. These three hyperparameters give you precise control over when to stop splitting nodes.

The Three Key Constraints

Max Depth

This limits how many levels deep your tree can grow. Think of it as setting a maximum number of questions you can ask before making a final decision.

  • max_depth=3 means the tree stops after 3 levels of splits
  • Shallow trees (small depth) = simpler models, less overfitting
  • Deep trees = more complex patterns captured, higher overfitting risk

Min Samples Split

This sets the minimum number of samples required to consider splitting a node further.

  • min_samples_split=20 means a node needs at least 20 samples to be eligible for splitting
  • Higher values prevent splits on small subsets, creating simpler trees
  • Analogy: Don't make business decisions based on just 2 customer opinions

Min Samples Leaf

This enforces the minimum number of samples that must end up in each leaf node after a split.

  • min_samples_leaf=10 means every final decision must represent at least 10 samples
  • Prevents creating leaves with just 1-2 samples (classic overfitting sign)
  • Ensures your predictions are based on sufficient evidence

How They Work Together

These hyperparameters often interact. Even if max_depth allows more growth, min_samples_split might stop it earlier. You'll tune them together using validation data to find the sweet spot between underfitting (too simple) and overfitting (too complex).

Key Takeaway: Max depth, min samples split, and min samples leaf are complementary "stopping rules" that constrain tree complexity, helping you balance model expressiveness with generalization ability.