Course contentsShow
← Machine Learning and Deep Learning
Lesson 301 of 3,5386. Tree-Based MethodsFree lesson

The sqrt(p) and log2(p) Rules

Why random forests typically consider sqrt(p) features for classification and p/3 for regression at each split.

The sqrt(p) and log2(p) Rules

What you'll learn: Why random forests use specific formulas to decide how many features to consider at each split.

The Core Idea

When building each tree in a random forest, we don't consider all features at every split—that would defeat the purpose of adding randomness. But how many features should we consider? Machine learning practitioners have discovered two simple rules of thumb that work remarkably well:

  • For classification: Consider sqrt(p) features (the square root of total features)
  • For regression: Consider p/3 features (one-third of total features)

Where p is the total number of features in your dataset.

Why These Numbers?

The Balance: These rules strike a sweet spot between diversity and predictive power.

  • Too few features → Trees become too random, like guessing blindly
  • Too many features → Trees become too similar, losing the "wisdom of crowds" benefit of ensembles

The Intuition: Think of it like a committee making decisions. If everyone considers all the same information, they'll all reach similar conclusions (high correlation between trees). If each person only looks at 1-2 random facts, their conclusions are useless. But if each expert examines a reasonable subset of evidence, you get diverse yet informed opinions.

Why the difference? Classification problems typically have clearer signal in fewer features (hence the smaller sqrt(p)), while regression problems benefit from considering more features to capture subtle numerical relationships (hence the larger p/3).

Example

If you have 100 features:

  • Classification: consider 10 features per split (√100 = 10)
  • Regression: consider 33 features per split (100/3 ≈ 33)

Key Takeaway: Random forests use sqrt(p) for classification and p/3 for regression as default rules because they balance tree diversity with predictive strength, preventing trees from being too similar or too random.