BiteSizedChunks.comLearn one small thing at a time.

Course contentsShow

Machine Learning and Deep Learning

285Decision Tree Fundamentals and Intuition
286Splitting Criteria: Information Gain and Entropy
287Gini Impurity as a Splitting Criterion
288Regression Trees and Variance Reduction
289The CART Algorithm
290Tree Pruning: Cost-Complexity Pruning
291Hyperparameters: Max Depth, Min Samples Split, Min Samples Leaf
292Feature Importance from Decision Trees
293Handling Categorical Features in Trees
294Decision Boundaries and Axis-Aligned Splits
295Advantages and Limitations of Decision Trees
296Implementing Decision Trees in Scikit-learn
297Ensemble Learning: The Wisdom of Crowds
298Bootstrap Aggregating (Bagging) Fundamentals
299Out-of-Bag Error Estimation
300Random Forests: Adding Feature Randomness
301The sqrt(p) and log2(p) Rules
302Feature Importance from Random Forests
303Hyperparameters in Random Forests
304Extremely Randomized Trees (Extra Trees)
305Bagging for Other Base Learners
306Random Forests in Practice with Scikit-learn
307Boosting Fundamentals: Ensemble by Sequential Learning
308AdaBoost: Adaptive Boosting Algorithm
309AdaBoost Weight Updates and Sample Reweighting
310Implementing AdaBoost with Decision Stumps
311Gradient Boosting: Framework and Intuition
312Gradient Boosting for Regression
313Gradient Boosting for Classification
314Learning Rate and Shrinkage in Boosting
315XGBoost: Extreme Gradient Boosting
316XGBoost Hyperparameters and Tuning
317LightGBM: Gradient Boosting with Histogram-Based Splitting
318CatBoost: Handling Categorical Features Natively
319Early Stopping and Monitoring in Boosting
320Comparing Boosting Libraries: XGBoost vs LightGBM vs CatBoost

Machine Learning and Deep Learning

285Decision Tree Fundamentals and Intuition
286Splitting Criteria: Information Gain and Entropy
287Gini Impurity as a Splitting Criterion
288Regression Trees and Variance Reduction
289The CART Algorithm
290Tree Pruning: Cost-Complexity Pruning
291Hyperparameters: Max Depth, Min Samples Split, Min Samples Leaf
292Feature Importance from Decision Trees
293Handling Categorical Features in Trees
294Decision Boundaries and Axis-Aligned Splits
295Advantages and Limitations of Decision Trees
296Implementing Decision Trees in Scikit-learn
297Ensemble Learning: The Wisdom of Crowds
298Bootstrap Aggregating (Bagging) Fundamentals
299Out-of-Bag Error Estimation
300Random Forests: Adding Feature Randomness
301The sqrt(p) and log2(p) Rules
302Feature Importance from Random Forests
303Hyperparameters in Random Forests
304Extremely Randomized Trees (Extra Trees)
305Bagging for Other Base Learners
306Random Forests in Practice with Scikit-learn
307Boosting Fundamentals: Ensemble by Sequential Learning
308AdaBoost: Adaptive Boosting Algorithm
309AdaBoost Weight Updates and Sample Reweighting
310Implementing AdaBoost with Decision Stumps
311Gradient Boosting: Framework and Intuition
312Gradient Boosting for Regression
313Gradient Boosting for Classification
314Learning Rate and Shrinkage in Boosting
315XGBoost: Extreme Gradient Boosting
316XGBoost Hyperparameters and Tuning
317LightGBM: Gradient Boosting with Histogram-Based Splitting
318CatBoost: Handling Categorical Features Natively
319Early Stopping and Monitoring in Boosting
320Comparing Boosting Libraries: XGBoost vs LightGBM vs CatBoost

← Machine Learning and Deep Learning

Lesson 286 of 3,538·6. Tree-Based MethodsPro lesson

Splitting Criteria: Information Gain and Entropy

How entropy measures impurity and information gain determines the best feature to split on at each node.

This lesson is for subscribers

You've completed the free preview. Subscribe to unlock every lesson in every course.

See pricing Back to course