Course contentsShow
← Machine Learning and Deep Learning
Lesson 287 of 3,5386. Tree-Based MethodsFree lesson

Gini Impurity as a Splitting Criterion

Understanding Gini impurity, how it differs from entropy, and when to prefer it for classification trees.

Gini Impurity as a Splitting Criterion

What You'll Learn

You'll discover an alternative to entropy for measuring node purity in decision trees, and learn when to use Gini impurity over entropy.

What is Gini Impurity?

Gini impurity measures the probability of incorrectly classifying a randomly chosen element if you labeled it according to the class distribution at a node. Think of it as asking: "If I blindly picked a sample from this node and randomly guessed its class based on the node's proportions, how often would I be wrong?"

For a node with classes, Gini impurity is calculated as:

Gini = 1 - Σ(p_i²)

where p_i is the proportion of samples belonging to class i.

Gini vs Entropy

Both measure impurity, but they differ subtly:

  • Entropy (from your last lesson) uses logarithms and measures information content
  • Gini impurity uses squared probabilities and is computationally simpler
  • Both reach minimum (0) when a node is pure (all one class)
  • Both reach maximum when classes are evenly distributed

Analogy: If entropy asks "How surprised am I?", Gini asks "How often would I guess wrong?"

When to Prefer Gini

  • Computational efficiency: Gini is faster (no logarithms)
  • Practical performance: Both usually produce similar trees
  • Default choice: Scikit-learn uses Gini by default for classification trees
  • Slightly different penalization: Gini tends to isolate the most frequent class, while entropy creates more balanced splits

Most practitioners stick with Gini unless they have specific reasons to use entropy.

Key Takeaway: Gini impurity provides a faster, simpler alternative to entropy for measuring node purity in decision trees, making it the practical default for classification tasks.