Course contentsShow
← Computer Science
Lesson 2844 of 2,87263. Classical ML AlgorithmsFree lesson

Impurity Measures: Gini Index

Understanding the Gini impurity measure as an alternative to entropy for evaluating classification splits.

Impurity Measures: Gini Index

What you'll learn: You'll discover how the Gini index provides a faster, simpler alternative to entropy for measuring node impurity in decision trees.

What Is the Gini Index?

The Gini index (or Gini impurity) measures how often a randomly chosen element from a set would be incorrectly classified if it were randomly labeled according to the distribution of labels in that set. Like entropy, it quantifies the "messiness" or impurity of a node, but uses a simpler mathematical formula.

The Formula

For a node with classes, the Gini index is calculated as:

Gini = 1 - Σ(p_i)²

where p_i is the probability of class i in that node.

How It Works

Imagine a bag of colored marbles. If all marbles are the same color, Gini = 0 (perfectly pure). If they're evenly mixed, Gini is higher. The Gini index ranges from 0 (pure) to 0.5 (maximum impurity for binary classification).

Example: A node with 70 "Yes" and 30 "No" samples:

  • Gini = 1 - (0.7² + 0.3²) = 1 - (0.49 + 0.09) = 0.42

Gini vs Entropy

Both measure impurity, but:

  • Gini is computationally cheaper (no logarithms)
  • Entropy penalizes impurity slightly more strongly
  • In practice, they usually produce similar trees
  • Gini is the default in many implementations because it's faster

Think of them as two thermometers that read the same temperature using different scales—both tell you how "mixed" your data is.

Key Takeaway: The Gini index offers a faster, simpler alternative to entropy for measuring node impurity, using squared probabilities instead of logarithms while producing comparable decision tree splits.