Course contentsShow
← Computer Science
Lesson 2852 of 2,87263. Classical ML AlgorithmsFree lesson

Advantages and Limitations of Decision Trees

Evaluating decision trees' interpretability and non-linearity handling against their instability and greedy nature.

Advantages and Limitations of Decision Trees

What you'll learn: You'll understand why decision trees are beloved for their clarity yet criticized for their brittleness, helping you choose when to use them.

The Bright Side: Why Decision Trees Shine

Decision trees offer unmatched interpretability. Unlike black-box models, you can literally draw out the decision-making process—perfect for explaining to non-technical stakeholders or satisfying regulatory requirements. If a loan application gets denied, you can trace exactly which features led to that decision.

They also handle non-linearity naturally. Without requiring you to manually engineer polynomial features or transformations, trees automatically capture complex, non-linear relationships by recursively splitting the feature space into regions. A tree can easily model "if age < 25 AND income > 50k, then approve" without you specifying interaction terms.

Decision trees work with mixed data types (categorical and numerical) and are robust to outliers since splits are based on ordering, not magnitude.

The Dark Side: Where Trees Stumble

Trees suffer from high instability. A tiny change in training data can produce a completely different tree structure—like a house of cards. Remove one training example, and the entire cascade of splits might reorganize.

They're inherently greedy. At each node, trees choose the locally optimal split without considering what splits might work better later. This myopic approach can miss globally better solutions. It's like choosing the shortest immediate path without checking if it leads to a dead end.

Trees also overfit easily without pruning, creating overly complex rules that memorize training noise rather than learning generalizable patterns.

Key Takeaway: Decision trees excel at interpretability and capturing non-linear patterns but struggle with instability and greedy optimization—making ensemble methods like Random Forests attractive for combining multiple trees to overcome individual weaknesses.