Course contentsShow
← Machine Learning and Deep Learning
Lesson 294 of 3,5386. Tree-Based MethodsFree lesson

Decision Boundaries and Axis-Aligned Splits

Understanding that trees create axis-aligned rectangular regions and their limitations with diagonal boundaries.

Decision Boundaries and Axis-Aligned Splits

What you'll learn: How decision trees create rectangular decision regions and why they struggle with diagonal patterns.

The Rectangular Nature of Tree Splits

Unlike logistic regression or SVMs that can create smooth, angled decision boundaries, decision trees make splits that are perfectly aligned with the feature axes. Each split asks a simple question: "Is feature X greater than value Y?" This creates vertical or horizontal cuts through your feature space, never diagonal ones.

Think of it like dividing a room using only walls that run perfectly north-south or east-west. You can never build a wall at a 45-degree angle.

Visualizing Axis-Aligned Regions

When a decision tree learns from data with two features (X₁ and X₂), every split creates a rectangular partition:

  • First split: "Is X₁ < 5?" divides space with a vertical line
  • Second split: "Is X₂ < 3?" divides one region with a horizontal line
  • Each subsequent split further subdivides existing rectangles

The final decision boundary looks like a staircase or grid of rectangles, not a smooth curve.

The Diagonal Boundary Problem

This axis-aligned constraint creates a fundamental limitation. Suppose your data is perfectly separated by a diagonal line (like X₂ = X₁). A decision tree would need many small rectangular steps to approximate that diagonal, resulting in:

  • A deeper, more complex tree
  • Potential overfitting
  • Less interpretable structure

Other algorithms like SVMs with kernels or logistic regression with engineered features can capture diagonal boundaries naturally with a single smooth line.

Key Takeaway: Decision trees split data using axis-aligned (horizontal/vertical) cuts only, creating rectangular decision regions that may require many splits to approximate diagonal or curved patterns that other models handle more efficiently.