Course contentsShow
← Machine Learning and Deep Learning
Lesson 285 of 3,5386. Tree-Based MethodsFree lesson

Decision Tree Fundamentals and Intuition

What decision trees are, how they partition feature space recursively, and why they're interpretable models.

Decision Tree Fundamentals and Intuition

What you'll learn: How decision trees make predictions by recursively splitting data based on simple yes/no questions about features.

What is a Decision Tree?

A decision tree is a flowchart-like model that makes predictions by asking a series of questions about your data. Imagine diagnosing why your car won't start: "Does it have gas?" → If yes: "Does the battery work?" → If yes: "Is the starter motor broken?" Each question narrows down the possibilities until you reach a conclusion.

In machine learning, decision trees work the same way but with data features. For classification, they predict a category (like "spam" or "not spam"). For regression, they predict a number (like house price).

How Decision Trees Partition Feature Space

Decision trees recursively partition the feature space into rectangular regions:

  1. Start with all data in one region
  2. Choose a feature and threshold (e.g., "Is age > 30?")
  3. Split data into two groups based on the answer
  4. Repeat for each group until stopping criteria are met (depth limit, minimum samples, or purity)

Each split creates a decision boundary parallel to a feature axis. Unlike logistic regression's diagonal lines, trees create axis-aligned boxes. Think of dividing a city map with only horizontal and vertical streets—no diagonal cuts allowed.

Why Decision Trees Are Interpretable

Decision trees are human-readable. You can literally trace the path from root to leaf and explain exactly why the model made a prediction: "This email is spam because it contains 'free' more than 5 times AND has no recipient name."

This transparency makes them valuable in domains requiring explainability, like medicine or finance.

Key Takeaway: Decision trees recursively split feature space using simple threshold questions, creating interpretable, axis-aligned decision boundaries that naturally handle both classification and regression tasks.