Decision Tree Fundamentals and Intuition
What you'll learn: How decision trees make predictions by recursively splitting data based on simple yes/no questions about features.
What is a Decision Tree?
A decision tree is a flowchart-like model that makes predictions by asking a series of questions about your data. Imagine diagnosing why your car won't start: "Does it have gas?" → If yes: "Does the battery work?" → If yes: "Is the starter motor broken?" Each question narrows down the possibilities until you reach a conclusion.
In machine learning, decision trees work the same way but with data features. For classification, they predict a category (like "spam" or "not spam"). For regression, they predict a number (like house price).
How Decision Trees Partition Feature Space
Decision trees recursively partition the feature space into rectangular regions:
- Start with all data in one region
- Choose a feature and threshold (e.g., "Is age > 30?")
- Split data into two groups based on the answer
- Repeat for each group until stopping criteria are met (depth limit, minimum samples, or purity)
Each split creates a decision boundary parallel to a feature axis. Unlike logistic regression's diagonal lines, trees create axis-aligned boxes. Think of dividing a city map with only horizontal and vertical streets—no diagonal cuts allowed.
Why Decision Trees Are Interpretable
Decision trees are human-readable. You can literally trace the path from root to leaf and explain exactly why the model made a prediction: "This email is spam because it contains 'free' more than 5 times AND has no recipient name."
This transparency makes them valuable in domains requiring explainability, like medicine or finance.
Key Takeaway: Decision trees recursively split feature space using simple threshold questions, creating interpretable, axis-aligned decision boundaries that naturally handle both classification and regression tasks.