Course contentsShow
← Computer Science
Lesson 2839 of 2,87263. Classical ML AlgorithmsFree lesson

Classification vs Regression Trees

Distinguishing between decision trees for categorical outcomes (classification) and continuous outcomes (regression).

Classification vs Regression Trees

What you'll learn: How decision trees adapt their behavior and evaluation criteria based on whether you're predicting categories or continuous numbers.

The Core Distinction

You've already learned that decision trees split data recursively to make predictions. But the type of problem fundamentally changes how the tree operates internally.

Classification Trees predict discrete categories (like "spam/not spam" or "species A/B/C"). At each leaf node, they output the most common class among training examples that landed there. To decide splits, they use measures like Gini impurity or entropy — metrics that quantify how "mixed" the classes are in a node.

Regression Trees predict continuous values (like house prices or temperatures). Leaf nodes output the mean (average) of all training values in that region. Splits are chosen to minimize variance or mean squared error within child nodes — finding divisions that create homogeneous numeric groups.

Why This Matters

Think of it like sorting mail versus weighing packages:

  • Classification: Sort letters into bins (categories). Success = "all letters in each bin go to the same destination"
  • Regression: Organize packages by weight ranges. Success = "packages in each group have similar weights"

Different goals require different measuring sticks. Classification cares about purity of categories, while regression cares about minimizing numeric spread.

Practical Implications

  • Splitting criteria differ: Classification uses information gain; regression uses variance reduction
  • Leaf predictions differ: Mode (most frequent) vs mean (average)
  • Evaluation metrics differ: Accuracy, precision for classification; MSE, R² for regression

Both use the same recursive tree structure you learned earlier — only the internal mechanics adapt to the prediction target.

Key Takeaway: Classification trees and regression trees share the same hierarchical structure but use different splitting criteria and leaf predictions tailored to categorical versus continuous outcomes.