Course contentsShow
← Computer Science
Lesson 2841 of 2,87263. Classical ML AlgorithmsFree lesson

The Splitting Process

Understanding how a decision tree recursively partitions the feature space by selecting optimal split points.

The Splitting Process

You'll learn: How decision trees recursively divide data by choosing the best questions to ask at each step.

Core Idea

At the heart of every decision tree is the splitting process—the method by which the algorithm decides how to partition your data at each node. Think of it like playing "20 Questions": at each turn, you want to ask the single best question that divides the remaining possibilities most effectively.

When a decision tree examines a node, it considers all possible ways to split the data based on available features. For a feature like "age," it might test thresholds: "Is age ≤ 30?" For categorical features like "color," it might ask: "Is color = red?"

How It Works

The algorithm evaluates every candidate split by measuring how "pure" the resulting child nodes would be. A pure node contains only one class (for classification) or similar values (for regression). The split that produces the greatest improvement in purity wins and becomes the decision rule at that node.

This process then recurses: each child node becomes a new parent, and the algorithm repeats the splitting process on the subset of data that reached that node. This continues until a stopping condition is met—like reaching maximum depth, having too few samples to split, or achieving perfect purity.

Example

Imagine classifying emails as spam/not-spam. At the root, you might split on "contains 'free'"—emails with that word go left, others right. The left child might then split on "has multiple exclamation marks," further refining the spam detection.

Key Takeaway: The splitting process recursively partitions the feature space by evaluating all possible splits at each node and choosing the one that best increases purity, building the tree from root to leaves.