Implementing Decision Trees in Scikit-learn
What you'll learn: How to build, train, and use decision tree models for both classification and regression tasks using Scikit-learn's ready-made classes.
The Core Idea
Now that you understand how decision trees work conceptually—splitting data based on features, using criteria like Gini impurity or entropy—it's time to put that knowledge into practice. Scikit-learn provides two main classes: DecisionTreeClassifier for predicting categories (like "spam" or "not spam") and DecisionTreeRegressor for predicting continuous values (like house prices).
Using DecisionTreeClassifier
For classification problems, you follow the standard Scikit-learn workflow:
from sklearn.tree import DecisionTreeClassifier
# Create the model with hyperparameters
clf = DecisionTreeClassifier(max_depth=3, min_samples_split=10)
# Fit the tree to your training data
clf.fit(X_train, y_train)
# Make predictions
predictions = clf.predict(X_test)
Using DecisionTreeRegressor
For regression tasks, the process is nearly identical:
from sklearn.tree import DecisionTreeRegressor
# Create and train
reg = DecisionTreeRegressor(max_depth=5, min_samples_leaf=5)
reg.fit(X_train, y_train)
# Predict continuous values
predictions = reg.predict(X_test)
Visualizing Your Tree
Scikit-learn lets you visualize the trained tree structure:
from sklearn.tree import plot_tree
import matplotlib.pyplot as plt
plt.figure(figsize=(12, 8))
plot_tree(clf, filled=True, feature_names=feature_names)
plt.show()
This shows each split, the splitting criterion value, and the resulting subsets—making your model interpretable and debuggable.
Key Takeaway: Scikit-learn's DecisionTreeClassifier and DecisionTreeRegressor let you build tree models with just fit() and predict(), while hyperparameters like max_depth and min_samples_split control tree complexity directly.