Extracting Residuals and Fitted Values in Python
You'll learn how to pull fitted values and residuals from your regression models using Python's statsmodels and scikit-learn libraries.
Why Extract These Values?
Once you've fitted a linear regression model, you need access to the fitted (predicted) values and residuals to perform diagnostics, check assumptions, and understand model performance. Both statsmodels and scikit-learn make this straightforward, though with slightly different syntax.
Using statsmodels
After fitting a model with statsmodels.api.OLS(), the results object gives you direct access:
import statsmodels.api as sm
import numpy as np
# Example data
X = np.array([1, 2, 3, 4, 5])
y = np.array([2, 4, 5, 4, 5])
X = sm.add_constant(X) # adds intercept
# Fit model
model = sm.OLS(y, X)
results = model.fit()
# Extract values
fitted_values = results.fittedvalues
residuals = results.resid
The .fittedvalues attribute contains ŷ (y-hat) predictions, while .resid contains the residuals (y - ŷ).
Using scikit-learn
With scikit-learn, you compute these manually after fitting:
from sklearn.linear_model import LinearRegression
X = np.array([[1], [2], [3], [4], [5]])
y = np.array([2, 4, 5, 4, 5])
model = LinearRegression()
model.fit(X, y)
# Extract values
fitted_values = model.predict(X)
residuals = y - fitted_values
Scikit-learn doesn't store residuals automatically—you calculate them by subtracting predictions from actual values.
Which Should You Use?
Use statsmodels when you need statistical details (p-values, confidence intervals). Use scikit-learn for prediction-focused workflows and machine learning pipelines.
Key Takeaway: Both statsmodels and scikit-learn let you extract fitted values and residuals, but statsmodels provides them as attributes while scikit-learn requires you to compute residuals manually using predictions.