Implementing Simple Linear Regression from Scratch
You'll learn how to translate the closed-form solution into working Python code that finds the best-fit line for your data.
The Core Idea
You've learned the math behind the closed-form solution (Normal Equation) for linear regression. Now it's time to actually implement it! You'll write Python functions that take toy data and compute the optimal slope (m) and intercept (b) using the formulas you've studied.
Remember, the closed-form solution gives us:
- Slope (m):
m = Σ((x - x̄)(y - ȳ)) / Σ((x - x̄)²) - Intercept (b):
b = ȳ - m * x̄
where x̄ and ȳ are the means of x and y.
Writing the Code
Here's a simple implementation using NumPy:
import numpy as np
def simple_linear_regression(x, y):
"""Compute slope and intercept using closed-form solution."""
x_mean = np.mean(x)
y_mean = np.mean(y)
# Calculate slope
numerator = np.sum((x - x_mean) * (y - y_mean))
denominator = np.sum((x - x_mean) ** 2)
m = numerator / denominator
# Calculate intercept
b = y_mean - m * x_mean
return m, b
# Toy data: hours studied vs test score
x = np.array([1, 2, 3, 4, 5])
y = np.array([2, 4, 5, 4, 6])
m, b = simple_linear_regression(x, y)
print(f"Slope: {m:.2f}, Intercept: {b:.2f}")
This code directly implements the mathematical formulas, computing means first, then the slope, and finally the intercept.
Key Takeaway: Implementing linear regression from scratch demystifies the algorithm—it's just computing means and applying two simple formulas to find the line that best fits your data.