Course contentsShow
← Machine Learning and Deep Learning
Lesson 194 of 3,5383. Linear RegressionFree lesson

Implementing Simple Linear Regression from Scratch

Writing Python code to compute slope and intercept using the closed-form solution on toy data.

Implementing Simple Linear Regression from Scratch

You'll learn how to translate the closed-form solution into working Python code that finds the best-fit line for your data.

The Core Idea

You've learned the math behind the closed-form solution (Normal Equation) for linear regression. Now it's time to actually implement it! You'll write Python functions that take toy data and compute the optimal slope (m) and intercept (b) using the formulas you've studied.

Remember, the closed-form solution gives us:

  • Slope (m): m = Σ((x - x̄)(y - ȳ)) / Σ((x - x̄)²)
  • Intercept (b): b = ȳ - m * x̄

where x̄ and ȳ are the means of x and y.

Writing the Code

Here's a simple implementation using NumPy:

import numpy as np

def simple_linear_regression(x, y):
    """Compute slope and intercept using closed-form solution."""
    x_mean = np.mean(x)
    y_mean = np.mean(y)
    
    # Calculate slope
    numerator = np.sum((x - x_mean) * (y - y_mean))
    denominator = np.sum((x - x_mean) ** 2)
    m = numerator / denominator
    
    # Calculate intercept
    b = y_mean - m * x_mean
    
    return m, b

# Toy data: hours studied vs test score
x = np.array([1, 2, 3, 4, 5])
y = np.array([2, 4, 5, 4, 6])

m, b = simple_linear_regression(x, y)
print(f"Slope: {m:.2f}, Intercept: {b:.2f}")

This code directly implements the mathematical formulas, computing means first, then the slope, and finally the intercept.

Key Takeaway: Implementing linear regression from scratch demystifies the algorithm—it's just computing means and applying two simple formulas to find the line that best fits your data.