Assumptions of Simple Linear Regression
You'll learn the four key assumptions that must hold for linear regression to produce reliable, valid results.
What Are Model Assumptions?
When you fit a linear regression model using the least squares criterion (which you learned previously), you're making implicit promises about your data. These assumptions aren't just mathematical formalities—they're conditions that, if violated, can make your predictions unreliable and your R² scores misleading.
The Four Core Assumptions
1. Linearity
The relationship between your feature (x) and target (y) must actually be linear. If the true pattern is curved or exponential, forcing a straight line through it will give poor predictions.
2. Independence
Each observation should be independent of others. Imagine measuring temperature every second for a minute—each reading is influenced by the previous one, violating independence. This matters because the normal equation assumes each data point contributes unique information.
3. Homoscedasticity (constant variance)
The residuals (prediction errors) should have roughly the same spread across all predicted values. Think of a cone vs. a tube: if your errors grow larger as predictions increase (cone-shaped), this assumption is violated.
4. Normality
The residuals should follow a normal distribution (bell curve). This assumption is especially important for statistical inference, though less critical for pure prediction tasks.
Why This Matters
These assumptions justify using MSE as your loss function and validate the closed-form solution you've learned. When assumptions break, your coefficient estimates may still be calculated, but they won't be reliable or optimal.
Key Takeaway: Simple linear regression rests on four assumptions—linearity, independence, homoscedasticity, and normality—that determine whether your model's predictions and statistics can be trusted.