Properties of Least Squares Estimators
What you'll learn: Why the least squares method produces estimators that are considered "optimal" and trustworthy for linear regression.
Why These Properties Matter
You've learned how to compute the slope (β₁) and intercept (β₀) using the least squares criterion. But why is this method the standard? It turns out that least squares estimators have special mathematical properties that make them the gold standard for linear regression.
Three Key Properties
1. Unbiasedness
The least squares estimators are unbiased, meaning that on average, across many samples, they hit the true population values. Think of it like a dart player who doesn't consistently aim too high or too low—their throws center around the bullseye.
If you could draw infinite samples and calculate β₁ each time, the average of all those estimates would equal the true population slope.
2. Minimum Variance (Efficiency)
Among all unbiased estimators, least squares has the smallest variance. This means it's the most precise—it gives you estimates that cluster tightly around the true value. Returning to our dart analogy: not only does the player aim correctly on average, but their throws are also tightly grouped.
3. BLUE: Best Linear Unbiased Estimator
These two properties combined earn OLS (Ordinary Least Squares) the title BLUE—the Best Linear Unbiased Estimator. Under standard assumptions (linearity, independence, constant variance, normality), no other linear method beats it for reliability and precision.
Why This Makes OLS the Standard
Because OLS estimators are both unbiased and have minimum variance, they give you the most trustworthy predictions possible with linear methods. This mathematical optimality is why nearly every regression package defaults to least squares.
Key Takeaway: Least squares estimators are unbiased (correct on average) and have minimum variance (most precise), making them the BLUE—the optimal choice for linear regression under standard assumptions.