Polynomial and Interaction Features
What you'll learn: How to capture nonlinear relationships and feature interactions by transforming your original features into polynomial terms and combinations.
The Core Idea
Linear regression assumes relationships are straight lines, but real-world data rarely cooperates. When y doesn't change proportionally with x, you need more flexibility. Polynomial features let you fit curves by adding powers of features (like x², x³), while interaction features capture how two features work together (like x₁ × x₂).
Think of it like cooking: sometimes individual ingredients (features) matter, but often the combination creates the flavor (prediction). Salt alone and sugar alone taste different than salt × sugar together.
How It Works
Polynomial features transform a single feature into multiple versions:
- Original:
x - Degree 2:
x, x² - Degree 3:
x, x², x³
Interaction features multiply features together:
- Original:
x₁, x₂ - With interactions:
x₁, x₂, x₁ × x₂
You can combine both! For x₁, x₂ with degree 2 polynomials, you get: x₁, x₂, x₁², x₁ × x₂, x₂²
These transformed features feed into the same linear regression you already know. You're not changing the algorithm—just enriching the input data so linear regression can fit nonlinear patterns.
Why This Matters
Remember from "Feature Scaling for Multiple Regression" that feature magnitudes matter? Polynomial features make this even more critical—x² can be vastly larger than x. Always scale your data before creating polynomial features.
Key Takeaway: Polynomial and interaction features let linear regression model curves and feature combinations by transforming the input space, turning nonlinear problems into linear ones through clever feature engineering.