Course contentsShow
← Data Science
Lesson 663 of 2,14517. Categorical Predictors and InteractionsFree lesson

Common Pitfalls with Polynomial Features

Recognize issues like overfitting at boundaries, extrapolation dangers, and when simpler transformations suffice.

Common Pitfalls with Polynomial Features

What You'll Learn

You'll learn to recognize three major problems that can arise when using polynomial features: overfitting at data boundaries, extrapolation dangers, and when simpler transformations would work better.

The Three Main Traps

1. Overfitting at Boundaries

Polynomial models often behave wildly at the edges of your data. High-degree polynomials can curve sharply upward or downward near the minimum and maximum values of your predictor, creating unrealistic predictions. Think of a roller coaster that suddenly shoots skyward at the end of the track—that's your polynomial desperately trying to fit those last few data points.

2. Extrapolation Dangers

Polynomials are notoriously unreliable outside the range of your training data. While a quadratic curve might fit your data beautifully between x=0 and x=10, it could predict absurd values at x=15. Unlike linear models that simply extend their trend, polynomials can spiral off in unexpected directions. Never trust polynomial predictions beyond your observed data range.

3. Overcomplicating When Simpler Works

Sometimes you reach for a cubic polynomial when a simple log transformation would capture the relationship better. If your relationship is exponential growth, a polynomial might "chase" the curve with multiple wiggles, while log(y) would linearize it cleanly. Ask yourself: does the shape really need multiple bends, or is there a natural transformation that makes more sense?

Practical Wisdom

Before adding polynomial terms, consider:

  • Does the relationship genuinely curve back on itself?
  • Will I need to predict beyond my data range?
  • Have I tried centering my predictors?
  • Could a simpler transformation (log, square root) work instead?

Key Takeaway: Polynomial features are powerful but fragile—they overfit at boundaries, fail spectacularly during extrapolation, and are often overkill when simpler transformations would capture the true relationship more reliably.