Course contentsShow
← Data Science
Lesson 660 of 2,14517. Categorical Predictors and InteractionsFree lesson

Choosing the Polynomial Degree

Learn strategies for selecting appropriate polynomial degrees using visual inspection, statistical criteria, and cross-validation.

Choosing the Polynomial Degree

You'll learn: How to select the right polynomial degree to balance model flexibility and reliability.

The Goldilocks Problem

After creating polynomial terms (degree 2, 3, 4, etc.), you face a crucial decision: which degree fits "just right"? Too low, and your model misses important curvature in the data (underfitting). Too high, and it memorizes noise, creating wild swings between data points (overfitting).

Three Strategies for Selection

1. Visual Inspection Plot your fitted polynomial curves against the actual data. A degree-2 curve might look smooth but miss a key bend. A degree-7 might wiggle erratically. Look for the simplest curve that captures the pattern without unnecessary complexity.

2. Statistical Criteria Use AIC or BIC to compare models with different degrees. Lower values indicate better balance between fit and complexity. BIC penalizes complexity more heavily than AIC, often favoring simpler models. Calculate these for degrees 1 through 5 (for example) and choose the minimum.

3. Cross-Validation Split your data and test how well each polynomial degree predicts unseen observations. The degree with lowest cross-validation error generalizes best. This is often the gold standard because it directly tests predictive performance rather than just fit to training data.

Practical Workflow

Start simple (degree 1 or 2), then gradually increase. At each step, check if adjusted R-squared improves meaningfully and if residual plots show better patterns. Stop when additional complexity doesn't justify the gains—remember parsimony and Occam's Razor.

Key Takeaway: Select polynomial degree by combining visual assessment of fitted curves, statistical criteria like AIC/BIC, and cross-validation error—choosing the simplest model that adequately captures the data's true curvature.