Course contentsShow
← Data Science
Lesson 589 of 2,14515. Regression DiagnosticsFree lesson

Deciding Whether to Remove Outliers

Develop decision criteria for handling outliers: investigate cause, assess impact, and consider domain knowledge before removal.

Deciding Whether to Remove Outliers

What you'll learn: You'll learn a structured approach to decide whether an outlier should be removed from your regression model—by investigating its cause, assessing its impact, and consulting domain knowledge.

Why This Decision Matters

Just because you've identified an outlier using standardized or studentized residuals doesn't mean you should automatically delete it. Removing data points can introduce bias, but keeping bad data can wreck your model. You need a principled decision framework.

The Three-Step Decision Framework

1. Investigate the Cause

Ask: Why is this point an outlier?

  • Is it a data entry error? (e.g., someone typed 150 instead of 15)
  • Is it a measurement error? (faulty sensor, transcription mistake)
  • Is it a legitimate extreme value? (a rare but real observation)

If you find a clear error, removal is justified—document it.

2. Assess the Impact

Use the tools you've learned:

  • Check Cook's Distance or DFFITS—does this point drastically change your regression line?
  • Fit the model with and without the outlier—compare coefficients and R-squared.

High influence + no good reason to trust the data = strong candidate for removal.

3. Consider Domain Knowledge

This is critical. Someone who understands the subject matter can tell you:

  • "That value is impossible given the physical constraints"
  • "Extreme values like this happen occasionally and are meaningful"

Analogy: Think of yourself as a detective, not a janitor. Don't just sweep outliers away—investigate whether they're criminals (errors) or eccentric but honest citizens (real extremes).

When in Doubt

Report results both ways: with and without the outlier. Transparency builds trust.

Key Takeaway: Never remove outliers automatically—investigate their cause, measure their influence, and rely on domain expertise to make an informed, documented decision.