What to Do with Influential Points
You'll learn: A practical decision framework for handling observations that disproportionately affect your regression model.
The Decision Framework
Finding influential points is just the beginning—what you do next matters more. You've identified them using Cook's Distance, DFFITS, or leverage metrics, but deleting data arbitrarily destroys scientific integrity. Instead, follow this systematic approach:
1. Investigate First
Always examine why a point is influential. Is it a data entry error (typing 1000 instead of 100)? A measurement failure? Or a legitimate but rare case? Check source records, cross-reference with other data, and consult domain experts.
2. Run Robustness Checks
Fit your model both with and without the influential point(s). If conclusions remain substantially the same, the point isn't problematic—just unusual. If your interpretation completely flips, you need caution.
3. Consider Transformation
Sometimes influence stems from scale issues. Log-transforming skewed variables or applying square-root transformations can reduce extreme leverage while preserving the data point.
4. Justified Removal (Last Resort)
Remove points only when you can document a legitimate reason: equipment malfunction, wrong population (e.g., adult data in a child study), or confirmed recording error. Never remove points simply because they're inconvenient.
The Golden Rule
Think of influential points like medical outliers—a patient with unusual symptoms might have a data error in their chart, or they might represent a rare condition that teaches you something important. You wouldn't ignore the patient; you'd investigate thoroughly.
Key Takeaway: Influential points demand investigation, not automatic deletion. Document your decision-making process, test model robustness, and only remove observations with clear, defensible justification—transparency is essential for reproducible science.