Avoiding Cherry-Picking Results
You'll learn why presenting all findings—including null and contradictory results—is essential for ethical data science.
Cherry-picking results means selectively highlighting only the findings that support your hypothesis or desired conclusion while hiding or downplaying inconvenient null results, contradictory evidence, or failed experiments. It's the data science equivalent of a prosecutor presenting only evidence that convicts while hiding exculpatory facts.
Why This Matters
When you've run twenty statistical tests and only one shows significance at p<0.05, reporting just that one result is misleading—it's likely a false positive. When Model A outperforms Model B on accuracy but underperforms on fairness metrics, showing only accuracy paints an incomplete picture. This practice corrupts the scientific record, misleads stakeholders, and can cause real-world harm when decisions are based on incomplete evidence.
The Ethical Imperative
Building on your understanding of communicating uncertainty honestly, avoiding cherry-picking means:
- Report all preregistered analyses, not just "successful" ones
- Include null results that show no effect or relationship
- Present contradictory evidence that challenges your hypothesis
- Disclose failed model iterations and why they didn't work
- Show sensitivity analyses that reveal how fragile findings are
Think of it like a medical trial: if a drug works for one symptom but worsens three others, hiding those side effects isn't just unethical—it's dangerous.
In Practice
This connects to your work on separating analysis from advocacy. Your role is to illuminate truth comprehensively, not build a case for a predetermined conclusion. Document negative findings in technical reports, include them in presentations, and explain their implications alongside positive results.
Key Takeaway: Comprehensive, honest reporting of all results—favorable and unfavorable—builds trust, prevents costly mistakes, and upholds the integrity of data science as a discipline.