Course contentsShow
← Data Science
Lesson 1879 of 2,14544. Data Ethics and PrivacyFree lesson

Selection Bias and Sampling Bias

How non-representative samples arise from flawed data collection methods and missing populations in datasets.

Selection Bias and Sampling Bias

What you'll learn: How flawed data collection methods and missing populations create non-representative samples that skew your analysis.

The Core Problem

Selection bias occurs when the data you collect doesn't accurately represent the population you're trying to study. This happens through two main pathways: sampling bias (who gets included in your dataset) and selection mechanisms (systematic exclusions during data collection).

Think of it like surveying "typical Americans" by only calling landline phones during business hours. You've systematically excluded mobile-only users, working people, and younger demographics—your sample is biased.

Common Sources

Sampling Bias arises when:

  • Your data collection method favors certain subgroups (online surveys exclude those without internet)
  • Voluntary response creates self-selection (angry customers review more than satisfied ones)
  • Convenience sampling ignores hard-to-reach populations

Selection Bias happens when:

  • Historical data reflects past discriminatory practices
  • Dropout or attrition affects certain groups differently
  • Data collection systems are only deployed in specific locations or contexts

Real-World Impact

A hiring algorithm trained on historical employee data inherits selection bias if past hiring was discriminatory. A health model trained only on patients who sought treatment misses asymptomatic populations. A credit model using zip codes as features encodes historical redlining patterns.

The danger: your model will be very accurate at predicting patterns in your biased sample, but fail catastrophically when applied to the real world.

Prevention Strategies

  • Document your data collection methodology thoroughly
  • Compare your sample demographics to known population statistics
  • Identify missing or underrepresented groups
  • Use stratified sampling or weighting techniques when possible
  • Question whether excluded populations differ systematically

Key Takeaway: Non-representative samples arise from how data is collected, not just what's collected—always ask "who's missing from this dataset and why?"