Course contentsShow
← Computer Science
Lesson 2805 of 2,87262. Machine Learning FundamentalsFree lesson

Data Leakage Prevention

Identifying and avoiding scenarios where test information contaminates training, leading to overly optimistic results.

Data Leakage Prevention

You'll learn how to identify and eliminate data leakage—one of the most insidious causes of artificially inflated model performance.

The Core Idea

Data leakage occurs when information from outside your training set "leaks" into your model during training, giving it an unfair advantage that won't exist in real-world deployment. It's like a student accidentally seeing tomorrow's exam questions while studying today—their practice test scores will look amazing, but they'll bomb the real exam.

Leakage happens when test set information contaminates training in subtle ways:

  • Target leakage: A feature wouldn't actually be available at prediction time (e.g., using "total_purchase_amount" to predict "will_make_purchase")
  • Train-test contamination: Preprocessing steps (like normalization or imputation) use statistics computed on the entire dataset including the test set, rather than training set only
  • Temporal leakage: Using future information to predict the past in time series data
  • Duplication: Same examples appearing in both training and test sets

Prevention Strategy

Always split your data first, then perform all preprocessing, feature engineering, and statistical calculations using only the training set. Apply those learned transformations to the test set separately. Think of it as a firewall: test data should never influence any decision made during training.

When using cross-validation, repeat preprocessing inside each fold—never compute statistics on the full dataset before folding.

Key Takeaway: Data leakage creates overly optimistic results that collapse in production. Prevent it by maintaining strict separation: split first, then transform training and test sets independently using only training-derived statistics.