Detecting Poisoned Training Data
What You'll Learn
You'll learn how statistical methods and anomaly detection techniques can identify corrupted training samples that attackers have injected into your ML datasets.
The Detection Challenge
After learning how attackers poison datasets and insert backdoors, the natural question is: how do we find the poisoned needles in the haystack of training data?
Think of it like food safety inspection: you can't taste every apple in the warehouse, but you can use statistical sampling, visual inspection, and anomaly detection to find the rotten ones before they spoil the batch.
Core Detection Approaches
Statistical Outlier Detection
Poisoned samples often exhibit statistical anomalies:
- Feature-space clustering: Clean data from the same class typically clusters together. Poisoned samples with backdoor triggers may appear as outliers in feature space.
- Activation analysis: Run training samples through a reference model and examine intermediate layer activations. Poisoned samples often produce unusual activation patterns.
Gradient-Based Methods
During training, poisoned samples can exhibit different gradient behaviors:
- Loss trajectory analysis: Samples that cause unusual loss spikes or gradient magnitudes may be poisoned.
- Influence functions: Measure how removing a specific sample affects model predictions—high-influence outliers warrant investigation.
Spectral Signature Detection
Backdoor triggers create detectable patterns in the covariance structure of representations. Techniques like spectral clustering can separate clean from poisoned samples by analyzing the top singular vectors of layer representations.
Practical Considerations
Detection isn't perfect—attackers design poisoning attacks to be stealthy. You're looking for subtle statistical deviations, not obvious corruption. Combine multiple detection methods and apply them at different pipeline stages for best results.
Key Takeaway: Detecting poisoned training data relies on finding statistical anomalies in feature spaces, gradients, activations, and spectral signatures—treating data quality as a security inspection problem rather than assuming training set integrity.