Differential Privacy in Training
You'll learn: How to train machine learning models with formal privacy guarantees using DP-SGD, protecting individual training examples from exposure.
The Challenge of Training Privacy
When you train a model on sensitive data (medical records, personal messages, financial transactions), the model can inadvertently memorize specific training examples. An attacker might extract private information by querying the model cleverly. Differential Privacy in Training solves this by mathematically guaranteeing that no single training example significantly influences the final model.
DP-SGD: Privacy-Preserving Gradient Descent
Differentially Private Stochastic Gradient Descent (DP-SGD) modifies the standard training algorithm with two key mechanisms:
1. Gradient Clipping
Before aggregating gradients from a batch, each individual gradient is clipped to a maximum norm (e.g., clip gradients to L2 norm ≤ 1.0). This prevents any single training example from having outsized influence.
Analogy: It's like limiting how loud any one person can speak in a group discussion, so no single voice dominates the decision.
2. Noise Addition
After clipping, calibrated Gaussian noise is added to the aggregated batch gradient. The noise scale is determined by your privacy budget (epsilon) and the clipping threshold. This noise masks the contribution of individual examples.
Privacy Accounting
The cumulative privacy loss across all training steps is tracked. Each gradient update "spends" some of your privacy budget. Training stops when you've exhausted your allocated epsilon.
Key Takeaway: DP-SGD provides provable privacy guarantees during training by clipping individual gradients and adding calibrated noise, ensuring no single training example can be identified or reconstructed from the final model—at the cost of some model accuracy.