Federated Learning with Differential Privacy
You'll learn how to protect individual client contributions in federated learning by integrating differential privacy guarantees into the training process.
The Core Challenge
Federated learning keeps raw data decentralized, but the model updates (gradients) shared by clients can still leak sensitive information about their local data. An attacker analyzing these updates might reconstruct training examples or infer membership. The solution? Add differential privacy noise to the updates before sharing them.
How It Works
In Federated Learning with Differential Privacy, each client applies the DP-SGD mechanism locally:
- Clip gradients to bound their sensitivity (per-example gradient clipping)
- Add calibrated noise (typically Gaussian) to the clipped gradients
- Share the noisy update with the central server
The server aggregates these noisy updates using FedAvg (Federated Averaging), producing a model that satisfies differential privacy with respect to any individual client's dataset.
Think of it like this: Instead of clients whispering their exact observations to the server, they add a controlled amount of random "static" to their message. The server can still learn the general pattern from many noisy messages, but no single client's exact data is recoverable.
Privacy Accounting Across Rounds
Each communication round consumes some privacy budget (ε). With composition theorems and privacy amplification by sampling (when only a fraction of clients participate per round), you track the cumulative privacy loss across all training rounds to ensure the total stays within acceptable bounds.
Key Takeaway:
Combining federated learning with differential privacy provides mathematically rigorous protection against data leakage from model updates, ensuring individual client contributions remain private even when gradients are shared with the server.