Course contentsShow
← Machine Learning and Deep Learning
Lesson 3357 of 3,53873. Privacy in Machine LearningFree lesson

Federated Learning with Differential Privacy

Combining federated learning with DP guarantees to protect individual client contributions.

Federated Learning with Differential Privacy

You'll learn how to protect individual client contributions in federated learning by integrating differential privacy guarantees into the training process.

The Core Challenge

Federated learning keeps raw data decentralized, but the model updates (gradients) shared by clients can still leak sensitive information about their local data. An attacker analyzing these updates might reconstruct training examples or infer membership. The solution? Add differential privacy noise to the updates before sharing them.

How It Works

In Federated Learning with Differential Privacy, each client applies the DP-SGD mechanism locally:

  1. Clip gradients to bound their sensitivity (per-example gradient clipping)
  2. Add calibrated noise (typically Gaussian) to the clipped gradients
  3. Share the noisy update with the central server

The server aggregates these noisy updates using FedAvg (Federated Averaging), producing a model that satisfies differential privacy with respect to any individual client's dataset.

Think of it like this: Instead of clients whispering their exact observations to the server, they add a controlled amount of random "static" to their message. The server can still learn the general pattern from many noisy messages, but no single client's exact data is recoverable.

Privacy Accounting Across Rounds

Each communication round consumes some privacy budget (ε). With composition theorems and privacy amplification by sampling (when only a fraction of clients participate per round), you track the cumulative privacy loss across all training rounds to ensure the total stays within acceptable bounds.

Key Takeaway:
Combining federated learning with differential privacy provides mathematically rigorous protection against data leakage from model updates, ensuring individual client contributions remain private even when gradients are shared with the server.