Course contentsShow
← Machine Learning and Deep Learning
Lesson 3332 of 3,53873. Privacy in Machine LearningFree lesson

Privacy Risks in Gradient Sharing

Examine how shared gradients in distributed or federated learning can leak information about local training data.

Privacy Risks in Gradient Sharing

What you'll learn: How gradients shared during distributed or federated learning can inadvertently leak information about the private training data used to compute them.

The Core Problem

In federated and distributed learning, multiple parties train a model collaboratively without directly sharing their data. Instead, they compute gradients locally and share only those gradients with a central server or other nodes. This sounds privacy-preserving—after all, you're not sending raw data, just mathematical derivatives.

But here's the catch: gradients are not anonymous summaries. They carry surprisingly rich information about the training examples used to compute them.

How Gradients Leak Information

Think of gradients as fingerprints of your data. When you compute ∂Loss/∂weight for a specific batch, that gradient vector is shaped by the exact features and labels in that batch. An adversary who intercepts these gradients can:

  • Reconstruct input features: Using techniques like gradient matching, attackers can iteratively reverse-engineer input data that would produce similar gradients
  • Infer sensitive attributes: Even partial gradient information can reveal whether certain individuals or records were in the training set
  • Extract labels: Classification gradients often leak ground-truth labels, especially in final layers

Real-World Scenarios

In federated learning for healthcare, a hospital shares gradients from patient records. An honest-but-curious server might use those gradients to reconstruct patient diagnoses or demographics. In collaborative research, competitors could extract proprietary dataset characteristics from shared gradient updates.

Key Takeaway: Gradients are not privacy-safe by default—sharing them in distributed learning exposes training data to reconstruction, membership inference, and attribute leakage, requiring explicit privacy protections like differential privacy or secure aggregation.