Course contentsShow
← Machine Learning and Deep Learning
Lesson 3349 of 3,53873. Privacy in Machine LearningFree lesson

Implementing DP-SGD with PyTorch and Opacus

Practical implementation using Opacus library; privacy engine setup and tracking privacy budget.

Implementing DP-SGD with PyTorch and Opacus

You'll learn how to integrate the Opacus library into PyTorch training loops to enable differentially private model training and track your privacy budget.

What is Opacus?

Opacus is Meta's library that makes implementing DP-SGD (Differentially Private Stochastic Gradient Descent) in PyTorch straightforward. Instead of manually clipping gradients and adding calibrated noise—as you've learned in theory—Opacus automates this entire process while tracking your privacy budget (ε and δ).

The Privacy Engine

The core component is the PrivacyEngine, which wraps your optimizer and modifies the training loop to enforce differential privacy:

from opacus import PrivacyEngine

# Standard PyTorch setup
model = MyModel()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
data_loader = DataLoader(dataset, batch_size=32)

# Attach privacy engine
privacy_engine = PrivacyEngine()
model, optimizer, data_loader = privacy_engine.make_private(
    module=model,
    optimizer=optimizer,
    data_loader=data_loader,
    noise_multiplier=1.1,  # Controls noise scale
    max_grad_norm=1.0,     # Gradient clipping bound
)

The PrivacyEngine handles gradient clipping per sample, adds Gaussian noise calibrated to your parameters, and leverages privacy amplification by sampling through the modified data loader.

Tracking Privacy Budget

During training, query the accumulated privacy cost:

epsilon = privacy_engine.get_epsilon(delta=1e-5)
print(f"Current ε = {epsilon:.2f} at δ = 1e-5")

This uses advanced composition to give you a tight bound on your total privacy loss across all training iterations.

Key Takeaway: Opacus simplifies DP-SGD implementation by automating gradient clipping and noise addition through the PrivacyEngine, while providing real-time privacy budget tracking to ensure your model meets specified (ε, δ) guarantees.