Privacy Auditing and Empirical Measurement
What You'll Learn
You'll learn how to test whether trained models actually achieved the privacy guarantees they claim, using membership inference attacks and empirical measurements to audit real-world privacy protection.
Why Privacy Auditing Matters
You've implemented differential privacy (DP) with DP-SGD, set your epsilon budget carefully, and deployed your model. But did it actually work? Privacy auditing is the practice of empirically measuring whether a model leaks private training data, rather than just trusting theoretical guarantees.
Think of it like a penetration test for privacy: you actively try to extract sensitive information to measure how much privacy your defenses actually provide.
Membership Inference as a Privacy Meter
The most common auditing technique uses membership inference attacks (which you studied earlier) as a measuring stick. You deliberately attempt to determine whether specific records were in the training set:
- Train shadow models on similar data with known membership
- Launch membership inference attacks against your audited model
- Measure attack success rate (accuracy, AUC, TPR@low FPR)
- Compare to baseline (random guessing = 50% accuracy)
The higher the attack accuracy, the more privacy leakage exists. If your DP model claims ε=1 but membership inference achieves 90% accuracy, something is wrong.
Empirical Privacy Loss Calculation
Instead of trusting theoretical epsilon values, auditing measures empirical privacy loss:
- Run multiple membership inference attempts
- Calculate the privacy amplification actually achieved
- Measure worst-case leakage on vulnerable subgroups
- Compare claimed versus measured privacy budgets
This reveals gaps between theory and practice—misconfigurations, implementation bugs, or insufficient privacy parameters.
Continuous Monitoring
Privacy auditing isn't one-and-done. As models are retrained or updated, you re-audit to ensure privacy degradation hasn't occurred over time.
Key Takeaway:
Privacy auditing uses membership inference attacks and empirical measurements to verify that deployed models achieve their claimed privacy guarantees, catching gaps between theoretical protections and real-world leakage.