Property Inference Attacks
What you'll learn: How attackers can discover aggregate statistical properties of a model's training dataset without ever seeing individual records.
The Core Idea
Property inference attacks target distributional characteristics of training data rather than individual samples. Unlike membership inference (which asks "was this record in the training set?") or model inversion (which reconstructs specific records), property inference asks broader questions: "What percentage of training users were under 30?" or "Did the training data come primarily from a specific geographic region?"
The attacker doesn't need direct access to training data—only the ability to query the trained model. By observing how the model behaves, they reverse-engineer aggregate properties that should remain confidential.
Why This Matters
Think of it like this: even if a hospital never reveals individual patient records, if you can determine that 80% of their training data consists of diabetic patients, you've learned something sensitive about the institution's patient population. The aggregate property itself may be proprietary, confidential, or reveal business intelligence.
Attack Mechanics
The attacker typically:
- Trains "shadow models" on datasets with known properties
- Observes how models behave differently when trained on data with vs. without the target property
- Queries the victim model and compares its behavior to shadow models
- Infers which property distribution the victim's training data likely had
For example, an attacker might learn that a fraud detection model was trained mostly on credit card transactions from high-income zip codes, revealing the bank's customer demographics.
Key Takeaway: Property inference attacks exploit the fact that aggregate statistical properties of training data leak through model behavior, allowing adversaries to infer sensitive distributional characteristics without accessing individual records—a privacy violation often overlooked when focusing solely on individual record protection.