Attribute Inference and Property Inference
What you'll learn: How attackers can deduce sensitive individual attributes or global dataset properties by analyzing a trained machine learning model's behavior.
The Core Concept
Even if a model doesn't directly reveal training data, clever attackers can still extract information through inference attacks. These fall into two categories:
Attribute Inference targets specific individuals in the training set. If an attacker knows some attributes of a person (e.g., age and zip code), they can query the model strategically to infer missing sensitive attributes (e.g., income or medical diagnosis). Think of it like filling in crossword puzzle blanks—you use what you know to deduce what's hidden.
Property Inference targets the entire dataset's characteristics. An attacker queries the model to learn global properties like: "Was this model trained on data from a specific hospital?" or "What percentage of training samples had a certain label?" This reveals aggregate information about the dataset composition that the model owner might want to keep confidential.
Why This Matters
These attacks exploit the fact that models memorize statistical patterns—and sometimes unintentionally "leak" them through prediction confidence scores, decision boundaries, or output distributions.
Example scenario: A health insurance company trains a model on customer data. An attacker with partial information about a customer queries the model repeatedly with varying inputs. By observing how predictions change, they infer the customer's undisclosed pre-existing condition (attribute inference) or discover that the training data came predominantly from patients with diabetes (property inference).
Defense Considerations
Mitigations include differential privacy during training, output perturbation, and limiting query access—topics you'll explore in defense strategies.
Key Takeaway: Attribute inference extracts sensitive features of specific training individuals, while property inference reveals global dataset characteristics; both exploit statistical leakage through model behavior without directly accessing training data.