Adversarial Robustness Fundamentals
What You'll Learn
You'll understand what adversarial robustness means, how to define threat models for it, and why building robust models often means sacrificing some accuracy.
The Core Concept
Adversarial robustness is the ability of a machine learning model to maintain correct predictions even when inputs are intentionally perturbed by an attacker. Unlike traditional ML metrics that focus purely on accuracy, robustness measures how well your model performs under attack.
Think of it like building a bridge: a normal bridge is designed for everyday traffic (standard accuracy), but a robust bridge must also withstand earthquakes and extreme weather (adversarial attacks). The sturdier you make it, the more expensive and sometimes less efficient it becomes for normal use.
Robustness Goals
The primary goal is certified robustness: guaranteeing that within a specific perturbation bound (like changing pixel values by at most ε), the model's prediction won't change. Secondary goals include maintaining reasonable clean accuracy and computational efficiency.
Threat Models Matter
You must define:
- Adversary's knowledge: White-box (full model access) vs. black-box (query-only)
- Perturbation budget: How much can inputs change? (L∞, L2, L1 norms)
- Attack objective: Untargeted (any wrong answer) vs. targeted (specific wrong answer)
The Accuracy-Robustness Tradeoff
Here's the hard truth: making models adversarially robust almost always reduces clean accuracy. Robust training forces models to learn smoother, more conservative decision boundaries that ignore small perturbations—but this means missing subtle legitimate patterns too. You're essentially teaching the model to be "paranoid" about small changes.
Key Takeaway: Adversarial robustness requires defining clear threat models and accepting accuracy-robustness tradeoffs; building models that resist attacks means making them more conservative, which comes at the cost of performance on clean data.