Course contentsShow
← Machine Learning and Deep Learning
Lesson 3412 of 3,53874. Adversarial RobustnessFree lesson

Evaluating Defense Effectiveness

Properly evaluating defenses with adaptive attacks and avoiding common pitfalls in robustness claims.

Evaluating Defense Effectiveness

What you'll learn: How to rigorously test adversarial defenses using adaptive attacks and avoid misleading robustness claims.

Why Standard Evaluation Fails

Many proposed defenses look impressive at first—they block FGSM or basic PGD attacks with ease. But here's the problem: attackers adapt. If you only test against fixed, off-the-shelf attacks, you're evaluating your defense against yesterday's threats, not tomorrow's.

Think of it like building a lock that only stops people from turning the doorknob. Sure, you'll pass the "doorknob test," but any attacker who tries kicking the door or picking the lock will succeed instantly.

The Adaptive Attack Principle

A proper evaluation requires adaptive attacks—attacks specifically designed to exploit your defense's structure. If your defense:

  • Preprocesses inputs → test attacks that account for the preprocessing
  • Uses randomness → test attacks that average over multiple queries
  • Masks gradients → use gradient-free or expectation-over-transformation methods

For example, if you defended using input JPEG compression (a preprocessing defense you learned earlier), an adaptive attacker would optimize adversarial examples through the compression operation, not before it.

Common Pitfalls to Avoid

  1. Gradient masking: Defense appears robust only because gradients are hidden, not because the model is truly robust
  2. Weak attack parameters: Testing with too few PGD steps or wrong epsilon values
  3. Single attack evaluation: Only trying one attack type (e.g., only FGSM)
  4. Ignoring transferability: Not testing whether examples from other models break your defense

A robust evaluation uses multiple strong adaptive attacks, tests various perturbation budgets, and compares against properly-tuned adversarial training baselines.

Key Takeaway: True defense effectiveness requires evaluation with adaptive attacks tailored to your specific defense mechanism—generic attack suites alone can give false confidence in robustness.