Course contentsShow
← Machine Learning and Deep Learning
Lesson 3401 of 3,53874. Adversarial RobustnessFree lesson

Defense Strategies Overview

Survey of adversarial defense approaches: preprocessing, adversarial training, detection, and certified defenses.

Defense Strategies Overview

You'll learn the major categories of defenses against adversarial attacks and how they approach the robustness problem differently.

After exploring how adversaries craft malicious inputs to fool models, we now turn to defenses: techniques that harden models against adversarial examples. Defense strategies fall into four broad categories, each with distinct philosophies and tradeoffs.

The Four Defense Paradigms

Preprocessing Defenses

These transform input data before it reaches the model, attempting to remove or neutralize adversarial perturbations. Think of this like airport security screening—you examine and sanitize inputs at the gate. Examples include image denoising, JPEG compression, or feature squeezing that reduces the input's complexity.

Adversarial Training

The most empirically robust approach: augment your training set with adversarial examples, teaching the model to classify them correctly. It's like inoculation—expose the model to "weakened" attacks during training so it builds immunity. This directly addresses the vulnerability at training time.

Detection Methods

Rather than classifying adversarial inputs correctly, these defenses attempt to detect when an input is adversarial and reject it. Analogous to a fraud detection system—if something looks suspicious, flag it rather than process it. This includes statistical tests, auxiliary detector networks, or confidence thresholding.

Certified Defenses

These provide provable guarantees about model behavior within a specified perturbation radius. Using techniques from formal verification or randomized smoothing, you can mathematically certify "no adversarial example exists within ε of this input." This is the gold standard but often comes with significant computational cost or accuracy penalties.

Each approach has strengths: preprocessing is computationally cheap, adversarial training is empirically strong, detection avoids misclassification, and certification provides guarantees. However, all involve tradeoffs between clean accuracy, robustness, and computational overhead.

Key Takeaway: Adversarial defenses span preprocessing, adversarial training, detection, and certified methods—each tackling robustness from a different angle with distinct tradeoffs in guarantees, performance, and computational cost.