Black-Box Evasion Attacks
What You'll Learn
You'll learn how attackers craft adversarial examples against ML models without seeing the model's internal structure or gradients—just by querying it like a regular user.
The Challenge: Attacking Without X-Ray Vision
In previous lessons, you learned gradient-based attacks like FGSM and C&W that directly manipulate model internals. But what if the attacker can't see inside the model—they only get predictions back, like using an API? This is the black-box scenario, and it's far more realistic for real-world attacks against commercial ML systems.
Three Core Approaches
Query-Based Methods
The attacker sends many inputs to the target model and observes outputs. By systematically perturbing inputs and watching how predictions change, they estimate which directions fool the model. Think of it like navigating a maze blindfolded—you learn by bumping into walls.
Transfer Attacks
The attacker trains a substitute model locally using synthetic data they create by querying the target. They then craft adversarial examples against their own model using gradient methods. Because ML models often share similar decision boundaries (especially within model families), adversarial examples frequently transfer from the substitute to the target.
Substitute Model Training
This involves building a local replica by:
- Starting with seed inputs
- Querying the target model to get labels
- Training a substitute model on this dataset
- Generating new inputs near decision boundaries
- Repeating until the substitute mimics the target's behavior
This is like learning to pick a lock by creating a practice lock that behaves similarly—you don't need the original's blueprints.
Why This Matters
Black-box attacks are the most practical threat model. They require no special access, just normal user privileges. Defenders can't rely on "security through obscurity"—hiding model architectures doesn't stop determined attackers.
Key Takeaway: Black-box evasion attacks use queries, transferability, and substitute models to fool ML systems without gradient access, making them the most realistic adversarial threat in production environments.