Course contentsShow
← Security
Lesson 2813 of 3,05253. AI and ML SecurityFree lesson

Black-Box Evasion Attacks

Attacking models without gradient access using query-based methods, transfer attacks, and substitute models.

Black-Box Evasion Attacks

What You'll Learn

You'll learn how attackers craft adversarial examples against ML models without seeing the model's internal structure or gradients—just by querying it like a regular user.

The Challenge: Attacking Without X-Ray Vision

In previous lessons, you learned gradient-based attacks like FGSM and C&W that directly manipulate model internals. But what if the attacker can't see inside the model—they only get predictions back, like using an API? This is the black-box scenario, and it's far more realistic for real-world attacks against commercial ML systems.

Three Core Approaches

Query-Based Methods

The attacker sends many inputs to the target model and observes outputs. By systematically perturbing inputs and watching how predictions change, they estimate which directions fool the model. Think of it like navigating a maze blindfolded—you learn by bumping into walls.

Transfer Attacks

The attacker trains a substitute model locally using synthetic data they create by querying the target. They then craft adversarial examples against their own model using gradient methods. Because ML models often share similar decision boundaries (especially within model families), adversarial examples frequently transfer from the substitute to the target.

Substitute Model Training

This involves building a local replica by:

  1. Starting with seed inputs
  2. Querying the target model to get labels
  3. Training a substitute model on this dataset
  4. Generating new inputs near decision boundaries
  5. Repeating until the substitute mimics the target's behavior

This is like learning to pick a lock by creating a practice lock that behaves similarly—you don't need the original's blueprints.

Why This Matters

Black-box attacks are the most practical threat model. They require no special access, just normal user privileges. Defenders can't rely on "security through obscurity"—hiding model architectures doesn't stop determined attackers.

Key Takeaway: Black-box evasion attacks use queries, transferability, and substitute models to fool ML systems without gradient access, making them the most realistic adversarial threat in production environments.