Equation-Solving Attacks on Linear Models
What you'll learn: How attackers can fully reconstruct linear and logistic regression models by cleverly crafting queries that turn model stealing into a simple algebra problem.
The Core Idea
Unlike black-box query-based attacks that approximate a model through thousands of samples, equation-solving attacks exploit the mathematical simplicity of linear models. Since a linear model is just a weighted sum plus a bias (y = w₁x₁ + w₂x₂ + ... + wₙxₙ + b), an attacker only needs (n+1) carefully chosen queries to set up a system of linear equations and solve for all weights directly.
Think of it like this: if someone has a secret recipe that combines ingredients by multiplication and addition, and you can submit ingredient lists to get the final "score," you only need as many tests as there are ingredients to reverse-engineer the exact formula.
How It Works
- Query strategically: Send inputs where all features are zero except one (basis vectors). Each response reveals information about one weight.
- Build the system: Each query creates one equation with the unknown weights.
- Solve algebraically: Use basic linear algebra (Gaussian elimination or matrix inversion) to extract exact model parameters.
For logistic regression, attackers query near decision boundaries where the sigmoid function is most informative, then use numerical methods to recover the linear component inside.
Key advantage: This requires minimal queries (often just dozens) compared to thousands needed for approximation-based stealing, making it stealthy and efficient against APIs with rate limits.
Key Takeaway: Linear and logistic regression models are vulnerable to exact parameter extraction through equation-solving attacks that require only as many queries as the model has features, turning model theft into an algebra problem rather than an approximation challenge.