Small Subgroup and Invalid Curve Attacks
What you'll learn: How attackers can manipulate Diffie-Hellman or elliptic curve parameters to weaken or break key exchanges, even when the math looks correct.
The Core Vulnerability
When using Diffie-Hellman (including ECDH from lesson 158), both parties exchange public keys and perform calculations to reach a shared secret. But what if an attacker sends you a malicious public key with special mathematical properties?
Small subgroup attacks exploit weak group parameters. In regular Diffie-Hellman, if an attacker sends you a public key that belongs to a small mathematical subgroup (a tiny subset of valid values), your resulting shared secret will be one of only a few possible values. Instead of having billions of possibilities, the attacker might narrow it down to just 2, 4, or 16 options — trivial to brute-force.
Invalid curve attacks target ECDH specifically. An attacker sends a point that doesn't actually lie on your agreed-upon elliptic curve, but on a different, weaker curve. If your implementation doesn't validate incoming points, it performs calculations on this weak curve, potentially leaking information about your private key through repeated exchanges.
Real-World Impact
These attacks bypass the encryption itself — they manipulate the key exchange phase. If your implementation accepts untrusted public keys without validation, an attacker can:
- Force your shared secret into a small, guessable set
- Extract bits of your private key through mathematical side channels
- Completely compromise confidentiality without breaking any encryption algorithm
Defense
Always validate received public keys:
- Check that DH public keys are in the correct range and not in small subgroups
- Verify ECDH points actually lie on the intended curve
- Use standardized, well-vetted parameter groups
Key Takeaway: Accepting untrusted public keys without validation lets attackers manipulate the mathematics of key exchange, turning strong cryptography into weak or broken security.