Adversarial Attacks on NLP Models
What you'll learn: How adversarial attacks manipulate text inputs at different granularities to fool NLP models while maintaining human readability.
The Core Challenge
Unlike images where pixel perturbations remain imperceptible, text is discrete and symbolic—every character matters for meaning. Adversarial attacks on NLP models must alter text in ways that preserve semantic meaning and grammatical correctness while deceiving the model. This creates unique attack surfaces at three distinct levels.
Character-Level Perturbations
These attacks introduce subtle typos, homoglyphs (visually similar characters), or character swaps that humans easily overlook but confuse models:
- Replacing 'o' with '0' or 'l' with 'I'
- Inserting invisible Unicode characters
- Strategic misspellings like "defnitely" instead of "definitely"
Think of it like optical illusions—the text looks right to humans but is technically different.
Word-Level Perturbations
These substitute words with synonyms, paraphrases, or semantically similar alternatives:
- Replacing "terrible" with "awful" in sentiment analysis
- Using contextually appropriate words that shift model predictions
- Exploiting word embeddings to find "nearby" replacements
This is like swapping ingredients in a recipe—the dish tastes similar but the model's "palate" detects something off.
Sentence-Level Perturbations
These restructure entire sentences through paraphrasing, insertion, or deletion while maintaining overall meaning:
- Reordering clauses
- Adding distracting but semantically neutral phrases
- Changing active to passive voice
These attacks are hardest to detect because they preserve grammatical structure and human comprehension.
Why Text is Different
Text attacks must balance three constraints: semantic preservation, grammatical validity, and adversarial effectiveness. Unlike continuous image perturbations, discrete text changes are instantly visible unless carefully crafted.
Key Takeaway: Adversarial attacks on NLP operate at character, word, and sentence levels, exploiting the discrete nature of language while attempting to preserve meaning for human readers but not machine models.