Semantic Relationships in Embedding Space
What you'll learn: How word embeddings capture meaningful semantic relationships that enable vector arithmetic like "king - man + woman = queen."
The Magic of Vector Arithmetic
One of the most fascinating properties of word embeddings is that they don't just represent words as random points in space—they organize words so that semantic relationships become geometric patterns.
When you train embeddings using Word2Vec or GloVe, words with similar meanings cluster together, but something even more remarkable happens: the differences between word vectors capture consistent relationships.
The Classic Example
The famous analogy "king is to man as queen is to woman" translates directly into vector math:
vector(king) - vector(man) + vector(woman) ≈ vector(queen)
What's happening here? The vector from "man" to "king" represents roughly "royalty + male." When you subtract "man" and add "woman," you're essentially saying "take the royalty concept, remove the male aspect, and add the female aspect"—landing you near "queen."
Why This Works
The distributional hypothesis means words appearing in similar contexts get similar embeddings. Since "king" and "queen" both appear in royal contexts, and "man" and "woman" appear in gender-specific contexts, these patterns become encoded as consistent directional relationships in the embedding space.
Other examples that work:
- Paris - France + Italy ≈ Rome (capital city relationships)
- walking - walked + swimming ≈ swam (verb tense patterns)
Limitations
These analogies work impressively well but aren't perfect. The relationships are approximations, and they can reflect biases present in training data.
Key Takeaway: Word embeddings organize semantic meaning geometrically, allowing meaningful relationships to be captured as consistent vector directions that enable analogy-solving through simple arithmetic.