Course contentsShow
← Machine Learning and Deep Learning
Lesson 1125 of 3,53825. Word Embeddings and Contextual RepresentationsFree lesson

Semantic Relationships in Embedding Space

Explore how embeddings capture analogies like 'king - man + woman = queen' through vector arithmetic.

Semantic Relationships in Embedding Space

What you'll learn: How word embeddings capture meaningful semantic relationships that enable vector arithmetic like "king - man + woman = queen."

The Magic of Vector Arithmetic

One of the most fascinating properties of word embeddings is that they don't just represent words as random points in space—they organize words so that semantic relationships become geometric patterns.

When you train embeddings using Word2Vec or GloVe, words with similar meanings cluster together, but something even more remarkable happens: the differences between word vectors capture consistent relationships.

The Classic Example

The famous analogy "king is to man as queen is to woman" translates directly into vector math:

vector(king) - vector(man) + vector(woman) ≈ vector(queen)

What's happening here? The vector from "man" to "king" represents roughly "royalty + male." When you subtract "man" and add "woman," you're essentially saying "take the royalty concept, remove the male aspect, and add the female aspect"—landing you near "queen."

Why This Works

The distributional hypothesis means words appearing in similar contexts get similar embeddings. Since "king" and "queen" both appear in royal contexts, and "man" and "woman" appear in gender-specific contexts, these patterns become encoded as consistent directional relationships in the embedding space.

Other examples that work:

  • Paris - France + Italy ≈ Rome (capital city relationships)
  • walking - walked + swimming ≈ swam (verb tense patterns)

Limitations

These analogies work impressively well but aren't perfect. The relationships are approximations, and they can reflect biases present in training data.

Key Takeaway: Word embeddings organize semantic meaning geometrically, allowing meaningful relationships to be captured as consistent vector directions that enable analogy-solving through simple arithmetic.