Evaluating Word Embeddings: Extrinsic Methods
You'll learn how to evaluate word embeddings by testing their performance on real-world NLP tasks.
Beyond Similarity Tests
While intrinsic evaluation measures word embeddings directly (like word similarity benchmarks), extrinsic evaluation asks a more practical question: Do these embeddings actually help solve real problems?
Think of it like testing a knife: intrinsic evaluation measures sharpness on a testing device, while extrinsic evaluation measures how well it cuts vegetables in an actual recipe. The recipe result is what truly matters.
How Extrinsic Evaluation Works
You evaluate embeddings by using them as input features for downstream NLP tasks:
- Choose a task: Named Entity Recognition (NER), sentiment classification, question answering, etc.
- Build a model: Use your word embeddings as the input layer
- Train and test: Measure task-specific metrics (accuracy, F1-score)
- Compare: Try different embeddings and see which gives better performance
For example, you might take Word2Vec embeddings and GloVe embeddings, plug each into a sentiment classifier, and see which achieves higher accuracy on movie reviews.
Why This Matters More
Extrinsic evaluation tells you whether embeddings capture the information your actual application needs. An embedding might score well on analogy tests but fail at sentiment analysis if it doesn't capture emotional nuance well.
The downside? It's slower and more expensive—you need labeled data and must train complete models. But the insight is worth it: you learn whether your embeddings work for your specific problem.
Key Takeaway: Extrinsic evaluation measures word embeddings by their performance on real downstream tasks like classification or NER, providing practical evidence of their quality for actual applications.