Course contentsShow
← Machine Learning and Deep Learning
Lesson 1306 of 3,53829. Core NLP TasksFree lesson

Dense Passage Retrieval for QA

Using dense embeddings to retrieve relevant passages before extracting or generating answers.

Dense Passage Retrieval for QA

What you'll learn: How to use dense embeddings to find relevant passages quickly before answering questions, making open-domain QA practical at scale.

The Problem with Open-Domain QA

In Open-Domain Question Answering (lesson 1305), we need to search through potentially millions of documents. Traditional keyword-based search (like BM25) often misses relevant passages because questions and answers use different words. If someone asks "Who invented the telephone?" a relevant passage might say "Alexander Graham Bell created the first practical telephone device" — no exact word match for "invented."

Dense Embeddings to the Rescue

Dense Passage Retrieval (DPR) solves this by encoding both questions and passages as dense vectors (embeddings) in the same semantic space. Think of it like a library where books aren't organized alphabetically, but by meaning — similar topics cluster together even if they use different words.

How It Works

  1. Two encoders: One BERT-based model encodes the question, another encodes passages (often sharing weights)
  2. Similarity search: Find passages whose embeddings are closest to the question embedding (using dot product or cosine similarity)
  3. Retrieval: Return top-k most similar passages
  4. Answer extraction: Feed retrieved passages to your QA model (like span prediction from lesson 1300)

The key insight: semantic similarity captures meaning better than keyword overlap. "invented" and "created" have similar embeddings even though they're different words.

Why This Matters

DPR makes open-domain QA feasible by narrowing millions of passages down to a handful of relevant candidates in milliseconds. You then run your (slower) extractive QA model only on these few passages, not the entire corpus.

Key Takeaway: Dense Passage Retrieval uses learned embeddings to retrieve semantically relevant passages based on meaning rather than keyword matching, enabling efficient large-scale question answering.