Course contentsShow
← Machine Learning and Deep Learning
Lesson 1301 of 3,53829. Core NLP TasksFree lesson

Context Encoding and Passage Retrieval

Encoding question-context pairs and handling passages that may or may not contain answers.

Context Encoding and Passage Retrieval

What You'll Learn

You'll learn how question answering systems encode question-context pairs and retrieve relevant passages before extracting answers.

The Core Idea

When performing extractive QA (like with BERT on SQuAD), the model needs to find the answer somewhere in the text. But what if you have a huge document or multiple documents? You can't feed everything at once—transformers have token limits (often 512 tokens).

Context encoding means creating dense vector representations of both the question and potential answer passages. Think of it like creating fingerprints for text: similar meanings produce similar fingerprints.

Passage retrieval is the step before span prediction. Instead of searching the entire library for an answer, you first grab the most relevant pages. The system:

  1. Splits long documents into manageable passages (chunks)
  2. Encodes each passage into a vector
  3. Encodes the question into a vector
  4. Compares question vector against all passage vectors (using similarity metrics)
  5. Retrieves the top-k most relevant passages
  6. Only then runs span prediction on those passages

Why This Matters

Imagine asking "Who invented the telephone?" across Wikipedia. Without retrieval, you'd need to check every article. With passage retrieval, you encode the question and quickly find passages about Alexander Graham Bell, then extract the precise answer span.

Some passages may not contain the answer at all—retrieval helps filter these out early. This two-stage approach (retrieve, then extract) makes QA scalable beyond single paragraphs.

Key Takeaway: Context encoding creates vector representations of questions and passages, enabling efficient passage retrieval to narrow down where answers might exist before running expensive span prediction models.