Course contentsShow
← Machine Learning and Deep Learning
Lesson 1176 of 3,53826. Pretrained Language Models: BERT FamilyFree lesson

Fine-Tuning for Question Answering

Implement span prediction heads for extractive QA tasks using BERT's contextualized representations.

Fine-Tuning for Question Answering

You'll learn how to adapt BERT's contextual representations to predict answer spans within passages for extractive question answering.

The Core Idea

In extractive question answering (QA), you're given a question and a passage of text, and your task is to identify the exact span (start and end positions) within that passage that answers the question. Unlike classification where you predict a single label, QA requires predicting two positions: where the answer begins and where it ends.

BERT's bidirectional context makes it perfect for this. You feed both the question and passage together (separated by [SEP]), and BERT creates rich contextual representations for every token. The trick is adding a span prediction head that uses these representations to score each token as a potential start or end position.

How Span Prediction Works

Think of it like highlighting text with two markers—one for the beginning, one for the end. The span prediction head consists of two simple linear layers:

  1. Start position classifier: Takes each token's BERT representation and outputs a score indicating how likely that token is to be the answer's start
  2. End position classifier: Similarly scores each token as a potential answer endpoint

During training, you know the correct start and end positions from labeled data. The model learns to assign high scores to the correct positions and low scores elsewhere. At inference, you pick the highest-scoring valid span (where start ≤ end).

The beauty is that BERT's contextualized representations already encode understanding of both the question and passage—your span prediction heads just need to learn which tokens form the answer boundary.

Key Takeaway: Fine-tuning BERT for QA adds two lightweight linear layers on top of BERT's token representations to predict answer start and end positions, leveraging the model's deep contextual understanding to identify text spans that answer questions.