Course contentsShow
← Machine Learning and Deep Learning
Lesson 1231 of 3,53827. Pretrained Language Models: GPT Family and BeyondFree lesson

Supervised Fine-Tuning Mechanics for Instructions

Training procedure for instruction tuning: loss masking, multi-turn formatting, and training dynamics.

Supervised Fine-Tuning Mechanics for Instructions

What you'll learn: How the actual training process works when teaching a base model to follow instructions through supervised fine-tuning.

The Core Training Process

Once you have an instruction dataset (input-output pairs), supervised fine-tuning (SFT) trains the model to predict the assistant's response given the instruction and any context. But there's a crucial detail: you don't want the model to learn to predict the instruction itself—only the response.

Loss Masking: Teaching What Matters

Loss masking ensures gradients only update weights based on the output tokens you want the model to generate. During training:

  • The instruction/prompt tokens are fed to the model but their loss is masked (set to zero)
  • Only the assistant's response tokens contribute to the loss calculation
  • This focuses learning on "what to say" rather than "what was asked"

Think of it like highlighting only the answers on a practice test—you're grading performance on responses, not on reading the questions.

Multi-Turn Formatting

Instruction datasets often include conversations with multiple exchanges. The formatting typically looks like:

<|user|> Question 1 <|assistant|> Answer 1 <|user|> Question 2 <|assistant|> Answer 2

Loss masking applies to each turn: mask user tokens, train on assistant tokens. This teaches the model conversational context awareness.

Training Dynamics

SFT usually requires:

  • Fewer epochs than pretraining (often 1-3)
  • Lower learning rates to avoid catastrophic forgetting
  • Smaller datasets (thousands to millions vs billions of pretraining tokens)

The model already knows language—you're just steering its behavior toward helpfulness and instruction-following.

Key Takeaway: Supervised fine-tuning for instructions trains on response tokens only (via loss masking), handles multi-turn conversations through special formatting, and uses gentle optimization to reshape base model behavior without erasing pretrained knowledge.