Supervised Fine-Tuning Mechanics for Instructions
What you'll learn: How the actual training process works when teaching a base model to follow instructions through supervised fine-tuning.
The Core Training Process
Once you have an instruction dataset (input-output pairs), supervised fine-tuning (SFT) trains the model to predict the assistant's response given the instruction and any context. But there's a crucial detail: you don't want the model to learn to predict the instruction itself—only the response.
Loss Masking: Teaching What Matters
Loss masking ensures gradients only update weights based on the output tokens you want the model to generate. During training:
- The instruction/prompt tokens are fed to the model but their loss is masked (set to zero)
- Only the assistant's response tokens contribute to the loss calculation
- This focuses learning on "what to say" rather than "what was asked"
Think of it like highlighting only the answers on a practice test—you're grading performance on responses, not on reading the questions.
Multi-Turn Formatting
Instruction datasets often include conversations with multiple exchanges. The formatting typically looks like:
<|user|> Question 1 <|assistant|> Answer 1 <|user|> Question 2 <|assistant|> Answer 2
Loss masking applies to each turn: mask user tokens, train on assistant tokens. This teaches the model conversational context awareness.
Training Dynamics
SFT usually requires:
- Fewer epochs than pretraining (often 1-3)
- Lower learning rates to avoid catastrophic forgetting
- Smaller datasets (thousands to millions vs billions of pretraining tokens)
The model already knows language—you're just steering its behavior toward helpfulness and instruction-following.
Key Takeaway: Supervised fine-tuning for instructions trains on response tokens only (via loss masking), handles multi-turn conversations through special formatting, and uses gentle optimization to reshape base model behavior without erasing pretrained knowledge.