Loss Masking for Instructions
What you'll learn: How to prevent your model from learning the wrong thing by only calculating loss on assistant responses, not user prompts.
The Problem with Naive Instruction Training
When fine-tuning a language model on instruction-following conversations, you might be tempted to compute loss across the entire sequence—both the user's question and the assistant's answer. But here's the issue: you don't want your model to learn to generate user questions. You want it to learn to respond to them.
How Loss Masking Works
Loss masking is like grading a student only on their answers, not on the test questions you gave them. During training, you:
- Feed the entire conversation through the model (user prompt + assistant response)
- Calculate predictions for all tokens
- Apply loss only to the assistant's tokens, setting loss to zero (or ignoring) the prompt tokens
This focuses the learning signal exclusively on improving generation quality for responses.
Why This Matters
Without masking:
- The model wastes capacity learning to predict prompt tokens
- Training becomes less efficient
- The model may confuse its role (am I asking or answering?)
With masking:
- All gradient updates improve response quality
- Faster convergence on instruction-following behavior
- Clearer role separation in conversations
Implementation Concept
You create an attention mask or loss mask that marks which tokens should contribute to the loss calculation. Tokens in the user prompt get masked out (weight = 0), while assistant tokens get full weight (weight = 1). The loss function only backpropagates through unmasked positions.
Key Takeaway: Loss masking ensures your instruction-tuned model learns to generate better responses by only computing gradients on assistant tokens, not user prompts—making training more efficient and focused.