Course contentsShow
← Machine Learning and Deep Learning
Lesson 1703 of 3,53837. LLM Fine-Tuning: Full and Parameter-EfficientFree lesson

Computing Loss for Fine-Tuning Objectives

Calculating next-token prediction loss or task-specific loss functions on your fine-tuning dataset.

Computing Loss for Fine-Tuning Objectives

What you'll learn: How to calculate the loss function that guides your LLM during fine-tuning, whether for next-token prediction or task-specific objectives.

The Core Idea

During fine-tuning, your model needs feedback on how well it's performing. The loss function quantifies the difference between what your model predicts and what you want it to predict. This numerical signal tells the optimizer which direction to adjust parameters.

For LLMs, you typically compute loss in one of two ways:

Next-Token Prediction Loss

This is the same objective used in pretraining. Given a sequence from your fine-tuning dataset, you:

  1. Feed tokens through the forward pass
  2. For each position, compare the model's predicted probability distribution over vocabulary against the actual next token
  3. Use cross-entropy loss to measure how "wrong" the prediction was

Think of it like a teacher grading an essay question where the student had to fill in blanks—you penalize confidently wrong answers more than uncertain ones.

Task-Specific Loss Functions

For specialized tasks, you might compute different losses:

  • Classification tasks: Cross-entropy over class labels (e.g., sentiment categories)
  • Instruction following: Loss only on the model's response portion, ignoring the instruction tokens
  • Preference learning: Ranking loss comparing preferred vs rejected outputs

The key insight: you're backpropagating through the exact same forward pass you just learned about, but the loss tells the model what to optimize for in your specific use case.

Why It Matters

The loss computation directly shapes what your model learns. Computing loss only on response tokens (masking instruction tokens) teaches the model to generate good answers without "memorizing" prompts unnecessarily.

Key Takeaway: The loss function transforms your fine-tuning objective into a numerical signal that drives parameter updates—typically next-token cross-entropy, but customizable for task-specific goals.