Course contentsShow
← Machine Learning and Deep Learning
Lesson 1704 of 3,53837. LLM Fine-Tuning: Full and Parameter-EfficientFree lesson

Backpropagation Through All Layers

How gradients flow backward through every transformer layer, updating all weights from output to input embeddings.

Backpropagation Through All Layers

You'll learn how gradients flow backward from the loss through every transformer layer during fine-tuning, updating weights from the output head all the way back to input embeddings.

The Reverse Journey of Learning

After computing loss in the forward pass, backpropagation traces the error backward through your entire LLM architecture. Starting from the final loss value, the chain rule computes gradients layer by layer in reverse order: first through the unembedding layer, then backward through each transformer block (attention, feedforward networks, residual connections), and finally reaching the input embeddings.

How Gradients Flow Backward

Think of backpropagation like tracing blame for a mistake through a chain of decisions. If the model's prediction was wrong, every weight that contributed needs to know "how much was this my fault?" The gradient tells each parameter how to adjust.

The process moves through:

  1. Output layers - gradients start here from the loss function
  2. Each transformer block - attention heads, feedforward networks, layer norms all receive gradients
  3. Residual connections - gradients split and flow through parallel paths
  4. Position embeddings and token embeddings - even the initial representations get updated

Why All Layers Matter

In full fine-tuning, every parameter receives gradient updates. Unlike approaches you'll learn later (like LoRA), nothing is frozen. This means the model can fundamentally reshape its learned representations to fit your specific task, but it also means updating billions of parameters requires substantial memory and compute.

The chain rule ensures gradients diminish or explode less than in older architectures, thanks to residual connections and careful normalization—but you're still propagating signals through dozens of layers.

Key Takeaway: Backpropagation in full fine-tuning computes and applies gradients to every weight in your LLM, flowing backward from loss through all transformer layers to input embeddings, enabling complete model adaptation.