Course contentsShow
← Machine Learning and Deep Learning
Lesson 2610 of 3,53856. Few-Shot and Meta-LearningFree lesson

MAML Gradient Computation

Deriving gradients through gradients: how MAML computes second-order derivatives for meta-optimization.

MAML Gradient Computation

You'll learn: How MAML computes second-order derivatives by taking gradients through gradients to meta-optimize across tasks.

The Core Challenge

In MAML's inner loop, you adapt model parameters using task-specific gradients. In the outer loop, you need to update the initial parameters to make future adaptations better. This requires computing gradients of gradients—a second-order derivative that measures how changing the starting point affects performance after adaptation.

Plain English Explanation

Imagine teaching someone to learn faster. You don't just evaluate their final performance—you evaluate how well they improved from your starting instructions. To optimize those starting instructions, you need to understand how small changes to them ripple through their entire learning process.

In MAML:

  1. Inner loop: compute θ' = θ - α∇L_task(θ) (adapt to a task)
  2. Outer loop: compute ∇_θ L_task(θ') (gradient through the adapted parameters)

The trick? θ' itself depends on θ, so computing ∇_θ L_task(θ') requires the chain rule through the inner gradient update. This is gradient through gradient.

The Mathematics

The meta-gradient is:

∇_θ L(θ') = ∇_θ L(θ - α∇L_task(θ))

Using the chain rule, this expands to include a Hessian term (second derivatives), making computation expensive but powerful—it captures how the adaptation trajectory itself should change.

Why It Matters

This second-order optimization is what makes MAML truly "meta"—it optimizes for adaptability, not just task performance. The computational cost is high, but the result is an initialization that enables rapid few-shot learning.

Key Takeaway: MAML computes gradients through the inner-loop gradient update, requiring second-order derivatives that optimize the starting parameters for maximum adaptability across tasks.