Course contentsShow
← Machine Learning and Deep Learning
Lesson 2609 of 3,53856. Few-Shot and Meta-LearningFree lesson

MAML's Inner and Outer Loop

Understanding the two-level optimization: task-specific adaptation (inner loop) and meta-parameter updates (outer loop).

MAML's Inner and Outer Loop

What you'll learn: How MAML uses two nested optimization loops to learn a model initialization that can quickly adapt to new tasks.

The Two-Level Optimization Structure

MAML's brilliance lies in its nested loop architecture that separates task-specific learning from meta-learning.

Inner Loop: Task-Specific Adaptation

The inner loop is where your model learns a specific task. Given an initial set of parameters θ (the meta-parameters), the model:

  • Takes a few examples from a task's support set
  • Performs one or more gradient descent steps to adapt θ to that specific task
  • Produces task-specific parameters θ'

Think of this as a student cramming for a specific exam using their general knowledge as a starting point.

Outer Loop: Meta-Parameter Updates

The outer loop optimizes the starting point itself. It:

  • Evaluates how well the adapted parameters θ' perform on the task's query set
  • Backpropagates through the inner loop updates
  • Adjusts the meta-parameters θ to make future task adaptations more effective

This is like adjusting your general study strategy based on how well cramming worked across many different exams.

The Meta-Learning Objective

MAML doesn't just want good performance on training tasks—it wants parameters θ that become good parameters θ' with minimal adaptation. The outer loop essentially asks: "Which initial weights lead to the fastest learning across all tasks?"

Key Takeaway: MAML's inner loop adapts to individual tasks through standard gradient descent, while the outer loop optimizes the meta-parameters to make that adaptation as effective as possible—learning not just what to learn, but how to learn quickly.