MAML's Inner and Outer Loop
What you'll learn: How MAML uses two nested optimization loops to learn a model initialization that can quickly adapt to new tasks.
The Two-Level Optimization Structure
MAML's brilliance lies in its nested loop architecture that separates task-specific learning from meta-learning.
Inner Loop: Task-Specific Adaptation
The inner loop is where your model learns a specific task. Given an initial set of parameters θ (the meta-parameters), the model:
- Takes a few examples from a task's support set
- Performs one or more gradient descent steps to adapt θ to that specific task
- Produces task-specific parameters θ'
Think of this as a student cramming for a specific exam using their general knowledge as a starting point.
Outer Loop: Meta-Parameter Updates
The outer loop optimizes the starting point itself. It:
- Evaluates how well the adapted parameters θ' perform on the task's query set
- Backpropagates through the inner loop updates
- Adjusts the meta-parameters θ to make future task adaptations more effective
This is like adjusting your general study strategy based on how well cramming worked across many different exams.
The Meta-Learning Objective
MAML doesn't just want good performance on training tasks—it wants parameters θ that become good parameters θ' with minimal adaptation. The outer loop essentially asks: "Which initial weights lead to the fastest learning across all tasks?"
Key Takeaway: MAML's inner loop adapts to individual tasks through standard gradient descent, while the outer loop optimizes the meta-parameters to make that adaptation as effective as possible—learning not just what to learn, but how to learn quickly.