Course contentsShow
← Machine Learning and Deep Learning
Lesson 2613 of 3,53856. Few-Shot and Meta-LearningFree lesson

Reptile: A Simpler Meta-Learning Algorithm

Understanding Reptile's approach: repeatedly moving toward task-specific parameters without explicit second-order gradients.

Reptile: A Simpler Meta-Learning Algorithm

What you'll learn: How Reptile achieves meta-learning by repeatedly moving model parameters toward task-specific solutions without computing expensive second-order gradients.

The Core Idea

Reptile simplifies MAML's approach by removing the need for second-order gradients (gradients of gradients). Instead of explicitly computing how gradients change across tasks, Reptile uses a beautifully simple idea: repeatedly take steps toward the parameters you get after training on individual tasks.

Think of it like this: imagine you're trying to find a central meeting point for friends scattered across a city. Instead of calculating the optimal spot mathematically, you simply walk a bit toward each friend's location, one at a time. Eventually, you converge to a good central spot. Reptile does exactly this with model parameters.

How It Works

  1. Initialize your meta-model with parameters θ
  2. Sample a task from your task distribution
  3. Train on that task for a few gradient steps, obtaining new parameters θ'
  4. Move your meta-parameters toward θ': θ ← θ + ε(θ' - θ)
  5. Repeat for many tasks

The key difference from MAML: you don't backpropagate through the inner-loop training steps. You just compute the direction from your starting point to where task-specific training led you, then move that way.

Why This Matters

Reptile achieves comparable performance to MAML while being:

  • Computationally cheaper: no second-order derivatives
  • Simpler to implement: just SGD at two levels
  • More memory efficient: no need to store inner-loop computation graphs

The algorithm works because moving toward task-specific solutions implicitly finds parameters that are good starting points for fast adaptation—the essence of meta-learning.

Key Takeaway: Reptile meta-learns by repeatedly updating parameters in the direction of task-specific training outcomes, achieving MAML-like benefits without expensive second-order gradient computation.