Course contentsShow
← Machine Learning and Deep Learning
Lesson 2571 of 3,53855. Self-Supervised LearningFree lesson

Masked Image Modeling: Core Concept

Understanding how masking patches in images enables self-supervised learning without labels.

Masked Image Modeling: Core Concept

What you'll learn: How masking random patches of an image forces a model to learn rich visual representations without any labels.

The Core Idea

Masked Image Modeling (MIM) borrows inspiration from masked language modeling in NLP (like BERT's approach to text). The concept is beautifully simple: hide parts of an image, then train a neural network to reconstruct what's missing.

Here's how it works:

  1. Split the image into patches (like cutting a photo into small squares)
  2. Randomly mask some patches (typically 60-80% of them)
  3. Feed the visible patches into an encoder (usually a Vision Transformer)
  4. Predict the missing patches using the learned representations

Why This Enables Self-Supervised Learning

When you force a model to reconstruct masked patches, it must learn to understand:

  • Spatial relationships: What typically appears next to what?
  • Object structure: If you see part of a wheel, the rest is probably circular
  • Contextual semantics: Grass patches likely connect to sky patches differently than building patches

Unlike contrastive learning (which focuses on distinguishing similar/different images), MIM learns by reconstructing local visual details from global context. The model develops rich internal representations because successful reconstruction requires understanding both low-level textures and high-level semantics.

Analogy

Think of it like completing a jigsaw puzzle with most pieces removed. To fill in the blanks, you need to understand what objects typically look like, how colors transition, and what makes visual sense—exactly the kind of knowledge useful for downstream vision tasks.

Key Takeaway: Masked Image Modeling teaches models to understand images by predicting missing patches from visible context, creating powerful representations without requiring any human-labeled data.