Course contentsShow
← Machine Learning and Deep Learning
Lesson 2548 of 3,53855. Self-Supervised LearningFree lesson

SimCLR: Simple Framework for Contrastive Learning

Understand SimCLR's architecture: data augmentation, encoder network, projection head, and contrastive loss.

SimCLR: Simple Framework for Contrastive Learning

What You'll Learn

You'll understand how SimCLR assembles four key components into an elegant, end-to-end contrastive learning system.

The Core Idea

SimCLR (Simple Framework for Contrastive Learning of Visual Representations) is Google's influential approach that brings together everything you've learned about contrastive learning into one cohesive pipeline. Think of it as a recipe where each ingredient serves a specific purpose: augmentation creates views, an encoder extracts features, a projection head maps to contrastive space, and the loss brings it all together.

The Four-Part Architecture

1. Data Augmentation

SimCLR applies two random augmentations to each image, creating positive pairs. Common augmentations include random cropping, color distortion, and Gaussian blur. These transformations create different "views" of the same image that should be recognized as similar.

2. Encoder Network (f)

A base encoder (typically ResNet) processes each augmented image, extracting rich visual features. This produces a representation vector h from the input.

3. Projection Head (g)

A small MLP (usually 2 layers) maps the encoder output h to a lower-dimensional space z where contrastive loss is computed. This is where the magic happens—the projection head is discarded after training, and only the encoder is kept for downstream tasks.

4. Contrastive Loss

SimCLR uses the InfoNCE loss with a temperature parameter, treating augmented pairs as positives and all other examples in the batch as negatives. Large batch sizes (thousands) are crucial for providing many negative examples.

The beauty is in the simplicity: no memory banks, no momentum encoders—just straightforward end-to-end training with these four components working together.

Key Takeaway: SimCLR combines stochastic data augmentation, a base encoder, a projection head, and NT-Xent (normalized temperature-scaled cross-entropy) loss into a simple yet powerful framework that learns strong visual representations without labels.