Class Token (CLS Token)
You'll learn: How Vision Transformers use a special learnable token to gather image-level information for classification tasks.
The Classification Challenge
When processing an image broken into patches (like we learned earlier), Vision Transformers treat each patch as a separate token. But here's the problem: how do we get a single prediction for the entire image when we have dozens of patch representations?
We need one unified representation that captures information from the whole image—not just individual patches.
The Class Token Solution
The CLS token (short for "class token") is a special learnable embedding that we prepend to the sequence of patch tokens before feeding them into the Transformer layers.
Think of it like adding a "team captain" to a group of players. While each patch token represents a specific region of the image, the CLS token has no fixed spatial location—its job is to attend to all patches and gather global image-level information through the self-attention mechanism.
How It Works
- Start with your sequence of patch embeddings (e.g., 196 patches from a 224×224 image)
- Prepend one additional learnable token—the CLS token
- Now you have 197 tokens total (1 CLS + 196 patches)
- After passing through all Transformer layers, the final representation of the CLS token is used as the image representation
- This CLS representation goes to a classification head (usually a simple linear layer) to predict the class
The CLS token learns to aggregate relevant information from all patches during training, becoming a summary of the entire image.
Key Takeaway: The CLS token is a learnable "summary token" prepended to patch sequences whose final representation captures global image information used for classification.