Course contentsShow
← Machine Learning and Deep Learning
Lesson 1339 of 3,53830. Vision TransformersFree lesson

Patch Embedding Layer

Linear projection of flattened patches into embedding vectors of fixed dimension.

Patch Embedding Layer

What you'll learn: How Vision Transformers convert image patches into vector representations that can be processed like word tokens in language models.

The Core Idea

In Vision Transformers, we first split an image into fixed-size patches (like 16×16 pixels), treating each patch as a "token"—similar to how words are tokens in NLP. But Transformers can't directly process raw pixel grids; they need fixed-length vectors.

The Patch Embedding Layer solves this by flattening each patch into a 1D vector and then applying a linear projection (a learnable matrix multiplication) to map it into an embedding vector of a chosen dimension (often 768 or 1024).

Why This Matters

Think of it like converting different-sized photos into standard passport photos. No matter what's in the original patch, it gets transformed into a uniform vector representation that the Transformer can work with.

The process:

  1. Flatten: A 16×16×3 RGB patch becomes a vector of length 768 (16 × 16 × 3)
  2. Project: Multiply by a learned weight matrix to produce the embedding dimension (e.g., 768 → 768)

This projection is typically implemented as a single convolutional layer with kernel size equal to the patch size and stride equal to the patch size—it simultaneously splits the image into patches and embeds them.

What Gets Learned

The weight matrix in this layer learns to extract meaningful features from raw pixel values, similar to how word embeddings learn semantic meaning. Different dimensions might capture edges, colors, textures, or more abstract visual concepts.

Key Takeaway: The Patch Embedding Layer linearly projects flattened image patches into fixed-dimensional embedding vectors, converting visual data into the token-like format that Transformers require.