Course contentsShow
← Machine Learning and Deep Learning
Lesson 1348 of 3,53830. Vision TransformersFree lesson

Interpolating Positional Embeddings

Adapting models to different input resolutions by interpolating learned position embeddings.

Interpolating Positional Embeddings

What you'll learn: How Vision Transformers adapt to different image resolutions by smoothly adjusting their learned position information.

The Resolution Problem

When you train a Vision Transformer (ViT) on images of one size (say 224×224 pixels), it learns positional embeddings for exactly that grid of patches. But what happens when you want to use the model on higher-resolution images at test time—say 384×384?

The number of patches changes! A 224×224 image with 16×16 patches gives you a 14×14 grid (196 patches). A 384×384 image gives you a 24×24 grid (576 patches). Your model only learned positions for 196 patches, not 576.

The Interpolation Solution

Instead of retraining from scratch, we interpolate the positional embeddings—stretching or compressing the learned position patterns to fit the new grid size.

Think of it like resizing a photograph: you have pixel values at specific locations, and you use interpolation (often 2D bicubic) to estimate what values should appear at the new, finer grid positions.

How It Works

  1. Reshape the learned 1D positional embeddings back into their 2D spatial grid (e.g., 14×14)
  2. Use 2D interpolation (typically bicubic) to upsample to the new grid size (e.g., 24×24)
  3. Flatten back to 1D and use these new embeddings

This lets you leverage pre-trained weights while gaining the benefits of higher resolution—more detail, better performance—without expensive retraining.

Key Takeaway: Interpolating positional embeddings allows Vision Transformers to flexibly handle different input resolutions by smoothly adapting the learned spatial position information to new grid sizes.