Course contentsShow
← AI Engineering
Lesson 207 of 1,8866. Embeddings and Semantic SearchFree lesson

Dimensionality in Embeddings

Explore common embedding dimensions (384, 768, 1536) and the trade-offs between size, performance, and storage.

Dimensionality in Embeddings

What you'll learn: How embedding vector sizes affect performance, storage, and the quality of semantic search in your AI applications.

The Concept of Embedding Dimensions

When you create an embedding (a vector representation of text), you must choose how many numbers will be in that vector—this is its dimensionality. Think of it like describing a person: you could use 3 traits (tall, funny, kind), or 1000 traits. More dimensions capture more nuance, but come with costs.

Common Embedding Dimensions

Most embedding models produce vectors with standard sizes:

  • 384 dimensions: Compact models like all-MiniLM-L6-v2
  • 768 dimensions: Mid-sized models like sentence-transformers/all-mpnet-base-v2
  • 1536 dimensions: Larger models like OpenAI's text-embedding-ada-002
  • 3072+ dimensions: State-of-the-art models for maximum accuracy

The Trade-offs

Higher dimensions (1536+):

  • ✅ Capture more semantic nuance and relationships
  • ✅ Better at distinguishing similar concepts
  • ❌ Require more storage (4x the space from 384→1536)
  • ❌ Slower similarity calculations
  • ❌ More expensive to store in vector databases

Lower dimensions (384):

  • ✅ Fast computations and queries
  • ✅ Minimal storage requirements
  • ✅ Good enough for many practical tasks
  • ❌ May miss subtle semantic differences

Choosing Your Dimension

Start with what your embedding model provides by default. Only worry about dimension size when you face storage constraints or need to optimize query speed. The quality of the embedding model matters far more than raw dimension count—a well-trained 384-dimensional model often outperforms a poorly-trained 1536-dimensional one.

Key Takeaway: Embedding dimensionality is a balance between semantic richness and computational efficiency; higher dimensions capture more meaning but cost more in storage and speed.