Course contentsShow
← AI Engineering
Lesson 248 of 1,8867. Vector Databases - FundamentalsFree lesson

The Curse of Dimensionality

Why high-dimensional vector search is computationally expensive and how vector DBs optimize for it.

The Curse of Dimensionality

You'll learn: Why searching through high-dimensional vector spaces becomes surprisingly difficult and computationally expensive.

The Core Problem

When embeddings have hundreds or thousands of dimensions (like OpenAI's 1,536-dimensional vectors), something counterintuitive happens: all points start looking equally distant from each other. This is the "curse of dimensionality."

Why High Dimensions Are Costly

Imagine searching for the nearest neighbor in a 2D space—you can visualize it easily. Now picture 1,536 dimensions. To find similar vectors, you must:

  1. Calculate distance to every vector in your database (cosine similarity or dot product)
  2. Compare across all 1,536 dimensions for each calculation
  3. Sort results to find the top-k matches

With a million documents, that's millions of calculations × 1,536 dimensions each. The compute grows linearly with both dimensions and dataset size.

The Distance Problem

In high dimensions, the difference between the "nearest" and "farthest" neighbor shrinks dramatically. Most points cluster in a thin shell at similar distances from any query point. This makes distinguishing truly relevant results harder and requires examining more candidates.

How Vector DBs Optimize

Vector databases like Pinecone, Weaviate, and Qdrant use specialized algorithms (which we'll cover later) to avoid checking every single vector. They trade perfect accuracy for massive speed improvements—often finding the approximate nearest neighbors in milliseconds instead of seconds.

Key Takeaway: High-dimensional vector search is expensive because distance calculations scale with dimensions and dataset size, and points become harder to distinguish—vector databases use clever approximation techniques to make this practical.