Course contentsShow
← AI Engineering
Lesson 254 of 1,8867. Vector Databases - FundamentalsFree lesson

The Curse of Dimensionality

Learn why high-dimensional vectors make distance calculations expensive and indexes necessary.

The Curse of Dimensionality

What you'll learn: Why adding more dimensions to your vectors makes similarity search exponentially harder and slower.

The Problem with High-Dimensional Space

Imagine searching for similar items in a one-dimensional line—easy! Now picture a 2D grid, then a 3D cube. With each dimension you add, the "space" grows exponentially larger. This is the curse of dimensionality.

When working with embeddings (like the 1536-dimensional vectors from OpenAI's API), you face a counter-intuitive reality: in high-dimensional space, almost everything becomes "far away" from everything else. Points that seem similar might still be distant, and the computational cost of comparing them skyrockets.

Why Distance Calculations Get Expensive

You learned earlier about cosine similarity and dot product for measuring vector similarity. These calculations require comparing every dimension between two vectors. For a 1536-dimensional vector, that's 1536 multiplication operations per comparison.

Now scale that to millions of documents. A simple search requires comparing your query against every document—millions of 1536-dimensional calculations. Even modern computers struggle with this at scale.

Why This Makes Indexes Necessary

This is exactly why flat (brute-force) indexing becomes impractical for large collections. Without smart indexing strategies, your search time grows linearly with dataset size. A million-document search takes a thousand times longer than a thousand-document search.

Vector databases solve this by using specialized indexes that approximate nearest neighbors without checking every single vector—trading perfect accuracy for massive speed gains.

Key Takeaway: High-dimensional vectors make exhaustive distance calculations prohibitively expensive at scale, which is why vector databases use approximation indexes instead of brute-force search.