Course contentsShow
← Machine Learning and Deep Learning
Lesson 1957 of 3,53842. Retrieval-Augmented Generation: FoundationsFree lesson

What Is a Vector Database and Why RAG Needs It

Understanding the role of vector databases in storing and retrieving embeddings for RAG systems.

What Is a Vector Database and Why RAG Needs It

What you'll learn: Why RAG systems rely on specialized vector databases to store and retrieve embeddings efficiently at scale.

The Problem with Traditional Databases

In RAG systems, you convert documents into high-dimensional embeddings (vectors of hundreds or thousands of numbers). Traditional relational databases like PostgreSQL or MySQL aren't designed to quickly find "similar" vectors among millions of entries. Searching for nearest neighbors in high-dimensional space using standard SQL would be painfully slow.

What Is a Vector Database?

A vector database is a specialized storage system optimized for:

  • Storing embeddings (dense numerical vectors)
  • Similarity search (finding vectors closest to a query vector)
  • Speed at scale (millions or billions of vectors)

Think of it like a library organized not alphabetically, but by meaning. When you ask "books about space exploration," it instantly finds books with similar semantic content, even if they don't share exact keywords.

Why RAG Needs It

During RAG's retrieval phase, you:

  1. Convert the user's query into an embedding
  2. Search the vector database for the top-K most similar document embeddings
  3. Return those documents as context for the LLM

Without a vector database, computing similarity scores against every stored embedding would take seconds or minutes. Vector databases use specialized indexing algorithms (like HNSW or IVF) to reduce search from O(n) to near O(log n) complexity.

Popular vector databases include Pinecone, Weaviate, Qdrant, Chroma, and FAISS (Facebook's library). They handle the heavy lifting of approximate nearest neighbor search, letting your RAG system scale from hundreds to millions of documents.

Key Takeaway: Vector databases are purpose-built for fast similarity search over embeddings, making real-time RAG retrieval practical at scale where traditional databases would fail.