Dense Vector Representations for Text
You'll learn: How neural models transform text into fixed-length numerical vectors that capture semantic meaning for retrieval systems.
From Words to Numbers
Traditional search uses exact keyword matching, but semantic search needs to understand meaning. Dense vector representations (also called embeddings) solve this by converting text—whether a single word, sentence, or paragraph—into a fixed-length array of floating-point numbers (typically 384 to 1536 dimensions).
Think of it like GPS coordinates: just as (latitude, longitude) maps any location to two numbers, embeddings map text to points in high-dimensional space where semantically similar texts cluster together.
How Dense Vectors Enable Semantic Search
When you encode "puppy" and "young dog" into vectors, they'll point in nearly the same direction in vector space—even though they share no exact words. This is the magic: neural embedding models learn semantic relationships during training, placing related concepts near each other geometrically.
Unlike sparse representations (think thousands of dimensions, mostly zeros for keyword counts), dense vectors pack meaning into every dimension. A 768-dimensional vector means 768 learned features, each capturing subtle aspects of meaning—topics, sentiment, entities, relationships.
The Embedding Process
A pre-trained embedding model (like sentence-transformers models) takes raw text and outputs a dense vector:
- Input: "How do I reset my password?"
- Output:
[0.23, -0.51, 0.07, ..., 0.19](768 numbers)
These vectors become your searchable representation in a vector database. When a user query arrives, you embed it the same way, then find the closest vectors using cosine similarity or Euclidean distance.
Key Takeaway: Dense vector representations transform text into fixed-size numerical arrays where semantic similarity becomes geometric proximity, enabling RAG systems to retrieve contextually relevant documents rather than just keyword matches.