Course contentsShow
← AI Engineering
Lesson 331 of 1,8869. Retrieval-Augmented Generation (RAG) - BasicsFree lesson

Query Time vs Index Time Operations

Separating preprocessing tasks done once during indexing from operations performed for each user query.

Query Time vs Index Time Operations

What You'll Learn

You'll learn how to separate one-time preprocessing work (done during indexing) from operations that run every time a user asks a question.

The Core Concept

In a RAG system, not all work happens at the same time. Some operations are expensive but only need to happen once when you build your knowledge base (index time), while others must happen quickly for every user query (query time).

Index Time Operations

These happen once when you're setting up your knowledge base:

  • Document chunking: Breaking documents into smaller pieces
  • Embedding generation: Converting text chunks into vectors
  • Vector indexing: Building the search structure (HNSW, IVF, etc.)
  • Metadata extraction: Pulling out dates, authors, categories
  • Building filters: Pre-computing filterable fields

Think of this as meal prep on Sunday — you chop vegetables, marinate meat, and organize ingredients once. It takes time, but you only do it once.

Query Time Operations

These happen every time a user asks a question:

  • Query embedding: Converting the user's question into a vector
  • Vector search: Finding relevant chunks
  • Metadata filtering: Applying user-specific filters
  • Reranking: Ordering results by relevance
  • LLM generation: Creating the final answer

This is like cooking dinner each night — you work with your prepped ingredients to quickly create a meal.

The Performance Impact

Getting this wrong is costly. If you try to embed 100,000 documents during every query, users wait minutes instead of seconds. If you don't pre-compute metadata filters, searches become painfully slow at scale.

The rule: Do heavy lifting at index time, keep query time lean.

Key Takeaway: Separate expensive one-time preprocessing (index time) from fast per-query operations (query time) to keep your RAG system responsive and scalable.