Query Processing Flow
You'll learn: How a search engine transforms a user's raw query into a ranked list of relevant documents through a series of well-defined steps.
The Journey from Query to Results
When a user types "best Italian restaurants NYC" and hits search, the query doesn't just magically find documents. It flows through a carefully orchestrated pipeline that bridges the gap between human language and the inverted index structure you've learned about.
Core Processing Steps
1. Query Reception
The search API receives the raw query string from the user interface, along with metadata like user location, language preferences, and filtering options.
2. Query Parsing and Normalization
Just as documents were tokenized during indexing, the query undergoes similar treatment:
- Tokenization: Split "best Italian restaurants NYC" into individual terms
- Normalization: Convert to lowercase, remove punctuation
- Stemming: Reduce "restaurants" to root form "restaurant"
- Stop word removal: Optionally drop common words like "the" or "a"
3. Term Lookup
Each processed query term is looked up in the inverted index. For example:
"italian"→ retrieves posting list with document IDs"restaurant"→ retrieves another posting list"nyc"→ retrieves a third posting list
4. Candidate Document Retrieval
The system intersects or unions these posting lists to identify documents that match the query criteria. Documents appearing in multiple lists are stronger candidates.
Think of this like searching your library's card catalog: you look up each subject card, gather the book numbers, then find which books appear under multiple relevant subjects.
Key Takeaway: Query processing mirrors the indexing pipeline in reverse—transforming human input into normalized terms, looking them up in the inverted index, and retrieving candidate documents for ranking.