Course contentsShow
← AI Engineering
Lesson 277 of 1,8867. Vector Databases - FundamentalsFree lesson

Pre-filtering vs Post-filtering

Understanding when metadata filters are applied before or after vector search and performance implications.

Pre-filtering vs Post-filtering

What you'll learn: How the timing of metadata filters—before or after vector search—dramatically affects performance and result quality.

The Two Filtering Strategies

When searching a vector database with metadata filters (like "author = 'Smith'" or "year > 2020"), you have two options for when to apply those filters:

Pre-filtering means applying metadata constraints before the vector search runs. The system only searches through vectors that already match your metadata criteria.

Post-filtering means running the full vector search first, then filtering out results that don't match your metadata requirements afterward.

The Library Analogy

Imagine searching for books similar to a mystery novel you love:

  • Pre-filtering: Ask the librarian to gather only mystery books published after 2020, then find the ones most similar to yours. You're searching a smaller, relevant collection.
  • Post-filtering: The librarian finds the 100 most similar books across the entire library, then removes everything that isn't a post-2020 mystery. You might end up with only 3 matches.

Performance Implications

Pre-filtering is faster when metadata drastically reduces your dataset (searching 1,000 items instead of 1,000,000). However, some index types (like HNSW) struggle with pre-filtering because they're optimized for searching the full dataset.

Post-filtering guarantees you get exactly K results if they exist, but wastes computation on vectors you'll discard. If your filter is very selective, you might retrieve 1,000 results but only keep 5.

The best choice depends on your filter selectivity and your vector database's index architecture.

Key Takeaway: Pre-filtering searches less data but may conflict with some indexes; post-filtering ensures result count but can waste computation on irrelevant vectors.