Query Performance Metrics
What you'll learn: How to track and interpret the key metrics that reveal how well your vector search system is performing.
When you run vector searches in production, you need to know if your system is healthy and fast. Think of it like monitoring a restaurant: you track how long customers wait (latency), how many orders you serve per hour (throughput), and how often orders get completed correctly (success rate).
Core Performance Metrics
Latency measures how long each query takes from submission to receiving results. This is typically measured in milliseconds. High latency means slow search responses, which directly impacts user experience. Track percentiles (p50, p95, p99) rather than just averages—that p99 spike might reveal a serious bottleneck affecting real users.
Throughput counts how many queries your system handles per second (QPS - queries per second). This tells you capacity: can your database handle current load? What about growth? Low throughput means your system will struggle under concurrent users.
Query Success Rate tracks what percentage of queries complete successfully versus timing out, erroring, or failing. A 95% success rate means 1 in 20 queries fails—unacceptable for most applications.
Identifying Bottlenecks
When latency climbs, check if your index type needs tuning (recall from Lesson 262). If throughput drops, you might need horizontal scaling or connection pool adjustments. Low success rates often indicate resource exhaustion or network issues.
These three metrics together paint a complete picture: fast queries (latency), high capacity (throughput), and reliability (success rate) define a healthy vector search system.
Key Takeaway: Monitor latency, throughput, and success rates continuously to catch performance degradation early and maintain a responsive, reliable vector search experience.