User Experience Metrics
What you'll learn: How to measure model impact through latency, throughput, and user satisfaction—not just accuracy.
Beyond Model Accuracy
You've mastered leading and lagging indicators, but accuracy alone won't tell you if users love—or hate—your ML system. A 95% accurate model that takes 10 seconds to respond might drive users away faster than an 85% accurate model that responds instantly.
User Experience (UX) metrics capture what actually matters to the people using your system:
The Three Pillars of UX Metrics
Latency measures response time from request to result. This includes model inference time, data fetching, preprocessing, and network overhead. Users perceive sub-200ms as instant; above 1 second feels sluggish. Track percentiles (p50, p95, p99) not just averages—one slow request can ruin an experience.
Throughput measures requests handled per second. High throughput ensures your system scales under load without degrading individual experiences. The balance matters: batching improves throughput but may increase individual latency.
User Satisfaction captures subjective quality through direct and indirect signals. Direct signals include explicit feedback (thumbs up/down), ratings, or surveys. Indirect signals include engagement metrics: click-through rates, time-on-page, completion rates, or abandonment. A recommendation system might be accurate but still frustrate users if results feel repetitive or irrelevant.
Why These Matter More Than You Think
An image classifier might have 98% accuracy, but if users abandon your app because results take 5 seconds, accuracy is meaningless. Conversely, a faster model with 90% accuracy that users engage with more frequently delivers more business value.
Key Takeaway: Production ML success requires measuring latency, throughput, and user satisfaction alongside model accuracy—technical excellence means nothing if users have a poor experience.