Course contentsShow
← AI Engineering
Lesson 1692 of 1,88641. Speech and Audio DeploymentFree lesson

ASR Quality Metrics and Evaluation

Measuring Word Error Rate (WER) and Character Error Rate (CER) to evaluate ASR pipeline performance.

ASR Quality Metrics and Evaluation

You'll learn: How to quantify ASR system accuracy using Word Error Rate (WER) and Character Error Rate (CER) to measure transcription quality objectively.

Why Quality Metrics Matter

After deploying an ASR pipeline (like Whisper), you need objective measures to know if your transcriptions are good enough. Just as a spell-checker needs a way to count mistakes, ASR systems require standardized metrics to evaluate performance against ground-truth transcripts.

Word Error Rate (WER)

WER measures how many words were transcribed incorrectly compared to a reference transcript. It's calculated using three error types:

  • Substitutions (S): Wrong word ("cat" → "bat")
  • Deletions (D): Missing word
  • Insertions (I): Extra word added

Formula: WER = (S + D + I) / N × 100%, where N = total words in reference.

Example: If the reference has 100 words and your ASR made 5 substitutions, 2 deletions, and 1 insertion, WER = 8/100 = 8%.

Character Error Rate (CER)

CER works identically but at the character level instead of words. It's especially useful for:

  • Languages without clear word boundaries (Chinese, Japanese)
  • Evaluating spelling/capitalization accuracy
  • Fine-grained error analysis

Lower is better for both metrics—0% means perfect transcription.

Practical Interpretation

Think of WER like a test score: 5% WER is excellent (production-ready), 15% is acceptable for some use cases, 30%+ suggests serious issues. Different domains have different tolerance levels—medical transcription demands near-zero errors, while voice search can tolerate more.

Key Takeaway: WER and CER provide objective, comparable measurements of ASR accuracy by counting transcription errors against known-correct reference texts, enabling data-driven pipeline improvements.