Word error rate compares what a speech recognizer produced (the hypothesis) with a human-labeled reference transcript. You align the two, count the words the system added (insertions), missed (deletions), and got wrong (substitutions), and divide the total by the number of words in the reference: WER = (I + D + S) / N (Microsoft Learn, Test accuracy of a custom speech model). Microsoft’s worked example has one insertion, one deletion and one substitution, which comes to a WER of 60%. Because insertions count against the length of the reference, the formula allows values above 100% for a system that adds many spurious words.
Microsoft’s guidance gives rough bands for its speech service: 5 to 10% is good quality, around 20% is acceptable but may need more training, and 30% or more needs customization. Those bands depend on the audio and the task, so treat them as a starting point rather than a standard. The same guidance makes a more useful point about diagnosis: the error mix tells you where to look. Many deletions usually trace to weak audio signal, insertions to noise or crosstalk, and substitutions to domain terms the model has not seen enough of. To reproduce scores locally, Microsoft points to the sclite tool in NIST’s Scoring Toolkit (SCTK).
WER treats every word the same, which is its main blind spot. Morris, Maier and Green pointed out a related problem with its scale: a system that outputs one wrong word for every input word scores 100%, and one that outputs two wrong words per input word scores 200%, yet both communicate no information at all, so the difference between them is meaningless; they proposed match error rate and word information lost as alternatives (Morris et al., From WER and RIL to MER and WIL). Mishearing a drug name or a dollar amount costs as much as dropping “the.” It also scores the lexical transcript, so formatting such as punctuation, capitalization and numerals is outside it; Microsoft’s token error rate extends the same formula to the displayed format. If a transcript feeds an agent or a search index, also check the words your workflow depends on, as part of the wider multimodal AI evaluation the AI, decoded explainer on multimodal AI describes.
Sources
- Microsoft Learn: Test accuracy of a custom speech model — Defines WER = (I + D + S) / N with a worked example and rough quality bands.
- Morris, Maier and Green: From WER and RIL to MER and WIL — Shows WER can exceed 100% meaninglessly and proposes match error rate and word information lost.