Mean Reciprocal Rank (MRR)

Mean reciprocal rank was the scoring measure for the first TREC question-answering track, which NIST ran in 1999. In TREC-8, each system returned up to five ranked answers per question; a question scored the reciprocal of the rank of the first correct answer (1 for first, 1/2 for second, 1/3 for third) or 0 if none of the five was correct, and a run’s score was the mean across all questions (Voorhees, The TREC-8 Question Answering Track Report). Voorhees noted that the measure is bounded between 0 and 1, averages well, and is closely related to the average precision used in document retrieval.

That definition carries over directly to retrieval for AI systems. If your retriever returns a ranked list of chunks and you know which chunk answers each test question, MRR tells you how high the first useful result typically lands. If the first relevant chunk ranked second for every query, MRR would be 0.5; so would a retriever that is right first time on half the queries and never right on the rest, which is why the average needs the distribution beside it. It matters for retrieval-augmented generation because the model only sees the top few results: a relevant chunk ranked fifth contributes nothing when you pass the top three.

MRR only looks at the first relevant result. Nick Craswell’s reference entry ties it to a user model in which the user wants to see one relevant document and reads down the ranking until finding it (Craswell, Mean Reciprocal Rank, Encyclopedia of Database Systems). If an answer needs several pieces of evidence, or you care about everything in the top k, use a measure that counts all of them, such as recall at k. It also needs labeled relevance judgments, which means a test set of questions with known answers. Pair it with context relevance, which scores whether what was retrieved is on topic at all. How to evaluate a RAG system shows where retrieval metrics fit alongside answer quality, and the RAG topic collects the episodes on retrieval.