Precision and recall measure different kinds of success. Precision is the share of the system’s positive calls that were correct. Recall is the share of actual positives it found. A spam filter with high precision has mostly real spam among its flagged messages; one with high recall catches most of the real spam. Precision does not measure the share of legitimate mail that was incorrectly flagged: that is the false-positive rate, with a different denominator.
Changing the decision threshold often trades precision against recall, but precision need not fall at every step. If the next message you flag is actually spam, you can improve both. Google’s classification metrics guide defines the two measures and describes their usual inverse relationship.
Which you favor is a product decision: a fraud detector may tolerate more false alarms to avoid missing fraud; a content filter may need to avoid blocking legitimate posts. The F1 score combines precision and recall with equal weight. It does not decide which mistake your use case can afford.
For ranked retrieval, precision at K (precision@K) and recall at K (recall@K) use only the first K returned documents. Suppose the first five contain three relevant documents, and the complete collection has twelve relevant documents for that query. Precision@5 is 3/5 = 60%; recall@5 is 3/12 = 25%. These are assigned binary relevance judgments with a known total, not measured retriever performance or a recommended cutoff. Recall needs that relevant-document total; precision at K only needs judgments for the returned results. This cutoff concerns retrieved documents, not top-K token sampling.
When most flags are false alarms
A classifier flags support tickets for follow-up. Start with an invented batch of 10,000 tickets where 1% really need it. The starting rule catches 90% of those tickets and falsely flags 5% of the rest.
This rule catches 90.0% of tickets needing follow-up and falsely flags 5% of tickets that do not.
| Needs follow-up | Does not | |
|---|---|---|
| Flagged | 90True positives | 495False positives |
| Not flagged | 10False negatives | 9,405True negatives |
- Precision
- 90 / 585 = 15.4%
- True flags / all flags
- Recall
- 90 / 100 = 90.0%
- True flags / all tickets needing follow-up
- Accuracy
- 9,495 / 10,000 = 95.0%
- All correct decisions / all tickets
585 tickets flagged: 90 need follow-up and 495 do not. Precision 15.4%; recall 90.0%.
Change the share needing follow-up while keeping the rule fixed. This assumes its recall and false-positive rate stay the same; real workload changes can alter both. All counts and rules here are assigned for teaching, not measured model results or recommended settings.
Check your understanding: does 90% recall mean 90% of flags are right?
No. At the starting 1% share, the rule finds 90 of the 100 tickets needing follow-up. It also flags 495 of the other 9,900. Only 90 of its 585 flags are right: 15.4% precision. At a 50% share with the same rule, precision becomes 94.7% while recall stays 90.0%.
Move the threshold, inspect the mistakes
Twelve invented messages, five actually spam. A message is flagged when its score is at least the threshold, including ties. Scores here are illustrative, not calibrated probabilities.
The worked example uses a threshold of 0.50. With JavaScript enabled, you can change it below.
6 of 12 messages flagged: 4 spam caught, 2 false alarms, 1 spam missed.
| Actual spam | Actual not spam | |
|---|---|---|
| Flagged | 4Correct flags (TP) | 2False alarms (FP) |
| Not flagged | 1Missed spam (FN) | 5Correct rejections (TN) |
- Precision
- 66.7%4 / 6 · correct among flagged
- Recall
- 80.0%4 / 5 · caught among actual spam
- F1
- 72.7%8 / 11 · 2TP / (2TP + FP + FN)
Nothing is flagged, so precision is undefined (0 / 0). Recall and F1 are zero for these five actual spam messages.
Inspect all 12 scored messages
| Message | Score | Actual label | Decision |
|---|---|---|---|
| A | 0.95 | Spam | Flagged (TP) |
| B | 0.90 | Spam | Flagged (TP) |
| C | 0.80 | Not spam | False alarm (FP) |
| D | 0.70 | Spam | Flagged (TP) |
| E | 0.50 | Spam | Flagged (TP) |
| F | 0.50 | Not spam | False alarm (FP) |
| G | 0.40 | Spam | Missed spam (FN) |
| H | 0.35 | Not spam | Not flagged (TN) |
| I | 0.30 | Not spam | Not flagged (TN) |
| J | 0.20 | Not spam | Not flagged (TN) |
| K | 0.10 | Not spam | Not flagged (TN) |
| L | 0.05 | Not spam | Not flagged (TN) |
Can lowering the threshold improve precision and recall together?
Yes. Here, moving from 0.50 to 0.40 catches message G, which is spam, without adding a false alarm. Precision rises from 4/6 to 5/7, and recall rises from 4/5 to 5/5. The tradeoff is common, but precision need not fall at every step.
F1 weights precision and recall equally; it does not encode the cost of a false alarm or a miss. Formulas: Google’s classification metrics guide. How thresholds work.
Sources
- Google, Classification metrics — Defines the metric denominators and explains why precision and recall often trade off as the threshold changes.
- scikit-learn: precision, recall and F-measures — Defines precision and recall from true positives, false positives and false negatives.
- Saito and Rehmsmeier (2015): precision-recall evaluation on imbalanced datasets — Table 2 and the class-distribution discussion explain why positive-class frequency matters when interpreting precision. The ticket example here uses our own assigned counts.
- Introduction to Information Retrieval: evaluating ranked results — Explains evaluating the top K results. The document counts below are assigned for teaching.
Go deeper
- Ragas: Context Recall docs
Apply recall to retrieved evidence. Compare reference-answer, reference-context and document-ID variants before choosing a scoring unit.
- Ragas: Context Precision docs
Compare ranked context precision with the document-ID variant. The ranked score accounts for where useful chunks appear.
