F1 Score

F1 combines precision and recall as 2 × precision × recall / (precision + recall). Google’s classification guide explains both the formula and why metric choice depends on the costs of false positives and false negatives (Google, Classification metrics). The equivalent count formula is 2TP / (2TP + FP + FN), which leaves out true negatives. F1 is a summary of precision and recall, not the percentage of all predictions that are correct.

For a concrete example, suppose a ticket classifier flags 100 tickets for escalation and 80 truly need it. Precision is 80/100 = 0.8. If 100 tickets in the whole test set need escalation, recall is also 0.8 and F1 is 0.8. If the system instead flags only 50 tickets and 40 are correct, precision stays at 0.8, recall falls to 0.4 and F1 falls to about 0.53.

Discussing what constitutes an acceptable evaluation score for a product, Greg Statton says, “you have to understand the end use” (episode 27, 30:30).

For builders, decide whether a missed escalation and an unnecessary escalation have equal costs before using F1 as your main score. Keep the underlying counts visible and state which class you measure. Precision and recall explains the two components; AI agent metrics beyond accuracy covers measurement beyond a single classification score.