A judge can score an answer against a reference, choose between two candidates, or inspect an agent trace. Its input should include the evidence needed for that judgment. A polished final response alone cannot establish that a tool action succeeded.
Zheng and colleagues’ MT-Bench study reports useful agreement with human preferences in its setting and documents position and verbosity biases. It also tested self-enhancement bias, a judge favoring its own answers, but says its limited data can’t determine whether the models show it. That result is not a universal accuracy guarantee for judges.
Example: test a judge on approved and rejected support answers, including subtly incorrect policy claims. Count false approvals as well as overall agreement. For pairwise comparisons, swap answer order and inspect whether the decision changes.
Our comparison guide covers the choice between model and human review. Hamel Husain’s episode 37 discussion at 8:23 emphasizes starting with actual failures. Galileo sponsored the show when episode 37 aired; co-host Atindriyo Sanyal, Galileo’s CTO, is introduced at 0:35. Use those failure categories to write a specific rubric, then validate the judge on held-out examples.
Agreement can hide false approvals
An invented validation set: 100 support answers labeled by a domain expert, 80 approved and 20 rejected. The judge agrees with the expert on 90 of them. That sounds strong. Now split the 10 disagreements: in 8, the judge approved an answer the expert rejected; in 2, it rejected one the expert approved.
| Expert approved | Expert rejected | |
|---|---|---|
| Judge approved | 78 | 8 |
| Judge rejected | 2 | 12 |
Overall agreement is 90/100 = 90%. But the judge passed 8 of the 20 bad answers, a 40% false-approval rate on exactly the cases the judge exists to catch. Report both numbers, and set the bar by what a passed bad answer costs you.
Hear it from the guest
“One of the things that's important with LLM as a judge is that you can trust the LLM as a judge. And to trust LLM as a judge, you have to compare it to some human labels.”
Quotes lightly edited to remove filler words.
Sources
- Zheng et al.: Judging LLM-as-a-Judge — Primary study of agreement with human preferences. Documents position and verbosity biases; its self-enhancement test was inconclusive on its limited data. Checked November 7, 2026.
Go deeper
- What is LLM-as-a-judge, and when can you trust it? AI, decoded · What Is LLM-as-a-Judge
- How do you evaluate an AI agent? AI, decoded · How to Evaluate an AI Agent: Steps, Outcomes and Side Effects
- Should you evaluate AI with an LLM-as-a-judge or with human review? AI, decoded · LLM-as-a-Judge vs. Human Evaluation: When to Use Which
- What is AI observability, and why do you need it in production? AI, decoded · What Is AI Observability
- Hamel Husain: validating an LLM judge article
Label a sample with a domain expert, separate prompt examples from held-out cases, and inspect false approvals. Live judge experiments need model access and may incur costs.
From the conversation
-
Mindset Over Metrics: How to Approach AI Engineering | Hamel Husain -
Explaining Eval Engineering | Galileo's Vikram Chatterji -
Architecting AI Agents: The Shift from Models to Systems | Aishwarya Srinivasan -
Practical Lessons for GenAI Evals | Chip Huyen & Vivienne Zhang -
The Infrastructure Behind Agents That Run for Months -
Building an AI-Native Startup | GrowthX's Marcel Santilli -
Inside IBM's watsonx: Building Enterprise AI That Ships | Dr. Maryam Ashoori