A law firm testing a contract-review assistant might ask two senior associates to read 100 of its summaries and mark each one as correct, missing a key clause, or wrong. No automated score can tell the firm as much about whether the tool is safe to use. That is human evaluation: people judging outputs, either on a rubric (“is every obligation listed?”) or by picking the better of two answers, which is pairwise evaluation.
It is slow and expensive, so most teams use people where judgment matters most and automate the rest. A common pattern is to have experts grade a sample, check how often they agree with each other (a statistic such as Cohen’s kappa helps), then tune an LLM judge until it agrees with them, and spot-check it from then on. The ratings are only as good as the instructions: vague rubrics produce noisy scores, and graders who don’t know the domain can’t tell a fluent wrong answer from a right one.
Human ratings also feed training, not just testing. In RLHF, people rank model outputs and those rankings train a reward model. The two uses get confused, but the question differs: evaluation asks whether the system is good enough to ship; feedback for training asks which answer the model should learn to prefer.
In episode 1 at 21:38, Galileo co-founder Yash Sheth estimates that at the big labs “90% of the time is going in human evaluation, human annotation and evaluation,” not waiting for GPUs. That is his estimate, not a measured figure. His co-founder Atindriyo Sanyal adds at 23:09 “the importance of human evals” and credits human feedback in training for the jump from GPT-3 to GPT-3.5. Galileo sponsored season one, and the panel participants are Galileo’s co-founders, whose company sells evaluation tooling.
Hear it from the guest
“90% of the time is going in human evaluation, human annotation and evaluation”
Quotes lightly edited to remove filler words.
Sources
- Zheng et al.: Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (2023) — Uses human preference ratings as the reference point for checking whether strong models can act as judges, and documents judge biases such as position, verbosity and self-enhancement; checked October 24, 2026.
- Ouyang et al.: Training language models to follow instructions with human feedback (2022) — The InstructGPT paper, which trains and evaluates models using labeler-written demonstrations and labeler rankings of model outputs; checked October 24, 2026.
- Anthropic: Define success criteria and build evaluations — Describes human grading as the most flexible and high-quality grading method but slow and expensive, and recommends automated grading where it is reliable; checked October 24, 2026.
