Instruction adherence evaluates the requirements of the prompt separately from answer quality. IFEval builds a benchmark around verifiable instructions, such as word counts and required keywords, rather than relying only on a model to judge another model (Zhou et al., Instruction-Following Evaluation). A factual answer can still fail an instruction test if it omits a required field or includes prohibited content.
For example, ask a ticket classifier for one of three allowed labels plus a short reason. Check the label against an enumeration, the reason against your length rule and the result against the reference category. Keep format compliance and classification correctness as separate results. Otherwise an improvement in one can hide a regression in the other.
Explaining the purpose of post-training, Maxime Labonne says, “a useful assistant that is able to answer questions and follow instructions.” (episode 43, 22:26).
For builders, use code to check constraints that are mechanically testable and a reviewed rubric for requirements such as tone. Keep the rubric explicit enough for a human to apply to the same examples. Structured output covers enforcing an output shape, while LLM-as-a-judge covers judgment that needs a model. How to test an AI system puts the checks into a repeatable workflow.
Sources
- Zhou et al., Instruction-Following Evaluation — Primary source for the explanation and distinctions in this entry.
