Which metrics make an AI evaluation useful?

Start with the promise you want to measure

An expense-policy assistant answers an employee's question. What counts as success? A plausible sentence is weak evidence. The answer should state the applicable deadline, support it with an authorized current passage, and abstain when that evidence is missing.

A metric measures one part of a promise like that. After reading this article, you should be able to distinguish one successful candidate from consistent repeated success, name the denominator behind a score, and spot a critical failure that an average hides.

HELM uses multiple metrics and scenarios to expose properties beyond accuracy. For your own product, begin with the claim and choose observable evidence for it. An access check can be exact; whether an explanation is helpful needs a defensible rubric.

One good answer and reliable repeated use differ

Suppose a trial has an assumed 80% chance of success. Three independent tries give a much greater chance of finding at least one success than of having all three succeed. A drafting tool that can validate and keep one candidate has a different requirement from three separate employees who each need a correct answer.

Change the number of trials below. The two probabilities move in opposite directions because the events being measured differ. Anthropic's agent evaluation guide explains this distinction through pass@k and pass^k, and separates tasks, repeated trials, traces, and final outcomes.

The assumptions belong beside the number

The calculator assumes identical, independent trial probabilities. If all three attempts use the same stale policy, their failures may share a cause. Retries do not supply a missing document, and an at-least-one score does not tell the product which candidate is correct.

In an actual evaluation, preserve the task IDs, number of trials per task, success criteria, and system configuration. “Nine out of ten trials” and “nine out of ten distinct tasks” are different denominators. Formula-based probabilities are also different from estimates derived from a finite sample of recorded runs. Report uncertainty and the sampling procedure instead of presenting a small sample as precise production reliability.

Keep critical failures visible

Return to the policy assistant. Imagine ten graded trials. Nine pass. The tenth might be an unnecessary abstention on an answerable question, or it might disclose a restricted policy. Both suites have 90% task success. Their failure consequences differ.

The next experiment makes that difference visible. The fictional product rule accepts at least 90% task success only if the suite has zero forbidden disclosures. Switch the failing case and inspect both gates.

A passing average does not settle the decision

A threshold is a product policy, chosen before inspecting the result. The demo's 90% threshold is illustrative. “Zero observed forbidden disclosures” describes these ten cases; it does not prove the system can never disclose something forbidden.

Show meaningful slices alongside the total: authorized versus denied requests, current versus stale evidence, supported answers versus expected abstentions. Keep critical-failure counts and their severity visible. Likewise, report latency and cost distributions rather than letting a cheap average hide slow or expensive tails.

Match each check to its evidence

Product claim	Useful evidence	What the metric leaves open
Output follows a schema	Exact validation on every output	A valid shape can contain a false claim
Retrieval finds necessary evidence	Recall/ranking against labeled relevant passages	A found passage may be ignored by generation
Answers follow the selected evidence	Claim-level support and citation review	The source itself may be incorrect
Agent completed an action	Final application state plus the trace	A convincing completion message is insufficient
Explanation is useful	A clear rubric with calibrated human or model grading	Grader bias and disagreement remain measurable

OpenAI's evaluation guide provides a provider-specific workflow for specifying test criteria and checking outputs. The underlying contract is general: record what happened and apply checks that can distinguish a valid result from a convincing claim.

Evaluate the grader too

A model judge extends what can be assessed automatically, but its score is another system output. G-Eval studied rubric-based evaluation on summarization and dialogue; LLM-as-a-Judge examined agreement and biases including position and verbosity. Those findings do not validate every judge on every task.

AgentRewardBench compares web-agent evaluators against expert-reviewed trajectories, illustrating why final-state evidence and human calibration matter. Check your grader against labeled examples and disagreements. Retain raw evidence so a surprising score can be investigated.

A useful report states the claim, suite and corpus versions, denominator, repeated-trial policy, grader, important slices, critical failures, and operational costs. It should make the next decision explainable, not merely produce a larger number.