Agent benchmarks & eval-driven dev

How do you know an agent is actually good? A static question-answering benchmark (does it know the capital of France?) tells you almost nothing about whether it can resolve a bug across ten files or complete a checkout flow on a website. Agents are judged on end-to-end task completion in a realistic environment, where success is an outcome you can verify — the tests pass, the right state is reached — not a string that matches a reference answer.

That shift, from text-matching to outcome-verification, is what makes agent benchmarks both more trustworthy and far more expensive than ordinary LLM evals.

The major agent benchmarks

Each leading benchmark drops the agent into a different real environment and checks a concrete outcome:

Why agent eval is hard — and noisy

Three things make it harder than scoring a quiz:

• Outcome verification needs the real environment. Running SWE-bench means spinning up each repo and executing its test suite; OSWorld means a real desktop. That’s heavy infrastructure — but the payoff is an objective pass/fail, not a fuzzy similarity score.
• Multi-step means run-to-run variance. An agent that takes twenty tool calls can succeed on one run and fail on the next from a single bad step. A single number is misleading — you report pass@k and average over seeds.
• Outcome vs process. “Did it succeed?” (outcome) is not the whole story; “was the trajectory sound?” (process — efficiency, no unsafe actions, no wasted cost) matters too. The best evals track both.

That variance is worth a number. If an agent succeeds on a hard task 40% of the time per attempt, its score depends entirely on how many attempts you let it have:

p = 0.40   # per-attempt success rate on a hard task
for k in [1, 2, 3, 5, 10]:
    passk = 1 - (1 - p) ** k          # P(at least one success in k attempts)
    print(f"pass@{k:<2}: {passk:.1%}")

pass@1 : 40.0%
pass@2 : 64.0%
pass@3 : 78.4%
pass@5 : 92.2%
pass@10: 99.4%

The same agent looks like a 40% system or a 92% system depending on whether you report pass@1 or pass@5 — so a benchmark number is meaningless without its k and seed count. Always state them, and compare like with like.

Eval-driven development

Benchmarks rank the field; your agent needs your eval. The discipline mirrors test-driven development:

1. Build a representative eval set of real tasks from your domain, each with a programmatic success check (the agent’s SWE-bench).
2. Run it on every change — a regression suite, so a prompt tweak that fixes one case but breaks five is caught immediately.
3. Analyze failed trajectories, not just the score: where did it go wrong (wrong tool, hallucinated argument, gave up early)? Failure clustering is where the next improvement comes from.
4. Iterate against the number.

In one breath

• Agents are judged on end-to-end task completion in a real environment, with success outcome-verified (tests pass, state reached) — not text-matched.
• Major benchmarks: SWE-bench (resolve GitHub issues, tests pass), GAIA (general assistant, web+tools), WebArena (web-app task state), OSWorld (desktop GUI) — each a different environment and verifiable outcome.
• Agent eval is expensive (needs the real environment) and noisy (multi-step, stochastic), so report pass@k and seeds — the same agent is “40%” or “92%” depending on k.
• Track outcome and process (did it succeed and was the trajectory sound).
• For your own agent, do eval-driven development: a representative eval set with programmatic checks, run on every change, with trajectory analysis — never ship on vibes.

Quick check

Next

Benchmarks measure the outcome; Evaluating agents covers the trajectory-scoring methods behind your eval set, and Reflection is how an agent uses a failed evaluation to improve itself.