Writing
40 essays on measuring AI systems honestly — what a benchmark has to be, whether a model can grade one, what the statistics will and will not support, and the defects we found in our own code while building the thing.
Most recent · 30 August 2026
evaluation foundations
What a benchmark actually is — and why yours probably isn't one
Three properties separate a benchmark from a demo. Most internal AI evaluations have one of them.
Can you trust an AI to grade an AI?
A practical guide to judge calibration: why raw agreement lies, what kappa corrects for, and why you gate on a lower bound.
The benchmarks are broken — and the audits are arriving late
What 2026's benchmark audits found, why every one of them was retrospective, and what gating at authoring time would have caught.
Evaluating an agent is a different discipline
Four things need grading, not one — and the scaffold is often doing more work than the model.
Your failures are your training data
A score isn't a plan. A failure map is — and the fastest way to get one is to cluster by the criterion that refused, not by topic.
How to write an evaluation that holds up
Scope from consequence. Weight by cost, not frequency. And write the oracle first — it's where you find out your experts disagree.
The statistics of evaluation: what your numbers actually support
Measure your noise floor first. Then intervals, multiple comparisons, pass@k, and where the variance really comes from.
Red-team your rubric, not just your model
Reward hacking is a specification defect. Here are six cheat probes to write, and the afternoon that catches most of them.
Evaluation doesn't end at launch
Offline and online answer different questions. Put the benchmark in CI, pin the model version, shadow before canary, and write postmortems back into the suite.
Trust that doesn't require trusting anyone
What a regulator actually asks, why provenance has to be recorded at authoring time, and why a result nobody outside can reproduce is a claim rather than a measurement.
You're six months into an AI programme. Is it working?
The pilot went well, a team uses it daily, and nobody can tell you whether it's safe to widen. That gap isn't capability — it's evidence.
Eight workflows, eight failure maps
The failure modes that actually hurt — by use case — and the expert data that fixes each one.
extended
The evaluation maturity model
Five stages from vibes to verifiable. Most organisations are at stage one and believe they're at stage three.
How to run a model bake-off that produces a decision
Most vendor evaluations produce a slide. Here's the protocol that produces a defensible choice.
Fourteen questions to ask a vendor about their benchmark
A buyer's checklist. Most vendors can answer four of these. The gap tells you what you're actually buying.
Should you build your evaluation in-house?
An honest decomposition. Some of it you must build. Some of it you shouldn't. The line is not where most teams draw it.
RAG evaluation is two problems, and most teams measure one
Retrieval failure and generation failure look identical in the output and need completely different fixes.
Evaluating multi-agent systems
When work is split across agents, failure attribution becomes the hard problem — and the handoff is where it lives.
The economics of evaluation
What it costs, what it saves, and why the second workflow is the number that matters.
Running human annotation that doesn't fall apart
Your experts are the scarcest resource in evaluation. Here's how to spend them so the labels are worth having.
Synthetic evaluation data: when it works and when it lies
Generation is fine for volume and dangerous for ground truth. The line is sharper than most teams draw it.
Benchmarks with a date on them
The questions enterprises actually care about are time-indexed. A live API call makes them unmeasurable. Capture and replay are separable.
A starting checklist for evaluation in regulated industries
What examiners actually look for, and the artefacts worth having before anyone asks.
The 30-day evaluation
What you can genuinely accomplish in a month, in order, without a platform or a budget line.
from the workbench
The number nobody publishes
Every benchmark reports the fraction of tasks a model got right at least once. Almost none report the fraction it gets right every time. For anything going to production, the second number is the one that matters.
An empty checklist passes every audit
The most dangerous bug in an evaluation system is the one where silence and success are the same value. We found nine of them in one codebase, each behind a green test suite.
Test the grader, not just the model
Your evaluation harness is production code that nobody evaluates. Five properties — stated once, enforced against every grader you have — catch defects that thousands of example-based tests miss.
Your traces are a benchmark you haven't compiled
You already have thousands of recordings of real work. Turning them into tasks is mostly mechanical — except for the one part that isn't, which is where every naive attempt fails.
When the simulated user leaks
Half of real agent work is getting information out of a person. Simulating that person with an LLM is the obvious move, and the obvious implementation hands the model the answer key.
Say what you're predicting before you know if you're right
The question that decides whether your evaluation was worth building is whether a better score predicted a better outcome. You can only answer it if you wrote down the claim first.
the refusals
Audit the passes, not just the failures
Every evaluation team reviews the failures and ships the passes. But the pass is the claim you are selling, and a pass nobody interrogated is not evidence that anything worked.
Every number should resolve to a receipt
A figure you cannot click through to the query that produced it is a claim, not a measurement. The fix is to make the evidence executable by the reader, and to diff the prose against what it actually printed.
How do you know the gate would catch it? Seed the corruption
A quality gate with no measured detection rate is decoration. The only way to learn what yours catches is to inject the defects it claims to stop, at known density, and count.
The grader needs a grader — and permission to refuse
If a model is scoring your evaluation, its agreement with human judgement is a measured quantity with a confidence interval. And when there is not enough data to measure it, the honest output is a refusal with a name on it.
An unverified oracle is not an answer key
If the reference solution has never been run through the graders that will score the model, you do not have an answer key — you have a plausible document. And a closed-book claim enforced by reading the command line is not a boundary.
Silence is never success
The zero that means no measurement, the null that gets formatted into a report, and the security setting a backend accepts and never enforces. Three shapes of the same defect, and the rule that closes all of them.
worlds and evaluation systems
A final-state benchmark cannot prove the world changed
Long-horizon agents work in worlds that interrupt them. A final score and an ordinary log cannot establish which change reached the agent, when it arrived, or whether restore preserved it. Here is the evidence contract that can.
A benchmark score also grades the harness
A task, model and final score are not the whole experiment. The runner, agent adapter, reset, timeout, network, grader and aggregation policy can change the answer. Audit both systems.
evaluation immune systems
The scorekeeper is an attack surface
An evaluator is executable policy. Before training against its reward, attack the scorer, localize each shortcut, repair it, and prove the repair did not damage honest behavior.
A failure map is not a training plan
Evaluation can reveal where an agent fails. Steering training requires a second contract: rights-safe data, isolated splits, controlled cells, uncertainty, regression gates and a refusal to call a proposal measured lift.