Your failures are your training data

Note: the figures in this post are illustrative examples, not measured results.

"We're at 84%." Fine. What do you do tomorrow morning?

Think about the actions actually available to you. Buy training data. Change the scaffold. Switch models. Give the agent a tool it didn't have. Rewrite a prompt. Accept the performance and keep a human in the loop.

Every one of those is invisible in the number 84. The score cannot tell you which to pick, and it can't tell you what any of them would be worth.

Measurement that doesn't change what you do next is expensive decoration.

Cluster by the criterion, not the topic

Most teams cluster failures by topic. Embed the transcripts, cluster the embeddings, see what's near what. It's easy and it produces clusters like "mortgage documents" or "edge cases" — interesting, almost never actionable.

Cluster by which criterion refused the episode instead.

Now every cluster arrives already attached to a cause:

• C1 failed — 61 episodes — income verified against one year where policy requires two
• C3 failed — 44 episodes — exception recorded with no written rationale
• C2 failed — 29 episodes — agency limit applied instead of the internal overlay

Three things follow immediately. Each cluster names its own fix, because it names the rule that wasn't met. Each is countable, so you can rank by volume. And each maps to a specific line of your rubric — which means the person who wrote that criterion is the person who can tell you whether it's a model problem or a specification problem.

That last distinction saves enormous amounts of money. A meaningful share of what looks like model failure is the rule being wrong, and topic clustering will never show you that, because it groups by what a case was about rather than what it violated.

The rubric you wrote is also the taxonomy you needed.

Buy data by the gap, not by the pound

The default enterprise data purchase is a volume. Fifty thousand examples of roughly the right shape, priced per unit, negotiated on price per thousand, aimed at nothing in particular — because at purchase time nobody knows what's missing.

A failure map turns that purchase into a specification:

Three things you get. You can size each ask, because you know how often the failure occurs. You can price it, because a specific ask is a smaller ask. And you can rank them, because you can estimate how much pass rate each buys — which turns a purchasing decision into an expected-value calculation.

That last part changes the conversation with finance. You're no longer asking for a data budget. You're presenting a ranked list with estimated returns.

Undirected data spend is the single largest wasted line item in enterprise AI, and it's downstream of one thing: not having a ruler.

Your graders are already a reward model

Here's the connection most teams miss, and it changes the economics of everything upstream.

To do reinforcement learning from feedback you need a reward signal — something that takes an attempt and scores it. Building a trustworthy one is normally the expensive part of the whole endeavour.

If you built calibrated graders for evaluation, you already have one. You just haven't served it as an endpoint.

Look at what a well-run evaluation produces:

• Oracle trajectories — the known-correct way to do each task, tool calls included. That's supervised fine-tuning data.
• Pairs of runs on the same task where one scored higher. That's preference data, generated as a by-product.
• Only the episodes your graders certified. That's a rejection-sampled set.
• The grader itself, which is a reward function already checked against your human experts.

That last point deserves a moment. The reason your grader is trustworthy as a training signal is exactly the reason it was trustworthy as a measurement: it cleared an agreement floor against people who know the job.

The signal that measured the failure is the same signal that trains it away.

The honest delta

You fine-tuned. The number went up. How much of that is real?

Three common ways to accidentally lie to yourself, all with good intentions:

• You evaluated on a fresh set of tasks, because reusing the old ones felt like cheating. But a new set has unknown difficulty. Some of your improvement is that the second exam was easier.
• You fixed some broken tasks in between, because you found them and fixing them was right. Now some of your gain is the benchmark becoming fairer.
• The judge version moved underneath you, and you're reading drift as progress.

The only clean comparison is the identical fingerprint: same tasks, same graders, same judge version, same environment, same budget. Change exactly one thing — the model — and the difference belongs to it by construction.

If more than one thing moved, you have a story, not a result.

Often the right answer is: don't fine-tune

When you read a failure map rather than a score, the causes sort into five kinds:

1. It lacked a tool it needed → give it the tool.
2. The context never reached it → fix retrieval, not weights.
3. The instruction was ambiguous and it made a reasonable call → rewrite the instruction.
4. The policy genuinely changed and the spec is stale → update the spec. The agent was right about the old world.
5. It genuinely doesn't understand this domain → now train.

Only the fifth is a training problem, and in practice it's usually the least common of the five.

This is why diagnosis has to precede intervention. Fine-tuning is the most expensive, slowest, hardest-to-reverse fix available — and it's the default reach for teams who only have a score, because when all you know is that the number is low, adding data feels like the only lever.

The most valuable thing an evaluation tells you is what not to spend money on.