Synthetic evaluation data: when it works and when it lies

Nobody has enough real cases with known-correct answers. So everybody generates some. The question isn't whether to — it's which parts you're allowed to generate.

The part you cannot generate

The correct answer on a hard case.

If a model generates the task and the answer, your benchmark measures agreement with that model, dressed as measurement of correctness. Any candidate sharing its training lineage will do suspiciously well, and you'll conclude something about capability that was actually about family resemblance.

This is the circularity that quietly invalidates a lot of internal evaluation, and it's invisible from the inside because the tasks look fine and the numbers look plausible.

The rule: a generated answer needs a human verdict before it counts as ground truth. Generation can propose. Only a person who could do the job can ratify.

The parts you can generate freely

Surface variation. Same underlying case, different phrasing, formatting, names, dates, ordering. This is genuinely valuable — it tests robustness to superficial form and it's tedious for humans to produce.

Distractors and noise. Extra documents, irrelevant records, plausible-but-wrong context. Real environments are cluttered; clean fixtures flatter the system.

Adversarial variants of a verified case. Once a human has established the correct answer, generating harder-to-read versions of the same case is safe, because the ground truth is anchored.

Volume around a verified core. Generate many cases from a frozen specification, then gate them: the known-correct answer must score full marks, deliberate cheats must score zero. Sample the survivors for human audit. This is the pattern that scales, and the gating is what makes it legitimate rather than circular.

The tells that your synthetic set is lying to you

Everything is too tidy. Complete documents, consistent formatting, no contradictions, round numbers, sensible dates. Real inputs are none of those things. A system evaluated on tidy data and deployed onto messy data will surprise you — and the surprise will look like a model problem when it was a fixture problem.

The distribution is too flat. Generated case sets tend toward uniformity — roughly equal numbers of each type, evenly spread difficulty. Real work is lumpy: one dominant pattern, a long tail, and a handful of pathological cases that generate most of the risk. A uniform set under-weights exactly the tail you built the evaluation for.

Nothing is impossible. In real work, some cases can't be resolved with the information available and the correct answer is to escalate or request more. Generators almost never produce those, because they're generating solvable problems. So your evaluation contains no test of the abstention behaviour that matters most in production.

The same idioms recur. Generated text has a fingerprint. If every case is written the way one model writes, a model with a similar style has an edge that has nothing to do with the job.

The hybrid that actually works

Real cases as the spine, generation for volume, humans at both ends:

1. Start with real cases — a small number, from actual operations, in the state they actually arrived.
2. Have experts write the correct answers for those. This is the irreducible bit.
3. Generate variants — surface changes, added noise, harder versions — anchored to the verified answers.
4. Gate everything: oracle scores 100%, cheats score 0%, the environment supports the task.
5. Audit a sample of the generated set with humans. Five percent is a reasonable starting rate.
6. Track the split. Tag every task as real or generated, and report scores separately.

That last step is the one people skip and it pays for itself immediately. If models score noticeably higher on your generated tasks than your real ones, your generated tasks are easier than reality — and you now know by how much, which is a correction factor you can apply to everything else.

A note on privacy

Synthetic data is often reached for as a privacy solution, and it's a decent one — but be precise about what it gives you.

Generating cases in the style of real ones removes the identifiers. It does not automatically remove everything sensitive; a rare enough combination of characteristics can still be identifying even with names changed. And "generated from real data" is not the same as "contains nothing from real data" — a model prompted with real examples can reproduce specifics.

If privacy is the driver, treat the generated set as still requiring review, and record its provenance honestly: derived from real cases, with what transformation, reviewed by whom.

The one-line version

Generate the wrapping. Never generate the verdict.

If a model both wrote the question and decided the answer, you built a mirror and called it a benchmark.