Evaluating multi-agent systems

A single agent that fails is a diagnosis problem. A system of agents that fails is an attribution problem, and attribution is genuinely harder.

The output was wrong. Which agent was wrong? Frequently the answer is none of them — each did its job correctly given what it received, and the failure lives in what got passed between them.

Grade the handoff, not just the endpoints

The most useful instrumentation in a multi-agent system is at the boundaries.

For each handoff, record what was passed and evaluate it as an artefact in its own right:

• Was the handoff complete? Did the upstream agent include everything the downstream one needed, or did it summarise away a detail that mattered?
• Was it accurate? Summarisation is lossy and confident. A researcher agent that says "no exclusions found" when it means "I didn't check exclusions" has produced a technically-shaped output that will be trusted downstream.
• Was uncertainty preserved? This one causes the most damage. Upstream says "probably a commercial policy." Downstream receives "commercial policy." The hedge is dropped at the boundary and never recovered, and the final answer carries a confidence nobody actually had.

That last failure is systematic rather than occasional. Each handoff is a compression step, and hedges compress badly — they're the first thing a summariser drops. In a four-agent chain, uncertainty expressed at step one is usually gone by step three.

Make it a criterion: downstream confidence may not exceed upstream confidence.

Attribution requires per-agent oracles

To say which agent failed, you need to know what each should have produced — not just what the system should have produced.

That means writing intermediate oracles: for this input, the research agent should have surfaced these three facts; the analysis agent should have reached this conclusion given those facts; the drafting agent should have produced something with these properties.

It's more work than a single end-to-end oracle. It's also the only thing that turns "the system was wrong" into "agent two dropped the exclusion."

A cheaper approximation, if full intermediate oracles are too expensive: substitute a perfect upstream. Hand agent three the ideal output of agent two and see whether it succeeds. If it does, the fault was upstream. Run that substitution at each boundary and you've localised the failure without writing a full oracle for every stage.

Failure modes specific to multi-agent systems

Error amplification. A small upstream mistake becomes a large downstream one, because each agent reasons confidently on what it received. Single-agent evaluation cannot produce this pattern at all.

Circular delegation. Two agents hand work back and forth without progress. This shows up in cost metrics long before it shows up in accuracy — another reason cost belongs in the scorecard.

Diffuse responsibility. Every agent did something reasonable and nobody did the necessary thing. Common when the orchestration is under-specified: the check nobody was explicitly assigned is the check that doesn't happen.

Consensus theatre. If your architecture has agents review each other, measure whether the reviewer ever disagrees. A reviewer that approves 98% of the time is not a control, it's latency — and its presence in the architecture diagram is actively misleading to whoever reads it.

Determinism gets harder, and matters more

Multi-agent systems compound non-determinism. Each agent contributes sampling variance; ordering effects appear when agents run concurrently; a retry at step two changes what step three sees.

Two consequences.

Your noise floor will be substantially higher than a single-agent system's. Measure it before interpreting any comparison — and expect that differences you'd have called meaningful in a single-agent setup are now inside the noise.

And replay needs the full trace, not just inputs and outputs. To debug a multi-agent failure you need the complete ordered record of what each agent saw and produced. Systems that log only final outputs make their own failures undiagnosable.

Is the architecture earning its complexity?

The question nobody asks in the enthusiasm of building.

Run the same benchmark against a single well-prompted agent with the same tools. Sometimes the multi-agent system wins clearly. Often it matches on accuracy at three times the cost and latency, with a much larger surface for the failure modes above.

That's a legitimate finding and it's worth knowing before the architecture is load-bearing. Multi-agent designs are frequently adopted for conceptual tidiness — separation of concerns, which is a good instinct in software and a costly one when every boundary is a lossy natural-language interface.

The honest test: can you show that decomposition improved a number you care about? If not, you have added coordination overhead and attribution difficulty in exchange for a cleaner diagram.

The one-line version

In a multi-agent system, the interesting failures don't live inside the agents. They live in what got passed between them — and if you only grade the ends, that's precisely the part you never see.