The Second Reviewer Problem - Why Adding an Agent Usually Adds Nothing

The problem

I built a financial-report QA workflow around a synthetic quarterly report I seeded with fifteen planted errors: misstated figures, stale footnote references, the kind of mistake that actually shows up in a real filing when nobody's looking closely enough. An Analyst agent reviews the report first. Then, because the obvious next move always looks like "let's get a second set of eyes on it," I added a Reviewer agent to catch what the Analyst missed. Same prompt as the Analyst. Same context. Same tools. No gate of any kind before its findings merge into the final QA report.

Here's what that second agent actually did. It mostly restated the Analyst's own findings back in its own words, which inflated the apparent confidence of the report without adding a single bit of real coverage, because two agents agreeing on a finding neither of them checked independently is just an echo wearing a second name tag. And on top of the restating, it occasionally overrode a correct Analyst finding with a confidently worded but completely unfounded restatement of its own, because the Reviewer had no tool or data access that would let it actually verify anything the Analyst hadn't already looked at. It could only reword what it saw, and rewording with confidence is not the same thing as being right.

I measured this precisely because "confidence" is such an easy thing to mistake for correctness when you're skimming a report rather than checking every line against ground truth. Incremental catch rate, the share of the fifteen planted errors the Reviewer caught that the Analyst missed, came in low. Redundant-restatement rate, the share of Reviewer findings that just repeated something the Analyst had already flagged, was high. And dollars spent per net-new error caught was ugly, because I was paying for a second full pass over the same report and getting almost nothing back that the first pass hadn't already surfaced.

The instinct to add a reviewer wasn't wrong on its face. Getting a second set of eyes on something genuinely does catch real mistakes, when the second set of eyes can see something the first one couldn't. The problem was that I built a Reviewer that structurally could not see anything the Analyst hadn't already seen. Same prompt, same context, same tools means the same blind spots, restated with a fresh coat of confident language.

The pattern

The fix is a bar I set before any second agent's finding is allowed into the merge step: it has to bring structurally distinct evidence or authority the existing agent couldn't already see, and it has to cite that distinct source directly. I gave the Reviewer a tool the Analyst never gets, direct access to the raw ledger behind the report rather than just the summarized figures the Analyst works from, and I added a gate that checks every Reviewer finding for a citation to that distinct source, or to a check the Analyst structurally couldn't have run, before the finding is allowed to merge. A finding that doesn't cite something genuinely new gets discarded, full stop, no matter how confidently it's worded.

Running the same fifteen planted errors through this gated version changed the shape of what the Reviewer actually contributed. Incremental catch rate went up, because the Reviewer's raw-ledger access could now surface errors the Analyst's tools structurally couldn't have caught in the first place, a stale footnote reference that only shows up if you cross-check against the underlying ledger entry rather than the summarized figure. Redundant-restatement rate and incorrect-override rate both dropped, because an ungrounded restatement never made it past the gate to begin with. Dollars spent per net-new catch improved, because I was no longer paying for a second pass that mostly reproduced the first one.

What the gate actually does is force the question that "let's add a second agent" always implicitly promises an answer to and almost never actually delivers: what, specifically, can this new agent see that the existing one couldn't. If the honest answer is nothing, the gate makes that visible immediately, as a discarded finding, instead of letting it quietly inflate the report's apparent thoroughness.

I found the strongest validation for this the way I like validation best, from a source making a harder claim than I actually needed. Recent research on multi-agent systems argues that most apparent gains from adding agents are redundant rearrangement of information the existing agent already had, and that this failure is structural, the default outcome of adding an agent, not an edge case caused by a sloppy prompt. There's a shorthand circulating alongside that finding, sometimes called the MIT rule, that states the same thing operationally: a specialist agent should bring a new exogenous signal, a better interface, or genuinely non-redundant review, or it's probably just compressing data you already had. Read against that, the gate I built functions less like a defensive workaround for a rare failure and more like a strict filter for what most proposed additional agents are actually going to be.

A study covered on the October 17 Mixture of Experts episode adds a cost dimension to that finding. Teams of agents cost two to five times as much to run as a single agent and failed to outperform one consistently. The panel pointed to orchestration overhead, added latency, and errors that propagate across handoffs as the places where the expected gains disappear. My Reviewer fits that description closely: a second full pass over the same material, paid for at full price, with the benefit concentrated in the few cases where it could see something new. The study's implication matches the default I now reach for. A single agent is the starting assumption, and each additional agent has to name the evidence it brings that the first one lacks.

Design considerations

The gate's limitation is by design, and I'd rather name it plainly than let the mechanism sound more complete than it is. Most proposed additional agents in production are going to fail this gate, and that's the point, not a flaw in the gate. If a team is adding a Reviewer, a second Analyst, a fourth model in the loop, and nobody can point to the specific tool, data source, or check that agent brings which the existing ones structurally lack, the honest answer is that the addition is theater. It looks like more rigor. It functions as an echo chamber with a bigger electric bill.

What the gate doesn't catch is a finding that cites the distinct source correctly and still draws the wrong conclusion from it. Citing something new is a real bar to clear, but it's not the same guarantee as interpreting that new evidence correctly, and a wrong-but-novel finding clears exactly the same gate a right-and-novel one does. I built this pattern to solve the redundancy problem specifically, not the accuracy-of-interpretation problem, and conflating the two would oversell what one gate can actually guarantee.

There's also a cost question I want to answer honestly before reaching for this pattern at all. Giving the Reviewer its own distinct tool and building the citation gate is real engineering work, and it only pays for itself when the report or decision in question is consequential enough that a redundant second pass is genuinely wasting money rather than just adding a small, tolerable overhead. On a low-stakes internal check where a rubber-stamp second look costs almost nothing and nobody's relying on it catching something novel, I wouldn't bother building the gate. The pattern earns its keep specifically where the second agent's cost is real and its actual contribution, absent a gate, tends to be close to zero.