The Wire Transfer and the Taxi Receipt - Tiering Autonomy Instead of Switching It

The problem

I built an expense-report automation agent to handle a seeded set of one hundred fifty reports, and I deliberately made that set span a genuinely wide range: a twelve-dollar taxi receipt sitting right next to a forty-five-hundred-dollar non-refundable international wire transfer for a vendor deposit. I gave the system exactly two modes, configured for the whole pipeline at once. Fully auto-approve, or fully route every report to a human.

Neither mode made any sense once I looked at what it actually did across the full range of reports. In auto-approve mode, the wire transfer, irreversible and expensive to get wrong, got the identical blanket approval as the trivial taxi receipt. Nothing distinguished them. In route-to-human mode, the trivial receipt burned the same reviewer attention as the wire transfer, because a human had to look at every single report regardless of how little judgment it actually required.

I built a metric to make the damage visible: a misrouted-decision rate, comparing whichever mode was actually applied against what a proper risk score, cost of error multiplied by reversibility, would have recommended for that specific report. It came out high in both configurations, in opposite directions. Auto-approve mode missed the reports that genuinely needed a human's attention. Route-to-human mode wasted reviewer time on reports that needed none. The pipeline had exactly two things it could say about any given report: trust everything, or trust nothing. It had no way to say "trust this one, look closely at that one," which is the sentence any competent human reviewer says a dozen times a day without thinking about it.

The pattern

The fix replaces the binary switch with a scoring function computed for every report before any approval action happens. Cost of error and reversibility get multiplied together, per report, and the resulting score maps into one of three oversight tiers instead of two: auto-approve, a lightweight spot-check, or full manual review.

A trivial, reversible taxi receipt scores low on both dimensions and auto-approves. A large, irreversible wire transfer scores high on both and routes straight to full manual review. Everything in between, the reports that are moderately expensive but easy to reverse, or cheap to fix but hard to unwind, gets a lightweight spot-check instead of either extreme. The middle tier is where most of the value in this pattern actually lives, because it's the option a binary switch structurally cannot offer.

Running the same one hundred fifty reports through this scoring function instead of a mode flag drops the misrouted-decision rate to near zero, because tier assignment is now derived directly from the same labels the metric uses to judge it. What matters more than the metric, though, is what happens to reviewer time. Total time spent reviewing drops substantially compared to the all-manual baseline, while the highest-risk reports still get the same full review they would have gotten under the strictest possible policy. Reviewer attention gets allocated by actual risk instead of by a single global setting applied uniformly to every report regardless of what it actually is.

Design considerations

Every scoring-based pattern lives or dies on whether the inputs to the score are honest, and this one has two inputs to get right instead of one. Cost of error is usually the easier of the two. It's often a dollar figure sitting right there in the transaction. Reversibility is the harder one, because it's a judgment call dressed up as a fact. A transaction can be technically reversible on paper, in the sense that a reversal process exists somewhere in a policy document, while being practically irreversible in the sense that nobody ever actually runs that process before real damage is done. A wire transfer sent to the wrong account is theoretically recoverable through a bank's fraud department. In practice, once the receiving account has moved the funds, it isn't coming back, and a scoring function that codes this transaction as reversible because a recovery process exists on paper will hand you a confidently wrong tier assignment with exactly the same clean structure as a correctly right one.

I want to be direct about what this pattern is actually solving, because it's easy to conflate with something adjacent. It decides, before any action happens, how much human attention a given transaction deserves given what it would cost to get wrong and how hard that wrongness would be to fix later. That decision happens entirely upfront, before anything runs. Whether an undo attempt actually succeeds once something has already gone through is a completely separate problem with its own failure modes, solved by a different mechanism entirely.

The tiering itself is only as useful as the organization's willingness to actually staff the middle tier properly. A lightweight spot-check tier that exists on a diagram but gets treated by overworked reviewers as "auto-approve with extra steps," rubber-stamped without the attention the design assumes it gets, quietly degrades back into the same binary failure this pattern was built to escape. The tier boundaries also aren't a one-time calibration. What counts as a high-cost-of-error transaction shifts as the business changes scale; a report that would have triggered full review at a smaller company might sit comfortably in the spot-check tier once average transaction size has grown by an order of magnitude, and a scoring threshold nobody revisits will drift out of sync with what the organization actually considers risky.

There's a related organizational habit that explains why teams reach for a binary switch in the first place rather than something graduated. Automation tends to get framed as a choice between reviewing everything and supervising nothing. The calibrated middle gets skipped because the binary habit of thinking about oversight never even considers that a third option exists, and building the third tier was never the hard part. I've watched a team build a perfectly good three-tier scoring function and still default back to talking about the decision as auto or manual the moment the conversation moved away from the code, which tells me the habit lives separately from whatever technical mechanism eventually answers it.