The Firewall Between the Number and the Words

The problem

Picture an auto insurer that built a claims-adjuster assistant. Feed it a claim record: the repair estimate, the deductible, the coverage cap, the depreciation schedule, the count of prior claims. Ask it to compute the payout per the policy terms and explain that payout to the policyholder in plain English. One prompt, one completion. The number and the letter come back together, generated in the same breath.

The model does this well. It subtracts the deductible. It clamps against the coverage cap. It applies the depreciation schedule. Then it writes a warm, clear paragraph telling the policyholder exactly why they're getting that number. Read the letter on its own and it sounds like a competent adjuster wrote it: confident, specific, procedurally correct in tone. You'd sign off on it without blinking.

I ran fifty of these claims against a hidden reference calculation, the actual arithmetic worked out independently and never shown to the model, and checked the two dollar figures against each other. Some matched. Some didn't. Here's the part that should worry you: nothing about reading the letter told you which was which. A confident wrong payout looks exactly like a confident right one, because the model produced both the same way, out of the same process, with the same fluency. There's no seam in the text where the arithmetic ends and the plausible-sounding prose begins. It's all plausible-sounding prose. Some of it happens to be right, the way a stopped clock happens to be right twice a day.

This is the generation problem in its purest, least dramatic form. Nothing about the setup is a bizarre edge case or a jailbreak or a model behaving badly. It's a model doing exactly what it was built to do, on a task that quietly needed something else from it. Charles Sanders Peirce named three kinds of inference: deduction, induction, and abduction. Deduction derives a necessary conclusion from premises, and it can't follow any other way. Abduction is inference to the best explanation: given an observation, what's the most plausible story that accounts for it. A large language model, at inference time, is an abduction engine, full stop. It generates the most plausible continuation given the context in front of it plus everything compressed into its weights. When a completion reads like a deduction, a necessary conclusion that had to follow from the premises, that's the abductive process producing text that happens to pattern-match what a deduction looks like. Nobody asked the model to prove anything. It was asked to continue plausibly, and a proof-shaped continuation is often the most plausible one available for arithmetic it has seen worked correctly thousands of times in training. Often. Not always. And there's no internal alarm bell that fires to tell you which case you're in.

The pattern

The fix leaves what the model is asked to write alone. What changes is what the model is allowed to touch, and that turns out to be the more important lever by far. A deterministic module, ordinary code with no model call anywhere in it, applies the deductible subtraction, the cap clamping, and the depreciation lookup directly against the claim and policy data. That module produces the payout. The model receives that number as a given fact, not as something to compute, and its only job is to draft the explanation letter around it. The payout field is never writable by the model, ever. There's no path, no prompt phrasing, no amount of fluency that lets the completion override the number, because the number was never generated by a completion to begin with.

That's the whole mechanism: route anything requiring proof, arithmetic, lookups, deterministic rule application, to a structurally separate, non-model verifier, and confine the model to the part of the task that only requires plausibility. The model can't be talked into overriding the number because it is never given write access to it. Not trained not to. Never given the access, period. That distinction carries a lot of weight for me, because it's the difference between a behavioral hope and a structural guarantee, and I've watched enough behavioral hopes fail to know which one I'd rather bet on.

The metric I care about is exact-match rate: does the generated payout equal the ground-truth payout to the cent, across the same fifty claims. A firewalled version should land at or near a perfect match with near-zero dollar variance, since the number is computed by code that gets unit-tested against ground truth independently of any model call. What makes that number plausible in the first place is that the calculator has no abductive step in it at all. Same code path, same inputs, same outputs, every single time. Correctness stops being a probability the moment you stop asking a language model to produce it.

This generalizes past insurance math in a way I find genuinely useful to name. Citation and reference fabrication, a well-studied hallucination subspecies, splits into two failure types. Where the failure is a citation that simply doesn't exist, a fabricated DOI, a URL that resolves to nothing, a bibliographic entry invented wholesale, the fix is a deterministic lookup against a real citation registry whose result the model can't override. A 2026 taxonomy of one hundred fabricated NeurIPS citations found that 66% were total fabrications, invented bibliographic entries with no source behind them at all, which means the majority failure mode is exactly the "does this thing exist at all" question a lookup answers cleanly. That's the identical mechanism I just described, applied to a bibliographic database instead of a calculator. Nothing about "the verifier is a citation index" changes the shape of the fix.

Design considerations

This pattern only works where the underlying task actually is deterministic, and that caveat does a lot of the real work. The firewall is only as good as the rules programmed into the calculator. If the policy language itself is ambiguous, if there's a genuine judgment call about which clause applies, the firewall enforces whatever the rule-writer encoded, confidently and consistently, whether or not that encoding is the right read of a contested contract term. A deductive firewall's job stops at one narrow failure: a model regenerating an already-known-computable fact and sometimes getting it wrong. Whatever ambiguity lives in the underlying policy language stays exactly where it was, for a person to adjudicate. I hold that claim narrow on purpose, because the pattern gets less useful the closer a task drifts from provable toward interpreted.

There's a related move that looks identical from a distance and does something completely different: routing work to executable code so a model can write a script against tools instead of trusting the model to produce a value directly. That's an efficiency play, cutting token and context cost by letting code do the heavy lifting instead of binding every tool schema into every prompt. Cost is the entire point of it. A deductive firewall exists for exactly the opposite reason: it makes a computed value un-overridable by the model. That's a correctness guarantee, and cost never enters the calculation. Same surface, let code do it, different reason for doing it, and conflating the two means you'll reach for a firewall when what you actually needed was a cheaper prompt, or you'll ship a cost optimization and call it a safety property it never earned.

The calibration question I keep coming back to is where the boundary sits between provable and interpreted, because that boundary is a judgment call made by whoever writes the calculator. Build the firewall too narrow and you leave real computable facts exposed to the model's fluency. Build it too wide, encode a contested interpretation as if it were settled arithmetic, and you've traded one kind of overconfidence for another: a wrong number that now looks unimpeachable because it came from code instead of from a completion. Getting the underlying rule right is still someone's job, done before any of this runs. What the firewall guarantees starts after that job is done: once the rule is right, the model can't quietly get it wrong on top of a correct foundation.