The Drift Gate - Catching Budget Creep the Day It Happens

The problem

I set up a contract-summarization workflow to run cheap. A small model tier, a declared cost of $0.015 per document, and a team that felt good about the number because they had picked it deliberately. Then I let ninety simulated days pass and watched two ordinary things happen, the way they happen in almost every production pipeline eventually.

The prompt template grew. Someone added an instruction to catch an edge case, then another team added a clarifying example, then a third person tacked on formatting guidance because the output looked wrong once. None of these changes were reckless. Each one made sense on the day it was made. Prompt size crept up anyway, the way it always does once real documents start showing up with real weirdness in them.

At the same time, a fallback path to a premium model started firing more often. The cheap model would time out on a longer document, and rather than fail the request, the pipeline quietly routed it to the expensive model instead. Somebody had built that fallback for a good reason: a slow failure is worse than an expensive success. But the fallback rate crept up right alongside the prompt size, because longer prompts time out more often, and nobody had connected those two dots.

Nowhere in the pipeline did anything compare what production was actually costing against the $0.015 the team had declared. By day ninety of that seeded trace, actual cost per document had climbed past $0.08, more than five times the original number. I tracked a metric I call days silently over budget, and it ran past sixty of the ninety days before anyone outside the pipeline knew anything had changed. The team found out the way most teams find out: a quarterly invoice landed, and it was bigger than anyone expected, and the ensuing conversation started with "wait, why."

That's the part that bothers me most about this failure mode. Nothing here was a bug. The prompt growth was reasonable. The fallback logic was reasonable. Every individual decision that added five cents of cost was defensible on its own terms. The only thing missing was a system that ever asked whether the sum of those reasonable decisions still matched what the workflow was supposed to cost.

The pattern

The fix leaves the underlying pipeline alone and changes what gets declared and what gets checked instead. At the moment a workflow gets built, I attach a structured budget envelope to it: an expected cost per unit, here $0.015 per document, and a tolerance percentage, here 20 percent. That envelope is a small piece of configuration that lives next to the workflow, versioned the same way the code is.

A scheduled job then runs once a day and does exactly one thing: it computes actual cost per document for that day and compares it against the envelope. The moment actual cost breaches the tolerance band, the job writes a structured drift event, expected cost, actual cost, percent deviation, and a timestamp. That event is the whole mechanism. It doesn't fix anything on its own. It just makes sure that the day drift crosses the line is the day somebody finds out, instead of the day the quarterly bill arrives.

Run the same ninety-day trace with this gate in place and the drift itself doesn't change. Prompt size still grows. The fallback still fires. What changes is when the divergence gets named. Instead of sixty-plus days of silence followed by an invoice, the gate raises its first event within a day or two of tolerance actually being crossed, with the exact numbers attached. The team still has to decide what to do about it. But now they're deciding on day four instead of day ninety, while the fix is a prompt-size review and a fallback-rate check instead of a full quarter's worth of accumulated spend nobody planned for.

Design considerations

The whole mechanism hangs on the word declared, and that word is doing more work than it looks like. A drift gate only checks production cost against a number a human wrote down at design time, and $0.015 per document was always an estimate. If that original estimate was too optimistic about fallback frequency, or too naive about how much a prompt template grows once real edge cases start arriving, the gate will alert faithfully on drift that was baked into the plan from day one. That signal is real and deserves attention every time it fires. But the attention it earns is different from what most people assume when an alert goes off: a drift gate catches divergence from a plan, full stop. Whether the plan itself was any good is a separate question, and this mechanism has no opinion on it.

There's also a timing distinction I want to be precise about, because it's easy to conflate two disciplines that look similar from a distance. Checking cost before a call is allowed to proceed, a run-scoped ceiling that blocks the call outright if a monthly budget is already exhausted, is a different discipline from what I'm describing here: comparing yesterday's actual cost against a declared per-unit target and flagging the divergence after the fact. Pre-flight blocking stops a specific dollar from being spent. A drift gate doesn't stop anything; it shortens the distance between a cost problem starting and a human learning about it. Both are legitimate, and I've seen teams build one and assume they've covered the other. They haven't. A pipeline can pass every pre-flight budget check on every individual call and still drift steadily upward in its per-unit economics over ninety days, because pre-flight checks only ask whether this call clears a ceiling. Whether the average call is getting more expensive over time is a question a pre-flight check never asks.

The calibration question that actually matters in practice is the tolerance percentage. Set it too tight and the gate fires on ordinary day-to-day noise, the kind of variance that comes from document length distribution shifting slightly week to week, and people start ignoring the alerts within a month. Set it too loose and real drift accumulates for weeks before it crosses the line, which defeats the entire point of building the gate in the first place. I don't think there's a universal right answer here. Twenty percent worked for a $0.015 baseline with the kind of document-length variance I was simulating. A workflow with tighter unit economics or less inherent variance in its inputs probably needs a narrower band, and a workflow with wildly variable document sizes might need a wider one just to avoid alert fatigue. The number has to come from looking at the actual variance in a specific pipeline's inputs. Copying whatever tolerance worked somewhere else just imports someone else's variance profile into a pipeline that doesn't share it.