The Outdated Fix That Kept Beating Its Own Replacement

I built a customer-support triage agent once that routed incoming tickets to a suggested fix by searching a "knowledge graph" built from five hundred resolved support tickets. I put that word in quotes for a reason, and the reason is the whole point of this post.

The problem

The graph in that first version was really just a cache wearing a fancier name. Each ticket got stored as raw resolved-ticket text, re-embedded and treated as a node, with no typed extraction of what the ticket was actually about. I seeded forty of the five hundred tickets as symptom-conflict pairs on purpose: the same symptom, resolved two different ways across two product versions, one an obsolete workaround from an older release, one the current fix.

Because tickets were stored raw, both resolutions surfaced as equally relevant nearest neighbors. Nothing anywhere in the graph said "applies to version 2.x only." The incoming-ticket classifier had nothing but raw-text similarity to decide between an outdated fix and a current one, and raw-text similarity has no concept of "outdated" at all. It only knows "similar" or "not similar." I measured a contradiction rate of at least fifty percent on the forty known conflict pairs, meaning at least half the time, an outdated fix surfaced in the top three results right alongside the correct current one, with nothing distinguishing them.

Here's what I find genuinely instructive about this failure. The system was returning a real fix, one that had actually resolved that exact symptom for a real customer at some point, so nothing about it looked like garbage. The problem was that it had no way of knowing that fix had since been superseded, because superseding something requires a system to track what current truth is, and a cache of raw text has no concept of "current." Everything it stores is equally old and equally new, forever, because it never distinguished between the two in the first place.

The pattern

The fix was to build the typed graph the first version only pretended to have, through two separate loops running on two separate timescales. A learning loop periodically processes resolved tickets, extracting typed nodes, Symptom, Cause, Fix, ProductVersion, and typed dependency edges, including one edge type that marks an older fix as outdated relative to a newer one without deleting it. That last detail matters. The old fix stays in the graph. It just gets marked as superseded, which preserves the history of what used to be true while keeping current truth queryable and unambiguous.

A separate execution loop classifies each incoming ticket's symptom and product version, then routes strictly against the typed graph, so a superseded fix for an old version is structurally excluded from ever outranking the current one for a current-version ticket. The word "structurally" is doing real work in that sentence. This is an exclusion, built into how the execution loop queries the graph in the first place, so the outdated fix never competes for the top slot at all.

Running the same eval set through this version, the learning loop correctly marked at least ninety percent of the forty known conflict pairs as superseded, the contradiction rate dropped to near zero, and routing accuracy improved measurably against the raw-cache version.

Design considerations

The failure the raw-cache version has is what I'd call no entity resolution, and that phrase sounds like it's describing the same gap a lot of graph-construction pipelines run into elsewhere. It isn't quite. This failure is raw-text-cache contradiction: no typed edges exist at all, so two full ticket texts sit as equally weighted nearest neighbors with nothing distinguishing current from outdated. A dedicated entity-resolution step solves an adjacent but genuinely different problem: deciding whether two already-typed entity mentions refer to the same real-world thing before either one gets written as a node.

Those two problems can absolutely coexist, and a production build combining a typed graph with continuous ingestion would likely need to handle both. A dual-loop graph built exactly as I've described here, with typed nodes and a working supersession mechanism, could still let a duplicate node sit uncaught inside its own curated store. Marking a fact superseded and merging two mentions of the same entity are different operations solving different failures, and fixing one doesn't get you the other for free.

The calibration question that actually matters here is how often the learning loop runs. Run it too infrequently and newly resolved tickets sit unprocessed, meaning a genuinely new fix hasn't been marked as current yet and an older one is still winning by default, which is a milder version of the exact problem this pattern exists to solve. Run it constantly and you're paying continuous processing cost for a graph that mostly doesn't change minute to minute, since support tickets get resolved and reviewed in batches, not in a steady stream. The right cadence tracks how fast your actual resolution volume moves.

Where this pattern doesn't earn its cost is anywhere the underlying knowledge genuinely doesn't go stale. If every resolved ticket in your domain stays permanently correct once written, there's nothing to supersede and no reason to build two loops instead of one. The two-loop split is a direct answer to a specific failure mode, a correct answer getting outranked by an answer that used to be correct, and it costs you real engineering complexity to build and maintain. Reach for it when your knowledge base is one where yesterday's right answer can become today's wrong one, because that's the exact condition a single-loop cache has no mechanism for handling.