The Webcam Light Question Nobody Scripted For

The problem

I built an IT helpdesk chatbot around three defined workflows: password reset, VPN access, hardware request. Each one is a scripted, multi-turn conversation with slots to fill, name, employee ID, the specific system involved, the kind of structured path that works cleanly as long as the person on the other end actually wants one of those three things.

I tested it against twenty-five tickets. Fifteen matched one of the three workflows cleanly and resolved the way you'd want. Ten didn't match anything. One asked whether it's normal for a webcam's light to stay on when the camera isn't in use. Another was a vague complaint that a laptop "feels slow lately," with nothing specific enough to point at any one of the three defined paths.

Watching what happened to those ten tickets was the useful part. The graph had no branch for "this doesn't match anything I know how to do," so it forced the closest available fit. The webcam question got treated as a hardware request, and the bot asked for an asset tag, a piece of information that has nothing to do with whether a webcam light staying on is a privacy concern or a hardware quirk. The vague slowness complaint either got crammed into hardware request too, or the bot re-asked a slot-filling question the user's actual words never answered, looping past four turns with no resolution in sight. At least eight of the ten off-script tickets ended up forced into the wrong workflow or stuck in a loop that never actually reached an answer.

What struck me about this failure is how confident it looked from the outside. The bot never said "I don't understand." It always had something to say, a next question to ask, a slot to request. It just kept asking questions that the actual ticket never had answers to, because the graph had exactly three roads and no shoulder to pull onto when a ticket didn't belong on any of them.

The pattern

The fix adds a fourth path ahead of the three workflow nodes. Before routing to password reset, VPN access, or hardware request, an intent-confidence check asks how well this ticket's actual content matches any of the three defined workflows. Above a confidence threshold, it proceeds exactly as before, into whichever workflow matched. Below that threshold, it routes to an explicit fallback node instead of forcing a fit.

The fallback node does four specific things, and all four matter. First, it acknowledges directly that this ticket doesn't match one of the three things it knows how to handle. Second, it offers general troubleshooting help where that's plausible, since plenty of off-script questions still have a reasonable, if unscripted, answer. Third, when the question needs a person, it escalates to a human queue. Fourth, it logs the ticket's actual text for later review, because a ticket the system doesn't recognize today is exactly the kind of signal that should inform what workflow number four eventually becomes.

Running the same twenty-five tickets through this version, the fifteen on-script tickets resolve exactly as they did before, with no regression from adding the new path. The ten off-script tickets move from mostly forced or looped to mostly reaching a clean handoff at the same rate the on-script tickets were already succeeding at. The webcam question gets acknowledged as outside the three defined workflows and either answered with general guidance or handed to a human, instead of interrogated for an asset tag it was never going to have an answer for.

Design considerations

The confidence threshold that decides what counts as off-script is the single number this entire pattern hinges on, and it's genuinely hard to get right on the first try. Set it too high and tickets that could have matched a real workflow start falling into the fallback path unnecessarily, adding friction and a human-escalation cost to requests that didn't need either. Set it too low and the fallback stops catching what it was built to catch, letting genuinely off-script tickets slip back into the forced-fit failure this pattern exists to close. I'd rather calibrate this against a labeled sample of real historical tickets than pick a number that sounds reasonable and hope the traffic behaves the way I assumed it would.

The fallback log is doing more long-term work than it looks like at first glance, and I'd resist the temptation to treat it as a nice-to-have. Every ticket that lands there is evidence about what the system's defined scope is actually missing, and a taxonomy review that periodically looks at what's accumulating in that log is how workflow number four eventually gets built with real evidence behind it instead of a guess about what users might ask someday. A fallback path that only ever deflects, without ever feeding what it catches back into how the system's scope evolves, is solving the immediate symptom while leaving the underlying gap exactly as wide as it was.

This pattern has a real limit, too. A confidence check catches input that doesn't map to any defined workflow. It has nothing to say about input that maps to the wrong defined workflow with high apparent confidence, a ticket that sounds like a password reset but is actually something subtly different that the classifier reads as a clean match anyway. That's a different failure, closer to a classification accuracy problem than a scope problem, and this pattern was never built to catch it. The fallback path protects against the gap between what a system knows how to do and what a user actually wants. Confident misclassification inside the three known workflows is a separate problem this pattern leaves untouched.

The temptation once this pattern is in place is to treat the fallback as good enough forever, a permanent home for anything unscripted instead of a holding pattern on the way to a fourth or fifth defined workflow. I resist that temptation deliberately. A fallback path that keeps catching the same category of off-script question month after month is telling you something specific: that category has earned its own defined path, and routing it to general troubleshooting indefinitely treats a known gap as an acceptable one long after the evidence says otherwise.