The Attribution Budget - Why Telemetry Never Answers Why I watched this play out on an agentic data-pipeline-repair system I built, an agent chain that diagnoses and patches seeded ETL failures, schema drift, null-handling bugs, timeout misconfigurations. I shipped genuinely complete, well-structured telemetry, one span per node, and treated that as sufficient for post-incident
The Approval That Expires - Building a Vendor Gate on Evidence The problem I built a small governance agent to answer one question employees kept asking: can I use this third-party AI vendor on a client's documents. The obvious first version checks a single allow-list, a flat table of vendor names marked approved or not approved, updated
The Approval Step I Wasn't Allowed to Collapse Yet The problem I built an onboarding workflow for new-hire access provisioning that had to survive a redesign, and I want to walk through what happened when I gave in to the obvious speed win. The existing human process was sequential: a manager signs off that a new hire needs
The Accuracy Number That Was True Once The Accuracy Number That Was True Once The problem I watched an internal knowledge-base Q&A agent launch against a RAG index built over the company's own policy documents, HR and IT material, the kind of thing employees ask about constantly. At launch, the team ran
The 503 That Kept a Chat Interface Honest The problem I gave myself two weeks to build an internal benefits assistant for a company's employees. I called it PolicyBot, because naming a pilot project is the one part of this job that never gets harder. The ask was simple: let an employee ask about PTO accrual
Ten Nodes, Forty-Five Connections, and a Throughput Drop Nobody Modeled The problem I built a five-agent content pipeline for a marketing production workflow: a research agent, a draft agent, a brand-check agent, an SEO agent, and a publish agent, each handing a piece of content to the next. Scaling that up looked like the obvious move: put a
Ten Days of Miscalibrated Sensors Before the Calendar Noticed Ten Days of Miscalibrated Sensors Before the Calendar Noticed I built a predictive-maintenance model once that scored vibration and temperature readings from forty machines to predict equipment failure seven days out. On day twenty, a firmware update recalibrated the vibration sensors, shifting the feature distribution without changing what normal
Sixty Snippets a Rule Check Could Have Caught Sixty Snippets a Rule Check Could Have Caught The problem I built a compliance review pipeline for marketing copy: two hundred snippets that need clearing before they can publish, checking for banned words, missing disclaimers, broken links, and the harder stuff that actually requires reading the copy and judging whether
Short, Bounded Pipelines - Trading a Seamless Run for a Shorter Leash on Drift The problem Ask a travel-planning agent to plan a five-day trip end to end in one uninterrupted run: interpret the traveler's preferences, search flights, search hotels, search a car rental, build a daily activity itinerary, check the result against a budget cap, reconcile schedule conflicts, and
Seventy Easy Questions Paying Premium Rates Seventy Easy Questions Paying Premium Rates The problem I ran a hundred requests through an internal help-desk triage agent, the kind of thing that answers whatever an employee types into the IT support channel. Seventy of those requests were trivial: what's the guest wifi password, how do
Seven Out of Ten Stopped Meaning What It Used To Seven Out of Ten Stopped Meaning What It Used To The problem I watched a marketing-copy pipeline auto-approve AI-generated ad copy for publication whenever an LLM judge scored it seven out of ten or higher on being on-brand and high quality. Anything under seven went to
Quality Gates at Every Transition - Stopping a Bad Value Before It Becomes the Next Step's Input The problem Take an eight-step insurance claims pipeline. OCR extraction reads the claim form. Field validation checks that the extracted values are well-formed. Policy lookup matches the claim to a record. Coverage determination decides what the policy pays for. A fraud flag screens the claim. Payout calculation sets
Order 31 and the Refund That Never Went Through Order 31 and the Refund That Never Went Through The problem I built a small order-support agent to watch a specific failure happen under controlled conditions, because I don't trust an intuition I haven't put through a project of my own. It handles a batch
One Flag Per Claim Isn't an Audit Trail - Telemetry at the Call Level One Flag Per Claim Isn't an Audit Trail - Telemetry at the Call Level The problem I built a research assistant for adjusters at a fictional insurance carrier, one I called ClaimAssist, and ran it through a short multi-step session per claim: a policy lookup, a medical-notes
One Exemplar Is Enough, and a Second One Leaks One Exemplar Is Enough, and a Second One Leaks The problem I built a receipt-digitization tool that pulls line items, name, quantity, unit price, total, out of OCR'd restaurant receipt text and into structured JSON. The prompt I started with included five few-shot examples of the
One Clause at a Time - Previewing Every Edit Before It Commits The problem I built a contract-redlining agent for legal-ops that reviewed vendor-contract clauses and proposed edits: tightening a liability cap here, adding a termination-for-convenience clause there. The first version I shipped auto-applied every proposed edit directly to the working document. There was no preview
One Bad Ticket, One Cleared Connector - Segregating Sensitivity by Chunk One Bad Ticket, One Cleared Connector - Segregating Sensitivity by Chunk The problem I built a customer-support assistant for a fictional broadband provider, one I called HelpDesk Copilot, retrieving from three different sources: a CRM export, a public product and policy handbook, and a set of support-ticket transcripts. Each
Load the Schema, Don't Paste It Load the Schema, Don't Paste It The problem A procurement automation agent I worked on extracts purchase-order documents into structured JSON against a forty-field schema, vendor info, line items, shipping terms, payment terms, all of it maintained by a data platform team as the canonical source
Every Ticket Went to the Expensive Model, Just in Case Every Ticket Went to the Expensive Model, Just in Case The problem A freight-logistics operation I worked with handles shipment exceptions: customs holds, missed connections, damaged-goods claims. An assistant drafts a resolution for a human dispatcher to approve on each one. I ran 600 tickets through it, spanning
Every Stage Logged Green While Acme Corp Split in Two Every Stage Logged Green While Acme Corp Split in Two I ran a legal-ops pipeline once that ingested sixty vendor contracts into a knowledge graph through five stages: parse, extract, resolve, classify, embed. I seeded the contracts with deliberately inconsistent party naming on purpose, the same vendor showing up
Eighteen Percent, Two Models Ago Eighteen Percent, Two Models Ago The problem I built a system called DrugQA to answer pharmacist-facing questions about drug interactions, and I shipped it with an abstention gate: a mechanism that teaches the model to say "I don't know" instead of guessing whenever its own
Coherent But Wrong - The Case for a Second Planning Checkpoint Coherent But Wrong - The Case for a Second Planning Checkpoint A customer writes in: "Refund to my original card. I already returned the item." I built a support agent to resolve tickets like this against an order database and a policy ruleset: a thirty-day return window, rules
Checkpointing Inside the Node - Resuming Instead of Redoing Checkpointing Inside the Node - Resuming Instead of Redoing The problem I built an insurance-claims intake pipeline to run fifty synthetic claim documents through a four-step process: extract, classify, enrich, file. The enrich step calls an external enrichment service that I mocked with an artificial delay and a configurable
Check It Before It Crosses - Verifying a Claim Before the Next Agent Trusts It The problem I built a three-agent news and intel pipeline, and running it unchecked felt like watching a game of telephone with a paycheck attached. A Source agent reads a seeded synthetic corpus of thirty articles, several of them carrying factual errors and outdated figures I'd planted
Can't Leak What Never Leaves - Air-Gapping the Highest-Sensitivity Tier Can't Leak What Never Leaves - Air-Gapping the Highest-Sensitivity Tier The problem I built a research assistant for a fictional law firm, one I called CaseLens, and gave it two kinds of work associates actually do: general contract-law research pulled from public case law and statutes,
