The Free Half of Speculative Decoding Ask most teams running speculative decoding on constrained hardware what they've quantized, and they'll tell you about the big model. Shrink the target model to INT4, keep the draft model at full precision, ship it. That's the instinct, and it leaves a second speedup
The Bottleneck Isn't the Fix, It's Finding What to Fix Ask a client team why their coding agent pilot is burning through budget on a modernization sprint and the answer is almost always the same: the model. Swap Sonnet for GPT, GPT for Gemini, whichever one had the best demo last quarter. I've sat in enough of these
The Benchmark Passed. The Chain Still Failed. Pull up whatever evaluation got your last agent deployment signed off. The number that convinced the room was almost certainly a benchmark score for the model, or a per-task success rate measured with the environment reset between runs. Neither number describes what that same model does across ten or
Running the Experiment Was Never the Hard Part Every move AlphaGo made during a game got graded before it was played. Not at the end, not after the game resolved into a win or a loss. Right then, move by move. Monte Carlo Tree Search runs the position forward thousands of times and hands back a target: this
Read the Paper, Not the Panel A client forwarded me a podcast clip last week and asked one question: is Claude conscious? He'd caught a clip, not the paper. That's normal. Nobody reads seventy pages of interpretability research before a Tuesday call. But it means the gap between what a research team
MCP Went Stateless. The State Didn't Go Anywhere. MCP crossed 400 million monthly SDK downloads in July, four times what it was moving at the start of the year, at almost the exact moment its authors ripped out the part of the protocol that was hardest to run in production. That's exactly what happens when a
It Can Read the Diagram. It Can't Reason About It. Hand a vision-language model a circuit diagram and ask it to name the components. It will get the resistor. It will get the capacitor. Ask it to trace what happens downstream of the switch, and something quietly breaks, in a way that never shows up on the benchmark slide
Hallucination Left the Sentence and Moved Into the Chain Every client conversation about AI governance eventually arrives at the same sentence: "we have a disclaimer in the system prompt." That's a permission slip, not a mitigation. It tells the user the model might be wrong, but it doesn't catch a wrong answer before
Five Questions I Ask Before a Vendor's Benchmark Goes in a Client Recommendation Next time a vendor's deck shows 81% on SWE-bench, don't write the number down yet. Ask what harness produced it. Ask who audited the eval. Ask whether an independent lab has ever reproduced it on the same tasks. If nobody in the room can answer,
Five Places a Document Pipeline Can Fail, and Four of Them Just Got Fixed A document AI pilot usually starts the same way. Someone points a vision-language model at a stack of PDFs, wires it into Azure AI Search or a Bedrock Knowledge Base, and calls the retrieval working because it answers three test questions correctly. Then it meets a real regulatory filing
Fine-Tune or Orchestrate: The Architecture Decision Every Agent Engagement Skips Pull up the last agent engagement I scoped and look for the sentence where I decided whether we were fine-tuning a model or building an orchestration layer around one. Most of the time that sentence isn't there. The decision got made anyway, by default, usually in favor
Escalation to a Human Is Not a Confidence Score Ask most agent frameworks how they decide when to hand a task to a human, and the answer is a threshold on a single number. The model reports a confidence score, something crosses 0.7 or fails to cross it, and the system either proceeds or escalates. That number is
Applause Is Not an Audit When IBM says its new coding tool lifted developer productivity 45 percent, run that number through the same test you'd run a benchmark slide through. Who checked it. Under what conditions. Compared to what baseline. If the answer is "nobody, IBM measured IBM," treat it as
Agent Containment Is Losing the Race to Capability Pick any three AI security writeups from the last month and you'll find the same sentence buried in each one. The agent did exactly what it was told to do, by someone who was never supposed to be talking to it. The instruction came from a public GitHub
A Rule in the Prompt Is a Suggestion Put four rules in an agent's system prompt. Check the book isn't recalled before renewing it. Confirm the library card on the request matches the one on file. Cap the renewal at thirty days. Keep the tone encouraging, even if the customer is rude. Test it
A Bigger Context Window Is Not Working Memory Ask a team running AI agents in production what their context window is, and they answer in seconds. A million tokens, sometimes two. Ask the same team what their agent's memory architecture is, and the room goes quiet. Nobody built one. They bought a bigger window and assumed
Five Firms Built Five AI Frameworks. None of Them Built a Ruler. Put BCG's AI framework next to Gartner's. Then put Bain's next to McKinsey's. Then put NIST's next to all four. Don't read them one at a time, the way they were designed to be read, in separate decks
Outcome-Based Pricing Is Real For Two Companies And A Rumor For Everyone Else Accenture lost 18% of its market value in a single session this year. Worst day since 2016, for a company that's been public since 2001. The headline reason was a guidance cut and soft bookings. The real reason, the one analysts kept circling back to, was doubt about
One Catalog, Every Consumer Ask a vendor whether their platform does everything this series has described, catalog and registry together, review gates before a skill ships, ownership that scales past one team, federation that holds up at the edge of the organization, and the answer is yes. Ask what "everything" means to
Off the Shelf Gets You Partway There More than 2,500 Claude Code plugin marketplaces are registered at claudemarketplaces.com as of mid-2026, ranging from polished, actively maintained collections to abandoned repositories nobody has touched since launch. Roughly 40 products across the industry now support the same open Agent Skills specification, which means a skill written
Somebody Has to Own the Skill Library Gartner projects that 40% of enterprise applications will embed task-specific AI agents by the end of 2026, up from under 5% a year earlier. Most of the organizations already in that number are sitting at what the Cloud Security Alliance describes as level 100 to 200: agents genuinely in
A Skill Registry Needs a Review Gate, Not Just a Folder A Git repository with a folder per skill is not a registry. It's a shared drive with better version history, and everything this series has already covered about why a shared drive isn't a system applies to it without modification. The difference between a folder of
Skill Sprawl Is the Shared Drive Again Elvis Sun, a former software engineer at Google, documented the moment his agent needed to read an image off his desktop, in what he called a nine-hour deep dive into his own agent setup. It tried a browser-read skill. Nothing. It tried a vision skill. Nothing. So it
A Skill Is Just Another Thing to Find Anthropic published Agent Skills in October 2025 and open-sourced the specification two months later. Strip away the framing and the format is almost defiantly plain: a folder containing a markdown file with some frontmatter, plus whatever scripts and reference material the task needs. Barry Zhang, who co-created the
Hybrid Retrieval Is the Floor, Not the Ceiling Neither method gets there alone, and the numbers are consistent enough across mixed enterprise corpora that they're worth treating as a baseline rather than a benchmark from one project. Keyword search by itself lands around 65% recall at ten results. Dense embeddings by themselves land around 75 to
