Agent Washing: How to Spot It in a Vendor Demo

Every vendor in the category now says the word "agent." The demo is where the word either survives contact with a real question or quietly turns back into the chatbot it always was. Here is what to ask.

The demo is going beautifully, which is the first thing that should make you uneasy. On the screen, a sales engineer types a request in plain English, and the system responds with a fluent, confident paragraph, then produces a tidy summary, then offers three next steps in a bulleted list. Everyone in the room nods. It looks like the future, and it looks that way because it has been rehearsed until it does. What you are watching is the happy path — the one scenario the vendor knows the product handles, walked at the exact pace and along the exact route where nothing breaks. The trouble is that the happy path is the one part of the job you were never worried about. You are buying software to handle the days that do not go according to script, and the demo has been engineered to make sure you never see one.

This is the environment in which "agent washing" thrives, and it is worth naming plainly because the analysts are already naming it. Gartner has predicted that over forty percent of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, inadequate risk controls — and, memorably, "agent washing," the practice of relabeling existing chatbots, rule engines, and workflow tools as autonomous agents without changing anything underneath. That number is not a verdict on the technology. It is a verdict on procurement. A large share of those cancellations will trace back to a room exactly like this one, where a buyer watched a polished happy-path demo, saw a chatbot wearing the word "agent," and signed anyway. The demo is not where you get impressed. It is where you do your diligence, and diligence in this category comes down to a small number of questions that a relabeled tool cannot survive and a real one answers without flinching.

Take it off the happy path and watch what it does

The single most revealing thing you can do in a demo is refuse to follow the script. When the sales engineer offers to show you the standard flow, ask instead for the ugly one — the case with a missing field, the request that contradicts itself, the customer record where two systems disagree about the truth. What you are testing is not whether the product is smart. It is whether the product was built to reason about a situation it was not explicitly told how to handle, or whether it was built to recognize a fixed set of inputs and route them down fixed paths. A relabeled workflow tool betrays itself the instant it leaves familiar terrain: it either falls back to a generic apology, hands the whole thing to a human with no attempt to make progress, or — worst and most common — proceeds confidently down the wrong branch because the only branches it has are the ones a designer drew in advance. The failure that costs you money in production is almost never one the designer anticipated, which means the demo scenario that matters most is precisely the one the vendor did not prepare.

Pay close attention to how the system behaves at the boundary of its knowledge, because that boundary reveals the architecture. A genuine reasoning system, encountering an unfamiliar case, does something recognizably intelligent: it gathers the context it is missing, states what it does and does not know, forms a plan, and either acts or asks a specific, well-formed question about the one thing it actually needs. A dressed-up chatbot, by contrast, has no model of its own uncertainty. It cannot tell the difference between a case it understands and one it is merely pattern-matching against, so it answers both with the same untroubled fluency. That fluency is the tell. Confidence uncorrelated with correctness is the native failure mode of a language model with no agency wrapped around it, and no amount of prompt engineering converts it into judgment. When you push into the unfamiliar and the system gets quieter, more careful, more specific about what it needs — that is reasoning. When it gets smoother, it is theater.

Can it act, or only suggest?

The second question separates the entire category into two piles, and it is deceptively simple: at the end of the task, does the system do the thing, or does it hand you a nicely written recommendation to go do the thing yourself? An enormous amount of what is sold as agentic AI is, on inspection, a very articulate suggestion engine. It drafts the email but does not send it, proposes the refund but does not issue it, identifies the record that needs updating but leaves the updating to you. There is nothing wrong with a suggestion engine, and for some jobs it is exactly right — but it is not autonomy, and it does not change your unit economics, because a human still sits on the critical path of every single transaction. If the product's real output is advice that a person must then execute, you have not removed the bottleneck. You have given your team a faster way to do work they still have to do.

The distinction you are probing for is whether the system can take a real action in a real system of record and own the consequence of that action. Ask to see it write, not just read — post the update, move the ticket, trigger the downstream process — against something that actually persists. This is where the conversation should turn immediately to control, and a serious vendor will welcome that turn rather than deflect it, because acting in the world is exactly where the engineering gets hard and where the relabeled tools have nothing to show. The right architecture does not choose between "acts autonomously" and "keeps a human in charge." It does both at once: the system executes the routine actions end to end and escalates the consequential ones to a person by design, with the boundary between the two set by policy rather than by accident. Human-in-the-Loop is not a limitation the vendor apologizes for; it is the feature that makes autonomy safe to deploy, and its absence — a system that either does nothing on its own or does everything with no gate — is itself a warning. What you want to hear is: it acts on what it should, and it stops on what it shouldn't, and you decide where that line sits.

Make it show you the trace

The last question is the one that a demo built on illusion cannot survive, and it is the one buyers most often forget to ask: show me exactly what it did, and let me replay it. A real autonomous system produces a trace — an inspectable, step-by-step record of what it observed, what it retrieved, what it decided, why it decided that, which action it took, and where a human signed off. That trace is not a nicety for the compliance team. It is the difference between software you can operate and software you can only hope about. When something goes wrong in production, and it will, the trace is how you find out whether the system misread the input, retrieved the wrong context, or reasoned poorly from good information — three different problems with three different fixes. A tool that cannot show you its own reasoning cannot be debugged, cannot be audited, and cannot be trusted with anything that touches money or a customer, because when it fails you will have no idea why and no way to keep it from failing the same way again.

Watch carefully how the vendor reacts when you ask to see the trace, because the reaction is often more informative than the answer. A team that built genuine agency will pull up the observations and the decision path almost eagerly, because that record is the thing they are proudest of and the thing that took the most work to build. A team that dressed up a chatbot will show you a chat log — the prompts and the responses — and call it a trace, hoping you will not notice that a transcript of what was said is not a record of what was done and why. Insist on the distinction. Ask to see the same case run twice and to compare the paths. Ask what happens to the trace when the case is messy, because a clean trace on a clean case proves nothing. The presence of a real, replayable, reason-bearing record of the system's own behavior is the strongest single signal that there is an actual agent underneath, precisely because it is the hardest thing to fake and the last thing a relabeled tool possesses.

Underneath these three questions is a single idea that reframes what a demo is for. You are not there to be shown how good the software is on the day it was rehearsed; you are there to find out what it is made of, and the way you find out is to attack the seams the rehearsal was designed to hide — the unfamiliar case, the moment of action, the demand for a trace. This is the same discernment that separates the organizations quietly building toward an autonomous enterprise from the ones that will contribute to Gartner's forty percent, and it is why platforms built for this — StudioX's Autonomous AI Workers running AI Missions through a Reasoning Core, recording their Observations, escalating through Human-in-the-Loop gates — are architected to answer exactly these questions rather than to avoid them. So carry a simpler test into the next demo than any feature checklist will give you. A chatbot performs on the path it was shown. An agent tells you what it did when you took the path away. Ask for the second one, and do not let the fluency of the first talk you out of the difference.