What a demo can't show you

Someone is about to show you a working system. Twenty minutes, screen shared, live, with a colleague narrating. It will go well.

That isn't dishonesty. Most demos are built by people who believe every word, and the thing on screen usually does work. The difficulty is structural: a demo is a performance of the best case, and the best case is not what you are being asked to buy.

Here is what the format cannot tell you, and what to ask instead.

Four things a demo structurally cannot show you

Whether the inputs were chosen. Every input in a demo has been run before, probably many times. Anything that produced an awkward result was quietly dropped weeks ago — not to deceive you, just because nobody demonstrates the example that goes wrong. What you are watching is the surviving subset.

What it does the second time. These systems sample from a range of plausible continuations rather than looking up an answer, so the same input can produce a different output on a second run. AWS lists nondeterminism among the disadvantages of generative AI, alongside hallucination and interpretability, and it is a design property rather than a defect awaiting a fix. Anthropic's own guidance treats consistency — similar responses to similar inputs — as a criterion you have to measure deliberately, which tells you it does not come free.

What it costs and how slow it gets when it isn't just you. A demo runs at an audience of one with nothing else in the queue. Latency and price per call are both named success criteria precisely because they are the two things that change most between a demonstration and a deployment.

What it does with your material. The demo corpus is clean, well-structured, and probably assembled for the demo. Yours is a shared drive with four versions of the same policy, two of them wrong, and a naming convention that broke in 2019.

The three requests that turn a demo into evidence

Each of these takes well under a minute, and asking all three is not rude. Anyone who has shipped one of these systems will expect them.

1. "Run this one." Hand over an input they have not seen — a real ticket, a real contract clause, a real question a customer actually asked. Choose something ordinary rather than adversarial; you are testing the middle of your distribution, not trying to break it.

This is the highest-value thirty seconds available to you in the entire meeting. It is also the request most likely to produce a reason why not, and the reasons are informative. "Our test environment doesn't have your data loaded" is fair and answerable later. "It needs to be phrased a particular way" is the actual finding.

2. "Run it again." Same input, twice, in front of everyone. You are looking for how much the answer moves. Some movement is normal and fine for drafting work. If it moves a lot and the intended use is a number someone will act on, you have learned something important about whether this belongs in that step at all.

3. "Show me one it gets wrong." The most revealing question in the room, and it works on internal teams as well as vendors.

How to read the answer to that third one

You are not really asking for a failure. You are asking whether they know where the failures live.

A strong answer is immediate and specific: it degrades on documents over about forty pages, it confuses these two product names, it does badly when the question contains two questions. That person has run it against a real test set and read the results. Everything else they told you gets more credible.

A weak answer is a reframe — that failures are rare, that a human reviews everything anyway, that the model is improving monthly. None of those are lies. None of them are an answer either.

The worst answer is a confident it doesn't really get things wrong. Every one of these systems gets things wrong. What that reply tells you is that nobody has looked systematically, which means the error rate is unknown rather than low.

What to ask for instead of a longer demo

The instinct after an unsatisfying demo is to ask for a bigger one. Resist it. A second demo is the same performance with more slides.

Ask for the evaluation. Anthropic's guidance on choosing between models is blunt about this — a good evaluation set is the most important step in the process, and you test with your actual prompts and your actual data. That advice was written for engineers picking a model, and it is exactly the right standard for a manager picking a supplier. The artefacts worth requesting:

• The test set: how many cases, where they came from, and whether the hard ones are in it.
• The score, with the definition of what counted as correct, and who decided.
• The failure cases, unedited.
• What it costs per transaction at your expected volume.

If those exist, you are dealing with a team who measured something, and the conversation gets a great deal more useful. If they do not exist, that is not automatically disqualifying — plenty of good projects are earlier than that — but it does mean you are being shown a prototype and should price the risk accordingly.

The distinction worth holding onto: a demo tells you the system can do the thing once. An evaluation tells you how often it does the thing. You are buying the second one.