Why AI pilots stall before production.

The demo took three weeks and impressed everyone. Eighteen months later, nothing runs in production. This pattern is so common it has a shape, and the shape is worth studying — because every cause is preventable.

The demo answered the wrong question

A demo answers “can the model do this?” Production asks a different question: “does this hold at our volume, on our data, inside our permissions, at a cost we accept?” Teams that never wrote down the production question keep answering the demo question in increasingly elaborate ways.

Nobody owned a number

Pilots scoped as “explore AI for support” stall. Pilots scoped as “cut cost per ticket on password resets” ship. If no business metric was named before the pilot, there is nothing to graduate against — and no budget owner with a reason to push it through.

Evaluation came last instead of first

The teams that reach production build the evaluation harness before they build the feature. Test cases from real traffic, quality scoring, regression suites. Without them, every model update is a gamble, and gambles do not pass change-management review.

The integration was the real project

The model is rarely the hard part. Access to clean data, permissions to act in real systems, audit trails, rollback paths — that is the engineering. Pilots that treat integration as a detail discover it as a wall.

What to do instead

Scope one workflow with a metric. Build the evaluation first. Wire the sandbox to real systems early. Put a human checkpoint where errors are expensive. Then graduate deliberately: supervised runs, measured quality, expanding autonomy. That is not slower than the eighteen-month stall — it is a quarter to production.