Diagnosis
Why your AI pilots never reach production
Last updated
Justas Butkus is an AI implementation advisor based in Vilnius, Lithuania, working with mid-market companies and scale-ups across the UK, EU and US. He builds and operates production AI systems, and takes engagements owning AI strategy and delivery for companies whose projects keep stalling.
Short answer
AI pilots stall for three reasons, and the technology is almost never one of them: the wrong use case was chosen, the data could not support it, or nobody owned the decision to put it live. Each has a different fix, and choosing a different model will not address any of them.
The three reasons, in order of frequency
After enough of these the pattern is consistent, and the specific technology in use rarely appears in it.
- The use case was chosen because it was interesting, not because it was expensive. Somebody demonstrated something impressive, and the organisation worked backwards from the demonstration to a justification. The result works and nobody misses it when it stops.
- The data could not support it. Not missing data — data that exists but is inconsistent, ambiguously defined, or means different things in different systems. This is discovered late because everyone assumes their own data is fine.
- Nobody owned the decision to go live. The pilot succeeded. Then the question of who accepts the risk of running it in front of customers had no answer, so it sat in a pilot state indefinitely, which is a decision by default.
How to tell which one you have
| Symptom | Likely cause | What actually fixes it |
|---|---|---|
| It works, but nobody asks for it | Wrong use case | Re-select against cost and frequency, not novelty |
| Accuracy is fine in testing, poor in reality | Data definitions differ across systems | Fix definitions before touching the model |
| Endless "one more evaluation round" | No owner willing to accept the risk | Name an accountable owner and define the rollback |
| Different departments each ran their own | No mandate at company level | One roadmap, ranked, with exclusions stated |
| Legal or compliance stopped it | Governance considered too late | Design oversight in from the start, not retrofitted |
Why choosing a different model rarely helps
The most common response to a stalled project is to try a better model. It is satisfying because it is tractable, and it almost never addresses the actual constraint.
If the use case was not worth doing, a better model produces a better solution to a problem nobody had. If the data is inconsistent, a better model learns the inconsistency more accurately. If nobody will own the decision to go live, model quality was never what was blocking it.
The exception is genuine capability limits, where the task was simply beyond what was available when you built it. That does happen, and it is worth re-testing periodically. It is just far rarer than it gets blamed for.
What unblocking actually involves
- Re-select the use case against cost and frequencyThe right first project is boring, expensive and repetitive. If it is exciting, be suspicious.
- Test the data before designing anythingNot whether it exists. Whether the same field means the same thing everywhere it appears.
- Name who accepts the riskBefore the build, not after. The absence of this name is why pilots become permanent.
- Define what "working" means, numericallyAgreed in advance, or evaluation becomes an argument about vibes that nobody can win.
- Ship the unglamorous one firstProve the pipeline end to end on something low-stakes before attempting the ambitious thing.
Frequently asked questions
Why do most AI projects fail to reach production?
Rarely because of the technology. The common causes are choosing a use case for novelty rather than cost, discovering late that data definitions are inconsistent across systems, and having nobody willing to accept the risk of putting the system in front of customers.
Would a better model fix our stalled project?
Usually not. A better model solves a capability problem, and most stalls are selection, data or ownership problems. If the use case was not worth doing, a stronger model produces a better answer to a question nobody was asking.
How do we choose the right first AI project?
Pick the process that is expensive, frequent and repetitive rather than the one that demonstrates well. The right first project is usually boring, and its value comes from volume rather than from sophistication.
Who should own AI implementation in a mid-sized company?
One named person with authority to decide what gets built and to accept the risk of putting it live. Distributing that across a committee is the most reliable way to produce pilots that never ship.
How long should it take to get something into production?
A well-chosen first use case with adequate data should reach production within a quarter. If it has been substantially longer, the constraint is almost always selection, data or ownership rather than engineering effort.
If this describes your last two projects
The diagnosis is usually quick, because the three causes look quite different from each other once you know to look.