Insight · AI pilots and scale

10 Reasons AI Pilots Stall Before They Scale

The model worked. The pilot succeeded. The rollout never followed. The reason is almost never the technology — it is that a pilot proves something different from what scaling needs.

For chief executives and IT directors All sectors 13 min read

In brief

Most mid-market AI pilots demonstrate that something works and then go nowhere. It is consistent across sectors, and the cause is rarely the model.
A pilot runs in conditions that are nothing like the conditions it would have to survive at scale. The ten reasons below describe what gets left out, and why it is expensive to add afterwards.
The position worth reaching is not “we have promising pilots”. It is “we know which of these could work at scale, what it would take, and what it would return”.

Most mid-market firms have run an AI pilot. Many have run several. The results looked promising, the team delivered, the numbers improved. The pilot succeeded.

What usually has not happened is the rollout. Some pilots stalled in evaluation. Some went live and never grew beyond the first handful of users. Some launched, quietly got worse, and were switched off without anyone announcing it. The pattern repeats across sectors.

The usual explanations — data quality, change management, skills — are real but partial. They describe symptoms. The cause is that a pilot is run in conditions that bear almost no resemblance to the conditions it would have to survive at scale, and the decisions that would have made scaling possible were never made.

The ten below describe what gets left out. None of them are fixed by improving the model. The starting point is recognising which apply to your own pilots.

01

The pilot was judged on technical performance, not on money

The proposal names a technical measure. How accurate it is. How fast. How much each run costs. The team optimises against it. The pilot finishes with the number improved, and the report celebrates the improvement.

What is rarely specified is which commercial number the thing would move at scale. Handling time. Conversion. Recovery rate on collections. Losses to fraud. Those are what a scaling decision actually turns on — and they were not what the pilot was measured on.

So when the rollout case is put, it rests on an accuracy improvement the board has no way of valuing. The fix is to name both numbers at the start, technical and commercial, and measure both. Most pilots name only the first.

02

The pilot was designed to fit how the business works today

The pilot is shaped around current practice. The support AI fits today’s ticket categories. The pricing AI fits today’s pricing. The forecasting AI slots into the finance routine as it stands. The pilot proves AI works inside the existing process.

But the value usually only appears when the process itself changes — different decision points, different handoffs, a different division of labour between people and machine. That change was not designed, scoped or costed. The pilot proved a narrow capability inside a business that stayed exactly the same.

When rollout arrives, the redesign surfaces as a far bigger piece of work than the pilot implied. The board sees the number and the case stalls. The fix is to work out how the job would have to be done differently first, then pilot against that — not against today. Most programmes do it the other way round, which is why the second year is harder than the first.

03

The team that built it is not the team that would run it

The marketing AI was built by the data team with marketing input. The pricing one by finance and the data team. The service one by a group that disbanded once the pilot ended.

What was rarely settled is who would own it day to day at scale. Marketing operations, not marketing strategy. The service floor, not the customer experience function. Different people, different accountability — and not the people who built it or signed it off.

So when rollout is proposed, the person being asked to own it inherits something they did not design, running on a data flow they do not fully understand, against measures nobody has explained. Nothing was built to hand it over. The rollout stalls in the handover, which looks like reluctance and is actually a design gap. The fix is to name the operational owner at kickoff, not at the rollout gate.

A pilot proves the model works. Scaling proves whether the business works. They are not the same question, and only one of them was tested.

04

The data was prepared by hand and nobody costed doing that forever

Someone spent weeks getting the pilot data ready. Manual extracts. One-off joins across systems. Custom fixes written in code rather than corrected at source. The pilot ran on data that looked clean because a person had cleaned it.

At scale that preparation has to happen continuously, automatically, at full volume. That is a materially harder thing to build: integration at source, governance, quality monitoring, knowing where each field came from, privacy controls.

Most firms discover this at the rollout gate. The cost of doing the data properly is often comparable to the cost of the AI itself, sometimes larger. The rollout case has to carry both, and the total is nothing like what the pilot suggested. The fix is to treat the data work as an equal investment from the start rather than a detail to sort out later.

05

Nobody knows what it costs to run at volume

The pilot costs are visible. Build, infrastructure, data preparation. The finance director has those numbers.

What the finance director rarely has is the cost of running it at real volume. Cost per use at production scale. Ongoing engineering. Retraining. Monitoring. Infrastructure at projected usage. That conversation arrives at the rollout decision, and the numbers have to be assembled in a few days from data nobody gathered.

So approval stalls, not because the case is bad but because the running cost is not credible. Enthusiasm does not survive a finance review. The fix is to measure cost per use during the pilot even at tiny volume — it is the best available indicator of what it will cost later, and it takes almost no effort to capture at the time.

Where do you sit?

Recognising the gap between pilot and rollout is the first step. Knowing which decisions were never made is the next.

A free Executive AI Opportunity Assessment. Forty-five minutes, online, with your leadership team. We arrive with a model of how firms in your sector make money and where AI has moved those numbers — and you tell us where it is wrong.

Book a free assessment

06

The pilot was governed lightly; a rollout cannot be

The pilot ran with almost no governance. The data team judged its own quality. One stakeholder reviewed the outputs. Risk was told about it and left alone.

At scale, everything switches on at once. Model risk. Documentation that would survive an audit. Fairness assessment. Compliance review. Continuous monitoring. What happens when it goes wrong. Being able to explain a decision to a customer who asks. None of it was built.

Retrofitting that is expensive, and occasionally the model has to be rebuilt to meet documentation standards it was never designed for. The fix is to build the pilot to the standard the rollout would need, even though most of it stays dormant at pilot scale. It costs a little then and saves a great deal later.

What changes between a pilot and a rollout

What	In the pilot	At scale
The data	Prepared by hand, once, for this use	Continuous, automated, at full volume
The users	A small group who volunteered	Everyone, including those who did not
Where it sits	On its own, off to one side	Wired into journeys and adjacent systems
Governance	Light, because the exposure is small	Risk, compliance, audit and monitoring all live
The money	Marginal, and nobody is counting closely	Cost per use decides whether it goes ahead
Who owns it	The people who built it	The people who have to run it every day
Getting people to use it	Voluntary; they chose to be there	Required; thousands affected; some will resist
If it goes wrong	A small problem, contained	A large one, in front of customers

07

It proved one thing works, not that it fits anything

The model works. It predicts accurately, finds the right customers, writes the right text. That demonstration is genuine and worth having.

What it did not show is how the thing fits into the rest of the business — into customer journeys, into decision flows, into audit trails, into the systems either side of it. That fit was never designed, so it has to be built afterwards.

And it is rarely costed properly. Each connection has to be designed, built, governed and run. Added together it often exceeds the original build. The rollout case absorbs all of it and becomes much harder to defend. The fix is to design the fit alongside the pilot, and to make it part of what the pilot has to prove.

For the chief executive and IT director

Three questions about your current pilots

For each one, can you name the commercial number it is being measured against — not the technical one? If only the technical answer exists, the rollout case will not survive the board.
For your most promising pilot, has the person who would run it every day agreed to take it on? If the answer is “we will sort that out when we scale”, it will stall at exactly that point.
What have you spent across all your pilots, and how many have reached real scale? If the second number is lower than you expected, the problem is not the pilots.

08

Getting people to use it was easy in the pilot and is the main cost at scale

The pilot ran with volunteers. People who wanted to try it. Their feedback was thoughtful, their workarounds inventive, their usage strong. It cost almost nothing to get them on board.

Rolling out means changing how hundreds or thousands of people work. The ones who did not volunteer. The ones who like the current way. The ones whose targets change. The ones who lose a decision they used to make. Each is manageable; together they dwarf the original build.

The pilot treated this as incidental. At scale it is often the largest line, sometimes bigger than the technology and data combined. The fix is to size it against the number of people whose jobs change, not against the pilot — and to do that sizing before the board sees a number it will later have to double.

09

The pilot succeeded because it was protected from reality

It ran in controlled conditions. Clean data, tight scope, willing colleagues, minimal contact with other systems. It succeeded inside those constraints, and the constraints were what made success possible.

Real conditions have none of them. Production data is dirtier. Scope widens the moment it is useful. The people involved did not choose to be. It touches other systems constantly. The same capability that worked in isolation behaves differently once it is connected to everything else.

This is where most rollouts fail quietly — not at the model, but at the joins. The model still works when you test it on its own, which makes the failure genuinely confusing to everyone involved. The fix is to test against real conditions during the pilot: production data quality, fragile neighbouring systems, actual handoffs between people.

10

Nobody owns the pilots as a portfolio

Each pilot has a sponsor. Marketing sponsors one. Finance another. Operations a third. Each is approved, run and reviewed inside its own department.

What does not exist in most mid-market firms is anyone who owns the whole set. The data investments that several of them share. The governance that would serve all of them. The decision about which ones deserve the next round of money and which should stop.

So the decisions get made one at a time, in isolation. Pilots compete individually for funding. Two that need the same data investment pay for it separately or not at all. Failing ones continue because stopping them is nobody’s job. The fix is to make the portfolio someone’s actual responsibility at executive level. Most mid-market firms have not, which is why their AI spend looks larger in aggregate than any single decision suggested.

What this means

The model works. The capability is real. The failure to scale is not a failure of the technology — it is the gap between the conditions a pilot runs in and the conditions a business actually operates in.

Every one of the ten describes a decision that got deferred: who owns it, how it is governed, what it connects to, where the data comes from, how the job changes, what it costs to run. Deferring was rational at pilot stage, because the volumes were small and the cost of deciding was real. At scale those deferred decisions become the whole cost, and they are far more expensive to settle after the fact than they would have been alongside.

Which is the argument in one line. Change the work before you change the technology. A pilot that only proves the technology works has answered the easy question. The harder one — what would have to be different for this to be worth doing — is the one that decides whether it ever scales.

The next step

Are your pilots built to scale, or built to demonstrate?

Forty-five minutes with your leadership team, online, at no cost. We bring a working model of how firms in your sector make money and where AI has moved those numbers. You tell us where it is wrong. Then we tell you what we think — including “don’t invest”, if that is the honest answer.