Stop, Narrow or Scale: Design Your AI Pilot for the Rollback

Here's a number worth sitting with before your next AI kickoff: in a September 2026 survey of 1,500 IT leaders, 98% said they had pulled back an AI deployment because of failure, hallucinations or unexpected behaviour (Cloudera, State of Enterprise AI 2026). The same survey found 89% had limited or redesigned AI use cases because of rising token costs.

Read that the right way. It doesn't mean AI doesn't work. It means pulling back is the normal path, not the exception. The question isn't whether one of your AI initiatives will need to stop or shrink. It's whether you decided in advance what that moment looks like, or whether you find out in a budget meeting six months later.

Most pilots are designed to succeed. Very few are designed to be judged.

Why pilots stall instead of ending

A pilot that's clearly failing is easy. You stop it. The expensive ones are the pilots that sort of work. The demo was impressive, a few people use it, nobody can say whether it's worth the money, and nobody wants to be the one who kills it. So it drifts. It keeps a licence, a vendor contact and a slice of someone's week, and it quietly becomes one more system to maintain.

That pattern usually has the same causes:

• No decision date. "Let's see how it goes" has no end.

• No baseline. If you didn't measure the process before the pilot, you can't prove the pilot changed it.

• No owner for the decision. The project has a sponsor. The go/no-go call has nobody.

• The data wasn't ready, and nobody checked. Gartner predicts that through 2026, organisations will abandon 60% of AI projects that aren't supported by AI-ready data (Gartner).

That last one deserves its own section.

Check the data before the pilot, not after

In my last post I described a company running three finance planning systems, one of them without actuals from SAP, with reporting done in Excel. Put an AI assistant on top of that and the pilot fails for reasons that have nothing to do with the model.

So the first gate isn't "is the AI good enough?" It's "can a machine rely on the data this process runs on?" If the honest answer is no, the right pilot might not be an AI pilot at all. It might be a data clean-up with a narrow AI use case at the end of it.

Gartner's own advice points the same way: define what AI-ready data means for the specific use case, and treat it as an ongoing practice rather than a one-time project. A pilot is the cheapest place to find out.

Three outcomes, decided up front

Before the pilot starts, write down what would make you choose each of these:

Stop. The pilot didn't move the metric, the error rate is too high for the risk involved, or the cost per use doesn't make sense. Stopping isn't failure. It's the pilot doing its job: you bought information cheaply.

Narrow. It works, but only for part of the problem. Maybe it handles the routine 70% of requests well and the hard cases badly. Narrowing means keeping the part that works, with a human handling the rest, instead of forcing it to cover everything.

Scale. It moved the metric, the error rate is acceptable, the people using it would be annoyed if you took it away, and you know what it will cost at ten times the volume.

That last point matters more than it used to. The same Cloudera survey found 45% of organisations had moved AI workloads because of higher or more volatile costs. A pilot that's cheap for 20 users can look very different at 2,000. Ask for the cost at scale before you approve the scale.

The one-page pilot brief

Here's the brief I'd want on the table before any AI pilot gets a budget. If you can't fill it in, the pilot isn't ready to start.

1. The process. Which process does this change, and who does it today?

2. The baseline. How long, how much, or how many errors today? Measured, not guessed.

3. The data check. Which systems feed this? Which one is the source of truth? Who owns it, by name?

4. The success metric. One number that should move, and by roughly how much, for this to be worth it.

5. The kill criteria. What result, error, or cost would make you stop? Write it before you see the results.

6. The decision date. A real date, usually 6 to 10 weeks out, not "end of Q2-ish".

7. The decision owner. One person who makes the stop, narrow or scale call on that date.

8. The rollback plan. If you stop, what goes back to how it was, and who tells the team?

9. The cost at scale. What would this cost at full volume, including usage-based fees?

Nine questions. Most of them aren't about AI at all. That's the point.

Make rollback cheap

The leaders who handle pullbacks well have one thing in common: they made rollback easy before they needed it. In practice that means:

• Keep the old process alive during the pilot. Don't decommission anything until the scale decision.

• Keep a human in the loop for anything customer-facing or financial until the error rate is known.

• Avoid long commitments at pilot stage. A month-to-month licence is worth paying a little more for.

• Tell the team up front that stopping is a possible, acceptable outcome. People report problems honestly when nobody's reputation depends on the pilot surviving.

Governance is where this usually breaks down later: 75% of the Cloudera respondents named governance and compliance as a constraint on scaling AI, and 49% said that if they could restart, they'd strengthen governance from the beginning. A pilot brief is the smallest possible version of that governance. It costs an hour.

What to do this week

Pick the AI initiative in your company that's been "in pilot" the longest. Fill in the nine questions above for it. If you can't answer the kill criteria, the decision date or the decision owner, you've found the real problem, and it isn't the model.

Then put a date in the calendar and make the call: stop, narrow or scale.

If you want help running this exercise with your leadership team, that's what our AI leadership workshops are for. Book a discovery call.