Skip to content

AI, decoded

Why do most enterprise AI projects fail to show ROI?

Most stall before they ever reach the scale where returns show up. The pilot demos well, then the project hits the costs nobody budgeted: evaluation, integration with messy real systems, data cleanup, governance sign-off, and the ongoing expense of running and monitoring the thing. Add a vague success metric — 'improve productivity' with no baseline — and you get projects that consume budget without producing a number anyone can point to. The failure is usually operational and organizational, not the model. On Chain of Thought, Accenture's Jay Subrahmonia cited stats that fewer than 10 percent of AI proofs of concept reach production, partly because the governance, testing, and monitoring aren't in place.

· Updated · Chain of Thought

Level 3: Leading with AI · 3.3 Pilot to production

Enterprise AIAI Evaluation & Reliability

  • 10 episodes
  • 9 guest clips
  • 5 glossary terms
An iceberg with a small tip labeled pilot above the waterline and a huge mass below, beside the words: under 10% of AI pilots reach production. The figure is a stat Accenture’s Jay Subrahmonia says the firm hears, from episode 4.

In detail

Pilots are easy; production is where ROI lives

A proof of concept proves the model can do the task once, in clean conditions. Returns mostly appear at scale, in production, running against real data and real users. The distance between those two is where most projects end — the pilot succeeds, the rollout never happens, and the spreadsheet shows cost with no offsetting return.

Accenture’s Jay Subrahmonia said in episode 4 at 33:00 that “we also hear stats” putting production deployment at less than 10% of proofs of concept, partly because the governance, testing and monitoring needed to make that shift are missing; he does not name the source of the statistic. In July 2024, Gartner predicted that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, citing poor data quality, inadequate risk controls, escalating costs or unclear business value.

The costs nobody budgets

The line item that kills ROI is everything after the model works: wiring it into legacy systems, cleaning the data it depends on, building evaluation so you can trust it, clearing governance and compliance, and paying to run and monitor it indefinitely. These are large, ongoing, and easy to leave out of the original business case — so the project looks cheaper going in than it is. Kyndryl Global Head of Strategy Kris Lovejoy says in episode 62 at 6:59 that critical infrastructure customers often run mainframes, newer systems and multiple clouds, with data scattered across them. A pilot does not settle the data integrity, security and reliability work required to operate across that mix.

Vague metrics guarantee invisible returns

Many projects can’t show ROI because they never defined it. “Make support more efficient” isn’t measurable; “cut average handle time by 20% against this quarter’s baseline” is. Without a baseline and a target set before you start, even a project that works can’t prove it did. In episode 76 at 4:33, Sonar CTO Andrea Malagodi found one engineer with more than 500 pull requests and a low AI cost per pull request, while another had high spend and few pull requests. Looking into their work showed they were solving different problems, so the cost metric alone could not establish whose work was more valuable. (5:02)

What the successful ones do differently

They pick a problem with a clear, measurable value, budget for the whole lifecycle rather than the pilot, and decide up front how they’ll measure success. The model is rarely the hard part. The operating discipline around it is.

Picking the problem can be the hard step. OutSystems co-founder Rodrigo Coutinho said in episode 19 at 36:22 that many of the company’s customers know AI matters but have not found the use cases where it pays off, partly because of a “wrong perception that these technologies fix everything” (37:14). His conclusion at 37:40 is that “the biggest challenge is not the technological” but ideation: working out with the business where the technology is strong. The episode aired in season two, which Galileo sponsored while Conor worked there; Galileo CTO Atindriyo Sanyal co-hosted (0:30).

Adoption also needs an owner. Claire Vo, then chief product and technology officer at LaunchDarkly and founder of ChatPRD, said in episode 36 at 6:05, published in August 2025, that large companies adopting AI in a meaningful way are operationally mature: their engineering practices already protect quality and velocity, which makes it easy to add agentic engineers or AI IDEs, because the same practices apply as when you add engineers or tools (6:33). Along with risk tolerance and that maturity, she names centralized focus and accountability. At LaunchDarkly, she says, the shift that made the biggest difference in engineering adoption was making one of its most senior engineering leaders the AI czar (7:22). She is describing engineering adoption, a precondition for returns, not measured ROI. The episode aired in season two, which Galileo sponsored while Conor worked there; Conor points listeners to the RunGalileo YouTube channel at 42:24.

Disclosure: In episode 4, Conor introduces himself as Galileo’s head of developer awareness, with Galileo CEO Vikram Chatterji co-hosting (0:00). Episode 76 credits Svix, Walrus Memory and Inngest as Season 4 presenting sponsors (Svix read at 14:55, Walrus Memory at 16:01, Inngest at 34:17) and carries a G2i episode sponsorship (read at 33:36).

Why enterprise AI projects fail to show ROI: the pilot is the tip An iceberg. Above the waterline, the small tip: the pilot, where the model does the task once in clean conditions and demos well; this is what the business case budgets. Below the waterline, the bulk that is easy to leave out of the budget: wiring it into legacy systems, cleaning the data it depends on, evaluation you can trust, governance and compliance sign-off, and running and monitoring it indefinitely. Beside it, returns mostly appear in production at scale, and can only be proved if success was defined up front: "make support more efficient" cannot be measured, while an example target, "cut average handle time by 20% against this quarter's baseline", can. Accenture's Jay Subrahmonia, episode 4, cites stats that fewer than 10 percent of proofs of concept reach production, partly for lack of governance, testing and monitoring. The pilot is the tip. The cost is underneath. Most enterprise AI projects stall between a pilot that works and the production scale where returns show up. easy to leave out of the budget the pilot works once, in clean conditions; what the business case budgets wiring into legacy systems cleaning the data evaluation you trust governance and compliance sign-off running and monitoring it, indefinitely RETURNS MOSTLY SHOW UP IN PRODUCTION …provable only if you defined them first. VAGUE ✗ “Make support more efficient.” works or not, it can’t prove it did MEASURABLE ✓ · FOR EXAMPLE “Cut average handle time by 20% against this quarter’s baseline.” Pick a problem with a clear, measurable value. Budget the whole lifecycle, not the pilot. Accenture’s Jay Subrahmonia, ep 4, citing stats Accenture hears: fewer than 10% of proofs of concept reach production, partly for lack of the right governance, testing and monitoring. The 20% target above is an illustration, not a benchmark.
Budget the whole lifecycle and set the metric before you start. The under-10% figure is a stat Accenture’s Jay Subrahmonia cites in episode 4; the 20% target is an illustration. Download the image

Hear it from the guest

“We also hear stats that say … while there are a lot of POC that have been done only less than 10 percent have actually been put into production and partly … it's because of not having the right sort of governance and the, testing and the monitoring in place to be able to make that shift.”
Jay Subrahmonia, Accenture EP 4 · around 33:48 · read the transcript
“But I feel like a lot of people skip over the hard part. It's less fun. It's less exciting. It's less sexy.”
Greg Statton, Cohesity EP 27 · around 23:35 · read the transcript
“95% of all AI work today done by enterprises remain as proof of concepts. They don't move into production.”
Siva Surendira, Lyzr EP 21 · around 9:27 · read the transcript
“The use cases where a customer is not aligned on what good looks like.”
May Habib, Writer EP 1 · around 43:35 · read the transcript
“They want to build something shiny and adopting some cool tech, right, but not necessarily solving a business problem.”
Aurimas Griciūnas, SwirlAI EP 38 · around 23:10 · read the transcript
“Most critical infrastructure industries are not cloud first. They may have some mainframe mid-range. They have your X-series. They've got cloud infrastructure. They probably have multiple clouds. They've got a truly hybrid environment. Some of it is very old. We say it's vintage.”
Kris Lovejoy, Kyndryl EP 62 · around 6:59 · read the transcript
Watch on YouTube
“I get to a reasonable understanding of the cost of PR. And we did that. And what I observed in the numbers, which I have to say, I mean, it made me incredibly curious immediately. was I had on the one side Individual at the one extreme that had an enormously high PR count, over 500.”
Andrea Malagodi, Sonar EP 76 · around 4:33 · read the transcript
Watch on YouTube
“The biggest challenge is not the technological. It's actually this ideation process and coming up with great use cases where they can benefit from the technology.”
Rodrigo Coutinho, OutSystems EP 19 · around 37:40 · read the transcript
“They're operationally mature in a way that their engineering practices already protect for quality, already protect for velocity”
Claire Vo, ChatPRD EP 36 · around 6:05 · read the transcript

Quotes lightly edited to remove filler words.

From the conversation

This explainer is drawn from these episodes — each carries its full transcript.

Concepts in this explainer

AI Pilot (Proof of Concept)AI BubbleCost per PRData QualityGenerative AI

Keep exploring