MIT AI Report is Widely Misinterpreted

Much has been made of a study suggesting 95 percent of enterprises deploying artificial intelligence are not seeing a return on investment.

There’s just one glaring problem: the report points out that just five percent of those entities have AI in a “production” stage. The rest are pilots or limited early deployments. 

That significant gap between AI experimentation and successful, large-scale deployment arguably explains most of the sensationalized claim that “only five percent of enterprises” are seeing return on AI investment. 

It would be much more accurate to say that most enterprises have not yet deployed AI at scale, and therefore we cannot yet ascertain potential impact. 

Limited deployments or trials often do not integrate into core business workflows or provide the necessary scale to demonstrate significant financial impact, leading to the perception of widespread failure. And, in any case, up to 70 percent of all information technology projects fail to produce expected results. 

So it would be entirely normal for AI projects to fail much more often than they succeed. 

The report notes that “despite $30 to $40 billion in enterprise investment into GenAI, this report uncovers a surprising result in that 95 percent of organizations are getting zero return.”

But one has to dig just a bit deeper to make sense of those figures. The report notes that “just five percent” of integrated AI pilots are extracting millions in value, while the vast majority remain stuck with no measurable P&L impact.”

The report says “tools like ChatGPT and Copilot are widely adopted,” and that  “over 80 percent of organizations have explored or piloted them, and nearly 40 percent report deployment.”

But then come the important caveats. Those use cases “primarily enhance individual productivity, not P&L performance,” the report says.

Meanwhile, enterprise-grade systems, evaluated by 60 percent of organizations, have only had about 20 percent in a “pilot stage.” 

Most tellingly, “just five percent” or surveyed entities have enterprise AI systems in full production and use, the report says. 

The real issue is the percentage of firms that have fully deployed AI in a business process and are seeing return on investment. The report does not address that issue, but many of us might expect failure rates of up to 70 percent for those deployments. 

The point is that many, if not most, interpretations of the report’s data are off the mark. The study does not show enterprise AI use cases are not producing ROI 95 percent of the time. The report shows that most entities have not yet deployed at scale.

So, of course measurable returns are not available. One cannot measure the impact of an innovation one has not yet deployed at scale.