More than 80% of AI Projects Fail, Rand Study Finds

“By some estimates, more than 80 percent of AI projects fail,” says a new study from Rand. “This is twice the already-high rate of failure in corporate information technology (IT) projects that do not involve AI.” 

That might seem shocking, but those of you familiar with the success rates of enterprise IT projects overall will not be surprised, as the general rule of thumb is that up to 70 percent of IT projects actually fail in some way. 

Likewise, studies suggest 74 percent of digital transformation projects fail. Innovation is hard. Some 75 percent of venture-funded startups also fail. 

From 2003 to 2012, only 6.4 percent of federal IT projects with $10 million or more in labor costs were successful, according to a study by Standish, noted by Brookings. 

IT project success rates range between 28 percent and 30 percent, Standish also notes. 

The World Bank has estimated that large-scale information and communication projects (each worth over U.S. $6 million) fail or partially fail at a rate of 71 percent. 

McKinsey says that big IT projects also often run over budget. Roughly half of all large IT projects, defined as those with initial price tags exceeding $15 million, run over budget. On average, large IT projects run 45 percent over budget and seven percent over time, while delivering 56 percent less value than predicted, McKinsey says. 

Beyond IT, virtually all efforts at organizational change arguably also fail. The rule of thumb is that 70 percent of organizational change programs fail, in part or completely. 

So wringing value out of AI will be as challenging as are most enterprise IT efforts and innovation projects.

With possibly one exception, the reasons for AI project failure are familiar. Of the five leading root causes of the failure of AI projects, unclear objectives are at fault, Rand researchers note. 

source: Rand 

Industry stakeholders often misunderstand—or miscommunicate—what problem needs to be solved using AI. Too often, trained AI models are deployed that have been optimized for the wrong metrics or do not fit into the overall business workflow and context, Rand researchers say. 

Simply, sometimes the AI use case does not produce meaningful outcomes because the wrong business problem was chosen as the focus. 

“For example, business leaders may say that they need an ML algorithm that tells them the price to set for a product—but  what they actually need is the price that gives them the greatest profit margin instead of the price that sells the most item,” Rand researchers note. 

In other cases, AI might be applied to problems that do not require its use. “As one interviewee explained, his teams would sometimes be instructed to apply AI techniques to datasets with a handful of dominant characteristics or patterns that could have quickly been captured by a few simple if-then rules,” Rand researchers note. 

In other cases, leaders switch priorities before a particular AI implementation can be put into production. 

But lack of data also is key. Many AI projects fail because the organization lacks the necessary data to adequately train an effective AI model. 

A third failure, related to the first, is that, in some cases, organizations focus more on using the 

latest and greatest technology than on solving real problems for its intended users. 

Also, organizations might not have adequate infrastructure to manage their data and deploy completed AI models. “Data engineering professionals need time to build up pipelines that can automatically clean data and continuously deliver fresh data to deployed AI models,” Rand says. 

Finally, in some cases, AI projects fail because the technology is applied to problems that are too difficult for AI to solve. 

The point is that AI projects--like all IT projects--are prone to fail. That is more a reflection of the human and organizational context than the value of AI itself.