Some Generative AI Disillusionment is Inevitable

Though chief opera;ting officers surveyed by Pymts believe the single greatest outcome from generative artificial intelligence is cost reduction--cited by 92 percent of survey respondents--they also believe there will be increased profits as well.

At this point, it is worth noting that those are expectations and beliefs, not statements about achieved outcomes. History suggests it is inevitable that some amount of disillusionment with generative AI will hit relatively soon. 

source: Pymts  

What the survey does not indicate is the magnitude of cost savings, profit margin or profit benefits. Also keep in mind that most respondents believed the greatest impact would come in areas including planning; risk management and customer satisfaction, not production management, for example. 

source: Pymts 

Such fuzziness is to be expected, this early into generative AI deployment. And we are likely to see both some amount of overinvestment and failed projects, as would be typical for any information technology project. 

Source	Key Points
Andrew Binns, Forbes	- Warns against overinvesting in generative AI too quickly
	- Emphasizes the importance of aligning learning pace with investment
	- Cites historical examples like GE's "big data" misstep
	- Recommends targeted business experiments before full-scale implementation
Goldman Sachs	- Executives expect enormous impact from generative AI
	- Most say they are unprepared for the technology
McKinsey	- 40% of respondents say their organizations will increase AI investment due to generative AI
	- Less than half of respondents say their organizations are mitigating even the most relevant risks
	- Suggests rapid adoption without adequate risk assessment

Some research suggests company big data investments often failed to generate observable and quantifiable outcomes. “In fact, six in 10 executives surveyed by Deloitte say it’s difficult to quantify the benefits of individual tech investments,” say Deloitte consultants. 

It would not be imprudent to suggest that much generative AI investment will fail to show measurable positive outcomes. As a rule of thumb, up to 70 percent of IT projects will fail. Studies have suggested that 74 percent of digital transformation projects have failed. 

In fact, most big information technology projects fail. BCG research suggests that 70 percent of digital transformations fall short of their objectives. 

From 2003 to 2012, only 6.4 percent of federal IT  projects with $10 million or more in labor costs were successful, according to a study by Standish, noted by Brookings.

source: BCG 

IT project success rates range between 28 percent and 30 percent, Standish also notes. The World Bank has estimated that large-scale information and communication projects (each worth over U.S. $6 million) fail or partially fail at a rate of 71 percent. 

So, expectations aside, most firm executives will probably be disappointed with their generative AI investments.