Friday, October 25, 2024

It's Okay to be Skeptical About Claimed AI Outcomes

podcast of this article


A KPMG study suggests technology, telecom and media firms are already seeing, or expecting, return on investment from artificial intelligence spending. In fact, survey respondents reported hefty revenue growth, with 38 percent of respondents suggesting AI already drove more than 10 percent of total entity revenue. 


source: KPMG


In my experience, most enterprise software buyers are quite skeptical of such claims, if intrigued. And few respondents who profess such outcomes are equally able to quantify the outcomes, when asked to do so. 


“We think this outcome can be attributed to a specific input” is one matter. Actual proof is something else altogether. And there is no shortage of reasons for respondents to make such claims. Leaders always are eventually required to justify investments, costs of those investments and attributable financial outcomes.


As the old adage goes, “nobody ever got fired for recommending we buy from IBM.” In other words, bad choices can be job-ending moves. 


As always, the assumptions are key. There is a difference between stakeholder expectations about how AI can contribute, and the actual outcomes. We expect positive outcomes or would not make the investments. But outcomes and productivity are notoriously difficult to quantify for any sort of knowledge or office work.


Neither is it easy to quantify the specific outcomes enabled by any single change a firm makes, when multiple inputs--all dynamic--might be involved. 


For example, the “Redefining TMT with AI” report talks about the benefits of AI-driven predictive network analysis, in tandem with robotic process automation to  enhance network operations and quality

of service. 


In a strict sense, there are two independent variables here: process automation and use of AI. Beyond that, KPMG consultants note that the use case involves automated scripts and algorithms; predictive models; fault prediction; alarm handling; trouble-ticket management;, configuration management; 

 customized network-level reports and workflow management. 


As many of you know, such process automation already is a feature of many operational support systems. AI should help, of course. 


But it might be hard to quantify the degree of impact. Still, the point is that ROI is created by a reduction in volume of alarms, faults and tickets; improved Mean Time to Repair and reduced downtime. I cannot think of a single OSS platform or system that fails to mention those outcomes as benefits. 


In the video content industry, the report suggests AI produces ROI by affecting the efficiency and accuracy of video dubbing (language translation) and synchronizing the dubbed dialogue with the onscreen actors’ vocal movements. The ROI then is produced by reduced production time and cost. 


Also, AI is used to “for understanding and translating complex scripts and  while supporting real-time lip-sync. Basically, in this use case AI aids the dubbing process. 


AI also is used to speed up software coding, so the ROI is based on faster development cycles, faster debugging, code quality and developer productivity. 


The issue is not AI and its ability to improve all those processes and use cases. That indeed is the attraction. Instead, the issue is that it is hard to isolate the AI contributions from the other value created by the processes AI enhances. 


No comments:

Agentic AI Could Change User Interface (Again)

The annual letter penned by Satya Nadella, Microsoft CEO, points out the hoped-for value of artificial intelligence agents which “can take a...