AI ROI Metrics are Coming, Even if They are Essentially "Soft" Measures of Impact

We might as well be honest and predict that enterprises are going to develop all sorts of metrics that purportedly show the positive impact of their artificial intelligence investments, but that the metrics will quite probably be proxies that measure all sorts of things other than direct AI impact.

Still, some common metrics are a starting point:

• Cost per unit of output — Does AI reduce the labor or compute cost to produce a document, resolve a ticket, process a claim, underwrite a loan?

• Throughput / cycle time — How many units processed per hour, or how much time shaved off a workflow (e.g., code review, contract drafting, customer onboarding)?

• Error rates and rework costs — Does AI reduce defect rates, compliance exceptions, or manual correction loops?

• Headcount avoidance — the ability to scale output without proportional headcount growth. Often measured as "FTE equivalents automated."

Revenue-side metrics are less common, but might include:

• Conversion lift — Does AI-personalized outreach or recommendation improve sales conversion rates?

• Revenue per sales rep — If AI handles pipeline qualification or proposal drafting, does rep productivity improve?

• Customer retention / churn reduction — Does AI-assisted support or proactive intervention improve net revenue retention?

• Time-to-market — Does AI-accelerated R&D or software development compress product cycles in ways that generate earlier revenue?

Other operational outcomes also sometimes are quantified:

• Accuracy or precision rates on specific tasks (e.g., document classification, anomaly detection in fraud)

• Audit findings or compliance exceptions reduced

• Model risk KPIs — false positive/negative rates in detection systems

• Employee time recaptured — hours per week freed from low-value tasks, redirected to higher-value work

• Employee satisfaction / retention — particularly in roles prone to burnout from repetitive work

• Decision quality — harder to measure, but some firms track downstream outcomes of AI-assisted decisions against historical baselines

As rational as all that sounds, the metrics are “soft.” The attribution problem is severe, as AI is almost never the sole variable changing in a deployment. 

AI might be deployed while other changes also are occurring:

• Process redesign — Most AI deployments force workflow reengineering. Efficiency gains may be 60% process change and 40% AI

• Training and change management — The same tool deployed with weak adoption programs vs. strong ones produces dramatically different outcomes

• Data quality improvements — Organizations often clean and structure data as a precondition to AI deployment; that alone drives gains

• Personnel changes — New hires, role restructuring, or management changes co-occur with AI rollouts

• Macroeconomic or market tailwinds — Revenue gains during an AI deployment may reflect market growth, not AI impact.

• Hawthorne effects — Measuring a team's performance changes behavior regardless of the tool.

The point is that it can be almost impossible to isolate the impact of AI cleanly. So most enterprise AI ROI figures are really "ROI of the initiative that included AI," not AI's marginal contribution.

The more interesting question might be "which specific processes have changed in ways we can measure, and do we understand why?"

Skeptics are correct to argue that attributing success purely to AI is often an oversimplification. But enterprises will have to try and do so, as investors will demand such “proof.”

So firms will supply such “proof” as best they can, even if the outcomes are not, strictly speaking, solely because of AI use. 

And that is not an unusual case. 

Research highlights that AI’s impact is heavily moderated by "complementary assets.” In other words, a firm’s  organizational structure, existing data quality and worker skill levels often do more to determine the outcome than the AI model itself.

Study Focus	Key Finding Regarding Attribution	Source
Productivity Paradox	AI adoption does not guarantee boosts; results are contingent on organizational structure and worker attributes.	Cho et al. (2026)
Social Penalty/Bias	Using AI for assistance causes observers to attribute success to the tool rather than the person, leading to negative competence assessments.	Reif (2025)
Supply Chain/Bias	In complex systems, responsibility is fragmented across vendors/platforms, making it nearly impossible to attribute specific outcomes to one source.	Sharma et al. (2026)
Task-based Impact	AI improves performance within its "capability frontier" but degrades it outside that range; attributing net gains requires granular task-level data.	Brynjolfsson et al. (2023)

The difficulty in quantifying the immediate return on investment for new technologies is a recurring theme in economic history.

During the 1970s and 1980s, despite massive corporate investment in information and communications technology, overall productivity growth in many industrialized nations remained stagnant. This led economists to question whether computers were truly providing the expected value.

Eventually, results were observed, but:

• Results lagged deployment: it took decades for firms to fully "reimagine" their organizational structures, business models, and workflows to leverage the new technology effective

• Value was indirect: better management, more efficient coordination or improved service quality, but correlation, not causation, remains a question. 

The measurable financial benefits of a transformative technology often became clear only after business processes were redesigned.

Technology	Scope of Impact	Key Findings	Source
ICT / General IT	U.S. Economy (1995–2000)	ICT accounted for 56% of labor productivity growth; added 1.18 percentage points to GDP growth.	Oliner & Sichel (2000)
Emerging Tech (AI/ML)	U.S. Public Firms (2009–2019)	Over a three-year period, “neither the mean nor the median abnormal ROE (expected performance) reaches statistical significance in the post-implementation period.” “The mean abnormal inventory turnover is −1.06, which is not significantly different from zero.” “Overall, our results…indicate no significant difference in performance between sample and control firms during the implementation period of emerging digital technologies.”	Li et al. (2024)
Internet / ICT	SME Growth (Global)	Web-savvy SMEs grew more than twice as fast as those with minimal web presence.	McKinsey (2011)

Still, in the meantime, we will see all sorts of metrics “demonstrating” AI impact. Enterprises making the investments have no choice but to try to do so, even if those metrics are “soft.”

Regulation and Deregulation Both Make Sense, at Different Times in an Industry's Lifecycle

In 1948, the Supreme Court ruled that five studios had monopolized the American film industry. Paramount, Warner Bros., MGM, RKO, and Fox owned the theaters that showed their own movies.

The court ordered them to sell.

For the next 72 years, the Paramount Consent Decrees kept the studios apart.

In August 2020, a federal judge terminated the decrees. The reasoning was that the market had changed beyond recognition.

Streaming had replaced theaters as the primary distribution channel. The studios were no longer dangerous monopolists. They were struggling incumbents.

Six years later, Paramount and Warner Bros. are merging. The deal is worth $111 billion including debt. The Justice Department approved it on June 12, 2026.

Two of the five studios that the Supreme Court forced apart are coming back together voluntarily. Not because they are too powerful, but because they are too weak to survive alone.

It’s a familiar story. Regulation is often designed to solve a specific market structure problem (monopoly power, natural monopoly characteristics, or high barriers to entry). 

Over time, technology, globalization, new business models, and substitute products can eliminate the original source of market power. Regulations that once made sense may then become unnecessary, counterproductive, or even protective of incumbents.

Industry	Original Monopoly Concern	Regulatory Response	What Changed?	Why Regulation Became Less Necessary
Railroads (1880s)	Railroads often held local transportation monopolies	Interstate Commerce Act of 1887 and creation of the ICC	Trucks, highways, pipelines, barges, airlines emerged	Railroads lost their transportation monopoly and faced extensive intermodal competition. The ICC was ultimately abolished in 1996. (PBS)
Airlines (1938–1978)	Fear that airlines would become monopolies and require centralized route and fare control	Civil Aeronautics Board regulated routes, prices, and entry	Industry matured; economists found regulation often restricted competition rather than promoting it	Congress passed the Airline Deregulation Act of 1978, eliminating most economic regulation. (Congress.gov)
Long-distance telephone service	AT&T dominance in national telephony	Rate regulation, entry restrictions, antitrust oversight	Fiber optics, microwave transmission, wireless networks, internet communications	Long-distance became highly competitive and prices collapsed. (Investopedia)
Telephone equipment	AT&T controlled devices connected to the network	FCC restrictions and later interoperability rules	Standardized interfaces and competitive equipment markets	Consumers now freely purchase phones and network devices from many suppliers. (WIRED)
Telegraph	Western Union's dominance	State and federal oversight of messaging services	Telephone, fax, email, messaging apps	Telegraph market essentially disappeared; monopoly concerns vanished with the technology itself.
Trucking (mid-20th century)	Concern about destructive competition and market concentration	ICC regulation of routes and pricing	Improved logistics, highways, nationwide competition	Most economic regulation was removed in the late 1970s and early 1980s. (LegalClarity)
Natural gas transportation	Pipeline monopolies in some regions	Extensive price and transportation regulation	Competitive gas production, spot markets, interstate trading hubs	Many pricing controls were relaxed as markets became more competitive.
Stock trading commissions	Dominant exchanges could maintain fixed commissions	SEC oversight and fixed-rate structures	Electronic trading and competing exchanges	Fixed commissions were abolished in 1975 ("May Day"), leading to intense competition.
Broadcast television	Scarce spectrum created limited competition	FCC ownership and content regulations	Cable TV, satellite TV, streaming services, internet video	The original scarcity rationale weakened substantially.
Local newspapers	Dominant local print monopolies	Special antitrust accommodations and ownership rules	Internet advertising, social media, digital news	Many newspaper monopolies disappeared due to competition from digital substitutes.

In the case of the studios, massive changes in the video and movie business make older restrictions unnecessary. 

Television was an alternative to “going to the movies, and therefore a threat. But studios discovered:

• TV licensing created new revenue

• Old film libraries became valuable assets

• Syndication emerged as a lucrative business. 

The additional changes in distribution (cable TV, home video, streaming) likewise emphasized the role of content ownership and creation for studios, even as new distributors emerged to capture value. 

Era	Largest Value Capture
Theater	Studios + theaters
Broadcast TV	Networks
Cable TV	Cable operators
DVD	Studios
Streaming	Platforms

Among the new issues with streaming is the importance of distribution versus “discovery,” as “scarcity value” migrates. 

Era	Scarce Resource
Theaters	Screens
Broadcast TV	Spectrum
Cable TV	Channel capacity
DVD	Shelf space
Streaming	Consumer attention

Frequently, the substitute products and competitors come from “outside” an industry’s chosen domain. 

Perhaps the classic example is railroads believing they were in the trains business, when they were actually in the transportation business. The substitutes did not come from inside the “railroad” business but from outside. 

Product	Apparent Monopoly	Important Substitute
Railroads	Railroads	Trucks, barges, airlines
Long-distance calls	AT&T	Mobile, VoIP, messaging apps
Broadcast TV	Local stations	Cable, satellite, streaming
Newspapers	Local newspaper	Internet and social media
Taxi medallions	Local taxis	Ride-sharing platforms
Video rental stores	Blockbuster	Streaming services

Each major distribution innovation created new winners, weakened existing gatekeepers, and shifted where revenue accumulated:

• broadcast television

• cable television

• home video

• DVD

• streaming. 

Era	Dominant Distribution	Key Gatekeeper	Main Revenue Source
1920s–1950s	Movie theaters	Theater chains	Ticket sales
1950s–1980s	Broadcast TV	TV networks	Advertising
1980s–2000s	Cable TV	Cable operators	Subscription fees + advertising
1980s–2010s	VHS/DVD	Retailers & studios	Unit sales/rentals
2010s–present	Streaming	Streaming platforms	Subscriptions
Emerging	AI-assisted distribution	Platforms & recommendation engines	Subscription + advertising + commerce

The point is that “where” monopoly danger exists will shift with time. And so must the regulatory concern.  Emerging industries might need one pattern. Declining industries virtually always need another: preventing concentration early; encouraging it in the industry decline phase.