Concerns about AI Infra Overinvestment are Rational: It has Happened Many Times in the Past

It is quite understandable that financial analysts covering public firms are concerned about the payback period for various forms of artificial intelligence. For example, venture capitalist David Cahn with Sequoia Capital argues that the big hyperscale cloud computing companies must earn about $600 billion in revenue to justify their investments in AI infrastructure, focused only on graphics processor investments and data center facilities and operating costs, plus an expected 50-percent profit margin on software sales. 

That noted, Cahn also says “a huge amount of economic value is going to be created by AI. Company builders focused on delivering value to end users will be rewarded handsomely,” as AI is a potentially “generation-defining technology wave.”

The larger point is that speculative frenzies are part of technology deployment. “Those who remain level-headed through this moment have the chance to build extremely important companies,” says Cahn. “But we need to make sure not to believe in the delusion that has now spread from Silicon Valley to the rest of the country, and indeed the world.”

In other words, the “get rich quick” mentality is going to disappoint, as did the mid-1880s gold rush in California. 

So will there be an AI investment bubble? Yes, he might argue. Such periods of investment frenzy have happened in the past, as well, before the benefits were realized. 

Engines That Move Markets: Technology Investing from Railroads to the Internet and Beyond by Alasdair Nairn describes the recurring investment patterns associated with major technological advancements. He notes that these innovations often follow a cycle, moving from skepticism to enthusiasm. Lots of venture capital investment follows, accompanied by inflated stock prices. 

Eventually, as the technology matures and financial realities set in, many companies fail, stock prices collapse, and naive investors lose money.

If the railroad investment pattern holds, there could be disappointment. Over the long term, investments in railways were not rewarding, he argues. Despite their economic impact, railways provided negative investment returns in real, relative, or absolute terms, however important the economic contribution. 

The point is that it is a safe bet to argue AI overinvestment will occur. That tends to be the pattern for major new technologies, especially those we generally recognize as being general-purpose technologies with wide economic impact. 

After all, huge capital investments in graphics processor units, for example, must be reflected in revenue upside at some point. The issue is whether expectations of near-term return are actually reasonable. 

As always, market forecasters, firm executives and others might lean towards the strategic implications, while financial analysts primarily look at the quarterly performance metrics. 

And, sometimes, investments are more “strategic” than “tactical.” In other words, a telco might have to invest heavily in fiber-to-home facilities simply to stay in business as competitors upgrade their home broadband infrastructures. 

The actual financial return on investment will matter, but might not be the driver. “You get to keep your business” or “you get to stay in business” might be the value, not simply increases in revenue after the investments are made. 

Most new information technologies take some time before we tend to see measurable benefits. That has been true for many technologies. So the issue is whether various forms of AI are more like social media or smartphones or PCs, the internet and automated teller machines. 

Technology	Approximate Lag Time to Measurable Outcomes (Years)
PCs	10-15
Internet	10-15
ATMs	8-10
HDTV	5-8
Cloud Computing	5-7
Wireless/Wi-Fi	5-7
Smartphones	3-5
Social Media	3-5

Applying various forms of AI to various use cases across industries might reasonably produce varied payback periods, from rapid to lengthy, suggesting that investment tied to particular use cases is a reasonable approach.

Most of us likely can imagine clear performance benefits in areas ranging from e-commerce, search and social media recommendations fairly quickly. As AI already is used to support such personalization features. 

Other use cases, including manufacturing or healthcare, might take longer, in part because many parts of the value chain have to be altered at the same time to take advantage of AI. 

Industry/Use Case	Estimated Payback Period
E-commerce (general)	1.2 - 1.6 years
E-commerce (search optimization)	6 months - 1 year
Social Media & Content	1 - 2 years
Manufacturing	2 - 3 years
Business Services (Law, Accounting, Consulting)	1.5 - 2.5 years
Smartphones (AI features)	1 - 2 years
PCs (AI-enhanced software)	2 - 3 years
Cloud Computing (AI services)	1 - 2 years
Healthcare (AI diagnostics)	2 - 4 years
Financial Services (Fraud Detection)	1 - 1.5 years

Obviously there are many variables. Larger-scale implementations may see faster payback due to economies of scale, so long as they are targeting major functions that can affect financial return. 

Some AI applications, such as fraud detection in financial services, may see quicker returns compared to more complex implementations in healthcare or manufacturing, and also be easier to measure. 

Existing information technology infrastructure and past success integrating information technologies, probably also will matter. Companies that have more-developed IT might see faster payback periods compared to firms whose existing infra is less well developed. 

Fast-moving industries such as  e-commerce and social media might realize benefits quicker than more traditional sectors, simply because they face fewer regulatory issues that must first be addressed. 

Regulatory environment: Industries with strict regulations (e.g., healthcare, finance) may have longer payback periods due to compliance requirements.

As always, the particular use cases will have different payback periods, when implemented at scale. 

Industry	AI Use Case	Payback	Source
E-commerce	Product Recommendation Engines	6-12 months	McKinsey, AI in Retail
Social Media & Content	Personalized Content & Ad Targeting	12-18 months	Forrester, AI in Marketing
Manufacturing	Predictive Maintenance	18-24 months	PwC, AI in Manufacturing
Business Services	Legal Document Review & Due Diligence	24-36 months	Accenture, AI in Professional Services
Smartphones	Voice Assistants & Virtual Companions	3-5 years (Long-term brand value)	CB Insights, AI in Mobile
PCs & Cloud Computing	Resource Optimization & Server Management	12-18 months	Bain, AI in Cloud Computing

Are Some Claimed Generative AI "Savings" for Customer Service Operations Fictitious? If so, When Does That Change, and Why?

There might be upsides and downsides as generative artificial intelligence systems--after crawling the whole internet--likely start to learn from each other. To be sure, some new data stores conceivably can be crawled, but that process will increasingly be expensive and involve much smaller, more-specialized sets of data, such as some proprietary enterprise content. 

But all that will be incremental. What is likely to happen is that models start to learn from each other, using “synthetic data” that is artificially generated mimicking real-world data in its statistical properties and structure, but without actual real-world data points. 

That could have both good and bad implications. Perhaps synthetic data can help compensate for scenarios where training data is under-represented or unavailable. That can help improve model performance and robustness.

Since synthetic data doesn't contain real individuals' information, it can be used to train language models on sensitive topics without risking privacy violations.

Carefully generated synthetic data can be used to balance datasets and reduce biases present in real-world data, potentially leading to fairer language models. 

In domains where real data is scarce or expensive to obtain, synthetic data might provide a viable alternative for training language models. Cost effectiveness is a possible advantage as well. 

Also, models could be pre-trained on large synthetic datasets before fine-tuning on smaller real-world datasets, potentially improving performance in data-limited domains. Likewise, synthetic data could be generated to support training for languages with limited real-world data available.

On the other hand, there are potential downsides. When AI systems learn from each other, there's a risk of amplifying existing biases present in the original training data. As models build upon each other's outputs, subtle biases can become more pronounced over time.

With AI systems learning from each other, there's a danger of converging on similar outputs and losing diversity of perspectives.

Of course, it might not always be the case that synthetic data accurately represents real-world scenarios. The same danger exists in terms of models learning incorrect information from other models.

Overall, the major danger of using synthetic data is that it could spread of misinformation or biased outputs. Traceability issues will exist as well. 

-------------------

The timing and degree of financial impact from applying generative artificial intelligence is a reasonable concern, as any similar major new investments in information technology would also raise. At a high level, the concern might be expressed as the return from using high-cost tools to automate low-cost processes, rather than using low-cost tools to automate high-cost processes.

Much hinges on whether generative AI, for example, remains a “high cost” tool and how much value it brings to business processes. 

Many could argue that the internet disrupted so many business models, revenue streams and industries because it was a low cost way to solve many higher-cost problems. We can point to IP communications as having disrupted high-cost long-distance voice calls and some amount of physical travel by substituting email, instant messaging and video conferencing.

   

Information dissemination costs also were reduced significantly, as physical media was replaced with online digital media. As the latency and cost of sending documents was reduced by the advent of the facsimile machine, so the internet reduced information float costs and time. 

The ability to reach potential customers--especially for niche products--also was radically altered, as the internet made global markets possible, where before local markets were hard to expand. In the past, retail outlet costs, relatively-expensive marketing and advertising costs existed for any firm selling a niche product. The internet made global markets almost as affordable to reach as local markets. 

Supply chain management; real estate, computing costs, analytic functions, social media and targeting plus product development all became easier.

 

The issue now is whether generative AI can become a low-cost tool to solve higher-cost problems. If not, it might remain a high-cost tool only useful for solving a smaller set of high-cost problems. Consider GenAI as the platform for smart chatbots supporting customer service operations. 

To be sure, most observers likely agree that the cost of GenAI will decline over time, as virtually all digital products have done. In addition to those “Moore’s Law” effects, open source should play a role as well. Meta’s open source Llama 3.1 405B model provides an example.  

Where it comes to using GenAI for supporting customer service chatbots, one immediately faces the challenge of determining whether that function is “high cost” or relatively “low cost,” and whether applying GenAI is a high cost solution or not, considering the degree of improvement; workforce implications and cost to implement. 

It might be the case that the use of chatbots is increasing. One study suggests monthly chats per agent increased by 43 percent on average in 2022. This resulted in a team of 26 or more people handling around 138 percent more inquiries than they did in the previous years.

Still, it is hard to determine what that means, where it comes to assessing GenAI costs and benefits. 

To what extent can the chatbots handle inquiries and resolve issues without human intervention? How much of the load is displaced? How complicated are the problems the chatbots can handle, and which problems still require human agent intervention? What percentage of inbound queries are routine enough to be handled properly by the chatbots?

Gartner, for example, predicts 85 percent of such engagements in 2025 will be handled by AI chatbots alone.McKinsey estimates that generative AI could increase productivity at a value ranging from 30 to 45 percent of current function costs. 

An NBER research paper also estimates productivity gains of up to 35 percent for support agents, with average increases of 14 percent. 

So Gartner predicts potential cuts of 20 percent to 30 percent of customer service agents by 2026, while analysts at Barclays see up to a 50 percent reduction in the contact center workforce.IBM has estimated that up to 30 percent of agent costs can be reduced using chatbots.

Assume the 30-percent cost reduction estimate is correct. Assume all other aspects (quality of interactions; customer satisfaction; time to resolve complaints) remain the same. Then the cost-benefit evaluation comes down to the cost of implementing the AI chatbot compared to the agent cost savings. 

And that is where the assumptions loom large. Some believe use of AI chatbots will reduce the number of agents required. So savings or benefits come in the form of reduced headcount. 

Consider a low-end total of about $44,321 per year, based on  $30,688 (salary) + $4,633 (benefits) + $7,000 (office space) + $2,000 (infrastructure). Consider a high-end total cost of $73,633 per year, based on  $57,000 (salary) + $4,633 (benefits) + $7,000 (office space) + $5,000 (infrastructure). 

Reduce costs if all agents work remotely and no office space is required. That still implies a cost between $37,321 and $66,633. 

If chatbots mean the number of CSRs can be reduced 10 percent, that might imply 10 agents at a firm with 100 agents. So payroll savings might range from $373,210 to $666,330. In that case the use of GenAI breaks even at about that recurring rate per year, ignoring development costs. 

Others might see savings in hiring and training as well. 

Assume the average cost to hire a new employee is $4,700 and the average cost of training an employee per year is $1700 per year. Assume a turnover rate of 20 percent for agents. 

A firm requiring use of 100 agents will lose 20 employees per year and will need to hire 20 new employees to replace them. So the cost of hiring and training these new employees would be $4,700 * 20 + $1,700 * 20, or about $128,000. 

Some will argue that hiring cost actually is not that high, as most of the costs might represent existing employee time spent on the hiring and intake processes, which are not actually new hard-dollar costs but represent pro-rated “time spent” by those workers. 

In that sense, the “cost to hire” might essentially be fictitious. And some amount of agent turnover still will happen, with or without use of GenAI. So reduce the “benefit” of using GenAI by half, to $64,000. 

That implies the use of GenAI, per year, simply related to reduced turnover, might lead to a lower “breakeven” proposition so long as it does not cost more than $64,000 a year for hiring and training. 

The point is that using GenAI is a breakeven proposition--using these simple assumptions--between $437,210 and $730,330 annually. 

Others might argue that the avoided costs of adding additional agents is the real benefit, so the savings come from avoiding hiring another 10 agents. The breakeven in that case comes from avoiding additional personnel expense, not headcount reductions. 

For most firms, GenAI to support CSR operations might come from use of a cloud computing “as a service” model, as the costs to develop an in-house “large” model might never make sense. Use of an existing model, available as a service, with some customization upfront is the more-likely deployment. 

Aspect	Small Language Models (SLMs)	Large Language Models (LLMs)
Model Size	500M - 20B parameters	100B+ parameters
Training Cost	$10,000 - $100,000	$1M - $10M+
Inference Cost (per 1K tokens)	$0.0001 - $0.001	$0.01 - $0.06
Fine-tuning Need	Always required	Often optional
Fine-tuning Time	Days to weeks	Weeks to months
Example Model	Mistral 7B	GPT-4

The questions about GenAI payback are reasonable enough. But it might not be possible to assess payback in various apps, use cases and deployments for some time.