AI Capex Might Not Grow as Much as Some Expect Over the Next Few Years

Is it possible some estimates of capital investment in artificial intelligence are inflated? Yes. According to Stanford University’s Human Centered AI institute, about $67 million was invested in AI in 2023. 

“To project such spending in the near term, we grossed up last year’s investments in AI by various annualized rates of growth ranging from 13 percent to 34 percent,” says Joe Davis, Vanguard global chief economist. “Those rates of growth would leave AI spending this year and next in the $76 billion to $121 billion range.”

source: Vanguard 

That’s significant, but nowhere near the “$1 trillion” some have estimated will be spent on AI capex over the next few years. 

source: Goldman Sachs

Or consider estimates of AI capex spending by a few of the big hyperscalers. To be sure, those firms are only some of the firms expected to make AI capex investments. 

Company	2024 US$ Billions		2025		2026
	Low Estimate	High Estimate	Low Estimate	High Estimate	Low Estimate	High Estimate
Alphabet	40	45	45	50	50	55
Microsoft	50	55	55	60	60	65
Amazon	35	40	40	45	45	50
Apple	20	25	25	30	30	35
Meta	35	40	40	45	45	50
Total	180	205	205	230	230	255

Also, a significant portion of the AI capex is likely to be shifted from existing information technology budgets, and might not represent incrementally-higher spending. That is virtually certain to be the case for most enterprises making AI capex investments, as most enterprise or smaller business IT budgets do not change all that much annually, with increases, if any, normally in single digits. 

Providers of “AI as a service” will have to make unusual investments, to be sure. But business users of AI will not do so. 

So the big change in AI capex is likely to be driven by a relatively few hyperscalers.

Estimating Edge Computing Capex is Tricky

It might be as difficult to forecast edge computing capital investment as it is to forecast edge computing demand. For starters, there are many different definitions of potential “edge” computing. Edge computing can happen directly on user devices; on a premises using a local data center; someplace three to five miles distant (base of cell tower); within a metro area or hundreds of miles distant. 

So a judgment has to be made about how to evaluate the edge computing capex “on device.” Is it the full cost of the appliance or the incremental cost of the AI capability? Do we include software? And is “capex” cost the relevant metric, or do we include the on-going costs of subscriptions, which might better capture the full cost of using AI capabilities? 

To be sure, if all we are measuring is capital investment, then software subscriptions are not relevant. But that full cost does matter if we are trying to compare the cost of creating and using remote and edge computing. 

The U.S. cloud computing services market was valued at approximately $216.91 billion in 2023 by Grand View Research, with projections indicating growth at a compound annual growth rate (CAGR) of 20 percent from 2024 to 2030. Meanwhile, the revenue from smartphone sales in the United States is projected to be around $109.8 billion in 2024.

If edge computing revenue grows, that should also imply that capex grows. The other issue is that we normally only measure “capex” on the producer side of a market. Consumer expenditures on computing gear, including smartphones, are not counted as “capex.”

source: Statista

AI Seems to be the Biggest Driver of Cloud Computing Spending Growth

A survey commissioned by Wipro of 500 business executives in Europe and the United States suggests continued growth of cloud computing spending, plus growing artificial intelligence spending. And the suirvey suggests AI is the single biggest growth driver.

Perhaps nobody would be surprised by that conclusion. Nor would many, if any, be surprised that cloud computing spending exceeds AI spending.

But the study does not specifically address the magnitude of the increases, so one cannot tell, from this survey, how much spending might be growing, if at all.

A majority of respondents (55 percent) report that their cloud adoption is outpacing their AI adoption, while 35 percent say they are moving at the same pace for both technologies. 

But 10 percent of respondents report that their AI adoption is outpacing cloud spending.

The “health and life sciences” sector has the highest percentage (41 percent) of organizations moving at the same pace for cloud and AI adoption. 

Retail has the highest percentage (63 percent) of industry response suggesting cloud spending ahead of AI. The energy and utilities sector has the highest percentage (18 percent) of organizations with AI adoption ahead of cloud.

The top reported drivers of cloud investment are AI/GenAI applications (54 percent), extended cloud infrastructure (47 percent), and increasing data demands (43 percent).

Separately, the Civo Cost of Cloud 2024 report suggests user organizations are unhappy with spiraling cloud computing costs.  According to the survey, 77 percent of the 500 industry professionals surveyed are using one of the Big 3 hyperscalers and many (37 percent, at least) believe cloud computing cost effectiveness has not been seen. 

source: Civo

How Might Generative AI Affect Software Profit Margins?

Are there  profit margin analogies for generative artificial intelligence as it is applied by software firms?  In other words, will generative AI profit margins prove to be equivalent to, or lower than, or higher than, profit margins in other industries that have faced new technology eras?

Generally speaking, media and connectivity firm profit margins have declined in the transition from pre-1990 to post-1990 periods when digital technology replaced analog. The exception, at least so far, is mobile service, which by most measures is more profitable today than in the analog era. 

As always, there are multiple possible driving forces, ranging from competition to deregulation to the impact of internet product substitution. 

 

Industry	Analog Era (Pre-1990s)	Digital Era (Post-1990s)
Newspapers	High (e.g., 25-30%)	Lower (e.g., 10-15%)
Magazines	High (e.g., 25-30%)	Lower (e.g., 10-15%)
Television (Broadcast)	High (e.g., 30-40%)	Lower (e.g., 15-25%)
Radio	High (e.g., 25-30%)	Lower (e.g., 15-25%)
Telecom (Landline)	High (e.g., 30-40%)	Lower (e.g., 15-25%)
Mobile Services	Emerging (e.g., 10-20%)	High (e.g., 25-35%)

Some might argue that productivity gains--and corresponding impact on profit margins--was at one level for software firms in the mainframe and then personal computing era; perhaps a different level in the cloud computing (software as a service) era and might change again in the AI era of software. 

Others are not so sure, but most might agree that hardware margins, which historically have trended downward over time, might continue to see margin erosion over time, particularly as large end users build their own chips for processing acceleration. 

source: MostlyMetrics, The Information, Meritech Capital 

But it remains unclear how AI will affect profit margins for software firms, but some would note that margins for both hardware and software have remained relatively unchanged for nearly 40 years. 

Right now, the issue might be we have no experience curve for AI operations and value. 

Era	Hardware Firms	Software Firms	AI/Cloud Firms
Mainframe (1960s-1980s)	40-50%	70-80%	N/A
PC (1980s-2000s)	20-30%	80-85%	N/A
Cloud Computing (2000s-2020s)	20-30%	70-80%	70-80% (SaaS)
AI Era (2020s-Present)	20-30%	70-80%	50-60%

Some will argue profit margins for generative AI service providers will fall over time, based largely on large capital investments and scarce evidence yet of robust revenue models. That might be the case, though. 

Software margins have been relatively consistent despite the move from mainframe to PC platforms; onboard to cloud processing. We cannot yet say how margins might change with AI, over time. 

The issue is whether AI products retain software margin characteristics or eventually resemble margin trends for content, connectivity and computing hardware. 

----------------

Amazon Says it Can Quantify Some Generative AI Outcomes; Target Not so Much

According to Andy Jassy, Amazon CEO, Amazon Q, Amazon’s generative artificial intelligence assistant for software development, has had a clear impact on Java upgrades. 

‘The average time to upgrade an application to Java 17 plummeted from what’s typically 50 developer-days to just a few hours,” said Jassy. “We estimate this has saved us the equivalent of 4,500 developer-years of work.”

“And, our developers shipped 79% of the auto-generated code reviews without any additional changes,” he noted. 

Beyond that, “the upgrades have enhanced security and reduced infrastructure costs, providing an estimated $260 million in annualized efficiency gains,” Jassy said. 

That is a good example of the expectation that GenAI will in many initial cases--perhaps most cases--be used to support ongoing business practices. 

While arguably helpful, are hard to quantify in ways other than “productivity” improvements, such as doing things faster. Amazon just happens to be able to apply GenAI in a way that is quantifiable, in this instance. 

Most firms will not have such outcome-oriented results. 

“Earlier this year, we integrated GenAI into the handheld devices in our stores, providing our team with rapid access to best practice documentation and the ability to quickly receive straightforward responses to common questions like, how do I sign a guest up for a Target Circle card, and how do I restart the cash register in the event of a power outage,” said Michael Fiddelke, Target CFO and COO. “Since the full chain rollout of this new tool, our team members have leveraged the technology more than 50,000 times, giving answers in a highly efficient average chat time of less than one minute.”

We can agree that is a productivity enhancement, but likely also agree that it is virtually impossible to correlate with financial results or operational outcomes.