Thursday, October 24, 2024

High AI Capex is Worrisome, But "Winner Take All" is the Prize

It is not hard to find estimates of investment in U.S. artificial intelligence infrastructure (computing capabilities) in the range of $300 billion or more between 2023 and 2030. IDC analysts have suggested $300 billion in investments between 2023 and 2026.


Nor is it hard to find critics who worry about uncontrolled spending without a clear revenue model. On the other hand, leaders of firms attempting to become leaders in the generative AI model business are likely to keep in mind the “winner take all” dynamic we have seen in the recent internet era, where just one or a few firms emerged as leaders in new markets. 


They might point to:

  • Amazon's years of heavy investment to dominate e-commerce

  • Google's massive spending to establish search leadership

  • Cloud providers' huge datacenter investments

  • Meta's acquisition strategy in social media.


In fact, many markets show scant ability to support three providers, as the market leader has twice the share--and up to an order of magnitude more-share compared to  the number-two provider.


Market

Dominant Player

Market Share

Runner-up

Market Share

Search Engines

Google

91.9%

Bing

3.0%

Desktop Browsers

Chrome

65.72%

Safari

18.22%

Mobile Browsers

Chrome

66.17%

Safari

23.28%

Social Media

Facebook

2.9B users

YouTube

2.5B users

E-commerce

Amazon

37.8% (US)

Walmart

6.3% (US)

Video Streaming

YouTube

2.5B users

Netflix

231M subscribers

Music Streaming

Spotify

31%

Apple Music

15%

Ride-hailing (US)

Uber

68%

Lyft

32%

Cloud Services

AWS

32%

Azure

22%

Mobile OS

Android

71.8%

iOS

27.6%


So even if McKinsey estimates AI infrastructure spending will exceed $500 billion between 2023 and 2030, and even if many of those investments do nor produce the expected results, model suppliers have incentives to risk quite a lot, knowing that there is a small  prize for being second best. 


Gartner forecasts global AI infrastructure investments will surpass $250 billion annually by 2030. 


The OECD estimates investments in AI infrastructure across industries, will reach $1 trillion by 2030, across the OECD countries. Bloomberg predicts that the global AI infrastructure market will $700 billion by 2030.


On the other hand, most of that investment will be by end users and others in the value chain, not the generative AI model providers. 


And some estimates made in 2023 might be considered conservative in 2024. Morgan Stanley’s  "The Economics of AI” study, published in October 2023 suggested more than $200 billion in AI infrastructure investments by 2030, including:

  • Data centers: $125B

  • Networking infrastructure: $50B

  • Chip fabrication: $25B

  • Cooling systems: $10B.


Boston Consulting Group in December 2023 suggested there would be $235 billion cumulative investments in 

  • Data center buildout: 45%

  • Compute infrastructure: 35%

  • Power infrastructure: 20%. 


The Goldman Sachs "AI Infrastructure Report," published in September 2023 estimated $275 billion in  cumulative investment, including:

  • Semiconductor investment: $100B

  • Data centers: $115B

  • Power systems: $35B

  • Network upgrades: $25B. 


The caution, though, is that early estimates of the size of new technology markets often lead to overinvestment across the value chain. 


Study/Report

Date

Publisher

Key Conclusions

The Dot-Com Bubble Burst: Causes and Implications

2001

U.S. Securities and Exchange Commission (SEC)

Overinvestment in internet startups led to a speculative bubble that burst in 2000. Many companies were overvalued despite having no profitability.

Boom and Bust: The Telecommunications Investment Bubble

2002

Federal Reserve Bank of San Francisco

Overinvestment in telecom infrastructure during the late 1990s led to a major industry downturn, with unsustainable levels of capital spending.

The Case for Less Innovation

2017

Harvard Business Review

Many companies overinvest in unproven technologies without clear demand, resulting in failed projects and wasted resources.

Lessons from the Clean Tech Bubble

2016

MIT Energy Initiative

Overinvestment in cleantech (2005-2011) led to massive failures, with many companies being too early to market and receiving excessive venture capital.

Investing in Innovation: Creating a Research and Innovation Policy That Works

2010

The NESTA Foundation (UK)

Over-investment in R&D for new technologies can create inefficiencies and fail to produce proportional economic benefits if not managed strategically.

The Nanotechnology Investment Bubble

2005

Journal of Nanoparticle Research

Speculative investments in nanotechnology during the early 2000s led to unmet expectations, as many products were not commercially viable.

Unleashing Productivity: Overinvestment in Information Technology

2005

McKinsey Global Institute

Overinvestment in IT during the late 1990s and early 2000s did not yield expected productivity gains, with firms often adopting technology prematurely.

The Illusions of Overinvestment in AI

2021

Brookings Institution

Many companies overinvest in artificial intelligence without clear applications, leading to inflated expectations and unrealized returns.

The Biotechnology Bubble: When Science and Finance Collide

2004

Nature Biotechnology

Excessive capital flow into biotech during the 1990s led to overvaluation, with many firms failing to achieve meaningful breakthroughs.


In recent years we have also seen examples of overinvestment by many platform suppliers as well. 


Technology

Company/Industry

Year

Description of Over-Investment

Artificial Intelligence

IBM Watson

2011-2022

IBM invested billions in Watson AI for healthcare, but struggled to generate significant revenue and ultimately sold off the health assets

Virtual Reality

Meta (Facebook)

2014-present

Meta has invested over $36 billion in VR/AR technology with limited returns, facing skepticism about the metaverse vision

Blockchain

Various

2017-2018

Many companies rushed to invest in blockchain during the crypto boom, only to scale back or abandon projects when the hype died down

Autonomous Vehicles

Uber

2016-2020

Uber invested heavily in self-driving technology, spending over $1 billion before selling the unit after a fatal accident and regulatory challenges

3D Printing

3D Systems

2013-2015

The company aggressively acquired 3D printing startups, leading to over $1.3 billion in losses and a stock price crash when consumer adoption didn't materialize

Cloud Computing

HP

2011-2012

HP's $11 billion acquisition of Autonomy for cloud services led to an $8.8 billion write-down 


So the rationale for investing heavily to secure the leading position in the generative AI model business is a reflection of the possible “winner take all” character of application and platform markets, where the number-one provider dominates. 


And since market share and profit margin generally are related, the rewards for market leadership also are significant. In many capital-intensive markets, the profit margin of the top provider is double that of number two. 


And provider number two can have margins double that of provider number three.


Wednesday, October 23, 2024

Will AI Have Impact More Like the PC or the Internet? Why it Matters

One reason it is conceptually hard to imagine the impact of artificial intelligence is that it is likely to have business impact along the same lines as did Moore’s Law or the internet: removing key cost barriers and enabling new business models. 


And though some outcomes are easy to envision, such as automating functions or removing geographic barriers, others are hard to grasp because they simply did not exist before. Search and social media are examples. 


In other words, as Moore’s Law led to the elimination of key constraints regarding the cost of computing and software, while the internet created new possibilities for product distribution and sales,, AI might well eliminate key barriers in a value chain.


That will allow lots of industries to evolve in ways that were not possible before, and possibly also create a few new industries that had not existed previously, as the search and social media businesses emerged with completely-new business models (ad supported technology and user-generated content). 


The way to think about it is to ask, in the context of any business, process or industry, what could be different if the key cost constraint, or a major cost constraint, were reduced to a point where it no longer was a constraint or barrier. .


In other words, the question is something like “what would my business look like if a key input were nearly free?” 


Perhaps the best example is Netflix. It is not entirely clear whether Netflix founder Reed Hastings initially and “always” thought the company would evolve into a video streaming service, but it is clear that he did believe a “deliver your DVDs by mail” service was viable in 1997. 


According to Barry McCarthy (Netflix's CFO from 1999 to 2010) and Neil Hunt (Netflix's Chief Product Officer from 1999 to 2017), they were at a 2005 dinner with Reed Hastings where they sketched out projections of bandwidth costs and speeds on a napkin. They plotted Moore's Law-like curves showing:

  • Internet speeds would keep increasing

  • Video compression technology would improve

  • The cost of bandwidth would continue falling


The key insight from their napkin math was that these trends would intersect at a point where streaming video would become economically viable for a mass market service. Netflix launched video streaming in 2007. 


So think of the ways AI might eventually remove key cost constraints in many industries, as the internet eliminated barriers in retailing.


Retailer Cost Constraint

Traditional Retail

Internet Retail

Inventory Costs

High costs associated with maintaining physical inventory, including storage, handling, and obsolescence

Reduced inventory needs due to drop-shipping models and virtual warehouses, leading to lower storage and handling costs

Real Estate Costs

High costs for physical store locations, including rent, utilities, and maintenance

Lower costs associated with online stores, as they require minimal physical space

Distribution Costs

High costs for shipping and transportation of products to physical stores

Lower costs for shipping directly to customers, especially for digital products

Marketing Costs

High costs for traditional advertising methods, such as print, television, and radio

Lower costs for online marketing, including search engine optimization, social media, and email marketing

Customer Service Costs

High costs for in-store customer service, including staffing and training

Lower costs for online customer service, often automated or outsourced


And we can note many similar constraint removals in other industries, including the creation of entirely-new business and revenue models for search and social media. Both search and social media were examples of “advertising-supported technology” models, something that had not been conceivable or possible before. 


But the internet also enabled a rearrangement of business models in most industries, often focused heavily on distribution methods. 


Industry

Traditional Cost Barriers

Internet Solutions

Retail

High overhead costs (rent, utilities), inventory management, distribution

E-commerce platforms, drop-shipping, digital products

Media

Printing costs, distribution logistics, limited reach

Online publishing, streaming services, social media

Software

Physical distribution, licensing costs

Digital distribution, SaaS models, open-source software

Education

Infrastructure costs, geographical limitations

Online courses, MOOCs, virtual classrooms

Finance

Branch network costs, transaction fees

Online banking, mobile payments, cryptocurrency

Travel

Agency fees, booking limitations

Online travel agencies, direct bookings, peer-to-peer platforms

Entertainment

Production costs, distribution channels

Digital content creation, streaming platforms, social media

Manufacturing

Supply chain costs, inventory management

3D printing, on-demand manufacturing, global sourcing

Customer Service

Infrastructure costs, geographical limitations

Online help desks, chatbots, AI-powered support

Professional Services

Geographical limitations, overhead costs

Remote work, online collaboration tools, freelance platforms


Consider the importance of Moore’s Law for the software industry’s “forward pricing” of its products.


Forward pricing is a strategy of setting prices for current products based on anticipated future costs and market conditions, rather than just current costs. 


Microsoft in the 1980s and 1990s, for example, is said to have deliberately released new products that both required more-powerful hardware and also with the expectation that the hardware would catch up. 


In the gaming Industry, products often were designed around advanced hardware that had not yet become mainstream, assuming that would happen and that costs for the platforms would drop. 


Suppliers of enterprise software arguably made the same assumptions, building features that required better hardware and platform upgrades.


On the other hand, initial high prices were expected to fall rapidly, creating the potential for mass market adoption though initially focusing on early adopters. 


The key issue at the moment is that it is very hard to conceive of entirely new ways an existing industry can innovate using AI, to revamp its value chains. It arguably is even harder to envision the emergence of at least a few entirely-new industries that do not presently exist. 


The personal computer and the internet have enabled the emergence of entirely industries or industry segments. For example, the independent software industry was enabled by the PC, along with lots of “PC-specific” industry functions. 


The internet arguably has had more-profound impact, enabling e-commerce, social media, search, cloud computing, digital advertising and streaming media. 


Personal Computer

Internet

PC Manufacturing

E-commerce

Operating Systems

Social Media

PC Software

Cloud Computing

Computer Peripherals

Digital Advertising

PC Gaming

Streaming Media

Desktop Publishing

Online Education

Computer-Aided Design (CAD)

Cybersecurity

PC Repair Services

Web Hosting

PC Retail

Search Engines

PC Magazines/Media

Digital Payment Systems


That should raise questions about the potential AI impact: will it mostly create new industry sub-sectors that support the use of AI itself, as did much of the PC ecosystem, or will it transform whole functions and industries, as arguably was the case for the internet?


Sunday, October 20, 2024

More than 80% of AI Projects Fail, Rand Study Finds

“By some estimates, more than 80 percent of AI projects fail,” says a new study from Rand. “This is twice the already-high rate of failure in corporate information technology (IT) projects that do not involve AI.” 


That might seem shocking, but those of you familiar with the success rates of enterprise IT projects overall will not be surprised, as the general rule of thumb is that up to 70 percent of IT projects actually fail in some way. 


Likewise, studies suggest 74 percent of digital transformation projects fail. Innovation is hard. Some 75 percent of venture-funded startups also fail. 


From 2003 to 2012, only 6.4 percent of federal IT projects with $10 million or more in labor costs were successful, according to a study by Standish, noted by Brookings. 


IT project success rates range between 28 percent and 30 percent, Standish also notes. 


The World Bank has estimated that large-scale information and communication projects (each worth over U.S. $6 million) fail or partially fail at a rate of 71 percent. 


McKinsey says that big IT projects also often run over budget. Roughly half of all large IT projects, defined as those with initial price tags exceeding $15 million, run over budget. On average, large IT projects run 45 percent over budget and seven percent over time, while delivering 56 percent less value than predicted, McKinsey says. 


Beyond IT, virtually all efforts at organizational change arguably also fail. The rule of thumb is that 70 percent of organizational change programs fail, in part or completely. 


So wringing value out of AI will be as challenging as are most enterprise IT efforts and innovation projects.


With possibly one exception, the reasons for AI project failure are familiar. Of the five leading root causes of the failure of AI projects, unclear objectives are at fault, Rand researchers note. 


source: Rand 


Industry stakeholders often misunderstand—or miscommunicate—what problem needs to be solved using AI. Too often, trained AI models are deployed that have been optimized for the wrong metrics or do not fit into the overall business workflow and context, Rand researchers say. 


Simply, sometimes the AI use case does not produce meaningful outcomes because the wrong business problem was chosen as the focus. 


“For example, business leaders may say that they need an ML algorithm that tells them the price to set for a product—but  what they actually need is the price that gives them the greatest profit margin instead of the price that sells the most item,” Rand researchers note. 


In other cases, AI might be applied to problems that do not require its use. “As one interviewee explained, his teams would sometimes be instructed to apply AI techniques to datasets with a handful of dominant characteristics or patterns that could have quickly been captured by a few simple if-then rules,” Rand researchers note. 


In other cases, leaders switch priorities before a particular AI implementation can be put into production. 


But lack of data also is key. Many AI projects fail because the organization lacks the necessary data to adequately train an effective AI model. 


A third failure, related to the first, is that, in some cases, organizations focus more on using the 

latest and greatest technology than on solving real problems for its intended users. 


Also, organizations might not have adequate infrastructure to manage their data and deploy completed AI models. “Data engineering professionals need time to build up pipelines that can automatically clean data and continuously deliver fresh data to deployed AI models,” Rand says. 


Finally, in some cases, AI projects fail because the technology is applied to problems that are too difficult for AI to solve. 


The point is that AI projects--like all IT projects--are prone to fail. That is more a reflection of the human and organizational context than the value of AI itself.


Directv-Dish Merger Fails

Directv’’s termination of its deal to merge with EchoStar, apparently because EchoStar bondholders did not approve, means EchoStar continue...