Here comes Large Language Model PC and Web Browser Control

Large Language Models are starting to shift capabilities from content creation to control of PC or web browser functions. 

Anthropic's updated Claude LLM gives users the option of granting the tool some control over a PC, including looking at a screen, moving a cursor, clicking buttons and typing text. 

Examples of what Claude can do include filling out forms, planning an outing, and building a website.

 

 Claude 3.5 Sonnet is the first frontier AI model to offer computer use in public beta. It goes without saying that such efforts will be subject to user concern about errors and mistakes as well as privacy and security.

Microsoft had to retreat on the Copilot+ PC Recall feature that stored a user's screen shots. Meant to help people find and remember things they've previously seen on their computer. But users seemed to dislike the privacy and security dangers. So the feature now is optional. 

Google, for its part, is said to be working on Jarvis, the next iteration of its Gemini generative AI model. Said to work with web browsers, Jarvis is said to be a tool to automate everyday web tasks such as by taking screenshots, clicking buttons or entering text. Perhaps more important, Jarvis is intended to help users make purchases, fill out forms, compile data into tables, open a series of webpages, or book flights online, for example. All those are examples of how AI can be integrated into useful common experiences for users. 

It's Okay to be Skeptical About Claimed AI Outcomes

podcast of this article

A KPMG study suggests technology, telecom and media firms are already seeing, or expecting, return on investment from artificial intelligence spending. In fact, survey respondents reported hefty revenue growth, with 38 percent of respondents suggesting AI already drove more than 10 percent of total entity revenue. 

source: KPMG

In my experience, most enterprise software buyers are quite skeptical of such claims, if intrigued. And few respondents who profess such outcomes are equally able to quantify the outcomes, when asked to do so. 

“We think this outcome can be attributed to a specific input” is one matter. Actual proof is something else altogether. And there is no shortage of reasons for respondents to make such claims. Leaders always are eventually required to justify investments, costs of those investments and attributable financial outcomes.

As the old adage goes, “nobody ever got fired for recommending we buy from IBM.” In other words, bad choices can be job-ending moves. 

As always, the assumptions are key. There is a difference between stakeholder expectations about how AI can contribute, and the actual outcomes. We expect positive outcomes or would not make the investments. But outcomes and productivity are notoriously difficult to quantify for any sort of knowledge or office work.

Neither is it easy to quantify the specific outcomes enabled by any single change a firm makes, when multiple inputs--all dynamic--might be involved. 

For example, the “Redefining TMT with AI” report talks about the benefits of AI-driven predictive network analysis, in tandem with robotic process automation to  enhance network operations and quality

of service. 

In a strict sense, there are two independent variables here: process automation and use of AI. Beyond that, KPMG consultants note that the use case involves automated scripts and algorithms; predictive models; fault prediction; alarm handling; trouble-ticket management;, configuration management; 

 customized network-level reports and workflow management. 

As many of you know, such process automation already is a feature of many operational support systems. AI should help, of course. 

But it might be hard to quantify the degree of impact. Still, the point is that ROI is created by a reduction in volume of alarms, faults and tickets; improved Mean Time to Repair and reduced downtime. I cannot think of a single OSS platform or system that fails to mention those outcomes as benefits. 

In the video content industry, the report suggests AI produces ROI by affecting the efficiency and accuracy of video dubbing (language translation) and synchronizing the dubbed dialogue with the onscreen actors’ vocal movements. The ROI then is produced by reduced production time and cost. 

Also, AI is used to “for understanding and translating complex scripts and  while supporting real-time lip-sync. Basically, in this use case AI aids the dubbing process. 

AI also is used to speed up software coding, so the ROI is based on faster development cycles, faster debugging, code quality and developer productivity. 

The issue is not AI and its ability to improve all those processes and use cases. That indeed is the attraction. Instead, the issue is that it is hard to isolate the AI contributions from the other value created by the processes AI enhances. 

High AI Capex is Worrisome, But "Winner Take All" is the Prize

It is not hard to find estimates of investment in U.S. artificial intelligence infrastructure (computing capabilities) in the range of $300 billion or more between 2023 and 2030. IDC analysts have suggested $300 billion in investments between 2023 and 2026.

Nor is it hard to find critics who worry about uncontrolled spending without a clear revenue model. On the other hand, leaders of firms attempting to become leaders in the generative AI model business are likely to keep in mind the “winner take all” dynamic we have seen in the recent internet era, where just one or a few firms emerged as leaders in new markets. 

They might point to:

• Amazon's years of heavy investment to dominate e-commerce

• Google's massive spending to establish search leadership

• Cloud providers' huge datacenter investments

• Meta's acquisition strategy in social media.

In fact, many markets show scant ability to support three providers, as the market leader has twice the share--and up to an order of magnitude more-share compared to  the number-two provider.

Market	Dominant Player	Market Share	Runner-up	Market Share
Search Engines	Google	91.9%	Bing	3.0%
Desktop Browsers	Chrome	65.72%	Safari	18.22%
Mobile Browsers	Chrome	66.17%	Safari	23.28%
Social Media	Facebook	2.9B users	YouTube	2.5B users
E-commerce	Amazon	37.8% (US)	Walmart	6.3% (US)
Video Streaming	YouTube	2.5B users	Netflix	231M subscribers
Music Streaming	Spotify	31%	Apple Music	15%
Ride-hailing (US)	Uber	68%	Lyft	32%
Cloud Services	AWS	32%	Azure	22%
Mobile OS	Android	71.8%	iOS	27.6%

So even if McKinsey estimates AI infrastructure spending will exceed $500 billion between 2023 and 2030, and even if many of those investments do nor produce the expected results, model suppliers have incentives to risk quite a lot, knowing that there is a small  prize for being second best. 

Gartner forecasts global AI infrastructure investments will surpass $250 billion annually by 2030. 

The OECD estimates investments in AI infrastructure across industries, will reach $1 trillion by 2030, across the OECD countries. Bloomberg predicts that the global AI infrastructure market will $700 billion by 2030.

On the other hand, most of that investment will be by end users and others in the value chain, not the generative AI model providers. 

And some estimates made in 2023 might be considered conservative in 2024. Morgan Stanley’s  "The Economics of AI” study, published in October 2023 suggested more than $200 billion in AI infrastructure investments by 2030, including:

• Data centers: $125B

• Networking infrastructure: $50B

• Chip fabrication: $25B

• Cooling systems: $10B.

Boston Consulting Group in December 2023 suggested there would be $235 billion cumulative investments in 

• Data center buildout: 45%

• Compute infrastructure: 35%

• Power infrastructure: 20%. 

The Goldman Sachs "AI Infrastructure Report," published in September 2023 estimated $275 billion in  cumulative investment, including:

• Semiconductor investment: $100B

• Data centers: $115B

• Power systems: $35B

• Network upgrades: $25B. 

The caution, though, is that early estimates of the size of new technology markets often lead to overinvestment across the value chain. 

Study/Report	Date	Publisher	Key Conclusions
The Dot-Com Bubble Burst: Causes and Implications	2001	U.S. Securities and Exchange Commission (SEC)	Overinvestment in internet startups led to a speculative bubble that burst in 2000. Many companies were overvalued despite having no profitability.
Boom and Bust: The Telecommunications Investment Bubble	2002	Federal Reserve Bank of San Francisco	Overinvestment in telecom infrastructure during the late 1990s led to a major industry downturn, with unsustainable levels of capital spending.
The Case for Less Innovation	2017	Harvard Business Review	Many companies overinvest in unproven technologies without clear demand, resulting in failed projects and wasted resources.
Lessons from the Clean Tech Bubble	2016	MIT Energy Initiative	Overinvestment in cleantech (2005-2011) led to massive failures, with many companies being too early to market and receiving excessive venture capital.
Investing in Innovation: Creating a Research and Innovation Policy That Works	2010	The NESTA Foundation (UK)	Over-investment in R&D for new technologies can create inefficiencies and fail to produce proportional economic benefits if not managed strategically.
The Nanotechnology Investment Bubble	2005	Journal of Nanoparticle Research	Speculative investments in nanotechnology during the early 2000s led to unmet expectations, as many products were not commercially viable.
Unleashing Productivity: Overinvestment in Information Technology	2005	McKinsey Global Institute	Overinvestment in IT during the late 1990s and early 2000s did not yield expected productivity gains, with firms often adopting technology prematurely.
The Illusions of Overinvestment in AI	2021	Brookings Institution	Many companies overinvest in artificial intelligence without clear applications, leading to inflated expectations and unrealized returns.
The Biotechnology Bubble: When Science and Finance Collide	2004	Nature Biotechnology	Excessive capital flow into biotech during the 1990s led to overvaluation, with many firms failing to achieve meaningful breakthroughs.

In recent years we have also seen examples of overinvestment by many platform suppliers as well. 

Technology	Company/Industry	Year	Description of Over-Investment
Artificial Intelligence	IBM Watson	2011-2022	IBM invested billions in Watson AI for healthcare, but struggled to generate significant revenue and ultimately sold off the health assets
Virtual Reality	Meta (Facebook)	2014-present	Meta has invested over $36 billion in VR/AR technology with limited returns, facing skepticism about the metaverse vision
Blockchain	Various	2017-2018	Many companies rushed to invest in blockchain during the crypto boom, only to scale back or abandon projects when the hype died down
Autonomous Vehicles	Uber	2016-2020	Uber invested heavily in self-driving technology, spending over $1 billion before selling the unit after a fatal accident and regulatory challenges
3D Printing	3D Systems	2013-2015	The company aggressively acquired 3D printing startups, leading to over $1.3 billion in losses and a stock price crash when consumer adoption didn't materialize
Cloud Computing	HP	2011-2012	HP's $11 billion acquisition of Autonomy for cloud services led to an $8.8 billion write-down

So the rationale for investing heavily to secure the leading position in the generative AI model business is a reflection of the possible “winner take all” character of application and platform markets, where the number-one provider dominates. 

And since market share and profit margin generally are related, the rewards for market leadership also are significant. In many capital-intensive markets, the profit margin of the top provider is double that of number two. 

And provider number two can have margins double that of provider number three.