What's the Best Analogy for LLMs?

For large language model use cases, one size does not necessarily fit all, all the time. On the other hand, to the extent that LLMs are conceived of as similar to operating systems, one size arguably is much more important.

Looking for historical analogies is one way of trying to understand large language models and other forms of artificial intelligence, when assessing business model implications. Where and when is scale essential versus merely helpful?

For example, are LLMs more akin to operating systems, platforms or applications? 

Are LLMs in part “picks and shovels,” which are more like OSes, and also, in part, applications that are always designed to run in loosely-coupled ways on any compliant platforms? Are LLMs sometimes also platforms? The importance of scale or market share might well hinge on which scenario matters most to particular providers. 

Feature	LLMs	OSs
Purpose	Generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way	Manage hardware resources, provide a platform for running software applications, and facilitate user interaction with computer systems
Underlying technology	Artificial neural networks	Kernel, device drivers, and user interface software
Scalability	Highly scalable, can be trained on massive amounts of data and run on distributed computing systems	Limited by hardware resources, require specialized software and configurations for different types of devices
Applications	Chatbots, virtual assistants, machine translation, content creation, code generation, research	Personal computers, servers, mobile devices, embedded systems
Maturity	Relatively new technology, still under development	Mature technology with a long history of development
Adoption	Growing rapidly, but still not as widely used as OSs	Ubiquitous, used by billions of people worldwide
Scale	Potential to be used on a wide range of devices and platforms	Typically designed for specific types of devices and platforms
Niche AI Models	Possible, as LLMs can be trained on specialized datasets	Less likely, as OSs need to be general-purpose and compatible with a wide range of hardware and software

OSs have tended to be a scale phenomenon, with a few dominant players controlling the market. This is due to the network effects that exist in the OS market: the more people use an OS, the more valuable it becomes to other users. As a result, it has been difficult for new OSs to gain traction.

However, the landscape for AI models may be different. Maybe the model is more social media, e-commerce, search, messaging than operating system, for example. In other words, instead of resembling an operating systems or device market, LLMs could resemble application markets, where age, culture, language, tone, content and function can vary widely.

Though scale still matters, apps are far less monolithic than operating systems or consumer devices such as smartphones. In other words, LLMs can enable app use cases that are more variegated than the OS market or device markets tend to be. 

Highly specialized LLMs usable by a single company and its own applications will be possible. So too will apps targeted to different age groups, language groups, cultures and functions. 

Edge computing might also mean it is possible to deploy AI models on devices with limited resources, such as smartphones and IoT devices, creating more “niche” use cases.

So we might argue that operating systems  require scale to be successful. Without scale, an operating system would have limited reach and adoption.

In the context of LLMs, scale is crucial for models that aim to be general-purpose solutions, catering to a wide range of tasks and domains. For instance, LLMs used for machine translation or text summarization need to be trained on massive amounts of data from various sources to handle diverse language contexts and content types. Scale allows these models to perform well across a broad spectrum of applications.

Platforms like social media networks, e-commerce sites, and content sharing platforms benefit from scale but don't necessarily require it. For LLMs, scale arguably is helpful but not essential for models that target specific platforms or applications. 

For example, an LLM integrated into a customer service chatbot might not require the same level of scale as a general-purpose language model, though scale generally is helpful. 

End-user applications like productivity tools, creative software, and games can succeed without scale. Similarly, LLMs can be incorporated into end-user applications without requiring massive scale.

As often is the case, LLM scale is a fundamental requirement in some use cases, but not in others. For suppliers of wholesale, general-purpose LLMs, scale likely will matter. 

When used as a platform, maybe not. And when used to enable apps and use cases, scale might not be particularly important. Model owners must care about scale. “Computing as a service” providers or data centers can afford to “support all” stances. 

App developers might not necessarily care which LLM is used, beyond the obvious matters of supplier stability, cost, reliability, reputation, ease of use, support and other parts of a value bundle. 

How Big is the Large Language Model Industry? How Big Could it Get?

As a brand-new market, large language models are likely still in the single-digit-billions range, not counting all the other markets that are, or can be, built on the use of LLM. 

Estimate	Study Name	Date of Publication	Publisher
$2.1 billion	Allied Market Research	2022	Allied Market Research
$1.3 billion	MarketsandMarkets	2022	MarketsandMarkets
$3.2 billion	Grand View Research	2022	Grand View Research
$5.6 billion	Mordor Intelligence	2022	Mordor Intelligence
$6.9 billion	Precedence Research	2022	Precedence Research

Just how big the core LLM model business might be remains a guess, but if one believes LLM is a general-purpose technology, then like other forms of enabling infrastructure, the market could be substantial. 

What also remains unclear is where key revenue elements--especially fees for generating inferences--will be reaped. Use of an LLM includes both the license to use a model plus the recurring fees paid for deriving inferences. 

As with any other form of “computing as a service,” some parameters will vary. License fees might be one-time payment, a recurring fee, or a fee based on usage.

Inference fees are variable. Support and maintenance fees might also often be charged, to cover bug fixes, security updates, and documentation updates.

The ultimate analogy might determine market size. Are LLMs most akin to servers, operating systems or end-user software? More generally, are LLMs going to be infrastructure somewhat similar in function to electricity networks, road systems, airports or seaports, as enablers of commerce?

Underlying Technology	Market Size	Study Name	Publication Date	Publisher
Electricity	$2.3 trillion	Global Electricity Market Report 2023	March 2023	IEA
Roads	$1.2 trillion	Global Road Infrastructure Market 2023	June 2023	Grand View Research
Airports	$1.8 trillion	Global Airport Infrastructure Market 2023	July 2023	Allied Market Research
Seaports	$1.1 trillion	Global Seaport Infrastructure Market 2023	August 2023	MarketsandMarkets
Smartphone Operating Systems	$128 billion	Mobile Operating System Market by Platform 2023	September 2023	Statista
PC Operating Systems	$44 billion	PC Operating System Market by Platform 2023	October 2023	Gartner
Server Processors	$25 billion	Server Processor Market 2023	November 2023	IDC
Platforms such as Linux, Windows or IoS	$1.5 trillion	Cloud Platform Market 2023	December 2023	Synergy Research Group
Software as a Service	$167 billion	Global Software as a Service (SaaS) Market 2023	January 2024	Gartner

Beyond all that, where is the incidence of payments? Is revenue generated by direct fees charged to business and consumer end users, directly by business partners or indirectly in the form of value for third parties and end users? 

In many cases, AI is a capability that enhances the value of some product a buyer consumes, but might not be a distinct extra charge or involve a subscription for use of the product. 

"Superbundling" Will Work, Until it Doesn't

“Superbundling”-- the practice of offering a package of services from different providers--has a long history in the cable TV and telco businesses. For many decades, the "triple play" of home broadband, linear video and voice was a staple in the business.

"Superbundling" that linclude streaming services in the older bundles has gotten more traction recently with the contract signed between Charter Communications and Disney relating to carriage of linear and streaming video services. 

In many ways, that search for the "right" products in a bundle is the search for a "natural" bundle that makes sense to most consumers.

Traditionally, bundles have included Internet access, linear TV and phone service. In recent days mobility service has been added to the “natural bundle.”

In their search for additional “natural bundle” services, ISPs have looked at app security, electricity services, home security and now video streaming services, with limited success to date. So attention now has turned to integrating video streaming platforms like Netflix, Hulu, or Disney+.

As always before, there are advantages for consumers and providers that eventually become negatives for both. 

Consumers have preferred bundles for a simple reason: they save money. Suppliers like bundles because they boost perceived value, average revenue per account and reduce churn, while boosting revenue. 

But bundles eventually can become their opposite. Consumer opposition to linear video subscriptions now turns precisely on the “paying for products I do not want.” In the past, for example, some customers took triple-play packages of video, home broadband and voice for the savings, but never even activated the voice services that were part of that bundle. 

And that is one danger for bundles: customers might come to feel they are paying for products they do not want. So where the cable TV bundle once was pitched as providing value--many channels for one price--it became a negative as consumers found they were paying too much for too many channels they never watched.  

Also, buyers eventually come to distrust and dislike product bundles that obscure the actual cost of each product within the bundle, as well as the vendor lock-in that the bundles represent. 

There is a delicate balance between cost, choice and simplicity that always has to be faced, where it comes to bundling. More products for one price--presumably representing equal increases in value--works. Until it doesn’t.

Parsing Disney's Annual Report

10-K forms (annual reports) filed with the U.S. Securities and Exchange Commission provide useful information about a firm’s prospects and threats, though we might also quip that 10-Ks also contain lots of statements related to business risk that collectively (and playfully on the part of the reader) suggest the firm faces steep obstacles.

It still is notable, in that sense, that Disney’s latest 10-K contains a discussion about content “misalignment.”

“We face risks relating to misalignment with public and consumer tastes and preferences for entertainment, travel and consumer products, which impact demand for our entertainment offerings and products and the profitability of any of our businesses,” the report states.

In a broad sense, that simply speaks to the unpredictability of content production. Story-telling is an art, not a science, so consumer tastes and preferences often are unpredictable. That is why some movies and shows do well, while others do not.

In the same vein, the cautionary statements about changes in economic conditions, technology or security and intellectual property rights might seem obvious. So too, “uncontrollable events such as the Covid epidemic, regulations, execution risks or competition.

Ths list of such dangers typically is quite extensive, as Disney’s is, so we should not read too much into such clauses.

Still, some will likely interpret the wording about misalignment as more than standard descriptions of business risk. The report also notes that “consumers’ perceptions of our position on matters of public interest, including our efforts to achieve certain of our environmental and social goals, often differ widely and present risks to our reputation and brands."

In other words, the big push for corporate activism might have hit a wall. We’ll have to look for other evidence of a public backlash to the “social activist agenda.” I don’t know that I’ve ever read a similar statement in any annual report.

The reasons for Disney's misalignment with public tastes and preferences arguably are complex. Some argue that the company has become too focused on producing sequels, remakes, and franchises, at the expense of original stories.

Others argue that Disney has lost touch with its core audience of children and families. Still others argue that Disney is simply not producing enough high-quality content overall. But some argue Disney content has suffered because it also has an activist agenda out of step with much of its intended audience.

AI KPIs Will Evolve, as did Those of Internet Firms

Since artificial intelligence is so new as a driver of firm revenues, growth and valuation, professionals have few ways to model AI equity reward and risk, as was true for professionals in the early days of the internet. 

Traditional valuation methods, such as price-to-earnings (P/E) ratios and discounted cash flow (DCF) analysis for firms with negative earnings or a clear path to profitability.

Instead, assessments will have to turn on other more-subjective angles, such as business model “potential," traffic or user engagement, much as early internet-watchers had to rely on the number of unique visitors, the average time spent on site, and the click-through rate for advertisements. 

Click-through rates were important proxies for apps in the early days of the internet. And customer acquisition cost might be less important for AI firms than repeat usage (lifetime value). 

Model owners and suppliers of computing as-a-service will measure volume of transactions and data processed. 

But key metrics may evolve. In the early days of the internet, user engagement and traffic seemed more important. In 1995, for example, page views, unique visitors, time spent on site and click-through rate were arguably the most important.

With greater maturity and actual profits, standard metrics such as revenue growth, profitability, customer acquisition cost or customer lifetime value became relevant.

In its mature phase, active users and user engagement seem more important. So is loyalty, as often measured using the net promoter score. 

Internet, 1995	AI (2023)
Website Traffic: Measured the number of unique visitors to a website. Indicated the popularity and reach of the website and its potential to attract users and advertisers.	User Engagement: Measures the level of interaction and involvement of users with an AI-powered product or service. Indicated the effectiveness of the AI in providing value to users and its potential for long-term adoption.
Page Views: Measured the number of times a page on a website was viewed. Indicated the depth of user engagement and the potential for advertising revenue.	Data Volume and Processing: Measures the amount of data processed by an AI system. Indicated the system's ability to handle large amounts of data and its potential for generating insights and value.
Click-Through Rate (CTR): Measured the percentage of users who clicked on an advertisement on a website. Indicated the effectiveness of the advertisement in attracting user attention and driving clicks.	Accuracy and Precision: Measures the ability of an AI system to produce correct and consistent results. Indicated the system's reliability and its potential for generating valuable insights.
Unique Visitors: Measured the number of individual users who visited a website. Indicated the reach of the website and its potential to attract a diverse user base.	Revenue Growth: Measures the increase in revenue generated by an AI-powered product or service. Indicated the commercial viability of the AI and its ability to generate financial returns.
Time Spent on Site: Measured the average amount of time that users spent on a website. Indicated the level of user engagement and the potential for monetization through advertising or subscriptions.	Return on Investment (ROI): Measures the financial return generated by an AI investment. Indicated the value of the AI in driving business outcomes and profitability.
Customer Acquisition Costs (CAC): Measured the cost of acquiring new customers for an internet-based business. Indicated the efficiency of marketing and sales efforts and the potential for profitability.	Customer Lifetime Value (CLV): Measures the long-term financial value of a customer to an AI-powered business. Indicated the ability of the AI to retain and grow a customer base and generate recurring revenue.
Brand Awareness: Measured the recognition and perception of a company's brand among potential customers. Indicated the company's ability to attract and retain users in a competitive market.	Industry Recognition: Measures the recognition of an AI company's achievements and innovations within the AI industry. Indicated the company's reputation and potential for leadership in the field.

Presumably, AI firms will move through similar stages as they mature. Early on, analysts will try to quantify the value of intellectual property, data assets or the danger competitors represent. 

In some cases, volume of data processed, accuracy of AI models, or adoption of AI solutions by customers could be proxies for growth. Customer acquisition costs could be another metric when traditional metrics do not yet apply. 

Revenue mix or reliance on few or broadly-diversified customers might also matter. But key metrics will change as the AI industry matures. Then the more-traditional financial metrics will apply. 

Usage matters now, but engagement will matter later.

How ChatGPT Works, "for Dumb End Users"