"Winner Takes All" or "Winner Takes Most" Market Structure for LLMs?

According to the Chatbot Arena leaderboard, a platform for evaluating AI determined by user votes, Gemini’s latest update--Gemini-Exp-1114--ranks best among large language models. 

It is worth noting that leaders change somewhat frequently, with the top-five models presently all versions of OpenAI or Google models. Perhaps notably, Grok-2-08-13 ranks sixth. 

source: Chatbot Arena 

It might also be worth noting that OpenAI's models (such as GPT-4) and Anthropic's Claude models have consistently ranked near the top of the leaderboard.

And leadership seems to have changed since the spring of 2023. Consider the leaderboard published by LMsys in the spring of 2023. ChatGPT 3.5 launched in late 2022 and seems to have been in the top five of the Arena leaderboard since its inception in the spring of 2023. 

source: lmsys.org 

Eventually, business history would suggest leadership of the market will condense, as have other technology markets. So. the LLM market is likely to evolve into a structure characterized by oligopolistic competition among a few major players, complemented by a range of specialized providers catering to specific industries or use cases.

In 2023 five LLMs had more than 88 percent market share. That leadership group might condense further, eventually. 

On the other hand, the room for specialized platforms might remain. How many of us would not see any way for OpenAI, Google, Microsoft, Meta, Amazon, Apple and IBM, for example, to continue as operators of domain-specific LLMs, no matter what happens ;with the broader market?

And who might doubt that specialized industry-specific platforms could number between 10 and 20 (catering to different sectors like healthcare, finance, legal)?

And of the leaders, might open-source initiatives include three to five significant contributors? 

Might AI-as-a-Service providers number 10 to 15 “significant” players, even if the top five or so positions include AWS, Google Cloud, Azure, Meta and Amazon? 

Also, if history is instructive, could there not exist five to 10 Integration and orchestration platforms as well?

The issue is what “winner takes all” will mean in the LLM ecosystem and platforms markets. Current examples include just one or perhaps two leaders in existing markets, which is more on the “operating system” model. On the other hand, most of us would have a hard time deciding on less than perhaps four leading LLMs for some time to come. 

And some structural differences between existing technology market structures and LLMs come to mind. Unlike the operating system market, LLMs don't require the same level of user lock-in or hardware integration. So the "two-leaders” pattern might not emerge. 

Roughly the same argument might be made about the e-commerce or search market structures, where one leader tends to emerge. The competitiveness of existing LLMs, with continual upgrades, tends to dispel the notion that any single provider will achieve technological superiority on a sustainable basis. 

LLMs also lack the network effects and user-generated content central to social media platforms. So it is possible the one leader model might not develop. Right now, differences between leading platforms are relatively subtle. 

So the likely direction is “winner take most” more than “winner take all.” Even if network effects are not so strong, high capital intensity, branding and trust issues and the ability to vertically integrate with existing ecosystems (Google, Apple, Microsoft, Meta) create enormous advantages for a few contenders. 

At least for the moment, “winner take all” is hard to see. A still-oligopolistic, but “winner take most” structure with a handful of leaders might be more plausible. 

Why Firm Productivity Might Drop in the Near Term as AI Gets Deployed

Among other issues, such as potential payback from deploying generative artificial intelligence, is the timing of the payback, and history suggests payback will take far longer than many expect. If AI does develop as a general-purpose technology, as were earlier GPTs including steam power and electricity, and even granting that many technological innovations--which are largely virtual--can propagate much faster than did earlier innovations.  

The initial impact of steam power and electricity on productivity was not as immediate or dramatic as expected. 

Consider steam power. Early adoption was slow. The first practical steam engine was invented by Thomas Newcomen in 1712. James Watt significantly improved the steam engine in 1765 and kicked off the process of commercialization. Still, by 1830, only 165,000 horsepower of steam was in use in Britain, for example.

Even in 1870, about two-thirds of steam power was concentrated in just three industries: coal mining, cotton textiles, and metal manufactures. So, while invented in the early 18th century, it took about 50 to 75 years for steam power to begin having truly widespread and transformative effects on industry and the economy.

The major productivity gains from electricity in the United States came in the 1920s, about 40 years after Thomas Edison first distributed electrical power in New York in 1882.

And there is ample prior evidence of actual productivity dips in  the early days of new technology diffusion. The J Curve, for example, illustrates the pattern that there is an early period of disruption and actual productivity decline when a major new technology is introduced. Only later are the tangible benefits seen. 

source: Flexible Production 

The J-curve effect in GPT adoption typically follows a few stages, from initial investment to realized productivity. AI clearly is in the early investment phase, which ought to imply significant costs without immediate financial returns.

Which ought to clue us in to the fact that investors are likely to be quite disappointed when most entities cannot show significant financial returns. 

And though the J curve might not apply when innovations do not require value chain disruption and displacement, Verizon’s experience with fiber-to-home upgrades still show that even innovations that do not require business model change can take a while to reach maturity. 

As significant as fiber-to-the-home was deemed to be by Verizon, one would be very hard pressed to show significant financial returns to Verizon for five years from mass deployment.

FTTH was not a GPT that required changes in consumer behavior or disruptions of Verizon’s supply and value chains. 

The thing about GPTs (and if AI is a GPT the J curve should apply) is that disruption is required. Still, Verizon arguably reached scale in about four to five years of construction, with very-significant revenue contributions for new video entertainment services enabled by the FiOS network. In the second quarter of 2011, for example, Verizon had about 4.5 million broadband accounts, as well as3.8 million video accounts. 

In the second quarter of  2011, FiOS generated 57 percent of consumer wireline revenues, up from 48 percent a year earlier, Verizon said that year. 

 

By the third quarter of 2011, FiOS accounted for nearly 60 percent of consumer wireline revenues. In the last quarter of 2014, FiOS contributed 75 percent of consumer wireline revenues. Keep in mind that statistic also includes the diminution of Verizon’s landline voice business, plus the maturation and decline of its linear video entertainment business as well. 

In other words, FiOS revenue became the driver of Verizon consumer fixed network revenue in part because the voice and video entertainment businesses declined. 

Year	Cumulative Capital Investment ($B)	Annual FiOS Revenue ($B)	FiOS Subscribers (Millions)
2006	3.6	0.5	0.7
2010	23.0	7.5	4.1
2014	30.0	12.7	6.6
2018	34.0	11.9	6.1
2022	36.5	12.8	6.3

The main take away is that productivity might actually dip in the near term as firms deploy AI technologies.

General-Purpose Technology	Initial Productivity Dip	Adaptation Period	Productivity Surge
Steam Engine	Slow adoption in early 19th century	1820s-1840s	1850s-1890s
Electricity	Limited productivity gains in 1890s-1910s	1920s-1930s	1940s-1950s
Computers	Productivity paradox in 1970s-1980s	1980s-1990s	Late 1990s-2000s
Internet	Initial investment costs in 1990s	Late 1990s-early 2000s	Mid 2000s-present

AI Will Eliminate Whole Industries, Not Just Some Jobs

Virtually all observers believe artificial intelligence is going to eliminate some jobs, in line with the ability AI might have to automate key job functions. The attrition could come because higher output can be achieved using fewer people, perhaps more so than because AI completely eliminates a particular job role. 

podcast of this content

But that is not even the most important "threat." Whole industries can disappear when a general-purpose technology appears, and AI is likely to be a GPT.

Industry	Disrupted by	New Industries/Roles
Horse-drawn carriage manufacturing	Internal combustion engine	Automobile manufacturing, transportation services
Typewriter manufacturing	Personal computer	Computer hardware and software manufacturing, word processing services
Film photography	Digital photography	Digital camera manufacturing, digital imaging services
Record stores	Digital music distribution (MP3, streaming)	Music streaming services, digital music production
Travel agents	Online travel booking websites	Travel technology companies, online travel booking services
Traditional retail	E-commerce	Online retail, logistics and delivery services

If AI does prove to be a general-purpose technology on the pattern of agriculture, steam power, the internal combustion engine, computing or electricity, that is inevitable. A look at the impact of various computing technologies, from personal computers to AI, illustrate the point, but huge changes in labor forces have always accompanied the emergence of a new GPT. 

Agriculture allowed humans to settle, rather than living as hunters and gatherers, creating the underpinnings for economic surplus that in turn enabled population growth, job specialization and settlements that enabled  the development of art, writing, legal systems, mathematics, new tools,  medicine and more-complex social structures. 

Study Title	Date	Publisher	Key Findings
The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies	2014	W. W. Norton & Company	Discusses how technologies like AI, robotics, and personal computers have automated routine jobs in clerical, manufacturing, and administrative sectors, while creating new roles in software development, IT management, and creative industries.
The Race between Education and Technology	2014	National Bureau of Economic Research (NBER)	Shows how general-purpose technologies, particularly computers, have reduced demand for middle-skill jobs (clerks, machine operators) and increased demand for high-skill (software engineers, data analysts) and low-skill jobs (service workers).
Technological Change and the Labor Market: A Survey	2017	Journal of Economic Surveys	Examines how the introduction of GPTs (such as AI, automation, and IT) has led to job polarization, with job losses in low-to-middle skill categories and gains in high-skill, technology-oriented positions, such as data scientists, AI researchers, and cybersecurity experts.
Automation, Jobs, and the Future of Work	2019	International Monetary Fund (IMF)	Reviews global trends in automation and AI, showing how GPTs have reduced the need for routine manual and cognitive jobs (e.g., typists, cashiers, machine operators), while increasing demand for jobs in tech, management, and highly skilled service sectors.
AI, Robotics, and the Future of Work	2018	Brookings Institution	Analyzes how AI and robotics, as general-purpose technologies, have transformed sectors like manufacturing, retail, and logistics by displacing low-skill jobs, but creating high-skill roles in programming, system analysis, and tech support.
Digital Transformation and the Future of Jobs	2020	World Economic Forum	Finds that GPTs like AI, blockchain, and the Internet of Things (IoT) have contributed to job displacement in traditional sectors like agriculture and manufacturing, while creating job opportunities in digital and tech sectors, such as cloud computing and cybersecurity.
The Impact of Information Technology on Labor Demand	2019	Journal of Economic Surveys	Examines the role of IT and general-purpose technologies in reshaping labor demand. It finds that routine jobs in finance, clerical work, and manual labor have declined, while demand for IT professionals, software developers, and project managers has risen.
How AI and Robotics are Transforming Labor Markets	2020	European Central Bank (ECB)	Demonstrates that AI and robotics have led to the disappearance of low-wage, routine manual jobs, and the growth of tech jobs in AI systems, machine learning, data processing, and AI ethics. Also highlights growing demand for high-skilled healthcare roles enabled by AI.
The Effects of Automation on Jobs and Wages: A Global Perspective	2023	OECD	Highlights that GPTs like AI and robotics have not only automated routine jobs in manufacturing and services but also created new job categories in software development, healthcare technology, and digital infrastructure management.
Automation and the Shift in Labor Markets: Evidence from AI and Robotics	2021	Review of Economics and Statistics	Provides evidence that AI and robotics have led to the shrinking of low-skilled jobs (e.g., assembly line workers) while new opportunities have emerged in tech-heavy fields like AI programming, cybersecurity, and data analysis, where skill requirements are much higher.
Technology, Jobs, and Inequality: A Survey of the Evidence	2018	Economic Policy Institute	Analyzes how the adoption of GPTs, including AI and automation, leads to job polarization, creating a split between low-wage, low-skill jobs in the service sector and high-wage, high-skill jobs in tech fields like cloud computing, data science, and AI.
Work in the Age of AI: The Impact of Automation and AI on Job Categories	2022	MIT Technology Review	Examines the effects of AI-driven GPTs on job categories, showing a decline in traditional roles such as truck drivers and retail workers, while new categories like AI ethics officers, machine learning trainers, and drone operators emerge.
The Future of Jobs in the Age of Automation	2017	McKinsey Global Institute	Explores how general-purpose technologies like automation and AI displace routine work (e.g., cashiers, clerks), but open up new career categories, particularly in fields such as AI system design, data analytics, and machine maintenance.
The Economic Impact of Automation and AI	2021	Stanford Center for Digital Economic Studies	Shows that GPT adoption has led to a shift in employment patterns: many low-skill roles have been automated (e.g., assembly line workers, clerks), while there is an increased demand for data scientists, digital marketers, and machine learning engineers.

So concerns about changes in job composition of the labor force are realistic, if quite possibly inevitable. U.S. dockworkers recently conducted a strike among which key demands included a complete ban on automation of dock work. Discussions about that portion of new contracts remain active, but the larger point is that demands by workers to ban the use of machinery of all types has arguably slowed, but never stopped, the deployment of new technology based on a GPT that automated formerly-human labor. 

Personal computers, for example, did not so much eliminate whole jobs as make possible the ability of each worker to produce some output that formerly might have been created by others (people write their own emails and documents, where in the past stenographers would have done so. 

PCs democratized access to tools that allowed workers at all levels to produce output that once would have been handled by others, such as document production, data entry, analysis, and design.

Before PCs became widespread, tasks like document production, data processing, and basic design were often handled by specialized staff such as secretaries, typists, clerks, and graphic designers. “Desktop publishing,” for example, was an early use case for Apple computers. 

As always, new jobs arose, as well. 

Sector/Occupation	Job Function Pre-PC	Job Function Post-PC	Change in Demand
Clerical/Administrative	Typist, Data Entry Clerk, Secretary	Office Manager, Executive Assistant, Office Coordinator	Decreased
Accounting/Finance	Bookkeeper, Ledger Clerk, Accounts Clerk	Financial Analyst, Accounting Software Specialist	Decreased
Manufacturing	Manual Laborer, Assembly Line Worker, Machine Operator	CNC Operator, Maintenance Technician, Robotics Specialist	Decreased
Retail	Inventory Clerk, Cashier, Stock Clerk	E-commerce Manager, Inventory System Analyst	Decreased
IT and Technology	None	Software Developer, Systems Administrator, IT Support	Increased
Customer Service	Telephone Operator, Customer Service Representative	Help Desk Technician, Customer Support via Online Platforms	Increased
Marketing and Sales	Sales Clerk, Telemarketer, Market Research Assistant	Digital Marketing Specialist, Social Media Manager	Increased
Education	School Secretary, Paper-Based Research Assistant	EdTech Specialist, E-learning Coordinator, Curriculum Developer	Increased
Health Services	Medical Records Clerk, Billing and Coding Clerk	Medical Data Analyst, Health IT Specialist, Telemedicine Support	Increased
Logistics/Transportation	Shipping Clerk, Inventory Handler, Dispatcher	Supply Chain Analyst, Logistics Software Manager	Increased

At a high level, most observers might agree that AI poses similar sorts of upside and downside, from the standpoint of jobholders. New jobs are going to be created, but some existing jobs will likely decrease in number. In that sense, we can expect continued opposition to the use of AI in many industries, though such opposition will fail, over time, as obvious productivity gains often compel all contestants in a market to adopt the new technologies. 

Study Title	Date	Publisher	Key Findings
The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies	2014	W. W. Norton & Company	The adoption of digital technologies leads to a polarization of the job market, creating high-skill, high-pay jobs while diminishing middle-skill jobs.
The Impact of Digital Technologies on Employment	2016	International Labour Organization (ILO)	Found that automation through digital technologies has the potential to displace a significant number of jobs, especially in manufacturing, but also create new job categories in tech and service sectors.
Automation, Skills, and the Future of Work	2019	McKinsey Global Institute	Predicts that up to 375 million workers may need to switch occupational categories due to the adoption of automation and AI, necessitating retraining and new skill development.
The Future of Jobs Report 2020	2020	World Economic Forum	Identified the net job creation potential from GPTs, predicting that 85 million jobs may be displaced while 97 million new roles could emerge, emphasizing the need for upskilling.
Artificial Intelligence and the Future of Work	2021	Brookings Institution	Discusses how AI adoption can create job growth in sectors requiring complex human interactions and creativity, but also warns of significant job displacement in routine tasks.
The Economic Impact of Artificial Intelligence on Work	2022	MIT Technology Review	Highlights that GPTs like AI can lead to job transformation rather than outright replacement, with new roles in data management, AI oversight, and ethics emerging as key areas of growth.
Industry 4.0 and Its Impact on the Labor Market	2023	Journal of Business Research	Analyzes the impact of Industry 4.0 technologies on job dynamics, indicating a shift towards more specialized roles and increased demand for technical and soft skills.
The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies	2014	W. W. Norton & Company	Examines how advances in automation and artificial intelligence (AI), including GPTs, lead to job displacement in routine tasks, but also creates opportunities for higher-skill jobs in sectors like healthcare and education.
The Impact of Information Technology on Labor Demand: A Review of the Literature	2019	Journal of Economic Surveys	Analyzes the role of IT and GPT adoption in reshaping labor demand across industries, revealing a shift toward higher-level cognitive tasks, while routine manual and clerical jobs decline.
Artificial Intelligence and the Economy	2020	Federal Reserve Bank of Dallas	Highlights that AI and GPTs automate a wide range of tasks, particularly in manufacturing and services, leading to job displacement but also the creation of new roles in data analysis, AI training, and tech support.
Technology, Jobs, and Inequality: A Survey of the Evidence	2018	Economic Policy Institute	Reviews evidence that while GPTs like AI and robotics reduce demand for low-skill jobs, they lead to greater inequality, with job growth concentrated in high-skill and managerial sectors.
Labor Market Polarization and Technological Change: A Historical Perspective	2016	National Bureau of Economic Research (NBER)	Finds that technological advancements, including GPTs, result in labor market polarization: growth in high-skill jobs and decline in middle-skill jobs, with a hollowing out of mid-level occupations.
Automation, Skills, and the Future of Work	2021	Brookings Institution	Concludes that GPT adoption accelerates demand for highly skilled labor (in areas like data science, programming), while jobs in routine sectors such as manufacturing and transportation face displacement.
The Effects of AI and Automation on Jobs and Wages: A Global Perspective	2023	OECD	Examines global trends in AI and automation adoption, indicating that the adoption of GPTs displaces low-wage jobs but creates more skilled positions in technology management, software development, and AI maintenance.
How Automation Affects Occupations: Assessing the Task Content of Occupations	2019	Quarterly Journal of Economics	Analyzes how automation via GPTs impacts occupations by reducing routine, repetitive tasks, but increasing demand for creative, problem-solving, and technical roles.
Digital Transformation and the Future of Jobs	2020	World Economic Forum	Provides evidence that GPTs spur job growth in technology-driven sectors (e.g., software development, cybersecurity) but reduce demand for jobs in routine, manual, and clerical functions.
Technological Change and the Labor Market	2017	Journal of Labor Economics	Identifies that technological progress, including GPTs, leads to increased job turnover and retraining requirements, with displaced workers often finding new roles in technology and service sectors.
Technological Shocks and Labor Markets: Evidence from GPTs and AI	2022	Review of Economics and Statistics	Examines the effect of GPTs on labor markets and finds a clear correlation between GPT adoption and shifts in employment toward knowledge-based industries, with a decline in jobs that involve manual labor.
The Impact of AI and Robotics on Labor Markets: A Review of Empirical Evidence	2018	International Journal of Robotics Research	Reviews empirical evidence on the impacts of AI and robotics (a form of GPTs) on job displacement in manufacturing and logistics, while showing job growth in healthcare, software, and robotics management.