It's Hard to Tell What Adding Generative AI Will Cost an Enterprise

Virtually everyone believes artificial intelligence use cases are going to drive important changes in employment, work processes, applications, use cases, processing operations, power consumption and data center requirements, though precisely how much change will occur, and when, remains unclear. 

More practically, firms and entities are having to estimate how much it will cost to create generative AI models and then draw inferences from those models. 

The answers, inevitably, are that “it depends” on what one wishes to accomplish, using which engines, which compute platform, scraping how much data, how much customization for a generic model is required, the number of users of the model; the complexity of the tasks that the model supports; the amount of data that is needed to train the model and the cost of computing resources. 

Context length, which determines the amount of information the LLM can consider when formulating an output, also affects pricing. if a generative AI model has a context length of 10, it will consider the 10 previous words when generating the next word.

The context length of ChatGPT-4 is 8,192 tokens for the 8K variant and 32,768 tokens for the 24K variant. This means that ChatGPT-4 can consider up to 8,192 or 32,768 previous words when generating the next word, depending on the variant. 

The cost for using the GPT-4 8K context model API is about $0.03 per 1,000 tokens for input and $0.06 per 1,000 tokens for output. 

Using the 32K context model, the cost is $0.06 per 1,000 tokens for input and $0.12 per 1,000 tokens for output.  

Applications	GenAI Costs	Studies
Marketing	$10,000 - $100,000	Gartner
Sales	$100,000 - $1 million	Forrester
Customer Service	$1 million - $10 million	McKinsey
Product Development	$10 million - $100 million	PwC

And the cost of building a model, offered as a platform, is not the same as the cost for an entity to use that model, when offered as a subscription; a pay-per-use model or bundled as a feature. 

Certainly, everyone expects model building, training and customization costs to come down over time. But the costs appear to be significant, whether enterprises choose to build using their in-house resources or use a cloud computing “as a service” provider. 

Business size	Cost of building generative AI model on-premises	Cost of building generative AI model on the cloud
Fortune 500	$10 million - $100 million	$5 million - $50 million
Mid-market	$1 million - $10 million	$500,000 - $5 million
Small business	$100,000 - $1 million	$50,000 - $100,000

The costs of building generic models will likely, over time, mostly be the province of LLM platform suppliers, as few entities will have the financial resources to build and train proprietary models. 

Cost estimate	Key assumptions	Study name	Date of publication	Publishing venue
$10M-$100M	100B parameters, trained on 100TB of text data, using 1,000 GPUs for 1 month		2022	OpenAI
$1B-$10B	1T parameters, trained on 1T TB of text data, using 10,000 GPUs for 1 year		2023	Google AI
$10B-$100B	10T parameters, trained on 10T TB of text data, using 100,000 GPUs for 10 years		2024	Microsoft AI
$10 million	175B parameter model, trained on 100TB of text data using 1024 GPUs for 1 month	"The Cost of Training a Large Language Model" by Brown et al.	2020	arXiv
$100 million	1 trillion parameter model, trained on 100PB of text data using 10,240 GPUs for 1 month	"Scaling Laws for Neural Language Models" by Chen et al.	2020	arXiv
$1 billion	10 trillion parameter model, trained on 10EB of text data using 100,240 GPUs for 1 month	"The Cost of Training a Large Language Model" by Webber	2023	Forbes
$1 billion	100 trillion parameters, 1 million GPUs	"The Cost of Large Language Models: A Scaling Law Analysis"	2022	Nature

For most entities, the relevant cost question will be “how much will it cost to use an existing platform,” including the cost of adapting (customizing) a generic model for a particular enterprise or entity. 

For example, costs of generating inferences when using "as a service" providers are based on the number of tokens. A generative AI token is a unit of text or code that is used by a generative AI model to generate new text or code. Generative AI tokens can be as small as a single character or as large as a word or phrase.

As a simplified rule, the number of tokens can be likened to the number of words in a generated response, for example. 

OpenAI offers a variety of generative AI models as a service through its API. Licensing costs range from $0.00025 to $0.006 per 1000 tokens for inference.

Google AI Platform offers a variety of generative AI models as a service through its Vertex AI platform. Licensing costs range from $0.005 to $0.02 per 1000 tokens for inference.

Microsoft Azure offers a variety of generative AI models as a service through its Azure Cognitive Services platform. Licensing costs range from $0.005 to $0.02 per 1000 tokens for inference.

Cost estimate	Key assumptions	Study name	Date of publication	Publishing venue
$0.006 per 1000 tokens	Inference on a single GPU	"Pricing Large Language Models as a Service"	2022	arXiv
$0.02 per 1000 tokens	Inference on multiple GPUs	"The Economics of Large Language Models"	2023	Medium
$0.05 per 1000 tokens	Inference on a TPU	"Comparing the Cost of Different Hardware Platforms for Large Language Models"	2023	arXiv
$0.02 per 1,000 tokens	GPT-3.5 model	"The Economics of Large Language Models"	2023	Medium
$0.10 per 1,000 tokens	GPT-4 (8K) model	"The Cost of Large Language Models: A Scaling Law Analysis"	2022	Nature
$0.40 per 1,000 tokens	GPT-4 (32K) model	"The Cost of Large Language Models: A Scaling Law Analysis"	2022	Nature

The point is that the cost of deploying generative AI for any particular business function is highly variable at the moment.

How Much Revenue Upside for Network APIs?

Many in the connectivity business have high hopes for network application program interface operations, which are intended to supply developers with a way to access and control the resources of a connectivity network, such as bandwidth, processing power and storage.

Forecast	Source	Date of Forecast
$20 billion by 2028	STL Partners	2023
$25 billion by 2028	Omdia	2022
$30 billion by 2028	ABI Research	2022
Over $100 billion by 2030	Nokia	2022
Multiple billions of dollars by 2030	Ericsson	2022
$5 billion by 2023	Informa Tech	2020
$10 billion by 2025	Juniper Research	2021

A network API could enable a developer to create a cloud-based application that stores and retrieves data from a database, for example. Likewise, an API could allow a developer to create a mobile application that allows users to check the status of their network connection or to manage their network settings.

Some believe APIS would allow developers to create IoT applications that enable users to control their smart home devices or to monitor the status of their industrial equipment.

Telcos hope APIs will allow them to charge developers for access to network resources, or allow them to create value-added services such as analytics tools, security features, and customer support.

Some believe telcos could use APIs, in partnership with cloud computing providers,  to offer developers a way to easily deploy and manage cloud-based applications.

The argument there seems to be that the APIs benefit developers by allowing them to avoid developing their own network infrastructure. Some might argue that most apps do not benefit so much from such integration. And most developers can find ways to work around needing such business relationships, and actually would prefer that option. 

That might be especially the case for apps that are expected to work globally, given the need for telcos to establish uniform APIs globally to reduce implementation hassles. 

The issue with such possible applications is that it is never so clear developers really need network APIs to do so. Industrial IoT sensors and apps arguably do not need network information to manage either smart home or industrial equipment. 

 In other cases, there is arguably value from having access to network information, but how much value is created? Will the additional value be viewed as worth payments by developers to telcos?

"10% of X" is Likely to Drive Investments in AI

Early in the funding process for big new potential markets, a common method for asserting firm market potential is to argue that X can get 10 percent of an existing revenue stream or market. 

So as monopoly telecom markets were about to be deregulated in the United States, competitive local exchange carriers essentially made the argument that they could grab 10 percent of the market for business voice and data services in a particular city from the incumbent. 

One might well make similar observations for artificial intelligence revenue upside for a few firms with very-large consumer or retail user footprints. Only the argument for artificial intelligence is likely to turn on boosting existing product revenues by 10 percent. 

Aside from suppliers of graphics processing units and cloud computing as a service suppliers being paid to house, train and provide inferences for  AI models, a few firms with consumer-facing or mass market operations could be winners when it comes to generating revenue from AI features. 

Note the ways AI already is used to support recommendation features for e-commerce apps, search and social media. Then think about a relative handful of firms that have huge user bases for popular apps, including both Apple and Microsoft. 

Then imagine new AI-based use cases offered to those users as a subscription add-on. Small percentages of very-large numbers also produce big numbers. 

Apple’s services segment, for example, is approaching a $100 billion annual run rate, accounting for nearly 26 percent of revenue with a 70 percent gross margin. Services are contributing 41.1 cents to each dollar of gross profit. 

Assume an AI feature costing about $3 a month is offered to Apple service customers. Assume 15 percent of users of Apple’s three billion active devices are persuaded to take that feature. Revenue would be in the $10 billion-plus range. 

Or consider an alternative where the prices for the existing subscription plans are boosted 50 cents per customer. That generates $5 billion or so, annually. 

Similar points might be made about Microsoft’s services business. Assume 10 percent of Microsoft 365 users decide to add the AI feature. Microsoft 365 has over 300 million commercial users, so 10 percent of that would be 30 million users. 

If each user pays a monthly subscription fee of $30, that would generate $900 million in monthly revenue, or $10.8 billion in annual revenue. 

Profit metrics would hinge on margins, of course, but an AI product might be assumed to have profit margins in the 50-percent range for those two firms. 

For Apple, conversational AI seems a likely area for exploration, given the amount of interaction users have with their phones and phone apps. Think Siri enhanced further using large language models. 

Google and Meta are other examples of firms with huge installed user bases; lots of data to use for training (assuming regulators allow it); existing revenue models that are enhanced by applied LLM and the ability to leverage applied LLM to improve the utility of their existing apps. 

All that before any of these firms are able to discover any truly-new products enabled by AI and LLM. 

If Google can increase ad rates by 10 percent due to AI, the revenue implications could be significant. In 2022, Google generated $257 billion in revenue, of which $209 billion was from advertising. A 10 percent increase in ad rates would therefore generate an additional $20.9 billion in revenue, or over eight percent growth.

In 2022, Meta generated $117.9 billion in revenue, of which $115.6 billion was from advertising. A 10 percent increase in ad rates would therefore generate an additional $11.56 billion in revenue, or nearly 10 percent growth.

One might not want to "get carried away" and assume that applied AI can drive a 10-percent across the board increase in revenue, but the numbers are suggestive.

Company	2022 Revenue ($B)	10% Increase ($B)
Apple	385.9	38.6
Amazon	470.1	47.0
Microsoft	192.9	19.3
Alphabet (Google)	257.6	25.8
Meta	117.9	11.8
Tesla	53.8	5.4
Berkshire Hathaway	276.1	27.6
JPMorgan Chase	125.8	12.6
Bank of America	115.9	11.6
Walmart	572.8	57.3
Exxon Mobil	286.1	28.6