Thursday, November 14, 2024

How Big is "GPU as a Service" Market?

It’s almost impossible to precisely quantify the addressable market for specialized “graphics processor unit as a service” providers such as CoreWeave, which specializes in providing GPU infrastructure to artificial intelligence developers.


CoreWeave might be considered an innovation in the area of high-performance computing for that reason, as it emphasizes GPUs rather than Central Processing Units. GPUs are considered essential for accelerating computationally intensive tasks like AI and machine learning. 

  

Traditional HPC often relies on CPUs for general-purpose computing. CoreWeave prioritizes GPUs, which are optimized for parallel processing. Right now, the market might be in low single-digit billions, but growing to possibly double-digit or triple-digit billions by the mid-2030s. 


Still, GPU as a service is a specialty within the cloud computing as a service business, and might remain two orders of magnitude smaller. 


Study Name

Date

Publisher

Key Estimates

GPU as a Service Market Size, Share & Growth Report

2023

Grand View Research

Global market valued at USD 3.35 billion in 2023, projected to grow at a CAGR of 21.6% from 2024 to 2030.

GPU as a Service Market Size, Growth

Forecast Analysis [2032]

Fortune Business Insights

Global market valued at USD 3.23 billion in 2023, projected to grow from USD 4.31 billion in 2024 to USD 49.84 billion by 2032, exhibiting a CAGR of 35.8%.

GPU as a Service Market Size & Share

Growth Forecast 2032

Global Market Insights

Global market valued at USD 6.4 billion in 2023, projected to grow at a CAGR of over 30% during 2024 to 2032.

GPU-as-a-Service Market Size, Trends & Outlook by 2033

2023

FMI - Future Market Insights

Global market valued at USD 3.91 billion in 2023, projected to grow at a CAGR of 40.8% between 2023 and 2033, totaling around USD 119.6 billion by 2032.

GPU as a Service Market Size & Share

Growth Analysis 2037

Research Nester

Global market valued at USD 4.34 billion in 2024, projected to exceed USD 95.07 billion by 2037, registering over 26.8% CAGR.


As with many other firms launching in new markets, perhaps the essential gamble is that X market will be huge and Y provider will get N percent of the market. So GPU as a service might be a subset of generative AI as a service. 


Study

Date

Publisher

Key Estimate

The State of AI 2023

June 2023

Anthropic

The global market for generative AI computing as a service is forecast to reach $15 billion by 2027.

Generative AI Market Outlook

September 2023

CoreWeave

The market for generative AI computing as a service is expected to grow at a CAGR of 35% from 2023 to 2028, reaching $18.2 billion in value by 2028.

Gartner Hype Cycle for AI 2023

July 2023

Gartner

Generative AI computing as a service is projected to have a market size of $14.5 billion by 2026.

Generative AI Market Size, Share

2023

Fortune Business Insights

Global market valued at USD 43.87 billion in 2023, projected to reach USD 967.65 billion by 2032, with a CAGR of 39.6%.

Generative AI Market Size To Hit USD 803.90 Bn By 2033

2023

Precedence Research

Global market size was USD 17.65 billion in 2023, expected to reach USD 803.90 billion by 2033, expanding at a CAGR of 46.5%.

Generative AI Market Size And Share

2024

Grand View Research

Global market led by North America, with a revenue share of 40.8% in 2024. Software segment dominates with a 64.2% share.

Generative AI Market Size, Trends, & Technology Roadmap

2023

MarketsandMarkets

Focuses on technology trends and roadmap, including advancements in transformer models and multimodal data.


BEAD Has Not Connected a Single Home for Broadband Interenet Access, After 3 Years

As an observer of the follies of government ineffectiveness, we note that the U.S. Broadband Equity, Access, and Deployment (BEAD) Program was enacted in November 2021 and allocated $42.45 billion to the National Telecommunications and Information Administration (NTIA) to work on the “digital divide” by facilitating access to affordable, reliable, high-speed internet throughout the United States, with a particular focus on communities of color, lower-income areas, and rural areas.


As of November 2024 not a single dollar has been spent in support of the program, for a variety of perhaps simple bureaucratic reasons. 


The program's implementation has been slowed by a complex approval process. States were required to submit Initial Proposals outlining their broadband deployment objectives. 


As of June 2024, only 15 states and territories had received approval. States have 365 days after approval to select projects and submit a final list to the National Telecommunications and Information Administration (NTIA) for review.


As you might imagine, all that has caused delays. 


Also, the NTIA had to wait for the FCC to release an updated national broadband map before allocating funds to states.


There have been other issues as well. The Virginia proposal has been delayed over affordability requirements and rate-setting. The program also has provisions related to accessibility, union participation, and climate impact, which have not helped speed things up. 

.

High interest rates and tight financing conditions have made it more difficult for broadband providers to secure funding for projects, even when approved. 


The result is that funding isn't expected to start reaching projects until 2025 at the earliest. .


Some might argue the program’s design was not optimal for rapid funds disbursal.


Some might argue it would have been far simpler to route money directly to Internet Service Providers (ISPs) based on their proven ability to deploy networks quickly in underserved areas. 


Competitive bidding could have been used. The program could have specified uniform national standards for broadband deployment to replace the current patchwork of state-specific and local requirements. 


The program could have been “technology neutral” instead of mandating use of some platforms over others, and might have used a simpler application and reporting system, in place of the cumbersome existing framework. 


The larger point is that the law arguably was poorly designed, in terms of its implementation framework. The fastest way to create infrastructure might have been to give buying power to potential customers, as did the Affordable Connectivity Program, or make direct grants to ISPs in position to build almost immediately. 


And since rural connectivity was deemed important, it might have been wise not to exclude satellite access platforms. 


It was a good impulse to “want to help solve this problem.” But intentions also must be matched by policy frameworks that are efficient and effective, getting facilities depl;oyed to those who need them fast. BEAD has not done so. 


Tuesday, November 12, 2024

ISP Marginal Cost Does Not Drive Consumer Prices

As the U.S. Federal Communications Commission opens an inquiry into ISP data caps, some are going to argue that such data caps are unnecessary or a form of consumer price gouging, as the marginal cost of supplying the next unit of consumption is rather low. 


Though perhaps compelling, the marginal cost of supplying the next unit of consumption is not the best way of evaluating the reasonableness of such policies.  


If U.S. ISPs were able to meet customer data demand during the COVID-19 pandemic without apparent quality issues, it suggests several things about their capacity planning and network infrastructure, and much less about the reasonableness of marginal cost pricing.


In fact, the ability to survive the unexpected Covid data demand was the result of deliberate overprovisioning by ISPs; some amount of scalability (the ability to increase supply rapidly); use of architectural tools such as content delivery networks and traffic management and prior investments in capacity as well. 


Looking at U.S. internet service providers and their investment in fixed network access and transport capacity between 2000 and 2020 (when Covid hit), one sees an increasing amount of investment, with magnitudes growing steadily since 2004, and doubling be tween 2000 and 2016.


Year

Investment (Billion $)

2000

21.5

2001

24.8

2002

20.6

2003

19.4

2004

21.7

2005

23.1

2006

24.5

2007

26.2

2008

27.8

2009

25.3

2010

28.6

2011

30.9

2012

33.2

2013

35.5

2014

37.8

2015

40.1

2016

42.4

2017

44.7

2018

47

2019

49.3

2020

51.6


At the retail level, that has translated into typical speed increases from 500 kbps in 2000 up to 1,000 Mbps in 2020, when the Covid pandemic hit. Transport capacity obviously increased as well to support retail end user requirements. Compared to 2000, retail end user capacity grew by four orders of magnitude by 2020. 


Year

Capacity (Mbps)

2000

0.5

2002

1.5

2004

3

2006

6

2008

10

2010

15

2012

25

2014

50

2016

100

2018

250

2020

1000


But that arguably misses the larger point: internet access service costs are not contingent on marginal costs, but include sunk and fixed costs, which are, by definition, independent of marginal costs. 


Retail pricing based strictly on marginal cost can be dangerous for firms, especially in industries with high fixed or sunk costs, such as telecommunications service providers, utilities or manufacturing firms.


The reason is that marginal cost pricing is not designed to recover fixed and sunk costs that are necessary to create and deliver the service. 


Sunk costs refer to irreversible expenditures already made, such as infrastructure investments. Fixed costs are recurring expenses that don't change with output volume (maintenance, administration, and system upgrades).


Marginal cost pricing only covers the cost of producing one additional unit of service (delivering one more megabyte of data or manufacturing one more product), but it does not account for fixed or sunk costs. 


Over time, if a firm prices its products or services at or near marginal cost, it won’t generate enough revenue to cover its infrastructure investments, leading to financial losses and unsustainable operations.


Marginal cost pricing, especially in industries with high infrastructure investment, often results in razor-thin margins. Firms need to generate profits beyond just covering marginal costs to reinvest in growth, innovation, and future infrastructure improvements. 


In other words, ISPs cannot price at marginal cost, as they will go out of business, as such pricing leaves no funds for innovation, maintenance, network upgrades and geographic expansion to underserved or unserved areas, for example. 


Marginal cost pricing can spark price wars and lead customers to devalue the product or service, on the assumption that such a low-cost product must be a commodity rather than a high-value offering. Again, marginal cost pricing only covers the incremental cost of producing the next unit, not the full cost of the platform supplying the product. 


What Would Artificial General Intelligence be Capable of Doing?

Dario Amodei, AnthropicCEO, has some useful observations about the development of what he refers to as  powerful AI (and which he suggests lots of people call  Artificial General Intelligence (AGI). As a one-sentence summary, Amodei suggests powerful AI has the capabilities of  a “country of geniuses in a datacenter”.


It would be “smarter than a Nobel Prize winner across most relevant fields – biology, programming, math, engineering, writing and so forth,” he says.” This means it can prove unsolved mathematical theorems; write extremely good novels; write difficult codebases from scratch.”


It would have “all the “interfaces” available to a human working virtually, including text, audio, video, mouse and keyboard control, and internet access. 


It could engage in any actions, communications, or remote operations enabled by this interface, including taking actions on the internet, taking or giving directions to humans, ordering materials, directing experiments, watching videos, making videos, and so on. It does all of these tasks with, again, a skill exceeding that of the most capable humans in the world.


It would not just passively answer questions. “Instead, it can be given tasks that take hours, days, or weeks to complete, and then goes off and does those tasks autonomously, in the way a smart employee would, asking for clarification as necessary,” Amodei argues.


The resources used to train the model can be repurposed to run millions of instances of it and the model can absorb information and generate actions at roughly 10 times to 100 times human speed. 


However, It may be limited by the response time of the physical world or of software it interacts with. There could well be energy availability constraints; hardware and software cost issues or deficiencies in mimicking human sensory or “fuzzy” reasoning capabilities. 


Also, since such a system would interact with the real physical world, it would remain within the constraints of the physical world. “Cells and animals run at a fixed speed so experiments on them take a certain amount of time which may be irreducible,” he notes. 


Also, “sometimes raw data is lacking and in its absence more intelligence does not help,” he says. In addition, “some things are inherently unpredictable or chaotic and even the most powerful AI cannot predict or untangle them substantially better than a human or a computer today.”


Ethically and legally, “many things cannot be done without breaking laws, harming humans, or messing up society,” he notes. “An aligned AI would not want to do these things.”


The points are that powerful AI (or AGI, as some would call it) will face constraints from the physical world and ethical, moral and legal constraints that could limit its application in many instances as a means of affecting output. 


Analyzing is one thing; the ability to translate that knowledge into outcomes is more tricky. Biological systems offer a case in point. “it’s very hard to isolate the effect of any part of this complex system, and even harder to intervene on the system in a precise or predictable way,” Amodei says. 


Directv-Dish Merger Fails

Directv’’s termination of its deal to merge with EchoStar, apparently because EchoStar bondholders did not approve, means EchoStar continue...