Have LLMs Hit an Improvement Wall, or Not?

Some might argue it is way too early to worry about a slowdown in large language model performance improvement rates. But some already voice concern, as OpenAI appears to see a slowdown in rates of  improvement. 

Gemini rates of improvement might also have slowed, and Anthropic might be facing similar challenges.   

To be sure, generative artificial intelligence language model size has so far shown a correlation with performance. More inputs--such as larger model size--have resulted in more output. 

source: AWS 

In other words, scaling laws exist for LLMs, as they do for machine learning and other aspects of AI. The issue is how long the current rates of improvement can last. 

Scaling laws describe the relationships between a model’s performance and its key attributes: size (number of parameters), training data volume, and computational resources. 

Scaling laws also imply that there are limits. At some point, the gains from improving inputs do not produce proportional output gains. So the issue is how soon LLMs might start to hit scaling law limits. 

Aside from the cost implications of ever-larger model sizes, there is the related matter of the availability of training data. At some point, as with natural resources (oil, natural gas, copper, gold, silver, rare earth minerals), LLMs will have used all the accessible, low-cost data. 

Other data exists, of course, but might be expensive to ingest. Think about the Library of Congress collection, for example. It is theoretically available, but the cost and time to “mine” it is likely more than any single LLM can afford. Nor is it likely any would-be provider could create (digitize) and supply such resources fast and affordably. 

source: Epoch AI 

Consider the cost to digitize and make available the U.S. Library of Congress collection. 

Digitization and metadata creation might cost $1 billion to $2 billion total, spread over five to 10 years, including the cost of digitizing and formatting:

• Textual Content: $50 million - $500 million.

• Photographic and Image Content: $75 million - $300 million.

• Audio-Visual Content: $30 million - $120 million.

• Metadata Creation and Tagging: Approximately 20-30% of total digitization costs ($200 million - $600 million).

I think the point is that with the speed of large language model updates (virtually continually in some cases, with planned model updates at least annually), no single LLM provider could afford to pay that much, and wait that long, for results. 

Then there are the additional costs of data storage, maintenance, and infrastructure, which could range from $20 million to $50 million annually. Labor costs might be in the range of $10 million to  $20 million annually as well.

Assuming the owner of the asset would want to license access to many other types of firms, sales, marketing, and customer support could add another $5 million to $10 million in annual costs.

The point is that even if an LLM wanted to spend $1 billion to $2 billion to gain access to the knowledge embedded in the U.S. Library of Congress, perhaps no LLM owner could afford to wait five years to a decade to derive the benefits. 

And that is just one example of a scaling law limit. The other issues are energy consumption; computing intensity and model parameter size. At some point, diminishing returns from additional investment would occur.

BEAD Has Not Connected a Single Home for Broadband Interenet Access, After 3 Years

As an observer of the follies of government ineffectiveness, we note that the U.S. Broadband Equity, Access, and Deployment (BEAD) Program was enacted in November 2021 and allocated $42.45 billion to the National Telecommunications and Information Administration (NTIA) to work on the “digital divide” by facilitating access to affordable, reliable, high-speed internet throughout the United States, with a particular focus on communities of color, lower-income areas, and rural areas.

As of November 2024 not a single dollar has been spent in support of the program, for a variety of perhaps simple bureaucratic reasons. 

The program's implementation has been slowed by a complex approval process. States were required to submit Initial Proposals outlining their broadband deployment objectives. 

As of June 2024, only 15 states and territories had received approval. States have 365 days after approval to select projects and submit a final list to the National Telecommunications and Information Administration (NTIA) for review.

As you might imagine, all that has caused delays. 

Also, the NTIA had to wait for the FCC to release an updated national broadband map before allocating funds to states.

There have been other issues as well. The Virginia proposal has been delayed over affordability requirements and rate-setting. The program also has provisions related to accessibility, union participation, and climate impact, which have not helped speed things up. 

.

High interest rates and tight financing conditions have made it more difficult for broadband providers to secure funding for projects, even when approved. 

The result is that funding isn't expected to start reaching projects until 2025 at the earliest. .

Some might argue the program’s design was not optimal for rapid funds disbursal.

Some might argue it would have been far simpler to route money directly to Internet Service Providers (ISPs) based on their proven ability to deploy networks quickly in underserved areas. 

Competitive bidding could have been used. The program could have specified uniform national standards for broadband deployment to replace the current patchwork of state-specific and local requirements. 

The program could have been “technology neutral” instead of mandating use of some platforms over others, and might have used a simpler application and reporting system, in place of the cumbersome existing framework. 

The larger point is that the law arguably was poorly designed, in terms of its implementation framework. The fastest way to create infrastructure might have been to give buying power to potential customers, as did the Affordable Connectivity Program, or make direct grants to ISPs in position to build almost immediately. 

And since rural connectivity was deemed important, it might have been wise not to exclude satellite access platforms. 

It was a good impulse to “want to help solve this problem.” But intentions also must be matched by policy frameworks that are efficient and effective, getting facilities depl;oyed to those who need them fast. BEAD has not done so. 

ISP Marginal Cost Does Not Drive Consumer Prices

As the U.S. Federal Communications Commission opens an inquiry into ISP data caps, some are going to argue that such data caps are unnecessary or a form of consumer price gouging, as the marginal cost of supplying the next unit of consumption is rather low. 

Though perhaps compelling, the marginal cost of supplying the next unit of consumption is not the best way of evaluating the reasonableness of such policies.  

If U.S. ISPs were able to meet customer data demand during the COVID-19 pandemic without apparent quality issues, it suggests several things about their capacity planning and network infrastructure, and much less about the reasonableness of marginal cost pricing.

In fact, the ability to survive the unexpected Covid data demand was the result of deliberate overprovisioning by ISPs; some amount of scalability (the ability to increase supply rapidly); use of architectural tools such as content delivery networks and traffic management and prior investments in capacity as well. 

Looking at U.S. internet service providers and their investment in fixed network access and transport capacity between 2000 and 2020 (when Covid hit), one sees an increasing amount of investment, with magnitudes growing steadily since 2004, and doubling be tween 2000 and 2016.

Year	Investment (Billion $)
2000	21.5
2001	24.8
2002	20.6
2003	19.4
2004	21.7
2005	23.1
2006	24.5
2007	26.2
2008	27.8
2009	25.3
2010	28.6
2011	30.9
2012	33.2
2013	35.5
2014	37.8
2015	40.1
2016	42.4
2017	44.7
2018	47
2019	49.3
2020	51.6

At the retail level, that has translated into typical speed increases from 500 kbps in 2000 up to 1,000 Mbps in 2020, when the Covid pandemic hit. Transport capacity obviously increased as well to support retail end user requirements. Compared to 2000, retail end user capacity grew by four orders of magnitude by 2020. 

Year	Capacity (Mbps)
2000	0.5
2002	1.5
2004	3
2006	6
2008	10
2010	15
2012	25
2014	50
2016	100
2018	250
2020	1000

But that arguably misses the larger point: internet access service costs are not contingent on marginal costs, but include sunk and fixed costs, which are, by definition, independent of marginal costs. 

Retail pricing based strictly on marginal cost can be dangerous for firms, especially in industries with high fixed or sunk costs, such as telecommunications service providers, utilities or manufacturing firms.

The reason is that marginal cost pricing is not designed to recover fixed and sunk costs that are necessary to create and deliver the service. 

Sunk costs refer to irreversible expenditures already made, such as infrastructure investments. Fixed costs are recurring expenses that don't change with output volume (maintenance, administration, and system upgrades).

Marginal cost pricing only covers the cost of producing one additional unit of service (delivering one more megabyte of data or manufacturing one more product), but it does not account for fixed or sunk costs. 

Over time, if a firm prices its products or services at or near marginal cost, it won’t generate enough revenue to cover its infrastructure investments, leading to financial losses and unsustainable operations.

Marginal cost pricing, especially in industries with high infrastructure investment, often results in razor-thin margins. Firms need to generate profits beyond just covering marginal costs to reinvest in growth, innovation, and future infrastructure improvements. 

In other words, ISPs cannot price at marginal cost, as they will go out of business, as such pricing leaves no funds for innovation, maintenance, network upgrades and geographic expansion to underserved or unserved areas, for example. 

Marginal cost pricing can spark price wars and lead customers to devalue the product or service, on the assumption that such a low-cost product must be a commodity rather than a high-value offering. Again, marginal cost pricing only covers the incremental cost of producing the next unit, not the full cost of the platform supplying the product. 

What Would Artificial General Intelligence be Capable of Doing?

Dario Amodei, AnthropicCEO, has some useful observations about the development of what he refers to as  powerful AI (and which he suggests lots of people call  Artificial General Intelligence (AGI). As a one-sentence summary, Amodei suggests powerful AI has the capabilities of  a “country of geniuses in a datacenter”.

It would be “smarter than a Nobel Prize winner across most relevant fields – biology, programming, math, engineering, writing and so forth,” he says.” This means it can prove unsolved mathematical theorems; write extremely good novels; write difficult codebases from scratch.”

It would have “all the “interfaces” available to a human working virtually, including text, audio, video, mouse and keyboard control, and internet access. 

It could engage in any actions, communications, or remote operations enabled by this interface, including taking actions on the internet, taking or giving directions to humans, ordering materials, directing experiments, watching videos, making videos, and so on. It does all of these tasks with, again, a skill exceeding that of the most capable humans in the world.

It would not just passively answer questions. “Instead, it can be given tasks that take hours, days, or weeks to complete, and then goes off and does those tasks autonomously, in the way a smart employee would, asking for clarification as necessary,” Amodei argues.

The resources used to train the model can be repurposed to run millions of instances of it and the model can absorb information and generate actions at roughly 10 times to 100 times human speed. 

However, It may be limited by the response time of the physical world or of software it interacts with. There could well be energy availability constraints; hardware and software cost issues or deficiencies in mimicking human sensory or “fuzzy” reasoning capabilities. 

Also, since such a system would interact with the real physical world, it would remain within the constraints of the physical world. “Cells and animals run at a fixed speed so experiments on them take a certain amount of time which may be irreducible,” he notes. 

Also, “sometimes raw data is lacking and in its absence more intelligence does not help,” he says. In addition, “some things are inherently unpredictable or chaotic and even the most powerful AI cannot predict or untangle them substantially better than a human or a computer today.”

Ethically and legally, “many things cannot be done without breaking laws, harming humans, or messing up society,” he notes. “An aligned AI would not want to do these things.”

The points are that powerful AI (or AGI, as some would call it) will face constraints from the physical world and ethical, moral and legal constraints that could limit its application in many instances as a means of affecting output. 

Analyzing is one thing; the ability to translate that knowledge into outcomes is more tricky. Biological systems offer a case in point. “it’s very hard to isolate the effect of any part of this complex system, and even harder to intervene on the system in a precise or predictable way,” Amodei says. 

Build Versus Buy is the Issue for Verizon Acquisition of Frontier

Verizon’s rationale for acquiring Frontier Communications, at a cost of  $20 billion, is partly strategic, partly tactical. Verizon and most other telcos face growth issues, and Frontier adds fixed network footprint, existing fiber access and other revenues, plant and equipment. 

Consider how Verizon’s fixed network compares with major competitors. 

ISP	Total Fixed Network Homes, Small Businesses Passed
AT&T	~70 million
Comcast	~60 million
Charter	~50 million
Verizon	~36 million

Verizon has the smallest fixed network footprint, so all other things being equal, the smallest share of the total home broadband market nationwide. If home broadband becomes the next big battleground for AT&T and Verizon revenue growth (on the assumption mobility market share is being taken by cable companies and T-Mobile from Verizon and At&T), then Verizon has to do something about its footprint, as it simply does not have enough ability to compete for customers across most of the Untied States for home broadband using fixed network platforms. 

And though Frontier’s customer base and geographies are heavily rural and suburban, compared to Verizon, that is characteristic of most “at scale” telco assets that might be acquisition targets for Verizon. 

Oddly enough, Verizon sold many of the assets it now plans to reacquire. In 2010, for example, Frontier Communications purchased rural operations in 27 states from Verizon, including more than seven million local access lines and 4.8 million customer lines. 

Those assets were located in Arizona, California, Idaho, Illinois, Indiana, Michigan, Nevada, North Carolina, Ohio, Oregon, South Carolina, Washington, Wisconsin and West Virginia, shown in the map below as brown areas. 

Then in 2015, Verizon sold additional assets in three states (California, Texas, Florida) to Frontier. Those assets included 3.7 million voice connections; 2.2 million broadband internet access customers, including about 1.6 million fiber optic access accounts and approximately 1.2 million video entertainment customers.

source: Verizon, Tampa Bay Business Journal 

Now Verizon is buying back the bulk of those assets. There are a couple of notable angles. First, Verizon back in the first decade of the 21st century was raising cash and shedding rural assets that did not fit well with its FiOS fiber-to-home strategy. In the intervening years, Frontier has rebuilt millions of those lines with FTTH platforms.

Also, with fixed network growth stagnant, acquiring Frontier now provides a way to boost Verizon’s own revenue growth.

For example, the acquisition adds around 7.2 million additional and already-in-place fiber passings. Verizon already has 18 million fiber passings,increasing  the fiber footprint to reach nearly 25 million homes and small businesses​. In other words, the acquisition increases current fiber passings by about 29 percent. 

There also are some millions of additional copper passings that might never be upgraded to fiber, but can generate revenue (copper internet access or voice or alarm services, for example). Today, Frontier generates about 44 percent of its total revenue from copper access facilities, some of which will eventually be upgraded to fiber, but perhaps not all. 

Frontier already has plans to add some three million more fiber passes by about 2026, for example, bringing its total fiber passings up to about 10 million. 

That suggests Frontier’s total network might pass 16 million to 17 million homes and small businesses. But assume Verizon’s primary interest is about 10 million new fiber passings. 

Frontier has estimated its cost per passing for those locations as between $1000 and $1100. Assume Verizon can also achieve that. Assume the full value of the Frontier acquisition ($20 billion) was instead spent on building new fiber plant outside of region, at a blended cost of #1050 per passing. 

That implies Verizon might be able to build perhaps 20 million new FTTH passings as an alternative, assuming all other costs (permits, pole leases or conduit access) were not material. But those costs exist, and might represent about 25 percent higher costs. 

So adjust the cost per passing for outside-of-region builds to a range of $1300 to $1400. Use a blended average of $1350. Under those circumstances, Verizon might hope to build less than 15 million locations. 

And in that scenario Verizon would not acquire the existing cash flow or other property. So one might broadly say the alternative is spending $20 billion to build up to 15 million new fiber passings over time, versus acquiring 10 million fiber passings in about a year, plus the revenue from seven million passings (with take rates around 40 percent of passings). 

Critics will say Verizon could do something else with $20 billion, to be sure, including not spending the money and not increasing its debt. But some of those same critics will decry Verizon’s lack of revenue growth as well. 

But Verizon also sees economies of scale, creating projected cost synergies of around $500 million annually by the third year. The acquisition is expected to be accretive to Verizon’s revenue, EBITDA and cash flow shortly after closing, if adding to Verizon’s debt load. 

Even if the majority of Verizon revenue is generated by mobility services, fixed network services still contribute a quarter or so of total revenues, and also are part of the cost structure for mobility services. To garner a higher share of moderate- to high-speed home broadband (perhaps in the 300 Mbps to 500 Mbps range for “moderate speed” and gigabit and multi-gigabit services as “high speed”), Verizon has to increase its footprint nationwide or regionally, outside its current fixed network footprint. 

One might make the argument that Verizon should not bother expanding its fixed network footprint, but home broadband is a relative growth area (at least in terms of growing market share). The ability to take market share from the leading cable TV firms (using fixed wireless for lower speed and fiber for higher speed accounts) clearly exists, but only if Verizon can acquire or build additional footprint outside its present core region.

And while it is possible for Verizon to cherry pick its “do it yourself” home broadband footprint outside of region, that approach does not offer immediate scale. Assuming all else works out, it might take Verizon five years to add an additional seven million or so FTTH passings outside of the current region. 

There is a value to revenue Verizon can add from day one, rather than building gradually over five years.