Why One-Bit LLMs Matter

Though much of the investor concern about high levels of infrastructure investment to support generative artificial intelligence, it might be easy to miss simultaneous moves to create lower-computational cost implementations, such as small language models.

It might seem incongruous to talk about “one-bit large language models,” since the whole point of such models is to reduce computational intensity and cost: they might not be “large,” in other words, or if based on LLMs, execute on limited-resource platforms (thus saving energy, computational resources and cost). 

For example, some point to the power consumption advantages of one-bit models, compared to LLMs of any size. Some argue BitNet might have an order of magnitude more efficiency in that regard. 

source: bitnet 

Edge computing and internet of things use cases (sensors, for example), might use one-bit LLMs profitably on edge devices for tasks such as simple anomaly detection, sensor data analysis, and local decision-making.

Such real-time monitoring and analysis of environmental conditions, equipment performance, or security threats, with minimal latency, often do not require the full use of LLMs and infrastructure. 

.

For similar reasons, mobile applications running directly onboard edge devices (smartphones, for example), can support real-time translation, voice assistants, and text analysis without relying on remote resources.

Some augmented reality use cases could support real-time object recognition and information overlays while conserving battery life.

The point is that AI inference operations do not all require the cost, energy consumption or computational precision of full frontier AI models to produce value. 

Smaller models seemingly could be used in financial analysis operations for trend analysis or fraud detection without the full overhead of LLMs.

In industrial settings, much predictive maintenance value could be wrung from simple, lighter-weight models. 

The point is that lots of useful inference use cases might be possible without the speed or accuracy advantages of bigger LLMs.

The Roots of our Discontent

Political disagreements these days seem particularly intractable for all sorts of reasons, but among them are radically conflicting ideas about the possibility of truth; application of reason and existence of universal truth. 

For example, we might differ on whether history or human experience can be known in a universal way, or whether only particularisms matter. .

That, in turn, is a difference of opinion about whether “nature” is subjective or objective.  Some consider "truth" to be shaped by language, culture, and power structures, leading to a variety of perspectives, all equally valid. Others essentially argue that objective and universal truths do exist, independent of our personal beliefs. 

Postmodernists argue that language shapes our understanding of reality, hence the importance of words and “names.

Likewise, some might contend that objectivity is an illusion. Others argue objectivity is necessary and obtainable to a substantial degree. Adherents of the former tend to believe that all knowledge is influenced by personal, cultural, and historical contexts. Those who take the latter view believe there is a difference between shared “reality” and individual beliefs. 

Some value diversity, plurality, and the coexistence of multiple viewpoints above shared common values, unity the necessity of choice among competing ideas. 

In other words, many embrace “Postmodernism” while others operate within the intellectual context of the Enlightenment. 

The rancor, hatred, bad manners and absolutism we often see in politics is rooted in fundamentally different universes of thought. 

Thinker	Enlightenment, Postmodernist Thinkers	Key Contributions	Key Works
René Descartes	Enlightenment	Rationalism, methodic doubt, "Cogito, ergo sum"	Meditations on First Philosophy (1641)
John Locke	Enlightenment	Empiricism, natural rights, social contract	Two Treatises of Government (1689)
Voltaire	Enlightenment	Freedom of speech, religious tolerance	Candide (1759)
Montesquieu	Enlightenment	Separation of powers, political theory	The Spirit of the Laws (1748)
Jean-Jacques Rousseau	Enlightenment	General will, social contract theory	The Social Contract (1762)
Immanuel Kant	Enlightenment	Critique of reason, moral autonomy	Critique of Pure Reason (1781)
David Hume	Enlightenment	Empiricism, skepticism, critique of causation	An Enquiry Concerning Human Understanding (1748)
Adam Smith	Enlightenment	Free market economics, "invisible hand" theory	The Wealth of Nations (1776)
Mary Wollstonecraft	Enlightenment	Women's rights, gender equality	A Vindication of the Rights of Woman (1792)
Denis Diderot	Enlightenment	Editor of the Encyclopédie, promoter of knowledge	Encyclopédie (1751–1772)
Jacques Derrida	Postmodernism	Deconstruction, critique of language	Of Grammatology (1967)
Michel Foucault	Postmodernism	Power/knowledge, discourse, social institutions	Discipline and Punish (1975)
Jean-François Lyotard	Postmodernism	Critique of meta-narratives, postmodern condition	The Postmodern Condition (1979)
Jean Baudrillard	Postmodernism	Hyperreality, simulacra, media theory	Simulacra and Simulation (1981)
Richard Rorty	Postmodernism	Pragmatism, rejection of objective truth	Philosophy and the Mirror of Nature (1979)
Fredric Jameson	Postmodernism	Cultural theory, analysis of late capitalism	Postmodernism, or, The Cultural Logic of Late Capitalism (1991)
Julia Kristeva	Postmodernism	Psychoanalysis, semiotics, feminist theory	Powers of Horror (1980)

Net AI Sustainability Footprint Might be Lower, Even if Data Center Footprint is Higher

Nobody knows yet whether higher energy consumption to support artificial intelligence compute operations will ultimately be offset by lower consumption by industries, firms and consumers that use AI to manage their energy footprints. 

But it seems inevitable that AI operations are going to contribute to a significant boost in data center energy requirements. 

source: CNBC 

source: Goldman Sachs

 

source: Goldman Sachs

Still, some researchers expect net gains in energy efficiency and potential greenhouse gas emissions as AI is applied to industries. 

Study	Date	Publisher	Key Conclusions
Revisit the Environmental Impact of Artificial Intelligence	2023	Springer	AI increases energy use, but optimizing energy systems with AI can lead to net emissions reduction. The carbon footprint varies depending on electricity sources.
Will AI Accelerate or Delay the Race to Net-Zero Emissions?	2023	Nature	AI applications in sectors like transportation and energy can reduce global emissions by up to 10%, but data centers' energy use may offset gains if unchecked.
Potential of AI in Reducing Energy and Carbon Emissions	2022	Nature	AI can lower global energy consumption by up to 19% by 2050 through efficiency improvements but needs sustainable practices for data center energy use.
Tackling AI’s Climate Change Problem	2023	MIT Sloan Review	Large AI models contribute significantly to emissions; however, AI-driven optimizations can reduce industrial and operational carbon footprints.
Reduce Carbon and Costs with the Power of AI	2023	BCG	AI helps companies track and reduce emissions, potentially cutting global GHG emissions by 5.3 gigatons by 2030 while improving efficiency.

The point is that it is hard to say whether net increases or decreases in carbon or other footprint will result from wide scale AI usage. More computation means more energy consumption, which increases data center emissions footprint. 

But AI should ultimately enable lower energy consumption by a wide range of industries.

Big Tech Chases Big AI for Big Reasons

The capital-intensive battle to lead the generative artificial intelligence market is important enough for Alphabet, OpenAI, Meta and a few others, as it arguably creates a platform for the next era of computing and computing-based applications and business models. 

In fact, AI might be a somewhat-rare general-purpose technology that ultimately affects virtually the entire economy. 

But big hopes are accompanied by huge capex spending. 

source: Sherwood News 

And that big spending is chasing big potential. 

Generative AI is  "a really unusually large, maybe once-in-a-lifetime type of opportunity," according to Andy Jassy, Amazon CEO. 

"When we go through a curve like this, the risk of underinvesting is dramatically greater than the risk of overinvesting for us here,” said Sundar Pichai, Alphabet CEO. 

"This next generation of AI will reshape every software category and every business, including our own,” said Satya Nadella, Microsoft CEO. 

That won’t calm unease about the huge upsurge in capital spending to create and support large language models, but the huge potential is the driver. 

But the strategic implications also might be profound, as GenAI might be key to defending or extending digital real estate. In other words, GenAI possibly is about protecting platforms and ecosystems: keeping users engaged within the ecosystem. 

As with computing platform battles of the past (operating systems, hardware and application ecosystems),  generative AI is viewed as a key tool for protecting business moats around existing ecosystems. 

Beyond that, the heavy investments in generative and other forms of AI--though unsettling--are made not only to secure leadership of a platform that might drive the next era of computing and technology, but also might do so in large part because AI might reset expectations about the “cost of doing things,” as has tended to be the case for earlier general-purpose technologies. 

GPT	Key Innovation	Cost Reductions Produced
Steam Engine (18th century)	Mechanized power generation	- Reduced transportation costs (rail, shipping) - Lowered manufacturing costs through mechanization
Electricity (19th century)	Reliable, distributed power	- Reduced costs of lighting, heating, and powering factories - Enabled more efficient production processes
Computers (mid-20th century)	Automation of data processing	- Lowered costs of calculation, record-keeping, and data management - Improved productivity in administrative and technical tasks
Internet (late 20th century)	Communication and information access costs	- Drastically reduced communication costs - Enabled e-commerce, lowering retail and transaction costs
AI (21st century)	Intelligent data processing and automation	- Potential for reduced labor costs in repetitive or cognitive tasks - Lower costs in decision-making through better predictions and insights