Which Edge Will Dominate AI Processing?

Edge computing advantages generally are said to revolve around use cases requiring low-latency response, and the same is generally true for artificial intelligence processing as well. 

Some use cases requiring low-latency response will be best executed “on the device” rather than at a remote data center, and often on the device rather than at an “edge” data center. 

That might especially be true as some estimate consumer apps will represent as much as 70 percent of total generative artificial intelligence compute requirements. 

So does that mean we see graphics processor units on most smartphones? Probably not, even if GPU prices fall over time. We’ll likely see lots of accelerator chips, though, including more use of tensor processing units or neural processing units and application specific integrated circuits, for reasons of cost.  

The general principle is always that the cost of computing facilities increases, while efficiency decreases, as computing moves to the network edge. In other words, centralized computing tends to be the most efficient while computing at the edge--which necessarily involves huge numbers of processors--is necessarily more capital intensive. 

For most physical networks, as much as 80 percent of cost is at the network edges. 

Beyond content delivery, many have struggled to define the business model for edge computing, however. Either from an end user experience perspective or an edge computing supplier perspective. 

Sheer infrastructure cost remains an issue, as do compelling use cases. Beyond those issues, there arguably are standardization and interoperability issues similar to multi-cloud, complexity concerns and fragmented or sub-scale revenue opportunities. 

In many cases, “edge” use cases also make more sense for “on the device” processing, something we already see with image processing, speech-to-text and real-time language translation. 

To be sure, battery drain, processors and memory (and therefore cost) will be issues, initially. 

On-Device Use Case	Benefits	Considerations
Image Processing (Basic)	Privacy: Processes images locally without sending data to servers.  Offline Functionality: Works even without internet connection. - Low Latency: Real-time effects and filters.	Limited Model Complexity: Simpler tasks like noise reduction or basic filters work well on-device. - Battery Drain: Complex processing can drain battery life.
Voice Interface (Simple Commands)	Privacy: Voice data stays on device for sensitive commands. - Low Latency: Faster response for basic commands (e.g., smart home controls).	Limited Vocabulary and Understanding: On-device models may not handle complex requests. - Limited Customization: Pre-trained models offer less user personalization.
Language Translation (Simple Phrases)	Offline Functionality: Translates basic phrases even without internet. - Privacy: Sensitive conversations remain on device.	Limited Languages and Accuracy: Fewer languages and potentially lower accuracy compared to cloud-based models.  Storage Requirements: Larger models for complex languages might not fit on all devices.
Message Autocomplete	Privacy: Keeps message content on device.  Offline Functionality: Auto-completes even without internet.	Limited Context Understanding: Relying solely on local message history might limit accuracy. - Personalized Experience: On-device models may not adapt to individual writing styles as well.
Music Playlist Generation (Offline)	Offline Functionality: Creates playlists based on downloaded music library. - Privacy: No need to send music preferences to the cloud.	Limited Music Library Size: On-device storage limits playlist diversity. - Static Recommendations: Playlists may not adapt to changing user tastes as effectively.
Maps Features (Limited Functionality)	Offline Functionality: Access basic maps and navigation even without internet. - Privacy: No user location data sent to servers for basic features.	Limited Features: Offline functionality may lack real-time traffic updates or detailed points of interest. - Outdated Maps: Requires periodic updates downloaded to the device.

Remote processing (edge or remote) will tend to favor use cases including augmented reality; advanced image processing; personalized content recommendations or predictive maintenance. 

Latency requirements for these and other apps will tend to drive the need for edge processing.

Connectivity Service Provider Revenue Growth to 2025 is About What You'd Expect

Connectivity provider revenue growth between 2024 and 2025 should be about as most would expect, with a global average of about three percent per year, with slower growth possibly in the one-percent range in North America and Europe, with higher growth in the four percent to 4.5 percent range in Asia-Pacific and Latin America, according to S&P Global Ratings.

source: S&P Global Ratings 

To be sure, executives might wish for faster growth rates, but growth rates in mature markets, especially in industries with “utility-type” characteristics, often are slow. 

Industry	Growth Rate (%)	Source
Telecom	3.2%	Deloitte
Passenger Airlines	7.4%	IATA
Seaborne Goods Transport	3.1%	World Maritime News
Retailing	4.1%	Statista
Retail Consumer Banking	2.7%	PwC
Electricity	4.8%	IEA
Natural Gas	2.1%	IEA
Wastewater Services	3.4%	Global Water Intelligence

Though growth rates in various utility-style industries vary over time, none of these industries are early in their adoption curves, when growth is much faster.

source: Corporate Finance Institute 

As the ILC applies to the connectivity service provider industry, while generally mature, segments within the industry that might be likened to “products” can be at different phases of their life cycles. 

The fixed network voice portion of the industry clearly is declining; the home broadband segment growing. The mobile industry routinely introduced a new generation of mobile services every decade, while sunsetting the older legacy generations as that happens. 

Within the mobile industry, growth is fastest in Asia-Pacific and Latin America; slowest in Europe. 

Industry	2000-2005	2005-2010	2010-2015	2015-2020	2020-2023	Source
Telecom	6.5	4.1	2.8	2.3	3.2	Statista
Electricity	3.8	4.2	3.6	2.4	4.8	IEA
Railroad	4.2	5.1	3.8	2.1	2.7	Statista
Aviation	5.8	5.3	4.2	4.6	7.4	IATA

If one looks at computing devices, “personal computing” clearly has moved through a personal computer stage to a mobile phone stage to a smartphone stage. 

The Economist

At a high level, only fixed network voice is clearly in its “decline” phase. Mobile service is expected to continue replacing its lead platform every decade.

Service	Product Life Cycle Stage	Trends
Fixed Network Telecom Service (e.g., Landlines)	Decline	Facing declining use due to substitution by mobile services and internet communication options (e.g., VoIP).  Limited market growth potential.
Mobile Service	Maturity	Widespread adoption and high market penetration.  Focus on differentiation through network coverage, data plans, and value-added services.  Potential for continued growth in emerging markets.
Home Broadband	Maturity/Growth	High market penetration, particularly in developed economies.  Growth potential in developing economies and through offering higher speeds and bundled services.
Virtual Private Networks (VPNs)	Maturity	Established technology with widespread adoption by businesses.  Potential growth in emerging markets and with increasing security concerns.
Managed Security Services	Growth	Growing demand for cybersecurity expertise and protection against evolving threats.
Data Center Services	Growth	Rising demand for cloud computing and data storage solutions.  Shift from on-premise infrastructure to cloud-based solutions.

GenAI Consumes Lots of Energy, But What is Net Impact?

Much has been made of a recent study suggesting ChatGPT operations consume prodigious amounts of electricity, as exemplified by the claim that ChatGPT operations consume 17,000 times more energy than a typical household.  

No question, cloud computing requires remote data centers, and data centers are big consumers of energy. In the United States, data centers now account for about four per cent of electricity consumption, and that figure is expected to climb to six per cent by 2026, according to reporting by The New Yorker. 

But that is not the whole story. Data centers, apps and cloud computing are used to design, manufacture and use all sorts of products that might also decrease energy consumption. Some would argue, for example, that there is a net energy reduction when people use ridesharing instead of driving their personal vehicles. 

Study Title	Location	Key Findings
Life Cycle Energy Consumption of Ride-hailing Services: A Case Study of Taxi and Ride-Hailing Trips in California (2020)	California, USA	- Ridesharing resulted in 11-23% lower energy consumption compared to private vehicles, primarily due to higher vehicle occupancy.
The Energy and Environmental Impacts of Shared Autonomous Vehicles Under Different Pricing Strategies (2023)	N/A (Hypothetical Scenario)	- Shared Autonomous Vehicles (SAVs) with high occupancy rates have the potential for significant energy savings compared to private vehicles.
Future Transportation: The Social, Economic, and Environmental Impacts of Ridesourcing Services: A Literature Review (2022)	N/A (Literature Review)	- Ridesharing can potentially reduce vehicle miles traveled (VMT) compared to private vehicles, leading to lower energy consumption. - However, concerns exist regarding: * Increased empty miles driven by rideshare vehicles searching for passengers. * Potential substitution of public transportation trips with ridesharing, negating some environmental benefits.
Life-Cycle Energy Assessment of Personal Mobility in China (2020)	China	Ridesharing with three passengers can reduce energy consumption.
The Energy and Environmental Impacts of Shared Autonomous Vehicles (2021)	N/A	Shared autonomous vehicles can reduce energy consumption.
Empty Urban Mobility: Exploring the Energy Efficiency of Ridesharing and Microtransit (2019)	Europe	High-occupancy ridesharing reduces energy consumption, compared to use of private vehicles, but we also must account for energy consumed when not transporting passengers.

So far as I can determine, nobody has really tried to model the net energy impact of generative artificial intelligence, data centers or cloud computing, where the energy footprint of GenAI, data centers or cloud computing is compared with the possible net reductions throughout an economy if the app outputs are used to reduce energy consumption in products using cloud computing, data center and GenAI  outputs. 

Study Title	Key Findings
Green Cloud? An Empirical Analysis of Cloud Computing and Energy Efficiency (2020)	Cloud computing adoption improves user-side energy efficiency, particularly after 2006. - SaaS (Software-as-a-Service) contributes most significantly to both electric and non-electric energy savings. IaaS (Infrastructure-as-a-Service) primarily benefits industries with high internal IT hardware usage.
The Internet: Explaining ICT Service Demand in Light of Cloud Computing Technologies (2015)	Cloud computing can lead to increased energy consumption in data centers. Potential for energy savings in other sectors due to: * Reduced need for personal computing devices.  Improved resource utilization and consolidation.
Decarbonizing the Cloud: How Cloud Computing Can Enable a Sustainable Future (McKinsey & Company, 2020)	Cloud adoption powered by renewables can significantly reduce emissions compared to on-premise IT infrastructure. Cloud enables the development of various sustainability solutions (smart grids, remote work).
Cloud Computing: Lowering the Carbon Footprint of Manufacturing SMEs? (2013)	Case studies of manufacturing SMEs shifting to cloud-based solutions.

But some related research suggests ways of looking at net energy footprint. 

Industry	Cloud-based Solution	Potential Fuel Savings	Source
Trucking	Route optimization with real-time traffic data	Up to 10%	DoT
Railroad	Predictive maintenance for locomotives	5-10%	Wabtec
Shipping	Optimized container loading and route planning	5-15%	Massey Ferguson

The point is that “net” impact is what we are after.