On-Device is the Overlooked AI Compute Venue

One way to look at 5G devices such as smartphones is as a developing computing architecture supporting artificial intelligence. In other words, some AI tasks are “best” handled on the device, while others are more appropriately handled at remote data centers. 

And some AI-related operations will make more sense at a metro data center (possibly regional) while yet others might be handled at smaller “edge” data centers.

On-device	Metro	Small "edge"	Remote data center
Real-time inference: Machine learning models that need to make predictions in real time, such as object detection and image recognition.	Data aggregation and analysis: Collecting and analyzing data from multiple devices and sensors in a localized area.	Data caching: Storing data that is frequently accessed by devices in a nearby location.	High-performance computing: Tasks that require a lot of processing power, such as video encoding and scientific computing.
Personalized experiences: Tailoring the user experience to individual preferences, such as recommending products and services.	Content delivery: Serving content to users in a localized area, such as streaming video and music.	IoE device management: Managing and monitoring IoT devices in a localized area.	Data backup and storage: Storing data for long-term retention and archiving.
Privacy-sensitive applications: Applications that handle sensitive data, such as financial transactions and medical records.	Disaster recovery: Providing a backup site for critical applications and data in the event of a disaster.	Edge analytics: Performing analytics on data at the edge of the network, such as filtering and identifying anomalies.	Batch processing: Tasks that can be processed in batches, such as data mining and machine learning training.

Most of us could predict that AI will affect data centers--in large part--by significantly increasing requirements to support generative AI model building, training and then inference operations. 

In some cases, metro or regional data centers will take the place of remote date centers. There is likely more disagreement about use of small edge data centers to support those operations. 

But often overlooked are the potential roles for device-based AI operations. 

On-device	Metro	Small "edge" data center	Remote data center
Voice assistants	Real-time traffic updates	Surveillance cameras	Batch processing
Augmented reality	Smart home devices	Self-driving cars	Data storage
Virtual reality	Industrial automation	IoT devices	Machine learning training
Mobile gaming	Content delivery networks	Real-time analytics	Disaster recovery
Natural language processing	Edge computing applications	Autonomous vehicles	High-performance computing