On-Device is the Overlooked AI Compute Venue

One way to look at 5G devices such as smartphones is as a developing computing architecture supporting artificial intelligence. In other words, some AI tasks are “best” handled on the device, while others are more appropriately handled at remote data centers. 

And some AI-related operations will make more sense at a metro data center (possibly regional) while yet others might be handled at smaller “edge” data centers.

On-device	Metro	Small "edge"	Remote data center
Real-time inference: Machine learning models that need to make predictions in real time, such as object detection and image recognition.	Data aggregation and analysis: Collecting and analyzing data from multiple devices and sensors in a localized area.	Data caching: Storing data that is frequently accessed by devices in a nearby location.	High-performance computing: Tasks that require a lot of processing power, such as video encoding and scientific computing.
Personalized experiences: Tailoring the user experience to individual preferences, such as recommending products and services.	Content delivery: Serving content to users in a localized area, such as streaming video and music.	IoE device management: Managing and monitoring IoT devices in a localized area.	Data backup and storage: Storing data for long-term retention and archiving.
Privacy-sensitive applications: Applications that handle sensitive data, such as financial transactions and medical records.	Disaster recovery: Providing a backup site for critical applications and data in the event of a disaster.	Edge analytics: Performing analytics on data at the edge of the network, such as filtering and identifying anomalies.	Batch processing: Tasks that can be processed in batches, such as data mining and machine learning training.

Most of us could predict that AI will affect data centers--in large part--by significantly increasing requirements to support generative AI model building, training and then inference operations. 

In some cases, metro or regional data centers will take the place of remote date centers. There is likely more disagreement about use of small edge data centers to support those operations. 

But often overlooked are the potential roles for device-based AI operations. 

On-device	Metro	Small "edge" data center	Remote data center
Voice assistants	Real-time traffic updates	Surveillance cameras	Batch processing
Augmented reality	Smart home devices	Self-driving cars	Data storage
Virtual reality	Industrial automation	IoT devices	Machine learning training
Mobile gaming	Content delivery networks	Real-time analytics	Disaster recovery
Natural language processing	Edge computing applications	Autonomous vehicles	High-performance computing

Computing Infra Not "Ready" for AI?

A survey of 500 information technology professionals sponsored by LogicMonitor shows at least half of those respondents believe their infrastructure is not equipped to handle increased artificial intelligence. 

And information technology analysts believe the amount of new investment to support AI could be substantial. 

A study by McKinsey Global Institute estimated that businesses would need to invest $3.5 trillion in AI by 2030 to realize AI benefits.

A study by the Boston Consulting Group suggested  that businesses would need to invest $1.7 trillion in AI by 2025 to realize AI benefits.

A study by PwC suggested that businesses would need to invest $2.2 trillion in AI by 2030 to retrain and reskill workers who are displaced by AI.

Of course, IT professionals often say the existing infra is not prepared for the platforms and functions, when a new technology emerges. In many cases, preparedness is an issue precisely because the capabilities and skills needed to introduce a new platform do not yet exist.  Many past examples include:

Electronic data interchange (EDI)

Enterprise resource planning (ERP)

Customer relationship management (CRM)

Supply chain management (SCM)

Content management systems (CMS)

Virtual private networks (VPNs)

Mobile computing

Cloud gaming

Telehealth

Robotic process automation (RPA). 

And there will always seem to be some new thing that IT professionals report they are not prepared to handle with the existing infra. CIO magazine’s annual surveys have tended to show new concerns every year, for example. 

Innovation	Study	Date of Publication
Cloud computing	The State of the CIO 2011	2011
Bring your own device (BYOD)	The State of the CIO 2014	2014
Social media	The State of the CIO 2015	2015
Big data	The State of the CIO 2016	2016
The Internet of Things (IoT)	The State of the CIO 2017	2017

Among the oft-cited concerns:

• Lack of skilled talent. AI is a complex technology that requires specialized skills and knowledge. Many organizations do not have the in-house expertise to develop and deploy AI solutions.

• Data quality and availability. AI algorithms need to be trained on large amounts of high-quality data. However, many organizations lack the necessary data or the resources to collect and clean it.

• Cost. AI can be a costly investment, especially for small and medium-sized businesses. The cost of developing, deploying, and maintaining AI solutions can be prohibitive for some organizations.

• Regulatory compliance. AI raises a number of regulatory concerns, such as data privacy and bias. Organizations need to ensure that their AI solutions comply with all applicable regulations.

• Security risks. AI systems can be vulnerable to cyberattacks. Organizations need to take steps to protect their AI systems from unauthorized access and tampering.

• Ethical concerns. AI raises a number of ethical concerns, such as bias and discrimination. Organizations need to develop ethical guidelines for the use of AI.

IT professionals believe that they need to make the following investments to adapt their infrastructures for AI, aside from acquiring new AI skills and continuing to streamline code development processes::

• Data infrastructure. AI algorithms need to be trained on large amounts of data. Organizations need to invest in data infrastructure that can store and process this data efficiently.

• Compute infrastructure. AI algorithms can be computationally demanding. Organizations need to invest in compute infrastructure that can run these algorithms quickly and efficiently.

• Networking infrastructure. AI applications often need to access and process data from multiple sources. Organizations need to invest in networking infrastructure that can support this connectivity.

• Security infrastructure. AI systems can be vulnerable to cyberattacks. Organizations need to invest in security infrastructure that can protect their AI systems from unauthorized access and tampering.

Some Will Benefit from AI More than Others

Though there is as yet no consensus about the degree of artificial intelligence impact on various industries, many could agree that some industries will benefit more, either in terms of process cost reduction or operating costs; others less so. 

Probably nobody would be surprised if financial services wound up seeing direct benefits, as has been the case in the past. But many would guess that healthcare will be a bigger beneficiary from AI than has traditionally been the case for information technology  investments. 

As with so many other metrics, it appears connectivity services and data centers are somewhere in the middle of industries where it comes to the degree of process automation and improvement. 

Industry	Impact on Operating Costs	Reasons
Healthcare	High	AI can automate many tasks in healthcare, such as medical record keeping, diagnosis, and treatment planning. This can free up healthcare workers to focus on more complex tasks and provide better care to patients. AI can also help to reduce the cost of healthcare by improving efficiency and reducing errors.
Financial Services	High	AI can automate many tasks in financial services, such as fraud detection, risk assessment, and investment management. This can free up financial professionals to focus on more complex tasks and provide better service to clients. AI can also help to reduce the cost of financial services by improving efficiency and reducing errors.
Manufacturing	High	AI can automate many tasks in manufacturing, such as assembly line production, quality control, and inventory management. This can help manufacturers to improve efficiency and reduce costs. AI can also help manufacturers to produce higher quality products by identifying and correcting defects early on.
Retail	High	AI can automate many tasks in retail, such as customer service, inventory management, and fraud detection. This can help retailers to improve efficiency and reduce costs. AI can also help retailers to increase sales by personalizing the shopping experience for customers.
Transportation	High	AI can automate many tasks in transportation, such as driving, traffic management, and logistics planning. This can help transportation companies to improve efficiency and reduce costs. AI can also help transportation companies to improve safety by reducing human error.
Telecommunications	Medium	AI can automate many tasks in telecommunications, such as network management, customer service, and fraud detection. This can help telecommunications companies to improve efficiency and reduce costs.
Data Centers	Medium	AI can automate many tasks in data centers, such as server provisioning, cooling system management, and anomaly detection. This can help data center operators to improve efficiency and reduce costs. AI can also help data center operators to improve the reliability and security of their data centers.
Construction	Low	AI can be used to automate some tasks in construction, such as design and planning. However, the construction industry is very labor-intensive, and it is not clear how much AI can automate.
Agriculture	Low	AI can be used to automate some tasks in agriculture, such as crop monitoring and harvesting. However, the agriculture industry is very weather-dependent, and it is not clear how much AI can automate.
Education	Low	AI can be used to personalize learning and provide feedback to students. However, the education industry is very people-centric, and it is not clear how much AI can automate.

In many industries, process improvements are expected to provide the clearest and perhaps most-significant financial benefits. Perhaps more interesting are ways AI can boost revenue, even when AI does affect both operating costs by streamlining processes.

Industry	Impact on Core Business Models	Revenue Impact
Healthcare	AI is transforming healthcare by automating tasks, improving diagnostics, and developing new treatments.	Large: AI-powered systems can help healthcare providers improve efficiency and accuracy, while also reducing costs. This could lead to lower healthcare costs for consumers and businesses. AI could also lead to new treatments.
Financial Services	AI is being used to automate tasks, detect fraud, and make better investment decisions.	Large: AI-powered systems can help financial institutions improve efficiency and reduce costs, while also generating new revenue streams. For example, AI-powered chatbots can provide customer service and answer questions 24/7, while AI-powered trading algorithms can generate profits for financial institutions.
Manufacturing	AI is being used to automate tasks, improve quality control, and optimize production lines.	Large: AI-powered systems can help manufacturers improve efficiency and reduce costs, while also producing higher quality products. For example, AI-powered robots can perform repetitive tasks on assembly lines, while AI-powered quality control systems can identify defects in products before they reach consumers.
Retail	AI is being used to personalize shopping experiences, recommend products, and optimize inventory levels.	Large: AI-powered systems can help retailers improve customer satisfaction and sales, while also reducing costs. For example, AI-powered chatbots can provide personalized recommendations to customers, while AI-powered inventory management systems can help retailers avoid stockouts and overstocking.
Transportation	AI is being used to develop self-driving cars, improve traffic management, and optimize supply chains.	Large: AI-powered systems can help transportation companies improve efficiency and reduce costs, while also offering new services to customers. For example, self-driving cars could revolutionize the taxi and trucking industries, while AI-powered traffic management systems could reduce congestion and travel times.
Telecommunications	AI is being used to improve network performance, detect fraud, and develop new services.	Medium: AI-powered systems can help telecommunications companies improve efficiency and reduce costs, while also generating new revenue streams. For example, AI-powered systems can be used to optimize network traffic and detect fraud. AI can also be used to provide personalized recommendations for customers.
Data Centers	AI is being used to improve energy efficiency, detect anomalies, and automate tasks.	Medium: AI-powered systems can help data center operators improve efficiency and reduce costs. For example, AI-powered systems can be used to optimize cooling systems and detect potential problems. AI can also be used to automate tasks, such as provisioning and managing servers.

That differential pattern of impact has been seen in prior eras of information technology investment as well. Of course, the industry most affected by IT is the IT industry itself, which is the supplier of the platforms and tools. In other industries, the financial services industry virtually always ranks high on any list of IT adopters, early IT adopters and estimated IT impact. 

Industry	Revenue Impact	Operating Cost Impact	Reasons
Technology	High	High	Information technology is the core business model of the technology industry, so new IT has had a profound impact on revenue, as that is the primary product. Technology firms also are early users of IT in most ways.
Financial Services	High	High	IT enabled the development of new financial products and services, as well as new ways to deliver them. This has led to increased revenue for many financial services companies. Additionally, IT has helped financial services companies to automate tasks and improve efficiency, which has reduced operating costs.
Retail	High	Medium	IT has also had a significant impact on the retail industry. IT has enabled the development of new retail channels, such as e-commerce. This has led to increased revenue for some retailers. Additionally, IT has helped retailers to automate tasks and improve efficiency, which has reduced operating costs.
Telecommunications	Medium	Medium	IT has also had a significant impact on the telecommunications industry. IT has enabled the development of new telecommunications services, such as mobile internet and broadband. This has led to increased revenue for many telecommunications companies. Additionally, IT has helped telecommunications companies to automate tasks and improve efficiency, which has reduced operating costs.
Data Centers	Medium	Medium	IT has also had a significant impact on the data center industry. IT has enabled the development of new data center services, such as cloud computing and big data analytics. This has led to increased revenue for many data center companies. Additionally, IT has helped data center companies to automate tasks and improve efficiency, which has reduced operating costs.

As always, other industries, such as agriculture and education, have benefitted more modestly from applied IT.