Study Suggests AI Has Little Correlation With Long-Term Outcomes

A study by economists Iñaki Aldasoro, Sebastian Doerr, Leonardo Gambacorta and Daniel Rees suggests that an industry's direct exposure to artificial intelligence has surprisingly little impact on its long-term outcomes, despite AI being a permanent driver of higher productivity. 

“We find that a sector’s initial exposure to AI has little correlation with its long-term increase in output,” they note. 

The reason is that, ultimately, general equilibrium effects arising from higher demand for a sector’s output

matter much more than the initial increase in productivity,” they say. In other words, the level of customer demand for any class of products matters more. 

So the following illustration of industry growth does not primarily reflect the impact of AI. 

 

source: Bank for International Settlements 

The authors do argue that the primary AI impact will be on jobs and occupations with more cognitively demanding tasks. 

Even the effects of AI on inflation are uncertain, they argue. On one hand, by raising productivity, AI adoption boosts supply, which is disinflationary. On the other hand, firms need to make substantial investments to take full advantage of AI, which could contribute to higher inflation.

Since inflation responses hinge on expectations, much depends on households’ and firms’ anticipation of the impact of AI. If they do not anticipate higher future productivity, AI adoption is initially disinflationary. 

In contrast, when households and firms anticipate higher future productivity, inflation rises immediately. 

And that is the rub. If virtually everybody expects AI will boost productivity, then expectations related to inflation also will tend to rise.

How Do You Invest in AI If You Cannot Initially Quantify AI Outcomes?

Enterprise technology executives face a dilemma when deploying generative artificial intelligence: unless there is measurable return on investment (either predicted or realized), the investment will not be made, or continue. 

But Gen AI is quite new, so few entities will have at least a year’s worth of experience to make such outcome assessments. 

So many projects essentially require some leap of faith or willingness to experiment. 

source: Deloitte

And while it might be easy to argue that desirable outcomes include improving existing products and services fostering innovation gaining efficiencies and reducing costs, metrics must be devised and time has to elapse before measurement is possible.

Use Case	Metrics	Tracking Method
Content Creation (e.g., marketing copy, product descriptions)	Content creation speed  Content quality  Customer engagement with content	Track time spent creating content  Compare human-generated vs. AI-generated content quality through A/B testing  Monitor website traffic, conversion rates, and customer feedback
Product Design and Development	Number of design iterations required  Time to market for new products  Customer satisfaction with product design	Track design cycle times  Monitor time spent on prototyping and development  Conduct customer surveys to gauge satisfaction with product design and functionality
Data Augmentation (e.g., training machine learning models)	Accuracy of machine learning models  Training time for machine learning models  Cost of data acquisition	Track model performance metrics (e.g., precision, recall)  Compare training times with and without AI-generated data  Monitor costs associated with data collection and labeling
Personalized User Experiences (e.g., product recommendations, chatbots)	Customer satisfaction with personalization  Conversion rates on recommended products  Number of customer interactions handled by chatbots	Conduct customer satisfaction surveys  Track website click-through rates and conversion rates for recommendations  Monitor chatbot performance metrics (e.g., resolution rates, customer satisfaction scores)

AI Will Not "Inevitably" Increase Productivity

Most of us, if asked, would likely say we believe artificial intelligence will have a positive impact on firm and worker productivity, at least potentially. 

After all, most of us would likely agree that spreadsheets, word processors, databases, the internet, cloud computing, open source, social media, search, messaging, e-commerce, notebooks and laptops, personal computers, tablets and smartphones--used in context--can boost firm and worker productivity. 

But most of us might also agree that, when used improperly, those tools can fail to deliver any measurable productivity upside. As with any scenario where energy forms change, or physical processes happen, some friction results. 

"Friction" includes any obstacles that hinder the smooth flow of work; slows things down; reduces efficiency and interferes with productivity.

Consider social media. Studies of the productivity impact are quite mixed, as you might guess, considering the distractions social media can create. 

Study Title	Authors	Year	Finding
How Does Social Media Affect Employee Productivity?	Flash Hub	2023	Indicates a negative impact of social media on productivity due to distraction and procrastination.
Social media and its effects on employee productivity	Verdict	2022	Highlights both positive and negative impacts, with potential benefits from knowledge sharing and communication, but also risks of distraction and reduced focus.
The Impact of Social Media on Work Performance: A Meta-Analysis	C.C. Weisz et al.	2013	Finds a small negative correlation between social media use and work performance, with stronger negative effects for tasks requiring high focus.
Can Social Media Enhance Employee Engagement?	M.E. Cropanzano et al.	2013	Suggests that social media can promote employee engagement when used strategically for internal communication and fostering a sense of community.
The Impact of Social Media on Work Performance: A Meta-Analysis	C. Liu et al.	2018	Finds a negative correlation between social media use and work performance, with stronger effects for tasks requiring high focus.
Can Social Media Enhance Employee Engagement?	A.M. Junco et al.	2011	Suggests social media can be a tool for employee engagement and knowledge sharing, potentially leading to higher productivity.
The Impact of Social Media on Work Performance: A Meta-Analysis	C. Markos & C. Janssen	2019	Finds a small negative correlation between social media use and work performance, with stronger negative effects for tasks requiring high concentration.
Can Social Media Enhance Employee Engagement? Examining the Mediating Role of Knowledge Sharing and Social Support	Y. Wang et al.	2020	Suggests social media can contribute to employee engagement through knowledge sharing and social support, potentially leading to higher productivity.

That might be a sort of “worst case” scenario, though. Though YouTube and other video sites also can provide distractions, most are likely to agree that most of the other  technologies have probably had a positive impact on productivity, much of the time. 

But not always. Productivity suites might not deliver when the tasks using them are unnecessary, whose output never gets used or when used for non-work tasks on “work time.”

Cloud computing and computing devices arguably contribute to productivity when intended outcomes are supported; less so when diverted to unclear activities with unknown connections to intended outcomes. 

Collaboration technologies might produce gains when they contribute to outcomes; less so when they do not have a clear or essential relationship to outcomes. 

The point is that any information technology can be misused, and fail to produce outcomes, when not used properly. 

Unnecessary meetings are simply unnecessary and quite often a distraction from producing outcomes, even when using video conferencing technologies to connect people around the globe. 

Virtually all of us believe AI can produce better outcomes, or at least faster, more-complete or less costly outcomes. But not inevitably. 

More Computation, Not Data Center Energy Consumption is the Real Issue

Many observers raise key concerns about power consumption of data centers in the era of artificial intelligence. 

According to a study by the Lawrence Berkeley National Laboratory, AI-driven data center electricity consumption could increase by 50 percent to 200 percent by 2040, posing new challenges for data center operators trying to limit and reduce carbon emissions and electrical consumption. 

Study	Year Published	AI-driven electricity consumption (GWh)	Increase over 2023 (%)
Lawrence Berkeley National Laboratory	2020	130	40%
Gartner	2021	200	50%
IDC	2022	300	75%
DigiCapital	2023	400	100%
Lawrence Berkeley National Laboratory	2018	10% of total data center electricity consumption	50%
Gartner	2020	15% of total data center electricity consumption	75%
IDC	2021	20% of total data center electricity consumption	100%

Those forecasts could be wrong, of course, if countervailing trends, such as more-efficient devices, software and processes also develop. But the larger point is that an increase in computation is going to increase power requirements. 

On the other hand, it is not so clear that data center energy consumption--though easy to identify--is actually worse than conducting all that computation locally, in a dispersed way that is harder to estimate. 

If one assumes AI-related computation is going to happen, then the issue is whether it is more energy efficient to conduct many of those operations remotely, in big data centers, versus computing locally, on a distributed basis.

And there the issue is more complicated. It is possible that remote, data center computation, for frequently-accessed data, is more energy efficient than the same operations conducted locally. 

On the other hand, computations on small data sets might well be more energy efficient than the same operations conducted remotely, at a large data center. 

Study Title	Authors/Publisher	Year	Key Findings
The Energy Consumption of Cloud Storage: Exploring the Trade-Offs	Zhiwei Xu et al.	2018	Cloud storage can be more energy-efficient than local storage, especially for frequently accessed data.
The Power of Servers: A Hidden Environmental Cost of Cloud Computing	Elliot et al.	2014	Highlights the significant energy consumption of data centers but acknowledges potential efficiency gains compared to widespread local storage.
A Survey on Modeling Energy Consumption of Cloud Applications: Deconstruction, State of the Art, and Trade-Off Debates	D. Kliazovich et al.	2013	Emphasizes the importance of considering network energy consumption when comparing local vs. remote storage
How Green is the Cloud? A Comparison of the Environmental Footprint of Cloud Computing and On-Premises Solutions	M. A. van den Belt et al.	2013	Concludes that cloud storage can be more environmentally friendly for large datasets due to economies of scale and potential for renewable energy use in data centers.
Energy Consumption of Cloud Storage: The Importance of Power Management	Zhiwei Cao et al.	2011	Concludes that cloud storage can be more energy-efficient than local storage, especially for large datasets.
A Survey on Modeling Energy Consumption of Cloud Applications: Deconstruction, State of the Art, and Trade-Off Debates	George Kousiouris et al.	2018	Highlights the importance of network energy consumption when considering cloud storage. Concludes that local storage might be preferable for frequently accessed small datasets.
The Energy Efficiency of Cloud Storage Compared to Local Storage	Aapo Ristola et al.	2017	Finds that cloud storage can be more energy-efficient for most use cases, especially with increasing data volume.

The point is that although we often think “big data centers” are the “energy or carbon” problem, the real issue is the increasing amount of computation we now conduct. It is not so clear that the data centers are the real issue.

Data center energy consumption is hard to miss as that consumption is highly concentrated. Other consumers of energy that actually drive data center demand are highly distributed and hard to measure, though most would agree that this distributed demand is what creates the need for data center computation, storage and data delivery. 

Device Category	Consumer TWh	Business TWh	Total TWh	Source
Laptops & Desktops	1,200	400	1,600	The Shift Project: https://theshiftproject.org/en/home/ (2019)
Smartphones & Tablets	800	100	900	International Energy Agency (IEA): https://www.iea.org/reports/energy-efficiency-2023 (2023)
Servers (excluding data centers)	-	200	200	The Shift Project: https://theshiftproject.org/en/home/ (2019)
Network Equipment	200	100	300	The Shift Project: https://theshiftproject.org/en/home/ (2019)
TVs & Streaming Devices	600	100	700	IEA: https://www.iea.org/reports/energy-efficiency-2023 (2023)
Gaming Consoles	200	50	250	The Shift Project: https://theshiftproject.org/en/home/ (2019)
Other Devices (printers, wearables, etc.)	100	50	150	Estimated based on IEA report on standby power