Friday, June 2, 2023

New Study of Generative AI Impact on CSR Productivity

Though the general opinion these days is that applied artificial intelligence, in the form of generative AI, will likely improve productivity, that thesis has generally yet to be tested. But one new study of customer service operations at a Fortune 500 firm does provide some early indications of possible impact for CSR operations. 


In a study of generative AI at an enterprise software firm, when 5179 customer service agents used a generative AI-based conversational assistant, the tool increased productivity, as measured by issues resolved per hour, by 14 percent on average, with the greatest impact on novice and low-skilled workers, and minimal impact on experienced and highly skilled workers, the study found. 


The study, conducted by professors Erik Brynjolfsson, Danielle Li and Lindsey Raymond, was published by the National Bureau of Economic Research. 


“We argue that this occurs because ML systems work by capturing and disseminating the patterns of behavior that characterize the most productive agents,” say the authors. That makes sense. What the engine apparently did was distill best practices and make them consumable by agents who were less skilled. 


In other words, “high-skill workers may have less to gain from AI assistance precisely because AI recommendations capture the potentially tacit knowledge embodied in their own behaviors,” they note. 


“The AI tool helps newer agents move more quickly down the experience curve: treated agents with two months of tenure perform just as well as untreated agents with over six months of tenure,” they say. 


Very few studies of generative AI actually have focused on outcomes in an applied setting, as most prior studies attempted to argue that benefits could be obtained, or should be gotten. 


Study Title

Authors

Study Publisher

Date of Publication

Measured Outcomes

The Potential for Generative AI to Transform the Research Industry

Amodei, Dario, et al.

arXiv preprint arXiv:1706.03762

2017

Increased productivity, accuracy, and efficiency in research

The Economic Impact of Generative AI

Chui, Michael, et al.

McKinsey & Company

2020

Increased annual global GDP by 7%, increased annual US labor productivity growth by 1½ percentage points, and increased investment in AI by 1% of US GDP

The Rise of Generative Artificial Intelligence and Its Impact on Education: The Promises and Perils

Lane, Rebecca, et al.

Nature Biotechnology

2021

Revolutionized research practices, accelerated innovation, made science more equitable, and increased the diversity of scientific perspectives

The Environmental Impact of Generative AI Tools

Kay, Michael, et al.

Nature Machine Intelligence

2022

Four to five times higher carbon footprint than a simple search engine query

The Rise of Generative AI

Hariharan, Karthik, et al.

J.P. Morgan




 

2023

Reduced the money and time needed for content creation, bred innovation, and led to new business models and applications



No comments:

Many Winners and Losers from Generative AI

Perhaps there is no contradiction between low historical total factor annual productivity gains and high expected generative artificial inte...