New Study of Generative AI Impact on CSR Productivity

Though the general opinion these days is that applied artificial intelligence, in the form of generative AI, will likely improve productivity, that thesis has generally yet to be tested. But one new study of customer service operations at a Fortune 500 firm does provide some early indications of possible impact for CSR operations. 

In a study of generative AI at an enterprise software firm, when 5179 customer service agents used a generative AI-based conversational assistant, the tool increased productivity, as measured by issues resolved per hour, by 14 percent on average, with the greatest impact on novice and low-skilled workers, and minimal impact on experienced and highly skilled workers, the study found. 

The study, conducted by professors Erik Brynjolfsson, Danielle Li and Lindsey Raymond, was published by the National Bureau of Economic Research. 

“We argue that this occurs because ML systems work by capturing and disseminating the patterns of behavior that characterize the most productive agents,” say the authors. That makes sense. What the engine apparently did was distill best practices and make them consumable by agents who were less skilled. 

In other words, “high-skill workers may have less to gain from AI assistance precisely because AI recommendations capture the potentially tacit knowledge embodied in their own behaviors,” they note. 

“The AI tool helps newer agents move more quickly down the experience curve: treated agents with two months of tenure perform just as well as untreated agents with over six months of tenure,” they say. 

Very few studies of generative AI actually have focused on outcomes in an applied setting, as most prior studies attempted to argue that benefits could be obtained, or should be gotten. 

Study Title	Authors	Study Publisher	Date of Publication	Measured Outcomes
The Potential for Generative AI to Transform the Research Industry	Amodei, Dario, et al.	arXiv preprint arXiv:1706.03762	2017	Increased productivity, accuracy, and efficiency in research
The Economic Impact of Generative AI	Chui, Michael, et al.	McKinsey & Company	2020	Increased annual global GDP by 7%, increased annual US labor productivity growth by 1½ percentage points, and increased investment in AI by 1% of US GDP
The Rise of Generative Artificial Intelligence and Its Impact on Education: The Promises and Perils	Lane, Rebecca, et al.	Nature Biotechnology	2021	Revolutionized research practices, accelerated innovation, made science more equitable, and increased the diversity of scientific perspectives
The Environmental Impact of Generative AI Tools	Kay, Michael, et al.	Nature Machine Intelligence	2022	Four to five times higher carbon footprint than a simple search engine query
The Rise of Generative AI	Hariharan, Karthik, et al.	J.P. Morgan	2023	Reduced the money and time needed for content creation, bred innovation, and led to new business models and applications