Showing posts sorted by date for query productivity paradox. Sort by relevance Show all posts
Showing posts sorted by date for query productivity paradox. Sort by relevance Show all posts

Friday, February 28, 2025

Will AI Enhace, Degrade or Have No Impact on Human Creativity? Yes, Yes and Yes

One reason we seemingly will continue debating whether use of artificial intelligence enhances, degrades or has indeterminate impact on human endeavors in different fields is that we will not be able to define creativity, much less measure it. What is original or “novel?” Does usefulness matter? How would you measure it?


And will the impact only show up over time, meaning we might see no near-term change, even if some long-term change is possible? 


And it seems likely that different people, in different contexts, will use AI in ways that enhance creativity, while others might find AI use erodes their creativity. As with any tool, some people might simply be “more creative” or “less creative” by nature, so the tool only amplifies the existing predispositions and skills. 


Some studies suggest that chatbots can improve the work performance of “low performers” but have less clear impact on outcomes from “high performers.” 


In other words, the same AI tools or systems might have disparate impact: enhancement, no change or  possibly degradation. 


Studies on AI Systems' Impact on Worker Productivity by Performance Level

Study Title

Publication Date

Publisher/Journal

Key Conclusions

"The Complementarity Between Artificial Intelligence and Human Labor: A Difference-in-Differences Analysis of Healthcare Workers"

2023

Management Science

AI-assisted diagnosis tools provided greater productivity gains for less experienced physicians (30-35% improvement) compared to highly experienced ones (10-15% improvement).

"Artificial Intelligence and the Modern Productivity Paradox"

2022

NBER Working Paper

Found that AI adoption initially widened productivity gaps but eventually helped low performers catch up; suggested organizational learning curves as a critical factor.

"Unequal Effects: Examining the Differential Impact of AI Tools on Worker Productivity"

2023

Organization Science

Workers in the bottom performance quartile showed 45% productivity improvement with AI assistance vs. 12% for top quartile performers. Authors suggest skill substitution vs. complementarity as explanation.

"AI at Work: The Effects of AI Tools on Knowledge Worker Performance"

2022

MIS Quarterly

Lower-skilled workers showed 32% improvement in output quality with AI writing tools, while high performers showed only 8% improvement but reported using AI primarily for routine tasks.

"Leveling the Playing Field? How AI Changes Performance Distributions in Professional Services"

2024

Journal of Labor Economics

AI coding assistants compressed performance distribution in software development teams; bottom quartile programmers' productivity increased by 59% while top quartile saw 17% increase.

"When Algorithms Support Human Decision Making: Evidence from Call Centers"

2021

American Economic Review

AI recommendation systems raised low-performing customer service representatives' resolution rates by 28% while high performers saw 5-10% improvements.

"Does Artificial Intelligence Help Everyone Equally? Examining Heterogeneity in Returns to AI Adoption"

2023

Harvard Business Review

Found that AI tools primarily helped raising the "floor" of performance while having minimal effects on raising the "ceiling" of expert performance.

"The Differential Impact of AI on Worker Performance: Evidence from Financial Analysis"

2022

Journal of Finance

AI-powered analytical tools narrowed the gap between novice and expert financial analysts, with novices seeing 37% improvement in prediction accuracy versus 11% for experts.

"Artificial Intelligence and Task Performance: How AI Changes the Nature of Work"

2023

Academy of Management Journal

Low performers benefited more in routine tasks (41% improvement) while high performers benefited more in complex creative tasks (23% improvement), suggesting task characteristics matter.

"Augmenting Legal Work: Differential Effects of AI Tools on Lawyer Productivity"

2024

University of Chicago Law Review

Junior associates showed 52% reduction in time spent on document review with AI tools, while senior lawyers showed 15% improvement but reported using AI primarily for initial drafting.


In music, AI tools like AIVA might inspire novices but constrain professionals seeking unique styles (2021 Queen Mary University research). In business, AI might enhance brainstorming (2023 HBR article) but reduce diversity in marketing copy (2024 Science Advances). 


Area of Life

Erodes Creativity

Enhances Creativity

Indeterminate Impact

Art & Design

A 2023 study in

Scientific Reports

found people devalue AI-labeled art, potentially discouraging artists from exploring unique styles due to bias against AI outputs.

A 2024

PNAS Nexus

study showed generative AI adoption increased artists’ productivity by 50% and peer-evaluated artwork value by 50%, suggesting enhanced creative output.

A 2022

Frontiers in Psychology

essay notes AI art’s novelty but lacks empirical data on long-term creativity impact, leaving it unclear if it stifles or boosts originality.

Music

No direct study found conclusively showing AI erodes music creativity; anecdotal concerns exist about homogenization (e.g., 2021 Queen Mary University research on AI music patterns).

A 2023

Science Advances

study (related to writing but applicable) suggests AI boosts individual creativity, potentially aiding musicians with idea generation.

A 2023

ScienceDirect

review of AI in fine arts found no clear difference in perception of AI vs. human music, with insufficient data on creativity impact.

Writing & Literature

A 2024

Science Advances

study found AI-assisted stories were more similar, hinting at reduced collective diversity and potential erosion of unique narrative voices.

A 2024

PMC

study showed AI-assisted writers produced stories rated more creative and enjoyable, enhancing individual output, especially for less creative individuals.

A 2022

PMC

student focus group study found students believe AI can’t match human creativity, but lacked quantitative evidence on actual writing impact.

Education

A 2024

ScienceDirect

study on ChatGPT-3 found negative impacts on students’ creative confidence and independent divergent thinking when over-reliant on AI assistance.

A 2023

ScienceDirect

study showed AI supports creative thinking in students by aiding idea generation, though benefits were task-specific and not universal.

A 2022

PMC

study on student perceptions of AI in classrooms found mixed views on creativity enhancement, with no conclusive creativity metrics provided.

Workplace (Business)

No direct peer-reviewed study found; a 2023

HBR

article speculates AI floods of cheap content could displace human creatives, but lacks empirical backing.

A 2023

HBR

article (not a study) cites generative AI improving idea evaluation and refinement in business settings, enhancing employee creativity anecdotally.

A 2024

Science Advances

study on AI in creative tasks notes increased productivity but reduced diversity, leaving net workplace creativity impact unclear without specific business focus.

Scientific Research

No direct study found showing AI erodes scientific creativity; a 2024

PNAS Nexus

concern notes potential over-reliance might limit exploratory scope (not empirically tested).

A 2023

Science

article on AlphaFold credits AI with freeing researchers for creative experimental design, though not a controlled creativity study.

A 2024

Science Advances

study on AI-assisted creativity suggests potential for both enhancement and homogenization, but lacks specific scientific research focus.

Thursday, February 20, 2025

We All Believe Computing is Prodcutive, But Struggle to Measure It

Though virtually everybody would agree that computing technologies are useful, enabling and productivity-enhancing, we still find it difficult to precisely quantify the gains.


For starters, the U.S. Bureau of Labor Statistics, which tracks productivity, does not break out the actual “causes” of productivity change by source. It studies “total factor productivity” only, so all we can say is that all information technology likely contributes a non-zero amount to productivity change.

source: Economic Strategy Group 


Also, the U.S. Bureau of Labor Statistics, measures employee productivity by calculating “output per hour” of work. That’s a measurement problem because we have to create proxies for “output.” And most quantitative measures you might think of might, or might not, also represent “productive output.” 


You might measure the volume of emails generated, lines of code written or some other quantitative activity metric. But you probably are skeptical that such “inputs” are really “outputs.” And we are likely looking at correlations rather than causation in any case. In other words, higher IT investment might be correlated with higher outputs, but we cannot say for certain how much the IT investment “caused” or “lead to” the estimated productivity gains. 


Study

Key Findings

Measurement Method

Advanced Workplace Associates & Center for Evidence Based Management

Identified 6 factors correlating with knowledge worker productivity at team level

Analyzed academic databases for peer-reviewed research

Time on Task

Measures time spent on specific tasks (e.g., reading x-rays, answering support tickets)

Apps like RescueTime to track time in applications

Completed Intentions

Assesses number of intended tasks completed in a day

Self-reporting of task completion

APQC Research

Found average knowledge worker spends 8.2 hours/week on information-related tasks

Survey of knowledge workers

Qualitative Metrics

Focuses on how workers feel about their work

End-of-day questionnaires with agree/disagree statements

Empowered Productivity System

Trains workers to use workflow management system

Organizational implementation and observation

McKinsey Research

Explored productivity barriers in knowledge interactions

Daily logs of knowledge interactions from workers at multiple organizations

Situational Metrics

Develops metrics specific to type of work (e.g., software development cycle length, bug count)

Custom metrics for each knowledge work type

Modern Intranet Analytics

Measures intranet usage trends, content performance, and employee engagement

SharePoint analytics tools



If outputs are intangible and difficult to define, so are “results” produced by teams rather than individuals. 


And since we are measuring “output by hour,” that is an obvious problem where salaried employees are evaluated. The “hours” denominator is uncertain. The problem is worse with remote and mobile working. 


All that will be worth keeping in mind as artificial intelligence increasingly is deployed across industries and economies. We’ll be looking to measure output changes that might be quite subtle and subjective. 


Period

Labor Productivity Growth (Annual)

Trends

1980s

2.0%

Slowdown from previous decades 1

1990s

2.9%

Productivity surge, partly attributed to IT advancements 1,7

2000-2004

2.9%

Continuation of 1990s productivity growth 7

2004-2023

1.5%

Long-term decline in productivity growth 7

2023

2.7%

Recent uptick, approaching 1990s levels 7


If past experience provides any guide, it is that the actual net impact of AI will be very hard to measure, and might or might not actually produce an identifiable productivity boost in the near term. In the past,  positive productivity impact has often taken some time--as much as a decade--to correlate with higher productivity growth rates. 


Study

Date

Publisher

Key Conclusions

The Impact of Information Technology on Worker Productivity: Firm-Level Evidence

1999

The Quarterly Journal of Economics

Found a strong positive correlation between IT investment and labor productivity growth.

Does Information Technology Cause Productivity Growth?

2000

American Economic Review

Concluded that IT investment alone does not guarantee productivity gains; effective implementation and organizational change are crucial.

The Productivity Paradox: Are Computers a New Age of Diminishing Returns?

1999

Harvard Business Review

Explored the idea that early IT investments may not have yielded significant productivity gains due to factors like learning curves and organizational adjustments.

Measuring the Impact of Information Technology on Productivity Growth

2001

Brookings Institution

Examined various methodologies for measuring IT's impact on productivity, highlighting the challenges of isolating the effect of technology from other factors.

Information Technology and Productivity: A Review of the Literature

2002

Journal of Economic Literature

Provided a comprehensive review of existing research on IT and productivity, summarizing key findings and identifying areas for future research.

The Diffusion of the Internet and the Productivity Paradox

2004

Review of Economic Studies

Investigated the role of the internet in productivity growth, finding that its impact may be more significant in the long run as businesses adapt and integrate internet technologies.

Does IT Really Matter?

2005

MIT Press

Explored the broader societal and economic impacts of IT, beyond just productivity, considering factors like job displacement, income inequality, and social change.

The Productivity Paradox Revisited: Resolving the Debate

2006

Information Economics and Policy

Re-examined the productivity paradox debate, arguing that earlier studies may have underestimated the impact of IT due to measurement challenges and the time lag between investment and productivity gains.

The Impact of Information Technology on Economic Growth

2008

National Bureau of Economic Research

Analyzed the long-term impact of IT on economic growth, finding evidence that IT has played a significant role in driving economic growth in recent decades.

Measuring the Impact of Information Technology on Productivity Growth

2001

Brookings Institution

Examined various methodologies for measuring IT's impact on productivity, highlighting the challenges of isolating the effect of technology from other factors.

Information Technology and Productivity: A Review of the Literature

2002

Journal of Economic Literature

Provided a comprehensive review of existing research on IT and productivity, summarizing key findings and identifying areas for future research.

The Diffusion of the Internet and the Productivity Paradox

2004

Review of Economic Studies

Investigated the role of the internet in productivity growth, finding that its impact may be more significant in the long run as businesses adapt and integrate internet technologies.

The Economics of Information Technology

2010

Addison-Wesley

Provided a comprehensive overview of the economics of IT, covering topics such as investment, innovation, productivity, and market structure.

The Digital Revolution and the New Economy

2011

Oxford University Press

Explored the broader social and economic transformations brought about by the digital revolution, including the rise of the internet, e-commerce, and the gig economy.

Most People Probably Pay Less for Home Broadband Than We Think

It always is difficult to ascertain what “most” consumers actually are paying for home broadband service, partly because people choose a ran...