Wednesday, November 20, 2024

It Will be Hard to Measure AI Impact on Knowledge Worker "Productivity"

There are over 100 million knowledge workers in the United States, and more than 1.25 billion knowledge workers globally, according to one Anatomy of Work estimate. And “work about work,” including unnecessary meetings, status checks and information searchers occupy as much as 60 percent of knowledge worker time. 


Hence the interest in AI agents that can conduct activities autonomously, presumably eliminating much of that “work about work.” Customer support, regulatory compliance, security and marketing are areas where agents are expected to contribute. 


To the extent we can measure knowledge worker productivity--and there is an argument to be made that we actually cannot measure it effectively--efforts to boost knowledge worker productivity since 2019 have been quite minimal. 


Title

Date

Publisher

Key Conclusions

How Do You Measure Knowledge Worker Productivity?

N/A

Serraview

- Outputs are intangible and difficult to define

- Results often based on team output rather than individual

- Companies not necessarily tracking hours for salaried employees

- Time spent working increasingly blurred in mobile workforce

The Best Way to Measure Knowledge Worker Productivity

2022

Maura Thomas

- Quantitative metrics don't help in short term for knowledge workers

- Qualitative metrics matter most in short term

- Best measure is questioning how employees feel about their work

How to Measure Employee Productivity in the Workplace

2024

Robin

- Knowledge work is intangible and difficult to categorize

- Existing productivity measures rooted in 'machine age' organizations

- Impossible to come up with single universal measure for knowledge worker productivity

Knowledge Worker Productivity

N/A

mediaX at Stanford University

- Serious productivity gap exists between available knowledge and how it is used

- Many enterprises fail to fully engage energy and intellect of employees

Research: Knowledge Workers Are More Productive from Home

2020

Harvard Business Review

- Knowledge workers' inputs and outputs can't be tracked like other workers

- They apply subjective judgment to tasks and decide what to do when

- Can withhold effort often without anyone noticing

"Boosting the Productivity of Knowledge Workers"

2006

McKinsey

Measuring productivity is complex due to the amorphous nature of tasks. Performance is constrained by physical, social, and contextual barriers. Interactions account for a significant portion of work, making targeted improvements essential.

"Measuring Knowledge Worker Productivity: A Taxonomy"

2004

Emerald Group Publishing

Identifies a lack of universally accepted metrics for productivity. Proposes a taxonomy categorizing productivity dimensions and highlights critical areas for future research.

"Rethinking Knowledge Work: A Strategic Approach"

2021

McKinsey

Structured approaches (e.g., workflow systems) improve some metrics but can undermine autonomy and collaboration. Freeform and creative aspects of work are harder to quantify.

Knowledge worker productivity: is it really impossible to measure it?

2021

ResearchGate

Argues that traditional productivity metrics are inadequate for knowledge workers and proposes a new methodology based on human capital efficiency.

Broken Speedometers: Quantifying Knowledge Worker Effectiveness?

July 1905

VeraSage Institute

Highlights the challenges of measuring knowledge worker productivity due to the intangible nature of their work and the limitations of traditional metrics.

What Really Matters for Knowledge Worker Performance

July 1905

Allsteel

Reviews existing research and concludes that a single, universal metric for measuring knowledge worker productivity does not exist.

Measuring Knowledge Worker Productivity

July 1905

Global Workspace Association

Discusses the complexity of measuring knowledge worker productivity and the limitations of traditional methods.


Knowledge workers are those who “think for a living,” making productivity challenging to measure. According to the U.S. Bureau of Labor Statistics, measuring employee productivity means calculating “output per hour” of work. 


But how does one quantify outputs, which often are intangible and difficult to define. Also, when “teams” produce the outcomes, how can individual contributions be assessed?


And there are other complications, such as quantifying “hours worked,” either in-office or remotely. 


None of that stops government agencies from doing their best to measure and quantify knowledge worker productivity. 


For example, total factor productivity in the United States is said to have grown 0.8 percent from 1987 to 2023, but only 0.5 percent from 2019 to 2023, according to the Bureau of Labor Statistics


All of which will raise questions when firms and entities start to report “productivity” gains from using AI.  If all we can be sure of is that we can measure or quantify some outcomes we believe to be measures of output.  


Whether that output actually represents knowledge worker productivity is less certain. Most of us would be circumspect about metrics such as “hours worked” or “email volume” or “meeting attendance.”


We’d probably have some greater confidence about tasks completed, revenue generated by a team, assuming it can be identified. 


But lots of common metrics are only quantitative, and cannot measure the quality of work performed or outcomes. People can produce lots of documents, lots of code or “ideas,” but it is hard to measure the quality of those outputs. 


Proxy Measure

Description

Time spent working

Hours logged or time tracked on tasks14

Email volume

Number of emails sent/received5

Meeting attendance

Number of meetings attended or hours in meetings

Task completion

Number of tasks or projects completed4

Revenue generated

Financial output attributed to individual or team2

Information searches

Time spent looking for information15

Internal collaboration

Time spent working with colleagues5

Documents produced

Number of reports, presentations, or other deliverables created

Client interactions

Number of client meetings or calls conducted

Ideas generated

Number of new ideas or innovations proposed


No comments:

It Will be Hard to Measure AI Impact on Knowledge Worker "Productivity"

There are over 100 million knowledge workers in the United States, and more than 1.25 billion knowledge workers globally, according to one A...