How much can artificial intelligence boost knowledge worker productivity? Maybe it can, but we will have trouble measuring that benefit, in all likelihood.
Self reports are likely to be key, and that means we will see a lot of subjectivity. People may believe they save time by using AI, but that does not necessarily mean they do save time; only that they believe they do.
We’ll also hear quite a lot about knowledge workers getting more done in the same amount of time. Most of us will likely agree that is a correct assessment, even if we remain circumspect about the value of much of the quantitative increase in output. It isn’t clear that more emails; more phone calls or text messages necessarily is proof of higher useful and impactful output.
Most of us might also agree that AI automating routine and recurring lower-level tasks will be a benefit, though. Preparing meeting notes; summaries of documents and reports might be examples.
Measuring “quality” of output is going to be harder to measure. More code is not necessarily better code, for example. Sometimes “quality” is going to mean “different” outputs. Those outputs might be “better” and arguably represent “better quality” on a subjective basis.
In my own analytical work, for example, I find AI helps me think about and research different topics than would otherwise be possible, because the research tasks can be completed so much faster. I don’t know whether that is a quality improvement or not, though I subjectively appreciate the ability to ask many questions I might formerly have avoided simply because the research time would be too onerous.
Still, knowledge worker productivity is quite hard to measure.
To the extent we can measure knowledge worker productivity--and there is an argument to be made that we actually cannot measure it effectively--efforts to boost knowledge worker productivity since 2019 have been quite minimal.
Title | Date | Publisher | Key Conclusions |
How Do You Measure Knowledge Worker Productivity? | N/A | Serraview | - Outputs are intangible and difficult to define - Results often based on team output rather than individual - Companies not necessarily tracking hours for salaried employees - Time spent working increasingly blurred in mobile workforce |
The Best Way to Measure Knowledge Worker Productivity | 2022 | Maura Thomas | - Quantitative metrics don't help in short term for knowledge workers - Qualitative metrics matter most in short term - Best measure is questioning how employees feel about their work |
How to Measure Employee Productivity in the Workplace | 2024 | Robin | - Knowledge work is intangible and difficult to categorize - Existing productivity measures rooted in 'machine age' organizations - Impossible to come up with single universal measure for knowledge worker productivity |
Knowledge Worker Productivity | N/A | mediaX at Stanford University | - Serious productivity gap exists between available knowledge and how it is used - Many enterprises fail to fully engage energy and intellect of employees |
Research: Knowledge Workers Are More Productive from Home | 2020 | Harvard Business Review | - Knowledge workers' inputs and outputs can't be tracked like other workers - They apply subjective judgment to tasks and decide what to do when - Can withhold effort often without anyone noticing |
"Boosting the Productivity of Knowledge Workers" | 2006 | McKinsey | Measuring productivity is complex due to the amorphous nature of tasks. Performance is constrained by physical, social, and contextual barriers. Interactions account for a significant portion of work, making targeted improvements essential. |
"Measuring Knowledge Worker Productivity: A Taxonomy" | 2004 | Emerald Group Publishing | Identifies a lack of universally accepted metrics for productivity. Proposes a taxonomy categorizing productivity dimensions and highlights critical areas for future research. |
"Rethinking Knowledge Work: A Strategic Approach" | 2021 | McKinsey | Structured approaches (e.g., workflow systems) improve some metrics but can undermine autonomy and collaboration. Freeform and creative aspects of work are harder to quantify. |
Knowledge worker productivity: is it really impossible to measure it? | 2021 | ResearchGate | Argues that traditional productivity metrics are inadequate for knowledge workers and proposes a new methodology based on human capital efficiency. |
Broken Speedometers: Quantifying Knowledge Worker Effectiveness? | July 1905 | VeraSage Institute | Highlights the challenges of measuring knowledge worker productivity due to the intangible nature of their work and the limitations of traditional metrics. |
What Really Matters for Knowledge Worker Performance | July 1905 | Allsteel | Reviews existing research and concludes that a single, universal metric for measuring knowledge worker productivity does not exist. |
Measuring Knowledge Worker Productivity | July 1905 | Global Workspace Association | Discusses the complexity of measuring knowledge worker productivity and the limitations of traditional methods. |
Knowledge workers are those who “think for a living,” making productivity challenging to measure. According to the U.S. Bureau of Labor Statistics, measuring employee productivity means calculating “output per hour” of work.
But how does one quantify outputs, which often are intangible and difficult to define. Also, when “teams” produce the outcomes, how can individual contributions be assessed?
And there are other complications, such as quantifying “hours worked,” either in-office or remotely.
None of that stops government agencies from doing their best to measure and quantify knowledge worker productivity.
For example, total factor productivity in the United States is said to have grown 0.8 percent from 1987 to 2023, but only 0.5 percent from 2019 to 2023, according to the Bureau of Labor Statistics.
All of which will raise questions when firms and entities start to report “productivity” gains from using AI. If all we can be sure of is that we can measure or quantify some outcomes we believe to be measures of output.
Whether that output actually represents knowledge worker productivity is less certain. Most of us would be circumspect about metrics such as “hours worked” or “email volume” or “meeting attendance.”
We’d probably have some greater confidence about tasks completed, revenue generated by a team, assuming it can be identified.
But lots of common metrics are only quantitative, and cannot measure the quality of work performed or outcomes. People can produce lots of documents, lots of code or “ideas,” but it is hard to measure the quality of those outputs.
Proxy Measure | Description |
Time spent working | Hours logged or time tracked on tasks14 |
Email volume | Number of emails sent/received5 |
Meeting attendance | Number of meetings attended or hours in meetings |
Task completion | Number of tasks or projects completed4 |
Revenue generated | Financial output attributed to individual or team2 |
Information searches | Time spent looking for information15 |
Internal collaboration | Time spent working with colleagues5 |
Documents produced | Number of reports, presentations, or other deliverables created |
Client interactions | Number of client meetings or calls conducted |
Ideas generated | Number of new ideas or innovations proposed |
No comments:
Post a Comment