It Will be Hard to Measure AI Impact on Knowledge Worker "Productivity"

There are over 100 million knowledge workers in the United States, and more than 1.25 billion knowledge workers globally, according to one Anatomy of Work estimate. And “work about work,” including unnecessary meetings, status checks and information searchers occupy as much as 60 percent of knowledge worker time. 

Hence the interest in AI agents that can conduct activities autonomously, presumably eliminating much of that “work about work.” Customer support, regulatory compliance, security and marketing are areas where agents are expected to contribute. 

To the extent we can measure knowledge worker productivity--and there is an argument to be made that we actually cannot measure it effectively--efforts to boost knowledge worker productivity since 2019 have been quite minimal. 

Title	Date	Publisher	Key Conclusions
How Do You Measure Knowledge Worker Productivity?	N/A	Serraview	- Outputs are intangible and difficult to define - Results often based on team output rather than individual - Companies not necessarily tracking hours for salaried employees - Time spent working increasingly blurred in mobile workforce
The Best Way to Measure Knowledge Worker Productivity	2022	Maura Thomas	- Quantitative metrics don't help in short term for knowledge workers - Qualitative metrics matter most in short term - Best measure is questioning how employees feel about their work
How to Measure Employee Productivity in the Workplace	2024	Robin	- Knowledge work is intangible and difficult to categorize - Existing productivity measures rooted in 'machine age' organizations - Impossible to come up with single universal measure for knowledge worker productivity
Knowledge Worker Productivity	N/A	mediaX at Stanford University	- Serious productivity gap exists between available knowledge and how it is used - Many enterprises fail to fully engage energy and intellect of employees
Research: Knowledge Workers Are More Productive from Home	2020	Harvard Business Review	- Knowledge workers' inputs and outputs can't be tracked like other workers - They apply subjective judgment to tasks and decide what to do when - Can withhold effort often without anyone noticing
"Boosting the Productivity of Knowledge Workers"	2006	McKinsey	Measuring productivity is complex due to the amorphous nature of tasks. Performance is constrained by physical, social, and contextual barriers. Interactions account for a significant portion of work, making targeted improvements essential.
"Measuring Knowledge Worker Productivity: A Taxonomy"	2004	Emerald Group Publishing	Identifies a lack of universally accepted metrics for productivity. Proposes a taxonomy categorizing productivity dimensions and highlights critical areas for future research.
"Rethinking Knowledge Work: A Strategic Approach"	2021	McKinsey	Structured approaches (e.g., workflow systems) improve some metrics but can undermine autonomy and collaboration. Freeform and creative aspects of work are harder to quantify.
Knowledge worker productivity: is it really impossible to measure it?	2021	ResearchGate	Argues that traditional productivity metrics are inadequate for knowledge workers and proposes a new methodology based on human capital efficiency.
Broken Speedometers: Quantifying Knowledge Worker Effectiveness?	July 1905	VeraSage Institute	Highlights the challenges of measuring knowledge worker productivity due to the intangible nature of their work and the limitations of traditional metrics.
What Really Matters for Knowledge Worker Performance	July 1905	Allsteel	Reviews existing research and concludes that a single, universal metric for measuring knowledge worker productivity does not exist.
Measuring Knowledge Worker Productivity	July 1905	Global Workspace Association	Discusses the complexity of measuring knowledge worker productivity and the limitations of traditional methods.

Knowledge workers are those who “think for a living,” making productivity challenging to measure. According to the U.S. Bureau of Labor Statistics, measuring employee productivity means calculating “output per hour” of work. 

But how does one quantify outputs, which often are intangible and difficult to define. Also, when “teams” produce the outcomes, how can individual contributions be assessed?

And there are other complications, such as quantifying “hours worked,” either in-office or remotely. 

None of that stops government agencies from doing their best to measure and quantify knowledge worker productivity. 

For example, total factor productivity in the United States is said to have grown 0.8 percent from 1987 to 2023, but only 0.5 percent from 2019 to 2023, according to the Bureau of Labor Statistics. 

All of which will raise questions when firms and entities start to report “productivity” gains from using AI.  If all we can be sure of is that we can measure or quantify some outcomes we believe to be measures of output.  

Whether that output actually represents knowledge worker productivity is less certain. Most of us would be circumspect about metrics such as “hours worked” or “email volume” or “meeting attendance.”

We’d probably have some greater confidence about tasks completed, revenue generated by a team, assuming it can be identified. 

But lots of common metrics are only quantitative, and cannot measure the quality of work performed or outcomes. People can produce lots of documents, lots of code or “ideas,” but it is hard to measure the quality of those outputs. 

Proxy Measure	Description
Time spent working	Hours logged or time tracked on tasks14
Email volume	Number of emails sent/received5
Meeting attendance	Number of meetings attended or hours in meetings
Task completion	Number of tasks or projects completed4
Revenue generated	Financial output attributed to individual or team2
Information searches	Time spent looking for information15
Internal collaboration	Time spent working with colleagues5
Documents produced	Number of reports, presentations, or other deliverables created
Client interactions	Number of client meetings or calls conducted
Ideas generated	Number of new ideas or innovations proposed

Nobody Knows if "Return to Office" Boosts Productivity

As a test, this content as a podcast here

It is virtually impossible to test human-driven processes in the same way a scientist might test hypotheses, despite our use of the term social “science,” but none of that inability to truly test cause and effect relationships alleviates the need to make decisions in a void. 

Consider the current debate over “return to office” policies, often touted because “in office” work builds company culture. Ignore for the moment our inability to measure “company culture.” Ignore the possible impact on productivity of office or knowledge workers (since we cannot really measure that, either. 

Researchers do not agree on whether the thesis that “in office” work creates or sustains company culture, even if it might affect other elements of “culture” such as collaboration, job satisfaction, innovation or employee assessment, training and advancement. 

Study Title	Date	Publisher	Key Conclusions
The Surprising Truth About InPerson Work	2024	Robin Powered	Inperson work boosts collaboration, job satisfaction, and innovation. Physical proximity increases knowledge sharing and motivation.
'Office culture' as we know it is dead. Workers have other ideas	2024	BBC	Employees are less interested in pre-pandemic office perks and culture. They prioritize flexibility, equitable pay, and being seen as whole humans.
Why a full return to office is gaining ground	2024	HRD Connect	In-office work enhances company culture through informal interactions and facilitates mentorship. However, employee preferences for flexibility must be considered.
Working in the office 5 days a week to build company culture is a myth	2024	Fortune	Hybrid workers report higher levels of satisfaction, engagement, and feelings of belonging compared to full time in-office workers. Forced return to office can negatively impact culture.
Workforce Radar Report	2024	PwC	Hybrid workers demonstrate the highest levels of satisfaction and engagement. The idea that fulltime in-office work is necessary for strong culture is a myth.
"The Impact of Physical Workspaces on Organizational Culture"	2023	Harvard Business Review	In-office work fosters a stronger sense of community, improves collaboration, and enhances knowledge sharing among employees.
"The Role of InPerson Interactions in Building Company Culture"	2022	Stanford Graduate School of Business	Face-to-face interactions are crucial for building trust, fostering a sense of belonging, and creating a shared company identity.
"The Negative Effects of Remote Work on Company Culture"	2021	MIT Sloan Management Review	Remote work can lead to decreased employee engagement, reduced informal communication, and a weaker sense of organizational connection.
"The Benefits of Hybrid Work Arrangements for Company Culture"	2020	McKinsey & Company	A hybrid work model can balance the advantages of in-office work with the flexibility of remote work, promoting a positive company culture.
"The Impact of Office Design on Employee Productivity and Wellbeing"	2019	Cornell University	A well-designed office environment can enhance employee satisfaction, productivity, and overall well being, contributing to a positive company culture.

Without much doubt, most employees prefer remote work when it is available. What is less clear is whether “return to office” mandates enable firms to function at a higher level of output or quality (assuming the work is amenable to remote venues and assuming you believe you can measure such effects. 

In some cases, studies suggest in-person work does improve productivity, to the extent that collaboration or knowledge sharing matters, and assuming such outcomes differ from remote instances in a material way. 

In other cases, outcomes likely hinge on the actual job functions. Writers, code developers, artists, marketing or advertising functions and many sales functions traditionally have used lots of remote workers, as the work is suited to remote basing. 

Study Title	Date	Publisher	Key Conclusions
The Science behind InPerson Productivity at the Office	2024	Robin Powered	Physical proximity boosts collaboration and knowledge sharing. Employees within 25 feet of high-performing colleagues saw a 15% productivity improvement.
Study finds hybrid work benefits companies and employees	2024	Stanford University	Hybrid work (2 days remote had no negative effect on productivity or career advancement. It reduced employee turnover by 33%.
Americans are embracing flexible work—and they want more of it	2024	McKinsey & Company	Well-organized hybrid work can improve productivity by up to 5%. 83% of employees cite increased efficiency as a primary benefit of remote work.
Study finds workers say they need to be in an office to be productive	2024	Scripps News	Workers report needing to be in the office for twothirds of the workweek to be productive. Top reasons include better focus and accessing technology.
The Surprising Truth About InPerson Work	2024	Robin Powered	In-person collaboration improves task completion by 50% compared to solo work. Face-to-face interactions are more impactful than virtual ones.

Perhaps the actual impact on productivity is not really the issue. If managers believe they are better off with employees in the office (for whatever reasons, that might be all that matters. Irrespective of the matter of productivity, which cannot be measured anyhow, if managers believe they benefit from “in office” work modes, that is likely to happen. 

Still, some jobs traditionally have been highly amenable to remote work. 

Job	Reasons for Remote Work Suitability
Software Developer	Work primarily involves coding and digital tools  Collaboration possible through version control systems  Can easily share and review code remotely
Graphic Designer	Digital design tools accessible from anywhere  Can share and collaborate on designs online  Client meetings can be conducted virtually
Writer/Editor	Writing and editing can be done independently  Digital collaboration tools for feedback and revisions  Research can be conducted online
Data Analyst	Data analysis software accessible remotely  Can access databases and datasets from anywhere  Results easily shareable through cloud platforms
Customer Service Representative	Phone and chat support can be done from home  Customer management systems accessible remotely  Call routing technology allows for distributed teams
Digital Marketer	Online marketing tools and platforms accessible anywhere  Analytics and reporting can be done remotely  Virtual collaboration for campaign planning
Translator	Translation work can be done independently  Digital resources and translation tools available online  Can communicate with clients virtually
Accountant	Accounting software accessible from anywhere  Digital document sharing for financial records  Virtual meetings for client consultations
Project Manager	Project management tools allow for remote team coordination  Virtual meetings for team updates and planning  Can track progress and deadlines online
Online Teacher/Tutor	Virtual classrooms and learning management systems  Can conduct lessons via video conferencing  Digital resources for curriculum and assignments
Remote Sales	Outside sales staff often are based wherever key clients are Inside sales can be based anywhere

According to one 2022 survey (which might still show the impact of Covid work-from-home policies, many industries were seeing high growth of remote work. 

Finance and Insurance   2,062%

Legal and Accounting   1,995%

Retail   1,651%

Aerospace   1,341%

Media   1,280%

Real Estate, Construction   1,150%

Information Technology     1,132%

Education, Govt., Non-profit     1,097%

Consumer Technology             1,028%

Consumer Goods   1,651%

Hospitals,Medical Centers       1,028%

Healthcare       859%

Telecommunications     1,020%

Energy and Utilities                      897%

Technical Services                      878%

Enterprise Technology                 701%

Manufacturing       869%

Food and Beverage         719%

Hospitality and Recreation           644%

Transportation         634%

Business Services         550%

Staffing         475%

Pharmaceuticals         248%

source: The Muse, Ladders 

And though large enterprises are shifting back to “in office” work, remote work job openings might still be increasing in 2024, compared to 2023. The number of “remote” job postings peaked in 2022, as Covid pandemic restrictions prohibited much in-office work.  

source: Bloomberry

All that noted, remote work is both an employee preference issue and a management comfort issue, at least for jobs that can plausibly be done on a remote basis.