How Much of What We See in Remote Work is "Hawthorne Effect?"

 We still actually know very little for certain about how productivity changed because of enforced remote work for knowledge and office workers. The problem is that we cannot measure the productivity of such workers easily, if at all.

Also, there are measurement effects, to the extent that enforced remote work is a bit of an experiment. The Hawthorne Effect is that subjects in an experiment tend to perform better. 

There also are demand characteristics. In experiments, researchers sometimes display subtle clues that let participants know what they are hoping to find. As a result, subjects will alter their behavior to help confirm the experimenter’s hypothesis.

Then there are novelty effects: The novelty can lead to an initial increase in performance and productivity that may eventually level off as an experiment continues.

Performance feedback is similar to the Hawthorne Effect. Increased attention from experimenters tends to boost performance. In the short term, that could lead to an improvement in productivity.

That assumes we can measure knowledge worker or office worker productivity, however. 

The problem with all studies of officer worker or knowledge worker productivity is measurement. What can be counted so we know whether inputs have changed. And how do we measure the output of knowledge work? 

Presumably a call center operation has quantifiable metrics, but most office or knowledge work does not have any obvious and convenient measurement criteria. We commonly measure “time working” with the assumption that additional time worked is better. Maybe. But hours worked is an input metric, not an output metric. It is the denominator, not the numerator. 

Logically, increasing input (denominator) can work to reduce productivity (output) unless output measures also increase faster than inputs increase. 

The other common issue is that we equate worker attitudes with outcomes. Happier workers might, or might not, be more productive. All we can measure is a subjective attitude. More happy or less happy does not necessarily correlate with outcomes. 

In principle, one could have happier but less productive workers; less happy but more productive workers. One would need a way to correlate output and outcomes with feelings in ways that outlive simple Hawthorne effects (people work better when they know they are part of an experiment). 

Work team collaboration might have fared better under full remote work conditions, but there is some evidence that firm-wide collaboration has decreased, though the amount of time spent collaborating (meetings, emails, messaging) has grown.  

Actual output is different from input or collaboration time and effort. It might be difficult to measure “creativity,” but there is some belief that has not done better under conditions of remote work.  

Meetings are inputs, not outputs. Having more meetings, or spending more time in meetings, does not make firms or organizations more productive. A Microsoft survey of 182 senior managers in a range of industries found support for that thesis. 

We might say the same for collaboration during the enforced remote work period. It is common to hear technology business or policy leaders argue that remote work has not harmed productivity. 

Leaving aside the issue of whether remote work productivity changes can be measured, collaboration--deemed by most to be vital for knowledge workers--might have gotten far worse because of Covid. 

People like the freedom to work from home, no question.  

That might have happened despite reports that suggest information, knowledge and office workers now are spending more time with electronic forms of communication. But “communication” is not necessarily “collaboration.”

If collaboration is defined as “people working in teams or with others,” then collaboration seemingly has suffered. 

According to Gensler, “high-performing people at top companies tend to do individual work and collaborative work in equal measures—45 percent each, according to our research--with the remaining 10 percent made up of learning and social time.” 

For better or worse, those balances were changed during the period of enforced work from home policies. “While at home during the pandemic, people reported working in individual focus mode 62 percent of the time and 27 percent in collaboration, a disparity that negatively impacts company creativity and productivity,” Gensler argues. 

Before the pandemic, U.S. workers spent an average of 43 percent of their work weeks collaborating either virtually or in person. That number fell to 27 percent for workers who worked from home in 2020, for example. 

“At the onset of the pandemic, our analysis shows that interactions with our close networks at work increased, while interactions with our distant networks diminished,” say Microsoft research. “This suggests that, as we shifted into lockdowns, we clung to our immediate teams for support and let our broader network fall to the wayside.”

There is a downside: similar companies almost certainly became more siloed than they were before the pandemic. 

“And while interactions with our close networks are still more frequent than they were before the pandemic, the trend shows even these close team interactions have started to diminish over time,” Microsoft researchers say. 

Younger workers (25 or younger) also reported more difficulty feeling engaged or excited about work, getting a word in during meetings, and bringing new ideas to the table when compared to other generations.

“Bumping into people in the office and grabbing lunch together may seem unrelated to the success of the organization, but they’re actually important moments where people get to know one another and build social capital,” says Dr. Nancy Baym, Microsoft senior principal researcher “They build trust, they discover common interests they didn’t know they had, and they spark ideas and conversations.”

Microsoft researchers noticed that “at the office: worker instant messages slowed  25 percent during lunchtime, but remote workers at home reduced IMs by 10 percent. Also, IMs grew by 52 percent between 6 p.m. and midnight, suggesting that at-home remote workers might have been working more total hours than employees in the office.

At-home workers also spent about 10 percent more time in meetings. Those results might be interpreted as either good or bad effects of collaboration. 

Microsoft research also suggests that while collaboration within work teams increased, collaboration outside of the teams, with the rest of Microsoft personnel, decreased.  

The point is that we actually know quite little about potential changes in productivity, especially longer-term impact. In the short term, there is a Hawthorne Effect at work, which would “boost productivity” in the short term.