1
00:00:00,640 --> 00:00:04,445
Two questions, first,
what happens to mentoring and
2
00:00:04,445 --> 00:00:10,250
to learning on the job when employees no
longer work together in the same location?
3
00:00:11,270 --> 00:00:15,378
Second, and related,
how does the rise of remote work affect
4
00:00:15,378 --> 00:00:18,388
productivity in the near term and
over time?
5
00:00:18,388 --> 00:00:27,198
[MUSIC]
6
00:00:27,198 --> 00:00:29,596
>> Steven Davis: Welcome
to Economics Applied,
7
00:00:29,596 --> 00:00:33,246
a podcast sponsored by
the Hoover Institution.
8
00:00:33,246 --> 00:00:36,700
My name is Steven Davis,
and I host the show.
9
00:00:36,700 --> 00:00:41,077
Our guest today has some very insightful
research that speaks to these two
10
00:00:41,077 --> 00:00:42,020
questions.
11
00:00:42,020 --> 00:00:47,140
She is Emma Harrington, an economics
professor at the University of Virginia.
12
00:00:47,140 --> 00:00:50,628
Emma earned her PhD in
economics at Harvard, and
13
00:00:50,628 --> 00:00:56,414
before her recent move to Virginia, she
held a postdoc position at Princeton and
14
00:00:56,414 --> 00:00:59,832
was on the faculty at
the University of Iowa.
15
00:00:59,832 --> 00:01:03,280
Welcome, Emma,
it's really great to have you on the show.
16
00:01:03,280 --> 00:01:05,072
>> Emma Harrington: Thanks so
much for having me.
17
00:01:05,072 --> 00:01:10,150
>> Steven Davis: So you co authored a
recent study titled the power of proximity
18
00:01:10,150 --> 00:01:15,760
to co workers, training for
tomorrow or productivity today.
19
00:01:15,760 --> 00:01:20,066
So the title kind of sets forth
what the paper is really after, and
20
00:01:20,066 --> 00:01:24,847
that's in this fundamental point
that proximity to coworkers involved
21
00:01:24,847 --> 00:01:29,782
a trade off between productivity today and
productivity in the future.
22
00:01:29,782 --> 00:01:32,382
So before we get into
the details of the study,
23
00:01:32,382 --> 00:01:36,606
just flesh out the nature of that trade
off a little bit for our audience.
24
00:01:36,606 --> 00:01:41,449
>> Emma Harrington: Yeah, definitely, so
we're particularly interested in studying
25
00:01:41,449 --> 00:01:44,953
how remote work plays out for
knowledge workers,
26
00:01:44,953 --> 00:01:50,510
where there's lots of things that we
mean when we say the word productivity.
27
00:01:50,510 --> 00:01:56,424
So some of it is what we mean is sort
of how much are you churning out today?
28
00:01:56,424 --> 00:01:58,792
In our particular setting
is software engineers.
29
00:01:58,792 --> 00:02:02,272
So you can think of that as how many
programs are you writing today?
30
00:02:02,272 --> 00:02:06,534
But then there's also this other
component of your job where you're
31
00:02:06,534 --> 00:02:10,600
thinking about mentoring other people or
learning yourself.
32
00:02:10,600 --> 00:02:15,672
And there we find that being out of the
office is a little bit more challenging.
33
00:02:15,672 --> 00:02:20,117
People actually write more programs
when they're not sitting together, but
34
00:02:20,117 --> 00:02:23,040
they seem to also be
engaged in less mentorship.
35
00:02:23,040 --> 00:02:25,670
>> Steven Davis: Sounds
a lot like professors, too.
36
00:02:25,670 --> 00:02:27,214
>> Emma Harrington: Yeah,
we've heard a lot.
37
00:02:27,214 --> 00:02:29,730
>> Steven Davis: Trade off is
very pronounced for professors.
38
00:02:31,190 --> 00:02:34,897
Okay, so, that's the issue,
but so tell us,
39
00:02:34,897 --> 00:02:41,590
give us the context in which you tried to
address this issue in a compelling way.
40
00:02:41,590 --> 00:02:44,268
I'll let you kind of lay
it out as you wish, but
41
00:02:44,268 --> 00:02:48,117
it's both the context of your study,
the data you had available,
42
00:02:48,117 --> 00:02:52,676
which is impressive in some respects,
and then the research design itself.
43
00:02:52,676 --> 00:02:54,044
So sit that out for us,
44
00:02:54,044 --> 00:02:57,820
how did you go about tackling
this really important issue?
45
00:02:57,820 --> 00:02:58,996
>> Emma Harrington: Yeah, definitely.
46
00:02:58,996 --> 00:03:04,056
So we have data from
a particular Fortune 500 firm,
47
00:03:04,056 --> 00:03:08,372
and a setting has a few key advantages for
us.
48
00:03:08,372 --> 00:03:12,472
One is that the firm had two
office buildings of engineers on
49
00:03:12,472 --> 00:03:16,890
its main campus that were about
a ten minute walk, apartheid.
50
00:03:16,890 --> 00:03:19,802
And because of limited desk availability,
51
00:03:19,802 --> 00:03:24,288
some teams ended up split across
those two different buildings.
52
00:03:24,288 --> 00:03:29,742
And that mattered for the teams dynamics
because it's pretty industry standard for
53
00:03:29,742 --> 00:03:33,825
engineers to have daily short
stand up meetings with the idea.
54
00:03:33,825 --> 00:03:36,488
>> Steven Davis: These are all
software engineers, right?
55
00:03:36,488 --> 00:03:38,120
That's what we're talking about here.
56
00:03:38,120 --> 00:03:41,722
>> Emma Harrington: Yeah, so they all
basically worked on creating the website
57
00:03:41,722 --> 00:03:47,040
of the firm and maintaining the databases
for the firm, it's an online retail firm.
58
00:03:47,040 --> 00:03:51,080
So these are pretty standard tasks for
engineers.
59
00:03:51,080 --> 00:03:54,384
>> Steven Davis: Okay, so you got some of
these guys working together all the time.
60
00:03:54,384 --> 00:03:57,960
Some of them are, some teams,
I should say are split apart.
61
00:03:57,960 --> 00:04:01,840
How many teams,
how many engineers overall in your study?
62
00:04:01,840 --> 00:04:05,837
>> Emma Harrington: So
it's 1000 software engineers and
63
00:04:05,837 --> 00:04:09,824
there's just a few people on each team.
64
00:04:09,824 --> 00:04:11,826
So also lots of teams, and
65
00:04:11,826 --> 00:04:17,072
about 60% of them were all co
located in one of the two buildings.
66
00:04:17,072 --> 00:04:23,166
And then the remaining 40% of them were
split across the two office buildings.
67
00:04:23,166 --> 00:04:27,545
>> Steven Davis: Okay, so you've got
a lot of useful variation in the data,
68
00:04:27,545 --> 00:04:29,742
as economists would put it.
69
00:04:29,742 --> 00:04:34,324
And then the setting, the time, the
temporal setting is also very important
70
00:04:34,324 --> 00:04:39,430
for your study, the period covered by
the data, so just lay that out for us.
71
00:04:39,430 --> 00:04:43,784
>> Emma Harrington: Yeah, so we span
the period before the office closure
72
00:04:43,784 --> 00:04:48,698
of the pandemic to a few months
after the office closures.
73
00:04:48,698 --> 00:04:53,900
And so we can see what happens to
the differences between these co located
74
00:04:53,900 --> 00:04:58,680
teams versus these distributed
teams once everyone's remote for
75
00:04:58,680 --> 00:05:03,370
the pandemic, we expect if the things
were causal to begin with,
76
00:05:03,370 --> 00:05:07,610
they should go away once
everyone is in the same boat.
77
00:05:07,610 --> 00:05:12,457
>> Steven Davis: Okay, and there are,
as you suggested, there are differences
78
00:05:12,457 --> 00:05:18,508
between the co located teams and the other
teams before the pandemic struck.
79
00:05:18,508 --> 00:05:21,753
And then there's going to be maybe
differences, maybe similarities,
80
00:05:21,753 --> 00:05:23,556
we'll talk about that afterwards.
81
00:05:23,556 --> 00:05:28,587
But you got this event, which is pretty,
this is an event we can safely
82
00:05:28,587 --> 00:05:33,668
say was largely unanticipated and
arrived with extreme abruptness.
83
00:05:33,668 --> 00:05:38,230
So you have a firm that is operating
in a certain way with respect to its
84
00:05:38,230 --> 00:05:39,876
software engineers.
85
00:05:39,876 --> 00:05:44,660
This big shock forces them to do something
different in terms of co location of
86
00:05:44,660 --> 00:05:46,190
workers.
87
00:05:46,190 --> 00:05:49,630
But some teams were highly
co located before and
88
00:05:49,630 --> 00:05:53,110
some were split across the two buildings.
89
00:05:53,110 --> 00:05:55,634
>> Emma Harrington: Yeah, and
so are already, in some senses,
90
00:05:55,634 --> 00:05:57,342
acting like remote teams.
91
00:05:57,342 --> 00:06:03,470
They held some of their meetings remotely,
even when they were in the office.
92
00:06:03,470 --> 00:06:05,420
>> Steven Davis: Before
we turn to your results,
93
00:06:05,420 --> 00:06:10,128
tell us a little bit more about exactly
what you can measure in this context.
94
00:06:10,128 --> 00:06:14,780
That's another very attractive
feature of your study.
95
00:06:16,280 --> 00:06:20,106
>> Emma Harrington: Yeah, I think the
really special thing about our study is
96
00:06:20,106 --> 00:06:24,970
that software engineers create this
sort of paper trail of mentorship,
97
00:06:24,970 --> 00:06:30,549
that it's really standard that if you're
writing a program at one of these firms,
98
00:06:30,549 --> 00:06:35,332
you're gonna have to get it peer
reviewed by another programmer who's
99
00:06:35,332 --> 00:06:39,557
gonna give you pretty detailed
feedback on your program before
100
00:06:39,557 --> 00:06:42,520
it can actually be used in production.
101
00:06:42,520 --> 00:06:47,084
These firms really don't want there to
be any bugs or glitches in the code, and
102
00:06:47,084 --> 00:06:50,360
so they have this peer review process.
103
00:06:50,360 --> 00:06:52,177
>> Steven Davis: Are you
confident that all or
104
00:06:52,177 --> 00:06:56,718
most of the feedback actually takes
this written form, or is there also
105
00:06:56,718 --> 00:07:01,552
feedback occurring that you're not
measuring because it's just oral?
106
00:07:01,552 --> 00:07:05,533
>> Emma Harrington: So, I think our
setting means that a lot of it is captured
107
00:07:05,533 --> 00:07:11,212
in the code review process because part
of it is that they want documentation for
108
00:07:11,212 --> 00:07:14,908
the reason why there were
changes to the program.
109
00:07:14,908 --> 00:07:18,605
So there are some references
to conversations that occur
110
00:07:18,605 --> 00:07:20,068
outside the system.
111
00:07:20,068 --> 00:07:26,300
So we can directly see things like
references to slack conversations.
112
00:07:26,300 --> 00:07:30,260
And those go in sort of the same
direction as our main results.
113
00:07:30,260 --> 00:07:34,316
>> Steven Davis: Okay, and
one more thing about the data.
114
00:07:34,316 --> 00:07:37,287
You have some information on
the quality of the feedback,
115
00:07:37,287 --> 00:07:39,711
as well as not just
measuring the quantity, but
116
00:07:39,711 --> 00:07:43,278
you've got some information on
the quality, the feedback as well.
117
00:07:43,278 --> 00:07:46,454
So that's gonna be important for
some aspects of your results.
118
00:07:46,454 --> 00:07:49,446
So how do you assess
the quality of feedback?
119
00:07:49,446 --> 00:07:51,102
How do you know it's even useful?
120
00:07:51,102 --> 00:07:54,170
>> Emma Harrington: Yeah, so, I mean, we
can look at that in a few different ways.
121
00:07:54,170 --> 00:07:58,795
So we can, we have the text of all
the feedback that is given and so
122
00:07:58,795 --> 00:08:03,089
we can look at measures of how
substantive the feedback is.
123
00:08:03,089 --> 00:08:09,228
Like, does it include a block of code, for
example, or a link to other documentation?
124
00:08:09,228 --> 00:08:15,836
We also had a subsample of the comments
reviewed by external reviewers who,
125
00:08:15,836 --> 00:08:20,689
by and large,
said that the comments were impactful and
126
00:08:20,689 --> 00:08:23,168
likely to change the code and
127
00:08:23,168 --> 00:08:28,865
also often provided actionable
steps to take to change the code.
128
00:08:30,725 --> 00:08:33,813
Yeah, those are our main
measures of the quality.
129
00:08:33,813 --> 00:08:35,883
>> Steven Davis: So
this is a data rich environment,
130
00:08:35,883 --> 00:08:39,609
an interesting physical setting,
an obvious shock that hits,
131
00:08:39,609 --> 00:08:43,680
that causes things to change over time,
and you're able to measure,
132
00:08:43,680 --> 00:08:47,677
follow individual engineers and
teams before and after the shock.
133
00:08:47,677 --> 00:08:49,317
So very nice setting.
134
00:08:49,317 --> 00:08:53,139
So before the shock hit, though, there
were already some noticeable differences
135
00:08:53,139 --> 00:08:55,583
between the co-located teams and
the other teams.
136
00:08:55,583 --> 00:08:56,663
Tell us about that.
137
00:08:56,663 --> 00:09:00,999
>> Emma Harrington: Yeah, so the
co-located teams received about 20% more
138
00:09:00,999 --> 00:09:03,855
comments on each of their programs.
139
00:09:03,855 --> 00:09:08,147
And as we've been talking about, that's
a pretty notable difference because what
140
00:09:08,147 --> 00:09:11,407
we're measuring is feedback
that it's occurring online.
141
00:09:11,407 --> 00:09:15,574
So to the extent to which there was some
feedback that was occurring face to face,
142
00:09:15,574 --> 00:09:18,583
presumably that would be
easier if you were co located.
143
00:09:18,583 --> 00:09:22,513
So we're likely providing a lower
bound on how big the differences in
144
00:09:22,513 --> 00:09:26,543
mentorship are between these
co-located and distributed teams.
145
00:09:26,543 --> 00:09:27,215
>> Steven Davis: Okay, and
146
00:09:27,215 --> 00:09:32,375
to what extent is the feedback going from
senior people to junior people as opposed
147
00:09:32,375 --> 00:09:38,135
to just horizontal people at roughly
the same experience level within the firm?
148
00:09:38,135 --> 00:09:40,263
>> Emma Harrington: There's some of both.
149
00:09:40,263 --> 00:09:45,184
The effects of proximity seem to be
really concentrated among feedback
150
00:09:45,184 --> 00:09:49,759
received by junior people and
feedback given by senior people.
151
00:09:49,759 --> 00:09:54,796
So some of the sort of horizontal
feedback that happens, but
152
00:09:54,796 --> 00:10:01,181
it doesn't seem to be as sensitive
to whether you're co-located or not.
153
00:10:01,181 --> 00:10:01,692
>> Steven Davis: Okay.
154
00:10:01,692 --> 00:10:05,288
>> Emma Harrington: I think some of that
may be because of the underlying mechanism
155
00:10:05,288 --> 00:10:09,853
that some of it is because it's important
to be co-located in terms of feeling
156
00:10:09,853 --> 00:10:14,073
comfortable asking for additional
feedback, and maybe if you feel like
157
00:10:14,073 --> 00:10:19,135
you're in a peer situation, you feel
comfortable doing that even remotely.
158
00:10:19,135 --> 00:10:22,756
>> Steven Davis: Or the flip side of
that is the senior person may be more
159
00:10:22,756 --> 00:10:24,286
comfortable saying,
160
00:10:24,286 --> 00:10:29,617
not providing feedback when they don't
have to see the person in the hallway.
161
00:10:29,617 --> 00:10:30,735
>> Emma Harrington: Yeah.
162
00:10:30,735 --> 00:10:35,746
>> Steven Davis: We'll get into later
more whether this feedback is actually,
163
00:10:35,746 --> 00:10:40,703
on the margin, the best use of time for
all parties involved.
164
00:10:40,703 --> 00:10:43,364
So I wanna come back to that, but okay, so
165
00:10:43,364 --> 00:10:48,770
there's more feedback received primarily
by junior people in co-located teams,
166
00:10:48,770 --> 00:10:52,321
and the feedback is largely
given by senior people.
167
00:10:52,321 --> 00:10:55,491
So senior people, well,
their time is pretty valuable, so
168
00:10:55,491 --> 00:10:58,477
there's an opportunity cost
here to this feedback, but
169
00:10:58,477 --> 00:11:02,385
presumably there's also quite a bit
of benefit for the junior people.
170
00:11:02,385 --> 00:11:06,266
And so we can already see even
in a kind of a static sense,
171
00:11:06,266 --> 00:11:09,585
there's a trade off here in attention.
172
00:11:09,585 --> 00:11:14,097
But then the dynamic one is, well, if the
junior people don't get enough feedback,
173
00:11:14,097 --> 00:11:16,761
they won't improve
rapidly enough over time.
174
00:11:16,761 --> 00:11:22,105
So then the pandemic hits played out for
us, what happened?
175
00:11:22,105 --> 00:11:25,598
>> Emma Harrington: So, feedback in
general declined across the board, but
176
00:11:25,598 --> 00:11:30,039
it declined much more for engineers
that had been co-located with their
177
00:11:30,039 --> 00:11:34,771
teammates than engineers who are already
distributed, and in some senses,
178
00:11:34,771 --> 00:11:37,433
already acting a bit like remote teams.
179
00:11:37,433 --> 00:11:42,057
And so, basically, that whole difference
between those co-located teams and
180
00:11:42,057 --> 00:11:46,793
those distributed teams when the offices
were open disappears once every once.
181
00:11:46,793 --> 00:11:48,313
>> Steven Davis: Okay, so
it completely disappears.
182
00:11:48,313 --> 00:11:50,538
But how much does the overall decline,
say, for
183
00:11:50,538 --> 00:11:52,705
the teams that were originally co-located?
184
00:11:52,705 --> 00:11:54,201
How big is that decline?
185
00:11:54,201 --> 00:11:59,225
>> Emma Harrington: I mean,
it's pretty enormous if you include.
186
00:11:59,225 --> 00:12:01,284
So, like, the gap goes away, and
187
00:12:01,284 --> 00:12:05,275
then also you just have a secular
decline across the board.
188
00:12:05,275 --> 00:12:10,367
So I think if you added it all together,
it would be like something
189
00:12:10,367 --> 00:12:14,755
along the order of a third
of the feedback disappears.
190
00:12:14,755 --> 00:12:19,286
Now, it's unclear exactly how much
you wanna think of that as all being
191
00:12:19,286 --> 00:12:24,027
a causal effect of proximity,
because you do have the pandemic going on.
192
00:12:24,027 --> 00:12:28,555
And so you might think that there's
other shocks to these engineers.
193
00:12:28,555 --> 00:12:31,215
It's not just because of
the loss of proximity.
194
00:12:32,615 --> 00:12:35,638
>> Steven Davis: Okay, so
if they were, for example,
195
00:12:35,638 --> 00:12:38,378
highly stressed by the pandemic,
196
00:12:38,378 --> 00:12:43,831
their overall work effectiveness
might have diminished, right?
197
00:12:43,831 --> 00:12:45,623
Is that a coming mind?
198
00:12:45,623 --> 00:12:49,439
Okay, but what happened to their
overall code production, though,
199
00:12:49,439 --> 00:12:51,047
in the wake of the pandemic?
200
00:12:51,047 --> 00:12:52,395
Did that go up or down?
201
00:12:54,175 --> 00:12:59,762
>> Emma Harrington: So, it went down
overall, but it went down by more for
202
00:12:59,762 --> 00:13:04,852
the people who had been
previously across the two
203
00:13:04,852 --> 00:13:11,087
buildings than people who had
been on one building teams.
204
00:13:11,087 --> 00:13:15,623
And so, relative terms, the engineers
who had been co-located saw their
205
00:13:15,623 --> 00:13:18,647
productivity in terms of
the number of programs,
206
00:13:18,647 --> 00:13:22,599
they won't go up relative to
those that have been distributed.
207
00:13:22,599 --> 00:13:26,259
>> Steven Davis: Okay, but as I
understand it, both mentorship measured,
208
00:13:26,259 --> 00:13:30,945
as you described earlier,
feedback might be a more accurate term.
209
00:13:30,945 --> 00:13:36,231
Feedback fell roughly a third for
the co-located teams fell less,
210
00:13:36,231 --> 00:13:41,377
but from a smaller original base for
the non co-located teams.
211
00:13:41,377 --> 00:13:46,409
But lines of code written also fell,
is that correct?
212
00:13:46,409 --> 00:13:51,567
Okay, so there was a general across
the board negative response,
213
00:13:51,567 --> 00:13:57,001
which, as you pointed out,
was a mix that we maybe can't fully unpack
214
00:13:57,001 --> 00:14:02,067
between the lost benefits of proximity and
the pandemic itself,
215
00:14:02,067 --> 00:14:08,527
disrupting people's lives, maybe even
their mental states in many respects.
216
00:14:08,527 --> 00:14:12,095
Okay, do I have it right there?
217
00:14:12,095 --> 00:14:13,335
>> Emma Harrington: Yeah,
no, that's right.
218
00:14:13,335 --> 00:14:17,795
I mean, I think that's why we
focus mainly on the difference and
219
00:14:17,795 --> 00:14:22,433
differences of thinking about how
the one building teams- Sure.
220
00:14:22,433 --> 00:14:27,607
Changed around the office closures
need to be the multi building teams.
221
00:14:27,607 --> 00:14:31,768
>> Steven Davis: Right, but I wanna put
myself in the shoes of the executives at
222
00:14:31,768 --> 00:14:35,879
this firm, the ones overseeing
the software engineers.
223
00:14:35,879 --> 00:14:38,727
So they must have seen
some of this in real time.
224
00:14:38,727 --> 00:14:42,915
And while for research purposes,
the difference in differences,
225
00:14:42,915 --> 00:14:46,703
it's extremely interesting
from a business perspective.
226
00:14:46,703 --> 00:14:51,695
They say, whoa, [LAUGH] l there's a lot
of lost outputs really down here.
227
00:14:51,695 --> 00:14:53,999
Now, how did they react?
228
00:14:53,999 --> 00:14:56,527
Or did they react to
this in near real time?
229
00:14:56,527 --> 00:14:57,455
What did they say?
230
00:14:57,455 --> 00:14:58,183
What did they do?
231
00:14:58,183 --> 00:15:00,671
I presume you've talked
to them about this.
232
00:15:00,671 --> 00:15:05,535
>> Emma Harrington: Yeah, we have
talked to some engineers at the firm.
233
00:15:05,535 --> 00:15:10,540
Some of them said it was really
resonant with their own experiences,
234
00:15:10,540 --> 00:15:15,457
that some senior engineers,
at least, felt, in relative terms,
235
00:15:15,457 --> 00:15:18,478
like they were being pretty productive in
236
00:15:18,478 --> 00:15:23,169
terms of how many programs they
were writing in the pandemic.
237
00:15:23,169 --> 00:15:24,853
>> Steven Davis: Relative
to their own past or
238
00:15:24,853 --> 00:15:29,632
relative to other [CROSSTALK] I think they
viewed it as relative to their own past.
239
00:15:29,632 --> 00:15:32,872
>> Emma Harrington: I mean, I guess the
data in an aggregate sense is inconsistent
240
00:15:32,872 --> 00:15:33,545
with that.
241
00:15:33,545 --> 00:15:37,485
Although on the individual level,
I'm sure for some people it is.
242
00:15:38,865 --> 00:15:42,700
And so for the people we talked to,
they felt subjectively
243
00:15:42,700 --> 00:15:47,905
like they were being more productive
than they had been in the office.
244
00:15:47,905 --> 00:15:52,043
But they had some sneaking suspicion that
that was partly because they were devoting
245
00:15:52,043 --> 00:15:53,592
less time giving-
>> Steven Davis: Okay.
246
00:15:53,592 --> 00:15:55,305
>> Emma Harrington: Feedback
to other people.
247
00:15:55,305 --> 00:15:58,313
>> Steven Davis: This is really
interesting, because I've done a lot of
248
00:15:58,313 --> 00:16:02,665
work relying on subjective assessment
Of performance, I'm just curious, and
249
00:16:02,665 --> 00:16:06,711
maybe you didn't look at this cuz I
don't remember seeing it in the study.
250
00:16:06,711 --> 00:16:12,379
What's the relationship between subjective
assessments of own performance and
251
00:16:12,379 --> 00:16:16,661
your external measures,
especially just code production?
252
00:16:16,661 --> 00:16:21,096
That's probably what they're
answering about when you ask them,
253
00:16:21,096 --> 00:16:23,913
are you getting more
done than you used to?
254
00:16:23,913 --> 00:16:28,514
Are those two things I'd be really
curious to know how they relate to?
255
00:16:28,514 --> 00:16:29,515
>> Emma Harrington: Yeah, I mean,
256
00:16:29,515 --> 00:16:33,215
unfortunately, we weren't able to,
in this setting,
257
00:16:33,215 --> 00:16:38,396
do a sort of systematic survey of people
to get their subjective expectations.
258
00:16:38,396 --> 00:16:39,526
>> Steven Davis: Okay.
>> Emma Harrington: Natalia,
259
00:16:39,526 --> 00:16:43,461
my co-author on this, and
I have another paper about call
260
00:16:43,461 --> 00:16:48,377
center workers where we were able to
more systematically elicit those.
261
00:16:48,377 --> 00:16:53,768
And find some correlation there, but
I'd have to check on the exact numbers.
262
00:16:53,768 --> 00:16:55,393
>> Steven Davis: Okay, well,
let's go back to this study.
263
00:16:55,393 --> 00:17:00,058
Now, you also found some very interesting
differences between men and women.
264
00:17:00,058 --> 00:17:03,504
So flesh that out for us,
cuz it's quite interesting.
265
00:17:03,504 --> 00:17:07,955
>> Emma Harrington: Yes, we find that
the tradeoffs from proximity are much more
266
00:17:07,955 --> 00:17:09,270
acute for women.
267
00:17:09,270 --> 00:17:13,987
So it seems to be more important for
junior women's feedback,
268
00:17:13,987 --> 00:17:19,156
the feedback they receive to be
co-located with their colleagues.
269
00:17:19,156 --> 00:17:20,962
And also more taxing for
270
00:17:20,962 --> 00:17:26,026
senior women's time to be sitting
near their junior colleagues.
271
00:17:26,026 --> 00:17:30,310
So for senior women,
it seems like they have a really big loss
272
00:17:30,310 --> 00:17:33,754
of output from sitting
near their co-workers.
273
00:17:33,754 --> 00:17:36,119
>> Steven Davis: But
how does it work dyadically?
274
00:17:36,119 --> 00:17:42,001
So is it [COUGH] that junior women
are getting a lot more feedback from
275
00:17:42,001 --> 00:17:48,421
senior men when they're in proximity,
or from senior women, or both?
276
00:17:48,421 --> 00:17:49,275
>> Emma Harrington: Both.
277
00:17:49,275 --> 00:17:50,802
>> Steven Davis: Both, okay.
>> Emma Harrington: So he thought it was
278
00:17:50,802 --> 00:17:54,028
possible that it would only
be across genders, but
279
00:17:54,028 --> 00:17:56,740
it looks like it's just across the board.
280
00:17:56,740 --> 00:17:59,957
More important for
junior women to be near their colleagues.
281
00:17:59,957 --> 00:18:02,837
>> Steven Davis: Okay, and for
the senior women who are giving,
282
00:18:02,837 --> 00:18:06,512
they're giving more feedback
when they're co-located.
283
00:18:06,512 --> 00:18:11,392
Because they're both more approachable
to the junior men and the junior women,
284
00:18:11,392 --> 00:18:13,914
or is it primarily from the junior women?
285
00:18:13,914 --> 00:18:15,335
>> Emma Harrington: It's both.
286
00:18:15,335 --> 00:18:20,125
So the only dyad that seems
to be pretty unaffected
287
00:18:20,125 --> 00:18:24,355
by proximity is men
giving feedback to men.
288
00:18:24,355 --> 00:18:27,525
Basically, all the other
combinations are effective.
289
00:18:27,525 --> 00:18:31,964
>> Steven Davis: Okay, do you have any
sense from your conversations with these
290
00:18:31,964 --> 00:18:37,678
software engineers, why the differences
between men and women take this form?
291
00:18:37,678 --> 00:18:38,369
What did they tell you?
292
00:18:38,369 --> 00:18:41,808
Or maybe you didn't ask them about this,
but again,
293
00:18:41,808 --> 00:18:46,331
it's a very interesting question
with relevance beyond your study.
294
00:18:46,331 --> 00:18:46,984
>> Emma Harrington: Yeah.
295
00:18:46,984 --> 00:18:51,868
So, I mean,
we didn't ask the software engineers
296
00:18:51,868 --> 00:18:55,967
of the firm that much about this question.
297
00:18:55,967 --> 00:19:00,487
They expressed a lot of concern at
the time that this was some form
298
00:19:00,487 --> 00:19:01,826
of mansplaining,
299
00:19:01,826 --> 00:19:07,278
which is a general concern that's been
raised when we've given this paper.
300
00:19:07,278 --> 00:19:11,728
And so we were able to do some
things using the text of the code
301
00:19:11,728 --> 00:19:14,587
reviews to try to get at that concern.
302
00:19:14,587 --> 00:19:21,040
So, I mean, one thing is that when we have
the external raters look at the comments.
303
00:19:21,040 --> 00:19:25,499
They actually rate comments that
are received by women as being more
304
00:19:25,499 --> 00:19:29,034
helpful and
more likely to cause changes to the code.
305
00:19:29,034 --> 00:19:33,700
So it doesn't seem like necessarily
women are just getting a bunch of
306
00:19:33,700 --> 00:19:35,637
really nitpicky comments.
307
00:19:35,637 --> 00:19:39,756
And that's true both when they're
proximate to their co-workers.
308
00:19:39,756 --> 00:19:42,973
>> Steven Davis: But even your
basic patterns don't really support
309
00:19:42,973 --> 00:19:45,739
a mansplaining story, as I understand it,
310
00:19:45,739 --> 00:19:49,052
because the senior women
are giving more feedback.
311
00:19:49,052 --> 00:19:52,173
You wouldn't call that
mansplaining [LAUGH].
312
00:19:52,173 --> 00:19:54,948
>> Emma Harrington: Yeah,
I guess, I mean [LAUGH].
313
00:19:54,948 --> 00:19:55,863
>> Steven Davis: And, [LAUGH] and
314
00:19:55,863 --> 00:20:00,724
the junior women are getting more feedback
from both senior women and senior men,
315
00:20:00,724 --> 00:20:04,116
which doesn't really sound
like a mansplaining story.
316
00:20:04,116 --> 00:20:06,500
Okay, so it's not a mansplaining story.
317
00:20:06,500 --> 00:20:08,673
So what do you think it is?
318
00:20:08,673 --> 00:20:12,619
>> Emma Harrington: I think it may be
something about comfort asking for
319
00:20:12,619 --> 00:20:13,547
feedback.
320
00:20:13,547 --> 00:20:18,555
So we particularly see that,
you can sometimes get feedback from
321
00:20:18,555 --> 00:20:23,301
multiple people or
you can get feedback from just one person.
322
00:20:23,301 --> 00:20:28,027
And when junior women
are proximate to their co-workers,
323
00:20:28,027 --> 00:20:33,426
they receive feedback from more
people than when they're remote.
324
00:20:33,426 --> 00:20:35,390
And so because in this setting,
325
00:20:35,390 --> 00:20:40,276
the software engineer themselves is
asking other people to review their code.
326
00:20:40,276 --> 00:20:45,077
It does seem consistent with a story
where when you're remote, you just, and
327
00:20:45,077 --> 00:20:49,152
you're a woman, you just don't
feel that comfortable asking for
328
00:20:49,152 --> 00:20:51,575
this additional feedback and advice.
329
00:20:51,575 --> 00:20:56,300
Whereas if you're in person, maybe you can
see someone in the halls and be like, hey,
330
00:20:56,300 --> 00:20:58,131
do you mind taking a look at this?
331
00:20:58,131 --> 00:21:01,355
And that just feels
a little less pushy to you.
332
00:21:01,355 --> 00:21:05,194
>> Steven Davis: Right, so an important
issue, I think, I know the answer, but
333
00:21:05,194 --> 00:21:08,548
I want to get it on the table for
the sake of the audience.
334
00:21:08,548 --> 00:21:13,748
Is there anything in your study that
tells us whether there was too little or
335
00:21:13,748 --> 00:21:18,053
too much feedback either before
the pandemic or afterwards?
336
00:21:18,053 --> 00:21:23,595
>> Emma Harrington: Yeah, it's hard for
us to sort of horse race the benefits and
337
00:21:23,595 --> 00:21:28,051
costs of the feedback
within our particular data
338
00:21:28,051 --> 00:21:32,957
because it's kind of
relatively short-time span.
339
00:21:32,957 --> 00:21:37,977
And so it's hard for us to see
the returns to the mentorship fully.
340
00:21:37,977 --> 00:21:43,492
From the individual's perspective,
it's useful to get the feedback and
341
00:21:43,492 --> 00:21:47,620
mentorship in terms of
downstream pay raises and also-
342
00:21:47,620 --> 00:21:48,405
>> Steven Davis: Sure, but
343
00:21:48,405 --> 00:21:53,205
it's feedback is by definition
a bilateral transaction [LAUGH] or
344
00:21:53,205 --> 00:21:57,494
not transaction exchange,
it's a bilateral exchange.
345
00:21:57,494 --> 00:22:01,107
So if we're gonna come up with some notion
of whether there's too much or too little.
346
00:22:01,107 --> 00:22:03,582
We got to add a minimum account for
347
00:22:03,582 --> 00:22:07,480
the two parties involved directly,
the feedback.
348
00:22:07,480 --> 00:22:11,734
We may wanna account for the firm as
an overall organization as well, and
349
00:22:11,734 --> 00:22:13,420
external parties, right?
350
00:22:13,420 --> 00:22:17,423
So I just think it's important to
understand, and correct me if I'm wrong,
351
00:22:17,423 --> 00:22:20,886
we don't know whether there was
a too much or too little feedback.
352
00:22:20,886 --> 00:22:23,106
We don't know whether
the men were closer or
353
00:22:23,106 --> 00:22:26,412
the women were closer to what some
optimal notion of feedback is.
354
00:22:26,412 --> 00:22:28,158
Or do you disagree with that?
355
00:22:28,158 --> 00:22:30,452
>> Emma Harrington: No,
I think I broadly agree with that.
356
00:22:30,452 --> 00:22:37,323
I mean, we can see some evidence from the
firm's own policies about what it thought.
357
00:22:37,323 --> 00:22:38,650
Now, it might be [INAUDIBLE].
358
00:22:38,650 --> 00:22:40,785
>> Steven Davis: I was gonna come
to that next, so go ahead, tell me,
359
00:22:40,785 --> 00:22:41,915
that was my next question.
360
00:22:41,915 --> 00:22:43,281
>> Emma Harrington: [LAUGH] Yeah.
361
00:22:43,281 --> 00:22:44,297
So, I mean, and
362
00:22:44,297 --> 00:22:48,668
it's unclear whether the firm was
perfectly well calibrated on this.
363
00:22:48,668 --> 00:22:53,623
But pre-COVID, it did allow some
people to work remotely, and
364
00:22:53,623 --> 00:22:58,051
those people tended to be
sort of mid-level engineers.
365
00:22:58,051 --> 00:23:01,108
So they were not the ones
receiving the most mentorship.
366
00:23:01,108 --> 00:23:03,201
They also were not the managers or
367
00:23:03,201 --> 00:23:07,099
super senior engineers who were
giving the most mentorship.
368
00:23:07,099 --> 00:23:11,087
And so that's consistent with
the firm wanting the most junior,
369
00:23:11,087 --> 00:23:13,199
most senior people to be located.
370
00:23:13,199 --> 00:23:15,767
>> Steven Davis: Okay, so, yeah,
that's a useful piece of information, but
371
00:23:15,767 --> 00:23:17,712
let me come at this from
a different direction.
372
00:23:17,712 --> 00:23:23,214
So I presume you've been
in conversations with
373
00:23:23,214 --> 00:23:29,548
some of the senior people at
the firm after your study.
374
00:23:29,548 --> 00:23:34,064
I'm sure they've read your study or
at least been briefed about it carefully.
375
00:23:34,064 --> 00:23:35,938
So how did they react?
376
00:23:35,938 --> 00:23:40,859
Cuz your study has information that might
cause them to change their behavior or
377
00:23:40,859 --> 00:23:44,983
to modify their policies or
try to reinstitute the old practices.
378
00:23:44,983 --> 00:23:47,575
What did they do?
379
00:23:47,575 --> 00:23:52,232
>> Emma Harrington: They have tried to get
people to come back in a hybrid format.
380
00:23:52,232 --> 00:23:53,213
>> Steven Davis: A hybrid format?
381
00:23:53,213 --> 00:23:55,905
Okay, so-
>> Emma Harrington: A hybrid format, yeah.
382
00:23:55,905 --> 00:23:59,489
>> Steven Davis: That's interesting, and
that's typical of what I see in many
383
00:23:59,489 --> 00:24:04,289
firms, which is, They don't try to go all
the way back, either because they don't
384
00:24:04,289 --> 00:24:08,821
think it's necessary or productive or
often because there's resistance from
385
00:24:08,821 --> 00:24:13,243
their employees, but they feel that
it's essential to have some FaceTime.
386
00:24:13,243 --> 00:24:16,512
And, that's basically
the reaction to your study?
387
00:24:16,512 --> 00:24:19,463
>> Emma Harrington: Yeah, it's unclear
to me whether it's coming from,
388
00:24:19,463 --> 00:24:22,862
they don't think it's helpful
to be in person all the time, or
389
00:24:22,862 --> 00:24:25,330
whether they just think it's infeasible.
390
00:24:25,330 --> 00:24:31,632
I mean, even the three days a week back
in the office has been a very hard sell.
391
00:24:31,632 --> 00:24:32,793
I think so.
392
00:24:32,793 --> 00:24:36,346
>> Steven Davis: Okay.
>> Emma Harrington: Yeah.
393
00:24:36,346 --> 00:24:40,845
>> Steven Davis: We didn't talk about this
earlier, but I know from reading your
394
00:24:40,845 --> 00:24:46,608
study that at least before the pandemic,
feedback was not explicitly rewarded.
395
00:24:46,608 --> 00:24:51,508
Of course, it can feed into performance
reviews and so on, but it's not like there
396
00:24:51,508 --> 00:24:56,128
was some formula that says, you get this
much compensation for the quality and
397
00:24:56,128 --> 00:25:00,481
frequent and I volume of your feedback and
this much for your code or so on.
398
00:25:00,481 --> 00:25:05,312
But after, now we settled into
this new norm where at this firm,
399
00:25:05,312 --> 00:25:10,681
at least it sounds like there's a lot
of hybrid working arrangements,
400
00:25:10,681 --> 00:25:15,693
has the firm tried to explicitly
incentivize or disincentivize,
401
00:25:15,693 --> 00:25:20,547
I suppose feedback,
especially from senior to junior people?
402
00:25:20,547 --> 00:25:23,211
>> Emma Harrington: Not to our knowledge.
403
00:25:23,211 --> 00:25:26,060
I think when we talked
to people at the firm,
404
00:25:26,060 --> 00:25:29,834
they said that that would be
very hard because basically,
405
00:25:29,834 --> 00:25:34,241
it's pretty hard to evaluate whether
comments are helpful or not.
406
00:25:34,241 --> 00:25:38,033
And so
if you incentivize giving more feedback,
407
00:25:38,033 --> 00:25:43,131
you would probably get a lot of
relatively low quality feedback.
408
00:25:43,131 --> 00:25:45,440
>> Steven Davis: Okay,
yeah, that makes sense,
409
00:25:45,440 --> 00:25:49,893
if you reward something explicitly
that can distort behavior,
410
00:25:49,893 --> 00:25:54,520
in ways that's kind of a well known
point in personnel economics.
411
00:25:54,520 --> 00:25:58,453
But still,
what has the firm done anything,
412
00:25:58,453 --> 00:26:01,767
if not through explicit incentives,
413
00:26:01,767 --> 00:26:07,062
to try to ensure that there is
an adequate level of feedback.
414
00:26:07,062 --> 00:26:12,568
Now that they're in a world where many
people are working in hybrid mode?
415
00:26:12,568 --> 00:26:16,714
>> Emma Harrington: I mean, something kind
of interesting we can see in the data is
416
00:26:16,714 --> 00:26:22,060
that it seems like there's a shift towards
a greater proportion of the feedback
417
00:26:22,060 --> 00:26:27,911
you're receiving coming from teammates as
opposed to from people outside your team.
418
00:26:27,911 --> 00:26:33,271
And so I think that's really consistent
with thinking that within a team,
419
00:26:33,271 --> 00:26:39,057
it's maybe a little bit easier for the
manager to observe those interactions and
420
00:26:39,057 --> 00:26:43,824
potentially reward them in something
like a performance review,
421
00:26:43,824 --> 00:26:48,015
even if they're not getting
explicit incentives for it.
422
00:26:48,015 --> 00:26:52,429
>> Steven Davis: The reason I'm
probing these questions is because,
423
00:26:52,429 --> 00:26:57,724
one of the key things into my mind
about the longer term effects of
424
00:26:57,724 --> 00:27:04,100
the rise of remote work, on productivity,
on human capital development and
425
00:27:04,100 --> 00:27:08,437
mentorship and so on,
is how firms adapt over time.
426
00:27:08,437 --> 00:27:13,317
And it's pretty clear that in many
settings, some adaptation is necessary.
427
00:27:13,317 --> 00:27:17,818
If you suddenly, there's a lot of
things that happen automatically,
428
00:27:17,818 --> 00:27:21,568
your study kind of shows this,
when people are co located,
429
00:27:21,568 --> 00:27:26,219
you get more feedback, even through
this formalized, written manner,
430
00:27:26,219 --> 00:27:28,866
than you do when they're not co located.
431
00:27:28,866 --> 00:27:32,949
And now if we go to a world where
there's a lot less co location,
432
00:27:32,949 --> 00:27:37,725
than there was before, firms need to
adapt to replace that which is lost,
433
00:27:37,725 --> 00:27:41,603
by the way, in this hybrid mode,
maybe the answer to this.
434
00:27:41,603 --> 00:27:43,993
When people are working
in hybrid mode now,
435
00:27:43,993 --> 00:27:48,062
are they encouraged to come to the office
on a certain set number of days, or
436
00:27:48,062 --> 00:27:52,539
they're kind of spread out over the week,
or they can come in whenever they want.
437
00:27:52,539 --> 00:27:55,219
>> Emma Harrington: Last time
we talked to people at the firm,
438
00:27:55,219 --> 00:27:59,278
it was at the discretion of
the employee what days they came in,
439
00:27:59,278 --> 00:28:03,355
which our studies suggest would
be potentially problematic.
440
00:28:03,355 --> 00:28:04,474
>> Steven Davis: Yeah, that's right.
441
00:28:04,474 --> 00:28:08,350
Because then [LAUGH] going from
five days a week in the office,
442
00:28:08,350 --> 00:28:10,863
to only three days a week in the office.
443
00:28:10,863 --> 00:28:14,563
Where it's not the same,
three days a week for everybody,
444
00:28:14,563 --> 00:28:17,157
is not a 40% reduction in colocation.
445
00:28:17,157 --> 00:28:21,594
It's much bigger than a 40%
reduction in colocation.
446
00:28:21,594 --> 00:28:24,905
>> Emma Harrington: Yeah, and
ours, anyone suggests much bigger,
447
00:28:24,905 --> 00:28:29,751
because we find that even having one
teammate in a different building,
448
00:28:29,751 --> 00:28:34,032
in our setting or remote,
it changes the interactions between
449
00:28:34,032 --> 00:28:38,416
coworkers who themselves
are actually in the office together.
450
00:28:38,416 --> 00:28:42,654
And we think that that's because if you
just have one guy who didn't come in
451
00:28:42,654 --> 00:28:45,426
that day,
you need to have your meeting online.
452
00:28:45,426 --> 00:28:49,532
And so then you don't have the schmoozing
on the way to the meeting,
453
00:28:49,532 --> 00:28:53,795
on the way back from the meeting,
and it just changes those dynamics.
454
00:28:53,795 --> 00:28:54,384
>> Steven Davis: Yeah, So
455
00:28:54,384 --> 00:28:58,420
tell us a bit more about exactly
what you found in this regard.
456
00:28:58,420 --> 00:29:01,770
So John's home today.
457
00:29:01,770 --> 00:29:04,043
His four or
five teammates are in the office, but
458
00:29:04,043 --> 00:29:05,879
they got to get on a Zoom call with John.
459
00:29:05,879 --> 00:29:11,157
So what does that do,
according to your analysis,
460
00:29:11,157 --> 00:29:16,818
to current code writing and
the extent of mentorship?
461
00:29:16,818 --> 00:29:21,695
>> Emma Harrington: Yeah, so we find,
most of our analysis that have focused on
462
00:29:21,695 --> 00:29:24,537
the mentorship aspect of things.
463
00:29:24,537 --> 00:29:30,137
And we find that about a third of
the effect of being on a team that's
464
00:29:30,137 --> 00:29:36,938
all in one building versus a team that's
distributed across the two buildings,
465
00:29:36,938 --> 00:29:42,539
comes actually from the spillovers
of having a distant teammate on
466
00:29:42,539 --> 00:29:47,862
the interactions between people
who are in the same building.
467
00:29:47,862 --> 00:29:52,386
We can see that basically, if Tom and
Susie are in the same building,
468
00:29:52,386 --> 00:29:57,300
but John is in the other building,
Tom and Susie are likely to exchange less
469
00:29:57,300 --> 00:30:01,909
feedback, than if they were on
a similar team that was all co located.
470
00:30:01,909 --> 00:30:05,155
>> Steven Davis: Right, so
that's a pretty big effect and
471
00:30:05,155 --> 00:30:10,074
it suggested much of the informal
interaction among co located
472
00:30:10,074 --> 00:30:14,992
people was actually taking place
right before or right after or
473
00:30:14,992 --> 00:30:18,456
during the physical meetings themselves.
474
00:30:18,456 --> 00:30:23,604
Cuz if they were just bumping into each
other at the water cooler, the fact that
475
00:30:23,604 --> 00:30:28,518
John's in another location and we have
to have meetings via Zoom with him,
476
00:30:28,518 --> 00:30:33,679
you wouldn't expect to get much of
a negative effect on mentorship from that.
477
00:30:33,679 --> 00:30:36,343
Okay, so that sounds plausible to me, but
478
00:30:36,343 --> 00:30:39,155
then kind of reinforces
a view I had already.
479
00:30:39,155 --> 00:30:43,576
I wanted to get your reaction to it,
which is, if you go from a world in which
480
00:30:43,576 --> 00:30:48,495
everybody's coming into the office five
days a week to one where there's hybrid,
481
00:30:48,495 --> 00:30:52,779
especially if it's people choosing
their own times to be in the office,
482
00:30:52,779 --> 00:30:57,699
you really got to replace the old informal
social interactions that took place with
483
00:30:57,699 --> 00:30:58,781
something else.
484
00:30:58,781 --> 00:31:05,499
And in your context, this would be
a way to overcome those kind of.
485
00:31:05,499 --> 00:31:10,164
Those social inhibitions
the junior person might feel,
486
00:31:10,164 --> 00:31:13,479
in asking a senior person for feedback.
487
00:31:13,479 --> 00:31:16,021
So does this firm,
are they making more efforts?
488
00:31:16,021 --> 00:31:21,044
Like, is there a,
Wednesday night drinks or something?
489
00:31:21,044 --> 00:31:25,251
I don't know what it might be,
but some way to encourage more
490
00:31:25,251 --> 00:31:29,707
social interactions when people
are actually in the office, so
491
00:31:29,707 --> 00:31:33,773
that these productive exchanges
will flow more readily.
492
00:31:33,773 --> 00:31:38,153
>> Emma Harrington: Yeah, we haven't
talked to the software engineers at this
493
00:31:38,153 --> 00:31:43,951
firm in a while, so I'm not entirely sure
what the state of the art is for them.
494
00:31:43,951 --> 00:31:48,950
The last time we talked to them,
they were really struggling to get
495
00:31:48,950 --> 00:31:52,967
people back into the office,
even three days a week,
496
00:31:52,967 --> 00:31:58,891
whichever days they chose, and so it was
a ways away from them institutional.
497
00:31:58,891 --> 00:32:04,075
>> Steven Davis: Okay, maybe there's
a future study coming, it's got a future
498
00:32:04,075 --> 00:32:10,230
study that is focused on how does your
organization in particular, adapt?
499
00:32:10,230 --> 00:32:16,104
It sounds like they might be settling into
some kind of hybrid working arrangements,
500
00:32:16,104 --> 00:32:20,592
but how are they adapting other
aspects of the work environment,
501
00:32:20,592 --> 00:32:23,286
including the extent of socializing,
502
00:32:23,286 --> 00:32:28,284
in order to mitigate some of the costs
associated with less co location?
503
00:32:28,284 --> 00:32:31,517
I think that's a really important
question for the long term.
504
00:32:31,517 --> 00:32:34,318
>> Emma Harrington: Yeah, I agree,
I mean, I think something also,
505
00:32:34,318 --> 00:32:38,792
additionally that's interesting that we
can see in the data that we have is that
506
00:32:38,792 --> 00:32:43,237
there's some suggestive evidence,
the firm shifted its hiring practices.
507
00:32:43,237 --> 00:32:46,037
So pre Covid, it hired a lot of new grads,
508
00:32:46,037 --> 00:32:50,844
who didn't have a lot of experience
as engineers and were quite young.
509
00:32:50,844 --> 00:32:53,554
And then when the office is all closed,
510
00:32:53,554 --> 00:32:57,096
it shifted towards hiring
more senior engineers.
511
00:32:57,096 --> 00:33:00,656
And there's lots of reasons that
that may have happened, but
512
00:33:00,656 --> 00:33:05,380
I think one interpretation is that when
it was hard to facilitate that proximity,
513
00:33:05,380 --> 00:33:08,402
it wanted to sort of buy
talent rather than build it.
514
00:33:08,402 --> 00:33:12,741
And so I think if you don't have these
successful strategies to potentially
515
00:33:12,741 --> 00:33:17,913
facilitate those mentorship relationships,
firms may just change who they wanna hire.
516
00:33:17,913 --> 00:33:21,486
>> Steven Davis: Right,
which could have implications for
517
00:33:21,486 --> 00:33:26,456
the wage structure and,
the wage tenure profile and so on.
518
00:33:26,456 --> 00:33:31,366
So, anything else you want
to tell us about your study,
519
00:33:31,366 --> 00:33:38,793
other key points that we've neglected or
over overlooked so far in our discussion?
520
00:33:38,793 --> 00:33:44,259
>> Emma Harrington: I think we've really
covered the main takeaways of the study.
521
00:33:44,259 --> 00:33:45,214
>> Steven Davis: Okay, great.
522
00:33:45,214 --> 00:33:50,154
Well, we will put a link to your
study on the episode website at
523
00:33:50,154 --> 00:33:54,246
the Hoover institution so
people can check it out.
524
00:33:54,246 --> 00:33:57,239
And Emma has lots of other
interesting research, too,
525
00:33:57,239 --> 00:33:59,602
that we didn't have
time to get into today.
526
00:33:59,602 --> 00:34:01,322
You might want to check that out, too,
527
00:34:01,322 --> 00:34:03,680
including the call center
study that she mentioned.
528
00:34:03,680 --> 00:34:08,963
It's one of the better known, studies in
this area and rewards a careful look.
529
00:34:08,963 --> 00:34:13,701
So thanks so much for Emma, it was lots of
fun to chat with you about this work, and
530
00:34:13,701 --> 00:34:16,644
we're really delighted
you could join us today.
531
00:34:16,644 --> 00:34:18,603
>> Emma Harrington: Yeah,
thanks so much for having me.
532
00:34:18,603 --> 00:34:20,488
>> Steven Davis: Okay, take care.
533
00:34:20,488 --> 00:34:20,988
>> Emma Harrington: Bye.
[MUSIC]
00:00:00,640 --> 00:00:04,445
Two questions, first,
what happens to mentoring and
2
00:00:04,445 --> 00:00:10,250
to learning on the job when employees no
longer work together in the same location?
3
00:00:11,270 --> 00:00:15,378
Second, and related,
how does the rise of remote work affect
4
00:00:15,378 --> 00:00:18,388
productivity in the near term and
over time?
5
00:00:18,388 --> 00:00:27,198
[MUSIC]
6
00:00:27,198 --> 00:00:29,596
>> Steven Davis: Welcome
to Economics Applied,
7
00:00:29,596 --> 00:00:33,246
a podcast sponsored by
the Hoover Institution.
8
00:00:33,246 --> 00:00:36,700
My name is Steven Davis,
and I host the show.
9
00:00:36,700 --> 00:00:41,077
Our guest today has some very insightful
research that speaks to these two
10
00:00:41,077 --> 00:00:42,020
questions.
11
00:00:42,020 --> 00:00:47,140
She is Emma Harrington, an economics
professor at the University of Virginia.
12
00:00:47,140 --> 00:00:50,628
Emma earned her PhD in
economics at Harvard, and
13
00:00:50,628 --> 00:00:56,414
before her recent move to Virginia, she
held a postdoc position at Princeton and
14
00:00:56,414 --> 00:00:59,832
was on the faculty at
the University of Iowa.
15
00:00:59,832 --> 00:01:03,280
Welcome, Emma,
it's really great to have you on the show.
16
00:01:03,280 --> 00:01:05,072
>> Emma Harrington: Thanks so
much for having me.
17
00:01:05,072 --> 00:01:10,150
>> Steven Davis: So you co authored a
recent study titled the power of proximity
18
00:01:10,150 --> 00:01:15,760
to co workers, training for
tomorrow or productivity today.
19
00:01:15,760 --> 00:01:20,066
So the title kind of sets forth
what the paper is really after, and
20
00:01:20,066 --> 00:01:24,847
that's in this fundamental point
that proximity to coworkers involved
21
00:01:24,847 --> 00:01:29,782
a trade off between productivity today and
productivity in the future.
22
00:01:29,782 --> 00:01:32,382
So before we get into
the details of the study,
23
00:01:32,382 --> 00:01:36,606
just flesh out the nature of that trade
off a little bit for our audience.
24
00:01:36,606 --> 00:01:41,449
>> Emma Harrington: Yeah, definitely, so
we're particularly interested in studying
25
00:01:41,449 --> 00:01:44,953
how remote work plays out for
knowledge workers,
26
00:01:44,953 --> 00:01:50,510
where there's lots of things that we
mean when we say the word productivity.
27
00:01:50,510 --> 00:01:56,424
So some of it is what we mean is sort
of how much are you churning out today?
28
00:01:56,424 --> 00:01:58,792
In our particular setting
is software engineers.
29
00:01:58,792 --> 00:02:02,272
So you can think of that as how many
programs are you writing today?
30
00:02:02,272 --> 00:02:06,534
But then there's also this other
component of your job where you're
31
00:02:06,534 --> 00:02:10,600
thinking about mentoring other people or
learning yourself.
32
00:02:10,600 --> 00:02:15,672
And there we find that being out of the
office is a little bit more challenging.
33
00:02:15,672 --> 00:02:20,117
People actually write more programs
when they're not sitting together, but
34
00:02:20,117 --> 00:02:23,040
they seem to also be
engaged in less mentorship.
35
00:02:23,040 --> 00:02:25,670
>> Steven Davis: Sounds
a lot like professors, too.
36
00:02:25,670 --> 00:02:27,214
>> Emma Harrington: Yeah,
we've heard a lot.
37
00:02:27,214 --> 00:02:29,730
>> Steven Davis: Trade off is
very pronounced for professors.
38
00:02:31,190 --> 00:02:34,897
Okay, so, that's the issue,
but so tell us,
39
00:02:34,897 --> 00:02:41,590
give us the context in which you tried to
address this issue in a compelling way.
40
00:02:41,590 --> 00:02:44,268
I'll let you kind of lay
it out as you wish, but
41
00:02:44,268 --> 00:02:48,117
it's both the context of your study,
the data you had available,
42
00:02:48,117 --> 00:02:52,676
which is impressive in some respects,
and then the research design itself.
43
00:02:52,676 --> 00:02:54,044
So sit that out for us,
44
00:02:54,044 --> 00:02:57,820
how did you go about tackling
this really important issue?
45
00:02:57,820 --> 00:02:58,996
>> Emma Harrington: Yeah, definitely.
46
00:02:58,996 --> 00:03:04,056
So we have data from
a particular Fortune 500 firm,
47
00:03:04,056 --> 00:03:08,372
and a setting has a few key advantages for
us.
48
00:03:08,372 --> 00:03:12,472
One is that the firm had two
office buildings of engineers on
49
00:03:12,472 --> 00:03:16,890
its main campus that were about
a ten minute walk, apartheid.
50
00:03:16,890 --> 00:03:19,802
And because of limited desk availability,
51
00:03:19,802 --> 00:03:24,288
some teams ended up split across
those two different buildings.
52
00:03:24,288 --> 00:03:29,742
And that mattered for the teams dynamics
because it's pretty industry standard for
53
00:03:29,742 --> 00:03:33,825
engineers to have daily short
stand up meetings with the idea.
54
00:03:33,825 --> 00:03:36,488
>> Steven Davis: These are all
software engineers, right?
55
00:03:36,488 --> 00:03:38,120
That's what we're talking about here.
56
00:03:38,120 --> 00:03:41,722
>> Emma Harrington: Yeah, so they all
basically worked on creating the website
57
00:03:41,722 --> 00:03:47,040
of the firm and maintaining the databases
for the firm, it's an online retail firm.
58
00:03:47,040 --> 00:03:51,080
So these are pretty standard tasks for
engineers.
59
00:03:51,080 --> 00:03:54,384
>> Steven Davis: Okay, so you got some of
these guys working together all the time.
60
00:03:54,384 --> 00:03:57,960
Some of them are, some teams,
I should say are split apart.
61
00:03:57,960 --> 00:04:01,840
How many teams,
how many engineers overall in your study?
62
00:04:01,840 --> 00:04:05,837
>> Emma Harrington: So
it's 1000 software engineers and
63
00:04:05,837 --> 00:04:09,824
there's just a few people on each team.
64
00:04:09,824 --> 00:04:11,826
So also lots of teams, and
65
00:04:11,826 --> 00:04:17,072
about 60% of them were all co
located in one of the two buildings.
66
00:04:17,072 --> 00:04:23,166
And then the remaining 40% of them were
split across the two office buildings.
67
00:04:23,166 --> 00:04:27,545
>> Steven Davis: Okay, so you've got
a lot of useful variation in the data,
68
00:04:27,545 --> 00:04:29,742
as economists would put it.
69
00:04:29,742 --> 00:04:34,324
And then the setting, the time, the
temporal setting is also very important
70
00:04:34,324 --> 00:04:39,430
for your study, the period covered by
the data, so just lay that out for us.
71
00:04:39,430 --> 00:04:43,784
>> Emma Harrington: Yeah, so we span
the period before the office closure
72
00:04:43,784 --> 00:04:48,698
of the pandemic to a few months
after the office closures.
73
00:04:48,698 --> 00:04:53,900
And so we can see what happens to
the differences between these co located
74
00:04:53,900 --> 00:04:58,680
teams versus these distributed
teams once everyone's remote for
75
00:04:58,680 --> 00:05:03,370
the pandemic, we expect if the things
were causal to begin with,
76
00:05:03,370 --> 00:05:07,610
they should go away once
everyone is in the same boat.
77
00:05:07,610 --> 00:05:12,457
>> Steven Davis: Okay, and there are,
as you suggested, there are differences
78
00:05:12,457 --> 00:05:18,508
between the co located teams and the other
teams before the pandemic struck.
79
00:05:18,508 --> 00:05:21,753
And then there's going to be maybe
differences, maybe similarities,
80
00:05:21,753 --> 00:05:23,556
we'll talk about that afterwards.
81
00:05:23,556 --> 00:05:28,587
But you got this event, which is pretty,
this is an event we can safely
82
00:05:28,587 --> 00:05:33,668
say was largely unanticipated and
arrived with extreme abruptness.
83
00:05:33,668 --> 00:05:38,230
So you have a firm that is operating
in a certain way with respect to its
84
00:05:38,230 --> 00:05:39,876
software engineers.
85
00:05:39,876 --> 00:05:44,660
This big shock forces them to do something
different in terms of co location of
86
00:05:44,660 --> 00:05:46,190
workers.
87
00:05:46,190 --> 00:05:49,630
But some teams were highly
co located before and
88
00:05:49,630 --> 00:05:53,110
some were split across the two buildings.
89
00:05:53,110 --> 00:05:55,634
>> Emma Harrington: Yeah, and
so are already, in some senses,
90
00:05:55,634 --> 00:05:57,342
acting like remote teams.
91
00:05:57,342 --> 00:06:03,470
They held some of their meetings remotely,
even when they were in the office.
92
00:06:03,470 --> 00:06:05,420
>> Steven Davis: Before
we turn to your results,
93
00:06:05,420 --> 00:06:10,128
tell us a little bit more about exactly
what you can measure in this context.
94
00:06:10,128 --> 00:06:14,780
That's another very attractive
feature of your study.
95
00:06:16,280 --> 00:06:20,106
>> Emma Harrington: Yeah, I think the
really special thing about our study is
96
00:06:20,106 --> 00:06:24,970
that software engineers create this
sort of paper trail of mentorship,
97
00:06:24,970 --> 00:06:30,549
that it's really standard that if you're
writing a program at one of these firms,
98
00:06:30,549 --> 00:06:35,332
you're gonna have to get it peer
reviewed by another programmer who's
99
00:06:35,332 --> 00:06:39,557
gonna give you pretty detailed
feedback on your program before
100
00:06:39,557 --> 00:06:42,520
it can actually be used in production.
101
00:06:42,520 --> 00:06:47,084
These firms really don't want there to
be any bugs or glitches in the code, and
102
00:06:47,084 --> 00:06:50,360
so they have this peer review process.
103
00:06:50,360 --> 00:06:52,177
>> Steven Davis: Are you
confident that all or
104
00:06:52,177 --> 00:06:56,718
most of the feedback actually takes
this written form, or is there also
105
00:06:56,718 --> 00:07:01,552
feedback occurring that you're not
measuring because it's just oral?
106
00:07:01,552 --> 00:07:05,533
>> Emma Harrington: So, I think our
setting means that a lot of it is captured
107
00:07:05,533 --> 00:07:11,212
in the code review process because part
of it is that they want documentation for
108
00:07:11,212 --> 00:07:14,908
the reason why there were
changes to the program.
109
00:07:14,908 --> 00:07:18,605
So there are some references
to conversations that occur
110
00:07:18,605 --> 00:07:20,068
outside the system.
111
00:07:20,068 --> 00:07:26,300
So we can directly see things like
references to slack conversations.
112
00:07:26,300 --> 00:07:30,260
And those go in sort of the same
direction as our main results.
113
00:07:30,260 --> 00:07:34,316
>> Steven Davis: Okay, and
one more thing about the data.
114
00:07:34,316 --> 00:07:37,287
You have some information on
the quality of the feedback,
115
00:07:37,287 --> 00:07:39,711
as well as not just
measuring the quantity, but
116
00:07:39,711 --> 00:07:43,278
you've got some information on
the quality, the feedback as well.
117
00:07:43,278 --> 00:07:46,454
So that's gonna be important for
some aspects of your results.
118
00:07:46,454 --> 00:07:49,446
So how do you assess
the quality of feedback?
119
00:07:49,446 --> 00:07:51,102
How do you know it's even useful?
120
00:07:51,102 --> 00:07:54,170
>> Emma Harrington: Yeah, so, I mean, we
can look at that in a few different ways.
121
00:07:54,170 --> 00:07:58,795
So we can, we have the text of all
the feedback that is given and so
122
00:07:58,795 --> 00:08:03,089
we can look at measures of how
substantive the feedback is.
123
00:08:03,089 --> 00:08:09,228
Like, does it include a block of code, for
example, or a link to other documentation?
124
00:08:09,228 --> 00:08:15,836
We also had a subsample of the comments
reviewed by external reviewers who,
125
00:08:15,836 --> 00:08:20,689
by and large,
said that the comments were impactful and
126
00:08:20,689 --> 00:08:23,168
likely to change the code and
127
00:08:23,168 --> 00:08:28,865
also often provided actionable
steps to take to change the code.
128
00:08:30,725 --> 00:08:33,813
Yeah, those are our main
measures of the quality.
129
00:08:33,813 --> 00:08:35,883
>> Steven Davis: So
this is a data rich environment,
130
00:08:35,883 --> 00:08:39,609
an interesting physical setting,
an obvious shock that hits,
131
00:08:39,609 --> 00:08:43,680
that causes things to change over time,
and you're able to measure,
132
00:08:43,680 --> 00:08:47,677
follow individual engineers and
teams before and after the shock.
133
00:08:47,677 --> 00:08:49,317
So very nice setting.
134
00:08:49,317 --> 00:08:53,139
So before the shock hit, though, there
were already some noticeable differences
135
00:08:53,139 --> 00:08:55,583
between the co-located teams and
the other teams.
136
00:08:55,583 --> 00:08:56,663
Tell us about that.
137
00:08:56,663 --> 00:09:00,999
>> Emma Harrington: Yeah, so the
co-located teams received about 20% more
138
00:09:00,999 --> 00:09:03,855
comments on each of their programs.
139
00:09:03,855 --> 00:09:08,147
And as we've been talking about, that's
a pretty notable difference because what
140
00:09:08,147 --> 00:09:11,407
we're measuring is feedback
that it's occurring online.
141
00:09:11,407 --> 00:09:15,574
So to the extent to which there was some
feedback that was occurring face to face,
142
00:09:15,574 --> 00:09:18,583
presumably that would be
easier if you were co located.
143
00:09:18,583 --> 00:09:22,513
So we're likely providing a lower
bound on how big the differences in
144
00:09:22,513 --> 00:09:26,543
mentorship are between these
co-located and distributed teams.
145
00:09:26,543 --> 00:09:27,215
>> Steven Davis: Okay, and
146
00:09:27,215 --> 00:09:32,375
to what extent is the feedback going from
senior people to junior people as opposed
147
00:09:32,375 --> 00:09:38,135
to just horizontal people at roughly
the same experience level within the firm?
148
00:09:38,135 --> 00:09:40,263
>> Emma Harrington: There's some of both.
149
00:09:40,263 --> 00:09:45,184
The effects of proximity seem to be
really concentrated among feedback
150
00:09:45,184 --> 00:09:49,759
received by junior people and
feedback given by senior people.
151
00:09:49,759 --> 00:09:54,796
So some of the sort of horizontal
feedback that happens, but
152
00:09:54,796 --> 00:10:01,181
it doesn't seem to be as sensitive
to whether you're co-located or not.
153
00:10:01,181 --> 00:10:01,692
>> Steven Davis: Okay.
154
00:10:01,692 --> 00:10:05,288
>> Emma Harrington: I think some of that
may be because of the underlying mechanism
155
00:10:05,288 --> 00:10:09,853
that some of it is because it's important
to be co-located in terms of feeling
156
00:10:09,853 --> 00:10:14,073
comfortable asking for additional
feedback, and maybe if you feel like
157
00:10:14,073 --> 00:10:19,135
you're in a peer situation, you feel
comfortable doing that even remotely.
158
00:10:19,135 --> 00:10:22,756
>> Steven Davis: Or the flip side of
that is the senior person may be more
159
00:10:22,756 --> 00:10:24,286
comfortable saying,
160
00:10:24,286 --> 00:10:29,617
not providing feedback when they don't
have to see the person in the hallway.
161
00:10:29,617 --> 00:10:30,735
>> Emma Harrington: Yeah.
162
00:10:30,735 --> 00:10:35,746
>> Steven Davis: We'll get into later
more whether this feedback is actually,
163
00:10:35,746 --> 00:10:40,703
on the margin, the best use of time for
all parties involved.
164
00:10:40,703 --> 00:10:43,364
So I wanna come back to that, but okay, so
165
00:10:43,364 --> 00:10:48,770
there's more feedback received primarily
by junior people in co-located teams,
166
00:10:48,770 --> 00:10:52,321
and the feedback is largely
given by senior people.
167
00:10:52,321 --> 00:10:55,491
So senior people, well,
their time is pretty valuable, so
168
00:10:55,491 --> 00:10:58,477
there's an opportunity cost
here to this feedback, but
169
00:10:58,477 --> 00:11:02,385
presumably there's also quite a bit
of benefit for the junior people.
170
00:11:02,385 --> 00:11:06,266
And so we can already see even
in a kind of a static sense,
171
00:11:06,266 --> 00:11:09,585
there's a trade off here in attention.
172
00:11:09,585 --> 00:11:14,097
But then the dynamic one is, well, if the
junior people don't get enough feedback,
173
00:11:14,097 --> 00:11:16,761
they won't improve
rapidly enough over time.
174
00:11:16,761 --> 00:11:22,105
So then the pandemic hits played out for
us, what happened?
175
00:11:22,105 --> 00:11:25,598
>> Emma Harrington: So, feedback in
general declined across the board, but
176
00:11:25,598 --> 00:11:30,039
it declined much more for engineers
that had been co-located with their
177
00:11:30,039 --> 00:11:34,771
teammates than engineers who are already
distributed, and in some senses,
178
00:11:34,771 --> 00:11:37,433
already acting a bit like remote teams.
179
00:11:37,433 --> 00:11:42,057
And so, basically, that whole difference
between those co-located teams and
180
00:11:42,057 --> 00:11:46,793
those distributed teams when the offices
were open disappears once every once.
181
00:11:46,793 --> 00:11:48,313
>> Steven Davis: Okay, so
it completely disappears.
182
00:11:48,313 --> 00:11:50,538
But how much does the overall decline,
say, for
183
00:11:50,538 --> 00:11:52,705
the teams that were originally co-located?
184
00:11:52,705 --> 00:11:54,201
How big is that decline?
185
00:11:54,201 --> 00:11:59,225
>> Emma Harrington: I mean,
it's pretty enormous if you include.
186
00:11:59,225 --> 00:12:01,284
So, like, the gap goes away, and
187
00:12:01,284 --> 00:12:05,275
then also you just have a secular
decline across the board.
188
00:12:05,275 --> 00:12:10,367
So I think if you added it all together,
it would be like something
189
00:12:10,367 --> 00:12:14,755
along the order of a third
of the feedback disappears.
190
00:12:14,755 --> 00:12:19,286
Now, it's unclear exactly how much
you wanna think of that as all being
191
00:12:19,286 --> 00:12:24,027
a causal effect of proximity,
because you do have the pandemic going on.
192
00:12:24,027 --> 00:12:28,555
And so you might think that there's
other shocks to these engineers.
193
00:12:28,555 --> 00:12:31,215
It's not just because of
the loss of proximity.
194
00:12:32,615 --> 00:12:35,638
>> Steven Davis: Okay, so
if they were, for example,
195
00:12:35,638 --> 00:12:38,378
highly stressed by the pandemic,
196
00:12:38,378 --> 00:12:43,831
their overall work effectiveness
might have diminished, right?
197
00:12:43,831 --> 00:12:45,623
Is that a coming mind?
198
00:12:45,623 --> 00:12:49,439
Okay, but what happened to their
overall code production, though,
199
00:12:49,439 --> 00:12:51,047
in the wake of the pandemic?
200
00:12:51,047 --> 00:12:52,395
Did that go up or down?
201
00:12:54,175 --> 00:12:59,762
>> Emma Harrington: So, it went down
overall, but it went down by more for
202
00:12:59,762 --> 00:13:04,852
the people who had been
previously across the two
203
00:13:04,852 --> 00:13:11,087
buildings than people who had
been on one building teams.
204
00:13:11,087 --> 00:13:15,623
And so, relative terms, the engineers
who had been co-located saw their
205
00:13:15,623 --> 00:13:18,647
productivity in terms of
the number of programs,
206
00:13:18,647 --> 00:13:22,599
they won't go up relative to
those that have been distributed.
207
00:13:22,599 --> 00:13:26,259
>> Steven Davis: Okay, but as I
understand it, both mentorship measured,
208
00:13:26,259 --> 00:13:30,945
as you described earlier,
feedback might be a more accurate term.
209
00:13:30,945 --> 00:13:36,231
Feedback fell roughly a third for
the co-located teams fell less,
210
00:13:36,231 --> 00:13:41,377
but from a smaller original base for
the non co-located teams.
211
00:13:41,377 --> 00:13:46,409
But lines of code written also fell,
is that correct?
212
00:13:46,409 --> 00:13:51,567
Okay, so there was a general across
the board negative response,
213
00:13:51,567 --> 00:13:57,001
which, as you pointed out,
was a mix that we maybe can't fully unpack
214
00:13:57,001 --> 00:14:02,067
between the lost benefits of proximity and
the pandemic itself,
215
00:14:02,067 --> 00:14:08,527
disrupting people's lives, maybe even
their mental states in many respects.
216
00:14:08,527 --> 00:14:12,095
Okay, do I have it right there?
217
00:14:12,095 --> 00:14:13,335
>> Emma Harrington: Yeah,
no, that's right.
218
00:14:13,335 --> 00:14:17,795
I mean, I think that's why we
focus mainly on the difference and
219
00:14:17,795 --> 00:14:22,433
differences of thinking about how
the one building teams- Sure.
220
00:14:22,433 --> 00:14:27,607
Changed around the office closures
need to be the multi building teams.
221
00:14:27,607 --> 00:14:31,768
>> Steven Davis: Right, but I wanna put
myself in the shoes of the executives at
222
00:14:31,768 --> 00:14:35,879
this firm, the ones overseeing
the software engineers.
223
00:14:35,879 --> 00:14:38,727
So they must have seen
some of this in real time.
224
00:14:38,727 --> 00:14:42,915
And while for research purposes,
the difference in differences,
225
00:14:42,915 --> 00:14:46,703
it's extremely interesting
from a business perspective.
226
00:14:46,703 --> 00:14:51,695
They say, whoa, [LAUGH] l there's a lot
of lost outputs really down here.
227
00:14:51,695 --> 00:14:53,999
Now, how did they react?
228
00:14:53,999 --> 00:14:56,527
Or did they react to
this in near real time?
229
00:14:56,527 --> 00:14:57,455
What did they say?
230
00:14:57,455 --> 00:14:58,183
What did they do?
231
00:14:58,183 --> 00:15:00,671
I presume you've talked
to them about this.
232
00:15:00,671 --> 00:15:05,535
>> Emma Harrington: Yeah, we have
talked to some engineers at the firm.
233
00:15:05,535 --> 00:15:10,540
Some of them said it was really
resonant with their own experiences,
234
00:15:10,540 --> 00:15:15,457
that some senior engineers,
at least, felt, in relative terms,
235
00:15:15,457 --> 00:15:18,478
like they were being pretty productive in
236
00:15:18,478 --> 00:15:23,169
terms of how many programs they
were writing in the pandemic.
237
00:15:23,169 --> 00:15:24,853
>> Steven Davis: Relative
to their own past or
238
00:15:24,853 --> 00:15:29,632
relative to other [CROSSTALK] I think they
viewed it as relative to their own past.
239
00:15:29,632 --> 00:15:32,872
>> Emma Harrington: I mean, I guess the
data in an aggregate sense is inconsistent
240
00:15:32,872 --> 00:15:33,545
with that.
241
00:15:33,545 --> 00:15:37,485
Although on the individual level,
I'm sure for some people it is.
242
00:15:38,865 --> 00:15:42,700
And so for the people we talked to,
they felt subjectively
243
00:15:42,700 --> 00:15:47,905
like they were being more productive
than they had been in the office.
244
00:15:47,905 --> 00:15:52,043
But they had some sneaking suspicion that
that was partly because they were devoting
245
00:15:52,043 --> 00:15:53,592
less time giving-
>> Steven Davis: Okay.
246
00:15:53,592 --> 00:15:55,305
>> Emma Harrington: Feedback
to other people.
247
00:15:55,305 --> 00:15:58,313
>> Steven Davis: This is really
interesting, because I've done a lot of
248
00:15:58,313 --> 00:16:02,665
work relying on subjective assessment
Of performance, I'm just curious, and
249
00:16:02,665 --> 00:16:06,711
maybe you didn't look at this cuz I
don't remember seeing it in the study.
250
00:16:06,711 --> 00:16:12,379
What's the relationship between subjective
assessments of own performance and
251
00:16:12,379 --> 00:16:16,661
your external measures,
especially just code production?
252
00:16:16,661 --> 00:16:21,096
That's probably what they're
answering about when you ask them,
253
00:16:21,096 --> 00:16:23,913
are you getting more
done than you used to?
254
00:16:23,913 --> 00:16:28,514
Are those two things I'd be really
curious to know how they relate to?
255
00:16:28,514 --> 00:16:29,515
>> Emma Harrington: Yeah, I mean,
256
00:16:29,515 --> 00:16:33,215
unfortunately, we weren't able to,
in this setting,
257
00:16:33,215 --> 00:16:38,396
do a sort of systematic survey of people
to get their subjective expectations.
258
00:16:38,396 --> 00:16:39,526
>> Steven Davis: Okay.
>> Emma Harrington: Natalia,
259
00:16:39,526 --> 00:16:43,461
my co-author on this, and
I have another paper about call
260
00:16:43,461 --> 00:16:48,377
center workers where we were able to
more systematically elicit those.
261
00:16:48,377 --> 00:16:53,768
And find some correlation there, but
I'd have to check on the exact numbers.
262
00:16:53,768 --> 00:16:55,393
>> Steven Davis: Okay, well,
let's go back to this study.
263
00:16:55,393 --> 00:17:00,058
Now, you also found some very interesting
differences between men and women.
264
00:17:00,058 --> 00:17:03,504
So flesh that out for us,
cuz it's quite interesting.
265
00:17:03,504 --> 00:17:07,955
>> Emma Harrington: Yes, we find that
the tradeoffs from proximity are much more
266
00:17:07,955 --> 00:17:09,270
acute for women.
267
00:17:09,270 --> 00:17:13,987
So it seems to be more important for
junior women's feedback,
268
00:17:13,987 --> 00:17:19,156
the feedback they receive to be
co-located with their colleagues.
269
00:17:19,156 --> 00:17:20,962
And also more taxing for
270
00:17:20,962 --> 00:17:26,026
senior women's time to be sitting
near their junior colleagues.
271
00:17:26,026 --> 00:17:30,310
So for senior women,
it seems like they have a really big loss
272
00:17:30,310 --> 00:17:33,754
of output from sitting
near their co-workers.
273
00:17:33,754 --> 00:17:36,119
>> Steven Davis: But
how does it work dyadically?
274
00:17:36,119 --> 00:17:42,001
So is it [COUGH] that junior women
are getting a lot more feedback from
275
00:17:42,001 --> 00:17:48,421
senior men when they're in proximity,
or from senior women, or both?
276
00:17:48,421 --> 00:17:49,275
>> Emma Harrington: Both.
277
00:17:49,275 --> 00:17:50,802
>> Steven Davis: Both, okay.
>> Emma Harrington: So he thought it was
278
00:17:50,802 --> 00:17:54,028
possible that it would only
be across genders, but
279
00:17:54,028 --> 00:17:56,740
it looks like it's just across the board.
280
00:17:56,740 --> 00:17:59,957
More important for
junior women to be near their colleagues.
281
00:17:59,957 --> 00:18:02,837
>> Steven Davis: Okay, and for
the senior women who are giving,
282
00:18:02,837 --> 00:18:06,512
they're giving more feedback
when they're co-located.
283
00:18:06,512 --> 00:18:11,392
Because they're both more approachable
to the junior men and the junior women,
284
00:18:11,392 --> 00:18:13,914
or is it primarily from the junior women?
285
00:18:13,914 --> 00:18:15,335
>> Emma Harrington: It's both.
286
00:18:15,335 --> 00:18:20,125
So the only dyad that seems
to be pretty unaffected
287
00:18:20,125 --> 00:18:24,355
by proximity is men
giving feedback to men.
288
00:18:24,355 --> 00:18:27,525
Basically, all the other
combinations are effective.
289
00:18:27,525 --> 00:18:31,964
>> Steven Davis: Okay, do you have any
sense from your conversations with these
290
00:18:31,964 --> 00:18:37,678
software engineers, why the differences
between men and women take this form?
291
00:18:37,678 --> 00:18:38,369
What did they tell you?
292
00:18:38,369 --> 00:18:41,808
Or maybe you didn't ask them about this,
but again,
293
00:18:41,808 --> 00:18:46,331
it's a very interesting question
with relevance beyond your study.
294
00:18:46,331 --> 00:18:46,984
>> Emma Harrington: Yeah.
295
00:18:46,984 --> 00:18:51,868
So, I mean,
we didn't ask the software engineers
296
00:18:51,868 --> 00:18:55,967
of the firm that much about this question.
297
00:18:55,967 --> 00:19:00,487
They expressed a lot of concern at
the time that this was some form
298
00:19:00,487 --> 00:19:01,826
of mansplaining,
299
00:19:01,826 --> 00:19:07,278
which is a general concern that's been
raised when we've given this paper.
300
00:19:07,278 --> 00:19:11,728
And so we were able to do some
things using the text of the code
301
00:19:11,728 --> 00:19:14,587
reviews to try to get at that concern.
302
00:19:14,587 --> 00:19:21,040
So, I mean, one thing is that when we have
the external raters look at the comments.
303
00:19:21,040 --> 00:19:25,499
They actually rate comments that
are received by women as being more
304
00:19:25,499 --> 00:19:29,034
helpful and
more likely to cause changes to the code.
305
00:19:29,034 --> 00:19:33,700
So it doesn't seem like necessarily
women are just getting a bunch of
306
00:19:33,700 --> 00:19:35,637
really nitpicky comments.
307
00:19:35,637 --> 00:19:39,756
And that's true both when they're
proximate to their co-workers.
308
00:19:39,756 --> 00:19:42,973
>> Steven Davis: But even your
basic patterns don't really support
309
00:19:42,973 --> 00:19:45,739
a mansplaining story, as I understand it,
310
00:19:45,739 --> 00:19:49,052
because the senior women
are giving more feedback.
311
00:19:49,052 --> 00:19:52,173
You wouldn't call that
mansplaining [LAUGH].
312
00:19:52,173 --> 00:19:54,948
>> Emma Harrington: Yeah,
I guess, I mean [LAUGH].
313
00:19:54,948 --> 00:19:55,863
>> Steven Davis: And, [LAUGH] and
314
00:19:55,863 --> 00:20:00,724
the junior women are getting more feedback
from both senior women and senior men,
315
00:20:00,724 --> 00:20:04,116
which doesn't really sound
like a mansplaining story.
316
00:20:04,116 --> 00:20:06,500
Okay, so it's not a mansplaining story.
317
00:20:06,500 --> 00:20:08,673
So what do you think it is?
318
00:20:08,673 --> 00:20:12,619
>> Emma Harrington: I think it may be
something about comfort asking for
319
00:20:12,619 --> 00:20:13,547
feedback.
320
00:20:13,547 --> 00:20:18,555
So we particularly see that,
you can sometimes get feedback from
321
00:20:18,555 --> 00:20:23,301
multiple people or
you can get feedback from just one person.
322
00:20:23,301 --> 00:20:28,027
And when junior women
are proximate to their co-workers,
323
00:20:28,027 --> 00:20:33,426
they receive feedback from more
people than when they're remote.
324
00:20:33,426 --> 00:20:35,390
And so because in this setting,
325
00:20:35,390 --> 00:20:40,276
the software engineer themselves is
asking other people to review their code.
326
00:20:40,276 --> 00:20:45,077
It does seem consistent with a story
where when you're remote, you just, and
327
00:20:45,077 --> 00:20:49,152
you're a woman, you just don't
feel that comfortable asking for
328
00:20:49,152 --> 00:20:51,575
this additional feedback and advice.
329
00:20:51,575 --> 00:20:56,300
Whereas if you're in person, maybe you can
see someone in the halls and be like, hey,
330
00:20:56,300 --> 00:20:58,131
do you mind taking a look at this?
331
00:20:58,131 --> 00:21:01,355
And that just feels
a little less pushy to you.
332
00:21:01,355 --> 00:21:05,194
>> Steven Davis: Right, so an important
issue, I think, I know the answer, but
333
00:21:05,194 --> 00:21:08,548
I want to get it on the table for
the sake of the audience.
334
00:21:08,548 --> 00:21:13,748
Is there anything in your study that
tells us whether there was too little or
335
00:21:13,748 --> 00:21:18,053
too much feedback either before
the pandemic or afterwards?
336
00:21:18,053 --> 00:21:23,595
>> Emma Harrington: Yeah, it's hard for
us to sort of horse race the benefits and
337
00:21:23,595 --> 00:21:28,051
costs of the feedback
within our particular data
338
00:21:28,051 --> 00:21:32,957
because it's kind of
relatively short-time span.
339
00:21:32,957 --> 00:21:37,977
And so it's hard for us to see
the returns to the mentorship fully.
340
00:21:37,977 --> 00:21:43,492
From the individual's perspective,
it's useful to get the feedback and
341
00:21:43,492 --> 00:21:47,620
mentorship in terms of
downstream pay raises and also-
342
00:21:47,620 --> 00:21:48,405
>> Steven Davis: Sure, but
343
00:21:48,405 --> 00:21:53,205
it's feedback is by definition
a bilateral transaction [LAUGH] or
344
00:21:53,205 --> 00:21:57,494
not transaction exchange,
it's a bilateral exchange.
345
00:21:57,494 --> 00:22:01,107
So if we're gonna come up with some notion
of whether there's too much or too little.
346
00:22:01,107 --> 00:22:03,582
We got to add a minimum account for
347
00:22:03,582 --> 00:22:07,480
the two parties involved directly,
the feedback.
348
00:22:07,480 --> 00:22:11,734
We may wanna account for the firm as
an overall organization as well, and
349
00:22:11,734 --> 00:22:13,420
external parties, right?
350
00:22:13,420 --> 00:22:17,423
So I just think it's important to
understand, and correct me if I'm wrong,
351
00:22:17,423 --> 00:22:20,886
we don't know whether there was
a too much or too little feedback.
352
00:22:20,886 --> 00:22:23,106
We don't know whether
the men were closer or
353
00:22:23,106 --> 00:22:26,412
the women were closer to what some
optimal notion of feedback is.
354
00:22:26,412 --> 00:22:28,158
Or do you disagree with that?
355
00:22:28,158 --> 00:22:30,452
>> Emma Harrington: No,
I think I broadly agree with that.
356
00:22:30,452 --> 00:22:37,323
I mean, we can see some evidence from the
firm's own policies about what it thought.
357
00:22:37,323 --> 00:22:38,650
Now, it might be [INAUDIBLE].
358
00:22:38,650 --> 00:22:40,785
>> Steven Davis: I was gonna come
to that next, so go ahead, tell me,
359
00:22:40,785 --> 00:22:41,915
that was my next question.
360
00:22:41,915 --> 00:22:43,281
>> Emma Harrington: [LAUGH] Yeah.
361
00:22:43,281 --> 00:22:44,297
So, I mean, and
362
00:22:44,297 --> 00:22:48,668
it's unclear whether the firm was
perfectly well calibrated on this.
363
00:22:48,668 --> 00:22:53,623
But pre-COVID, it did allow some
people to work remotely, and
364
00:22:53,623 --> 00:22:58,051
those people tended to be
sort of mid-level engineers.
365
00:22:58,051 --> 00:23:01,108
So they were not the ones
receiving the most mentorship.
366
00:23:01,108 --> 00:23:03,201
They also were not the managers or
367
00:23:03,201 --> 00:23:07,099
super senior engineers who were
giving the most mentorship.
368
00:23:07,099 --> 00:23:11,087
And so that's consistent with
the firm wanting the most junior,
369
00:23:11,087 --> 00:23:13,199
most senior people to be located.
370
00:23:13,199 --> 00:23:15,767
>> Steven Davis: Okay, so, yeah,
that's a useful piece of information, but
371
00:23:15,767 --> 00:23:17,712
let me come at this from
a different direction.
372
00:23:17,712 --> 00:23:23,214
So I presume you've been
in conversations with
373
00:23:23,214 --> 00:23:29,548
some of the senior people at
the firm after your study.
374
00:23:29,548 --> 00:23:34,064
I'm sure they've read your study or
at least been briefed about it carefully.
375
00:23:34,064 --> 00:23:35,938
So how did they react?
376
00:23:35,938 --> 00:23:40,859
Cuz your study has information that might
cause them to change their behavior or
377
00:23:40,859 --> 00:23:44,983
to modify their policies or
try to reinstitute the old practices.
378
00:23:44,983 --> 00:23:47,575
What did they do?
379
00:23:47,575 --> 00:23:52,232
>> Emma Harrington: They have tried to get
people to come back in a hybrid format.
380
00:23:52,232 --> 00:23:53,213
>> Steven Davis: A hybrid format?
381
00:23:53,213 --> 00:23:55,905
Okay, so-
>> Emma Harrington: A hybrid format, yeah.
382
00:23:55,905 --> 00:23:59,489
>> Steven Davis: That's interesting, and
that's typical of what I see in many
383
00:23:59,489 --> 00:24:04,289
firms, which is, They don't try to go all
the way back, either because they don't
384
00:24:04,289 --> 00:24:08,821
think it's necessary or productive or
often because there's resistance from
385
00:24:08,821 --> 00:24:13,243
their employees, but they feel that
it's essential to have some FaceTime.
386
00:24:13,243 --> 00:24:16,512
And, that's basically
the reaction to your study?
387
00:24:16,512 --> 00:24:19,463
>> Emma Harrington: Yeah, it's unclear
to me whether it's coming from,
388
00:24:19,463 --> 00:24:22,862
they don't think it's helpful
to be in person all the time, or
389
00:24:22,862 --> 00:24:25,330
whether they just think it's infeasible.
390
00:24:25,330 --> 00:24:31,632
I mean, even the three days a week back
in the office has been a very hard sell.
391
00:24:31,632 --> 00:24:32,793
I think so.
392
00:24:32,793 --> 00:24:36,346
>> Steven Davis: Okay.
>> Emma Harrington: Yeah.
393
00:24:36,346 --> 00:24:40,845
>> Steven Davis: We didn't talk about this
earlier, but I know from reading your
394
00:24:40,845 --> 00:24:46,608
study that at least before the pandemic,
feedback was not explicitly rewarded.
395
00:24:46,608 --> 00:24:51,508
Of course, it can feed into performance
reviews and so on, but it's not like there
396
00:24:51,508 --> 00:24:56,128
was some formula that says, you get this
much compensation for the quality and
397
00:24:56,128 --> 00:25:00,481
frequent and I volume of your feedback and
this much for your code or so on.
398
00:25:00,481 --> 00:25:05,312
But after, now we settled into
this new norm where at this firm,
399
00:25:05,312 --> 00:25:10,681
at least it sounds like there's a lot
of hybrid working arrangements,
400
00:25:10,681 --> 00:25:15,693
has the firm tried to explicitly
incentivize or disincentivize,
401
00:25:15,693 --> 00:25:20,547
I suppose feedback,
especially from senior to junior people?
402
00:25:20,547 --> 00:25:23,211
>> Emma Harrington: Not to our knowledge.
403
00:25:23,211 --> 00:25:26,060
I think when we talked
to people at the firm,
404
00:25:26,060 --> 00:25:29,834
they said that that would be
very hard because basically,
405
00:25:29,834 --> 00:25:34,241
it's pretty hard to evaluate whether
comments are helpful or not.
406
00:25:34,241 --> 00:25:38,033
And so
if you incentivize giving more feedback,
407
00:25:38,033 --> 00:25:43,131
you would probably get a lot of
relatively low quality feedback.
408
00:25:43,131 --> 00:25:45,440
>> Steven Davis: Okay,
yeah, that makes sense,
409
00:25:45,440 --> 00:25:49,893
if you reward something explicitly
that can distort behavior,
410
00:25:49,893 --> 00:25:54,520
in ways that's kind of a well known
point in personnel economics.
411
00:25:54,520 --> 00:25:58,453
But still,
what has the firm done anything,
412
00:25:58,453 --> 00:26:01,767
if not through explicit incentives,
413
00:26:01,767 --> 00:26:07,062
to try to ensure that there is
an adequate level of feedback.
414
00:26:07,062 --> 00:26:12,568
Now that they're in a world where many
people are working in hybrid mode?
415
00:26:12,568 --> 00:26:16,714
>> Emma Harrington: I mean, something kind
of interesting we can see in the data is
416
00:26:16,714 --> 00:26:22,060
that it seems like there's a shift towards
a greater proportion of the feedback
417
00:26:22,060 --> 00:26:27,911
you're receiving coming from teammates as
opposed to from people outside your team.
418
00:26:27,911 --> 00:26:33,271
And so I think that's really consistent
with thinking that within a team,
419
00:26:33,271 --> 00:26:39,057
it's maybe a little bit easier for the
manager to observe those interactions and
420
00:26:39,057 --> 00:26:43,824
potentially reward them in something
like a performance review,
421
00:26:43,824 --> 00:26:48,015
even if they're not getting
explicit incentives for it.
422
00:26:48,015 --> 00:26:52,429
>> Steven Davis: The reason I'm
probing these questions is because,
423
00:26:52,429 --> 00:26:57,724
one of the key things into my mind
about the longer term effects of
424
00:26:57,724 --> 00:27:04,100
the rise of remote work, on productivity,
on human capital development and
425
00:27:04,100 --> 00:27:08,437
mentorship and so on,
is how firms adapt over time.
426
00:27:08,437 --> 00:27:13,317
And it's pretty clear that in many
settings, some adaptation is necessary.
427
00:27:13,317 --> 00:27:17,818
If you suddenly, there's a lot of
things that happen automatically,
428
00:27:17,818 --> 00:27:21,568
your study kind of shows this,
when people are co located,
429
00:27:21,568 --> 00:27:26,219
you get more feedback, even through
this formalized, written manner,
430
00:27:26,219 --> 00:27:28,866
than you do when they're not co located.
431
00:27:28,866 --> 00:27:32,949
And now if we go to a world where
there's a lot less co location,
432
00:27:32,949 --> 00:27:37,725
than there was before, firms need to
adapt to replace that which is lost,
433
00:27:37,725 --> 00:27:41,603
by the way, in this hybrid mode,
maybe the answer to this.
434
00:27:41,603 --> 00:27:43,993
When people are working
in hybrid mode now,
435
00:27:43,993 --> 00:27:48,062
are they encouraged to come to the office
on a certain set number of days, or
436
00:27:48,062 --> 00:27:52,539
they're kind of spread out over the week,
or they can come in whenever they want.
437
00:27:52,539 --> 00:27:55,219
>> Emma Harrington: Last time
we talked to people at the firm,
438
00:27:55,219 --> 00:27:59,278
it was at the discretion of
the employee what days they came in,
439
00:27:59,278 --> 00:28:03,355
which our studies suggest would
be potentially problematic.
440
00:28:03,355 --> 00:28:04,474
>> Steven Davis: Yeah, that's right.
441
00:28:04,474 --> 00:28:08,350
Because then [LAUGH] going from
five days a week in the office,
442
00:28:08,350 --> 00:28:10,863
to only three days a week in the office.
443
00:28:10,863 --> 00:28:14,563
Where it's not the same,
three days a week for everybody,
444
00:28:14,563 --> 00:28:17,157
is not a 40% reduction in colocation.
445
00:28:17,157 --> 00:28:21,594
It's much bigger than a 40%
reduction in colocation.
446
00:28:21,594 --> 00:28:24,905
>> Emma Harrington: Yeah, and
ours, anyone suggests much bigger,
447
00:28:24,905 --> 00:28:29,751
because we find that even having one
teammate in a different building,
448
00:28:29,751 --> 00:28:34,032
in our setting or remote,
it changes the interactions between
449
00:28:34,032 --> 00:28:38,416
coworkers who themselves
are actually in the office together.
450
00:28:38,416 --> 00:28:42,654
And we think that that's because if you
just have one guy who didn't come in
451
00:28:42,654 --> 00:28:45,426
that day,
you need to have your meeting online.
452
00:28:45,426 --> 00:28:49,532
And so then you don't have the schmoozing
on the way to the meeting,
453
00:28:49,532 --> 00:28:53,795
on the way back from the meeting,
and it just changes those dynamics.
454
00:28:53,795 --> 00:28:54,384
>> Steven Davis: Yeah, So
455
00:28:54,384 --> 00:28:58,420
tell us a bit more about exactly
what you found in this regard.
456
00:28:58,420 --> 00:29:01,770
So John's home today.
457
00:29:01,770 --> 00:29:04,043
His four or
five teammates are in the office, but
458
00:29:04,043 --> 00:29:05,879
they got to get on a Zoom call with John.
459
00:29:05,879 --> 00:29:11,157
So what does that do,
according to your analysis,
460
00:29:11,157 --> 00:29:16,818
to current code writing and
the extent of mentorship?
461
00:29:16,818 --> 00:29:21,695
>> Emma Harrington: Yeah, so we find,
most of our analysis that have focused on
462
00:29:21,695 --> 00:29:24,537
the mentorship aspect of things.
463
00:29:24,537 --> 00:29:30,137
And we find that about a third of
the effect of being on a team that's
464
00:29:30,137 --> 00:29:36,938
all in one building versus a team that's
distributed across the two buildings,
465
00:29:36,938 --> 00:29:42,539
comes actually from the spillovers
of having a distant teammate on
466
00:29:42,539 --> 00:29:47,862
the interactions between people
who are in the same building.
467
00:29:47,862 --> 00:29:52,386
We can see that basically, if Tom and
Susie are in the same building,
468
00:29:52,386 --> 00:29:57,300
but John is in the other building,
Tom and Susie are likely to exchange less
469
00:29:57,300 --> 00:30:01,909
feedback, than if they were on
a similar team that was all co located.
470
00:30:01,909 --> 00:30:05,155
>> Steven Davis: Right, so
that's a pretty big effect and
471
00:30:05,155 --> 00:30:10,074
it suggested much of the informal
interaction among co located
472
00:30:10,074 --> 00:30:14,992
people was actually taking place
right before or right after or
473
00:30:14,992 --> 00:30:18,456
during the physical meetings themselves.
474
00:30:18,456 --> 00:30:23,604
Cuz if they were just bumping into each
other at the water cooler, the fact that
475
00:30:23,604 --> 00:30:28,518
John's in another location and we have
to have meetings via Zoom with him,
476
00:30:28,518 --> 00:30:33,679
you wouldn't expect to get much of
a negative effect on mentorship from that.
477
00:30:33,679 --> 00:30:36,343
Okay, so that sounds plausible to me, but
478
00:30:36,343 --> 00:30:39,155
then kind of reinforces
a view I had already.
479
00:30:39,155 --> 00:30:43,576
I wanted to get your reaction to it,
which is, if you go from a world in which
480
00:30:43,576 --> 00:30:48,495
everybody's coming into the office five
days a week to one where there's hybrid,
481
00:30:48,495 --> 00:30:52,779
especially if it's people choosing
their own times to be in the office,
482
00:30:52,779 --> 00:30:57,699
you really got to replace the old informal
social interactions that took place with
483
00:30:57,699 --> 00:30:58,781
something else.
484
00:30:58,781 --> 00:31:05,499
And in your context, this would be
a way to overcome those kind of.
485
00:31:05,499 --> 00:31:10,164
Those social inhibitions
the junior person might feel,
486
00:31:10,164 --> 00:31:13,479
in asking a senior person for feedback.
487
00:31:13,479 --> 00:31:16,021
So does this firm,
are they making more efforts?
488
00:31:16,021 --> 00:31:21,044
Like, is there a,
Wednesday night drinks or something?
489
00:31:21,044 --> 00:31:25,251
I don't know what it might be,
but some way to encourage more
490
00:31:25,251 --> 00:31:29,707
social interactions when people
are actually in the office, so
491
00:31:29,707 --> 00:31:33,773
that these productive exchanges
will flow more readily.
492
00:31:33,773 --> 00:31:38,153
>> Emma Harrington: Yeah, we haven't
talked to the software engineers at this
493
00:31:38,153 --> 00:31:43,951
firm in a while, so I'm not entirely sure
what the state of the art is for them.
494
00:31:43,951 --> 00:31:48,950
The last time we talked to them,
they were really struggling to get
495
00:31:48,950 --> 00:31:52,967
people back into the office,
even three days a week,
496
00:31:52,967 --> 00:31:58,891
whichever days they chose, and so it was
a ways away from them institutional.
497
00:31:58,891 --> 00:32:04,075
>> Steven Davis: Okay, maybe there's
a future study coming, it's got a future
498
00:32:04,075 --> 00:32:10,230
study that is focused on how does your
organization in particular, adapt?
499
00:32:10,230 --> 00:32:16,104
It sounds like they might be settling into
some kind of hybrid working arrangements,
500
00:32:16,104 --> 00:32:20,592
but how are they adapting other
aspects of the work environment,
501
00:32:20,592 --> 00:32:23,286
including the extent of socializing,
502
00:32:23,286 --> 00:32:28,284
in order to mitigate some of the costs
associated with less co location?
503
00:32:28,284 --> 00:32:31,517
I think that's a really important
question for the long term.
504
00:32:31,517 --> 00:32:34,318
>> Emma Harrington: Yeah, I agree,
I mean, I think something also,
505
00:32:34,318 --> 00:32:38,792
additionally that's interesting that we
can see in the data that we have is that
506
00:32:38,792 --> 00:32:43,237
there's some suggestive evidence,
the firm shifted its hiring practices.
507
00:32:43,237 --> 00:32:46,037
So pre Covid, it hired a lot of new grads,
508
00:32:46,037 --> 00:32:50,844
who didn't have a lot of experience
as engineers and were quite young.
509
00:32:50,844 --> 00:32:53,554
And then when the office is all closed,
510
00:32:53,554 --> 00:32:57,096
it shifted towards hiring
more senior engineers.
511
00:32:57,096 --> 00:33:00,656
And there's lots of reasons that
that may have happened, but
512
00:33:00,656 --> 00:33:05,380
I think one interpretation is that when
it was hard to facilitate that proximity,
513
00:33:05,380 --> 00:33:08,402
it wanted to sort of buy
talent rather than build it.
514
00:33:08,402 --> 00:33:12,741
And so I think if you don't have these
successful strategies to potentially
515
00:33:12,741 --> 00:33:17,913
facilitate those mentorship relationships,
firms may just change who they wanna hire.
516
00:33:17,913 --> 00:33:21,486
>> Steven Davis: Right,
which could have implications for
517
00:33:21,486 --> 00:33:26,456
the wage structure and,
the wage tenure profile and so on.
518
00:33:26,456 --> 00:33:31,366
So, anything else you want
to tell us about your study,
519
00:33:31,366 --> 00:33:38,793
other key points that we've neglected or
over overlooked so far in our discussion?
520
00:33:38,793 --> 00:33:44,259
>> Emma Harrington: I think we've really
covered the main takeaways of the study.
521
00:33:44,259 --> 00:33:45,214
>> Steven Davis: Okay, great.
522
00:33:45,214 --> 00:33:50,154
Well, we will put a link to your
study on the episode website at
523
00:33:50,154 --> 00:33:54,246
the Hoover institution so
people can check it out.
524
00:33:54,246 --> 00:33:57,239
And Emma has lots of other
interesting research, too,
525
00:33:57,239 --> 00:33:59,602
that we didn't have
time to get into today.
526
00:33:59,602 --> 00:34:01,322
You might want to check that out, too,
527
00:34:01,322 --> 00:34:03,680
including the call center
study that she mentioned.
528
00:34:03,680 --> 00:34:08,963
It's one of the better known, studies in
this area and rewards a careful look.
529
00:34:08,963 --> 00:34:13,701
So thanks so much for Emma, it was lots of
fun to chat with you about this work, and
530
00:34:13,701 --> 00:34:16,644
we're really delighted
you could join us today.
531
00:34:16,644 --> 00:34:18,603
>> Emma Harrington: Yeah,
thanks so much for having me.
532
00:34:18,603 --> 00:34:20,488
>> Steven Davis: Okay, take care.
533
00:34:20,488 --> 00:34:20,988
>> Emma Harrington: Bye.
[MUSIC]
