WEBVTT
1
00:00:00.240 --> 00:00:04.519
This week I'll be speaking with Chris
Alban about getting your first data science job.
2
00:00:04.559 --> 00:00:08.669
Chris is a data scientist at devoted
health, where he uses data science
3
00:00:08.710 --> 00:00:12.990
and machine learning to help fix America's
healthcare system. Chris is also doing a
4
00:00:13.070 --> 00:00:17.429
lot of hiring a devoted and that's
why he's so excited today to talk about
5
00:00:17.510 --> 00:00:21.460
how to get your first data science
job. You may know Chris as Co
6
00:00:21.620 --> 00:00:26.140
host to the podcast partially derivative from
his educational resources, such as is blog
7
00:00:26.300 --> 00:00:30.059
and machine learning flash cards, or
is one of the funniest data scientists on
8
00:00:30.100 --> 00:00:35.140
twitter. Welcome to data framed,
the weekly data camp podcast exploring what data
9
00:00:35.179 --> 00:00:38.810
science looks like on the ground for
working data scientists and what problems that can
10
00:00:38.890 --> 00:00:42.689
solve. I'm your host, Hugo
bound Anderson. You can follow data camp
11
00:00:42.770 --> 00:00:46.090
on twitter, that data camp and
me at Hugo bound. You can find
12
00:00:46.130 --> 00:00:54.240
all our episodes and show notes at
Data Campcom community podcast. This is data
13
00:00:54.359 --> 00:01:17.909
framed. Hi there, Chris,
and welcome to data framed. Hey,
14
00:01:17.950 --> 00:01:19.459
how's it going? It's great,
man. How are you? I'm good.
15
00:01:19.540 --> 00:01:22.900
This is like one of the first
podcasts I've done in a while.
16
00:01:23.060 --> 00:01:26.459
This is like I stopped my podcast
and now I've gone on a few ones,
17
00:01:26.540 --> 00:01:30.219
but then I had a kid and
I did a move and that kind
18
00:01:30.219 --> 00:01:33.129
of stuff, and now I'm back
back on the podcasting circuit. This is
19
00:01:33.209 --> 00:01:36.170
it fantastic. How long has it
been? Who? It's been like?
20
00:01:36.609 --> 00:01:38.370
You know. So my kid is
two months old, so I think I
21
00:01:38.489 --> 00:01:42.209
did my last podcast like two months
before that. So it's been four months
22
00:01:42.450 --> 00:01:46.569
without run them or congratulations on the
new member of your family. Thank you,
23
00:01:46.650 --> 00:01:49.239
thank you. She is doing great
all as well. Awesome, and
24
00:01:49.319 --> 00:01:52.760
congrats on the new job and the
move. There's been a lot of change.
25
00:01:52.799 --> 00:01:55.959
Right, it's been a lot of
change. There's definitely been a lot
26
00:01:55.959 --> 00:01:59.599
of change. I feel like it's
one of those things where I have a
27
00:01:59.640 --> 00:02:02.670
very hard time saying no to DJ. So when he does think that you
28
00:02:02.750 --> 00:02:06.870
should come do something, get to
think really heard about whether or not you
29
00:02:07.390 --> 00:02:08.750
you want to say no to him, which I've said no to him in
30
00:02:08.789 --> 00:02:12.189
the past, that I could not
say no to him this time. Thus
31
00:02:12.349 --> 00:02:15.860
I'm with DJ on a crazy adventure. So this is DJ Patil. This
32
00:02:15.020 --> 00:02:21.219
is DJ DJ Patil, former chief
data scientists of the US, former head
33
00:02:21.219 --> 00:02:23.580
of data a linkedin. I don't
know his whole resume, but a wellknown,
34
00:02:23.780 --> 00:02:27.900
wellknown person in the in the data
science world, very well known,
35
00:02:27.979 --> 00:02:30.969
very well respected, doing a lot
of interesting work at devoted and otherwise as
36
00:02:31.009 --> 00:02:35.849
well. I mean he's recent series
of articles with Michael Katie's and Hillary mayson
37
00:02:36.289 --> 00:02:39.289
around kind of forming a conversation around
that designs. I thinks he's really interesting
38
00:02:39.289 --> 00:02:44.360
as well. Yeah, I think
he's doing a lot of really important thinking
39
00:02:44.360 --> 00:02:46.680
in a space that we're still figuring
out. I mean this was something that
40
00:02:46.879 --> 00:02:51.360
I think a lot of us have
been talking about sort of off and on
41
00:02:51.439 --> 00:02:53.879
for a long time of like hey, we don't actually you know, we
42
00:02:53.120 --> 00:02:55.360
have lots of goals for that what
this field can do, but what does
43
00:02:55.360 --> 00:02:59.469
it actually mean in practice? And
the more we get to do that and
44
00:02:59.590 --> 00:03:01.150
the more that people sort of carved
out space in their work day to say,
45
00:03:01.189 --> 00:03:04.750
Hey, I'm going to think about
this topic, I'm going to produce
46
00:03:04.830 --> 00:03:07.990
something that other people can read and
agree with and disagree with and uses a
47
00:03:08.069 --> 00:03:12.900
point of discussion or build something off
of, is great and I hope he
48
00:03:13.020 --> 00:03:15.180
does it more and I hope Hillary
does it more and I think more people
49
00:03:15.659 --> 00:03:20.819
should spend a little bit more time
thinking about that. And it's part of
50
00:03:20.860 --> 00:03:23.979
it. That was a nice to
go work for him, because it is
51
00:03:23.060 --> 00:03:25.699
sort of Nice to go work for
someone who's thinking about that kind of stuff
52
00:03:25.819 --> 00:03:30.370
and really is big on ethics and
it's big on trying to use technology for
53
00:03:30.490 --> 00:03:35.449
for social good. Because my I
mean my for people on this packets,
54
00:03:35.490 --> 00:03:39.250
like my background is not in business, like I have started a startup,
55
00:03:39.689 --> 00:03:46.080
but my main background is nonprofits,
humanitarian nonprofits. I work on data on
56
00:03:46.120 --> 00:03:50.759
humanitarying round not profits. Did that
for most of my career, and so
57
00:03:51.319 --> 00:03:54.520
to work for a team that's led
by someone who spends a lot of time
58
00:03:54.639 --> 00:04:00.870
thinking about how to build companies with
a soul was very refreshablesome. So maybe
59
00:04:00.909 --> 00:04:04.270
you can sort up by telling us
just about devoted in general and the work
60
00:04:04.310 --> 00:04:10.939
you're doing. Sure. So devoted
a health insurance company that was started by
61
00:04:11.939 --> 00:04:15.740
Todd and Ed Park, who Todd
Park was the former CTO of the United
62
00:04:15.740 --> 00:04:21.060
States and Ed Park was the CEO
of another health insurance company or another healthcare
63
00:04:21.100 --> 00:04:26.850
company before this. And what devoted
tries to do is tries to friendly,
64
00:04:26.930 --> 00:04:30.730
like create a health insurance company that
you would want your own family members to
65
00:04:30.810 --> 00:04:32.170
be at. It is it is
a company. It is a startup.
66
00:04:32.649 --> 00:04:35.810
EXAC operates exactly like the startup,
is funded exactly like the startup. I
67
00:04:35.850 --> 00:04:39.879
think there's there's areas that would set
it apart from the start up, but
68
00:04:39.879 --> 00:04:42.279
I think overall you would look at
it me like, yes, this is
69
00:04:42.360 --> 00:04:46.920
a startup. However, devoted is
on emission. We are trying to make
70
00:04:46.040 --> 00:04:50.879
healthcare that works, that works for
people, particularly right now senior citizens.
71
00:04:50.920 --> 00:04:55.829
So we work on Medicare. If
you don't know, Medicare is the health
72
00:04:55.949 --> 00:04:59.990
insurance there's government health insurance program for
senior citizens. That is what devoted is
73
00:05:00.110 --> 00:05:02.389
focused on. That sort of what
we consider ourselves as a Medicare company.
74
00:05:03.509 --> 00:05:06.699
But on the daily basis, if
you work inside devoted, you can see
75
00:05:06.699 --> 00:05:12.500
that we are trying to build something
that would be a company with the soul,
76
00:05:12.540 --> 00:05:15.939
a company that's trying to do something
real and trying to do something that
77
00:05:15.060 --> 00:05:18.540
matters in people's lives and do right
by people and is hyper compliant with the
78
00:05:18.579 --> 00:05:24.689
law and do more than just simply
profit. Awesome. That was with draw
79
00:05:24.810 --> 00:05:27.769
drew me to it. I think
that's what draws a lot of people to
80
00:05:27.889 --> 00:05:30.810
devoted. That's great and, as
will discuss, you're working a lot on
81
00:05:30.970 --> 00:05:32.329
hiring at the moment, and what
we're going to talk about today, among
82
00:05:32.370 --> 00:05:38.160
other things, is people getting their
first Dietosians jobs and advice for such people,
83
00:05:38.519 --> 00:05:40.879
how it works, what it looks
like from your your side of the
84
00:05:40.920 --> 00:05:46.079
conversation as well, and I think
particularly this point, where junior dietosaians parts
85
00:05:46.079 --> 00:05:49.000
aren't necessarily fleshed out with sign to
see a bunch of specialization in the industry.
86
00:05:49.189 --> 00:05:53.629
They such things will be incredibly important
going forward. But before we dive
87
00:05:53.670 --> 00:05:56.509
in to that, I just want
to get a bit more background about you
88
00:05:56.750 --> 00:05:59.269
and if you could tell me,
like you said you'll, your background isn't
89
00:05:59.269 --> 00:06:01.990
necessarily in ditosase. I'm wondering how
you got involved in Dicosians in the first
90
00:06:02.029 --> 00:06:08.220
place. My backgrounds in quantitative political
science, so political science studying of politics.
91
00:06:08.259 --> 00:06:11.939
I studied civil wars, but it's
political science from completely from the perspective
92
00:06:11.980 --> 00:06:16.860
of statistics, of quantitative research,
of experimentation, rather than qualitative work,
93
00:06:17.180 --> 00:06:21.730
interviewing people, looking at historic documents, that kind of thing. And when
94
00:06:21.810 --> 00:06:27.970
I was getting my PhD, I
kept on having drinks with these people in
95
00:06:28.009 --> 00:06:30.129
San Francisco, where I was sort
of living at the time. And they
96
00:06:30.129 --> 00:06:32.800
were working at places like Linkedin.
So there was DJ and there's number of
97
00:06:32.800 --> 00:06:38.240
other people and they were doing such
cool applied stuff, so many, you
98
00:06:38.319 --> 00:06:43.120
know, amazing projects, amazing uses
of data in a very, very applied
99
00:06:43.160 --> 00:06:46.680
way. And there was about then
that I kind of decided that if I
100
00:06:46.959 --> 00:06:51.110
really wanted to have a real impact
and really wanted to do something that would
101
00:06:51.509 --> 00:06:56.589
matter, that I needed to not
be an academia. I needed to go
102
00:06:56.670 --> 00:07:00.110
out and actually apply the skills in
a way that was beyond research. And
103
00:07:00.189 --> 00:07:03.259
don't get me wrong, I love
research. I every single person who applies
104
00:07:03.339 --> 00:07:05.579
for devoted who has any kind of
PhD. I just want to talk to
105
00:07:05.620 --> 00:07:10.459
about their phcl day. I think
you know that that's a a bias of
106
00:07:10.540 --> 00:07:13.459
mine. But there's so many ways
that you could apply it. And then
107
00:07:13.500 --> 00:07:16.569
at the time there was some chances
for me to go work for some Joint
108
00:07:16.649 --> 00:07:20.009
Canyon US nonprofits, Canyon nonprofits,
that kind of stuff, and I spent
109
00:07:20.089 --> 00:07:25.089
a number of years working, being
sort of the first day to hire over
110
00:07:25.129 --> 00:07:28.610
there at say, Ushihiti, which
is a Canyon nonprofit that works on election
111
00:07:28.649 --> 00:07:32.360
monitoring and disaster relief. Went to
go work for brick, which is a
112
00:07:32.439 --> 00:07:39.160
canyon, startup that works on providing
free Wi fi to lower inco people in
113
00:07:39.160 --> 00:07:43.759
Kenya. CARLINE SE MEST did a
lot of workaround election monitoring. But,
114
00:07:44.079 --> 00:07:46.029
you know, taking data, real
data about, say, say, an
115
00:07:46.069 --> 00:07:50.949
election in a country with an authoritarian
leadership, and actually seem like is this
116
00:07:50.990 --> 00:07:55.069
day of true? Are People filing
in fake you know our peel filing and
117
00:07:55.149 --> 00:07:58.550
fake elections? Are People filing in
fake reports, like what is actually happening
118
00:07:58.660 --> 00:08:03.019
on the ground, with real data, with people who are in places where
119
00:08:03.060 --> 00:08:05.899
they can get arrested if things go
wrong? You know, that was a
120
00:08:05.980 --> 00:08:09.259
big eye opener to be around issues
of safety, around its use of ethics,
121
00:08:09.620 --> 00:08:13.290
rund issues of like will like,
for example, if I go to
122
00:08:13.370 --> 00:08:18.089
a place that has election monitoring and
we're running election monitoring campaign with the local
123
00:08:18.170 --> 00:08:22.410
Ngoh, if the copsbuts down the
door for some reason because something happened,
124
00:08:22.889 --> 00:08:26.730
I would be arrested and then flown
back to the US. They would be
125
00:08:26.810 --> 00:08:31.839
arrested and God knows what would happen. And that mean that our threat models
126
00:08:31.879 --> 00:08:35.360
are very different and sort of understanding
that your threat model isn't their threat model,
127
00:08:35.840 --> 00:08:37.960
I think has a lot of applications
for data science, you know in
128
00:08:39.039 --> 00:08:41.159
the US and everywhere. That,
I think, is it was. It
129
00:08:41.240 --> 00:08:43.190
was an important lesson for me.
Absolutely, and I've a question around.
130
00:08:43.710 --> 00:08:46.389
Just seems like on these time this
type of work, you'll learn a lot
131
00:08:46.470 --> 00:08:50.750
on the job, both in terms
of domain expertise but also in terms of
132
00:08:50.789 --> 00:08:54.590
data scientific techniques. I'm just wondering, before you got your first day to
133
00:08:54.629 --> 00:08:58.139
science job, did you know how
to program in Python, or did you
134
00:08:58.220 --> 00:09:01.340
have it like an idea of what
the landscape look like from your time in
135
00:09:01.539 --> 00:09:05.100
in research? Where I was when
I was finishing at my phd was I
136
00:09:05.299 --> 00:09:11.570
had done some web work. So
I'm not going to say web development because
137
00:09:11.570 --> 00:09:13.490
it wasn't a lot of javascript that
I stuff, but I done like html
138
00:09:13.769 --> 00:09:16.730
and CSS, which aren't really programming, but just you know that kind of
139
00:09:16.769 --> 00:09:20.690
stuff, like making some of the
pages with a little bit of Javascript in
140
00:09:20.730 --> 00:09:26.080
there and that kind of stuff.
And I done all of my research in
141
00:09:26.279 --> 00:09:31.399
Stata and are but I was a
very good at I don't think. I
142
00:09:31.600 --> 00:09:33.759
think my my code was very poor
at that moment, but I did see
143
00:09:35.039 --> 00:09:37.789
through my conversation to other people that
there was this world of coding wasn't just
144
00:09:37.909 --> 00:09:41.950
a little script that you run and
like leave your laptop, but you know
145
00:09:41.990 --> 00:09:43.190
if it needs to run for more
than like six hours, you just leave
146
00:09:43.190 --> 00:09:46.950
your laptop open for six hours.
That I realized that there was much more
147
00:09:48.470 --> 00:09:52.419
into the world of software engineering and
I started to move in that direction because
148
00:09:54.019 --> 00:09:56.860
when you want to build stuff with
other engineers, it's nice to use the
149
00:09:56.899 --> 00:09:58.940
tools that they use and to think
about things that they use and to take
150
00:10:00.019 --> 00:10:03.299
code that they can insert into their
into their projects, and so I really
151
00:10:03.299 --> 00:10:09.289
started to push more along the lines
of developing more software engineering skills, more
152
00:10:09.409 --> 00:10:13.090
languages that are pretty common in software
engineering, such as Python. But I
153
00:10:13.169 --> 00:10:18.169
definitely when I left my PhD,
I was not the wizard programmer that I'm
154
00:10:18.169 --> 00:10:20.679
not today, but I could.
I love that. So I usually ask
155
00:10:20.759 --> 00:10:24.519
a question around what your colleagues think
you do as opposed to what you actually
156
00:10:24.559 --> 00:10:28.559
do. That might be slightly different
in what you do. Devoted people might
157
00:10:28.559 --> 00:10:31.399
have more of an idea. I
think historically, though, in terms of
158
00:10:31.399 --> 00:10:35.830
the setups you've worked in doing analytics
and diato signs, is there a mismatch
159
00:10:35.950 --> 00:10:39.230
between what people think you do and
what you actually do? I think people
160
00:10:41.269 --> 00:10:45.710
so we'll take brick brick is a
Kenyan startup that does free Wi fi for
161
00:10:45.789 --> 00:10:48.950
low and could people it. The
whole team is Kenyan. They're based out
162
00:10:48.990 --> 00:10:52.899
of Kenya and I was the first
day to hire at brick. I think
163
00:10:52.940 --> 00:10:56.940
a lot of them thought that I
was doing wizard mathematics, like behind you
164
00:10:56.980 --> 00:11:00.940
know, there was a white board
and my office and I'm writing equations and
165
00:11:01.860 --> 00:11:05.409
solving riddles of mathematics to get them
some kind of thing, where in fact
166
00:11:05.490 --> 00:11:09.690
what I was doing was a lot
of software engineering stuff, a lot of
167
00:11:09.809 --> 00:11:13.570
building functions, running things, crown
jobs, like using a lot of python
168
00:11:15.250 --> 00:11:18.679
or a lot of pandas some psycit
learned that kind of stuff. But I
169
00:11:18.840 --> 00:11:22.200
was I was mostly using established tools, but it would provide them with actually
170
00:11:22.200 --> 00:11:26.360
something that was useful in there in
their work. But I think from their
171
00:11:26.399 --> 00:11:28.519
perspective I was like wizardry right,
because I would do something like imputation,
172
00:11:28.720 --> 00:11:33.669
where you take missing values in your
data and you impute the value. You
173
00:11:33.789 --> 00:11:37.629
like fake what the value would probably
be, and it was like wizardry right,
174
00:11:37.710 --> 00:11:39.669
like wow, I can't believe that
happened, like you were predicting this
175
00:11:39.789 --> 00:11:45.590
stuff, and so I think it
feels very normal to the data scientists and
176
00:11:45.750 --> 00:11:48.379
to me, but I think it
felt very it was. It was very
177
00:11:48.419 --> 00:11:50.580
exciting for a team that didn't have
that to start to have that kind of
178
00:11:50.659 --> 00:11:54.500
option for stuff, and I think
it worked well intasty. So, as
179
00:11:54.539 --> 00:11:58.740
we've said, we here to talk
about getting your first Dita Science job and
180
00:11:58.220 --> 00:12:01.850
you're working on hiring dita scientists,
devoted and thinking a lot about people getting
181
00:12:01.850 --> 00:12:05.889
the first data science job today and
I want to kind of figure out what
182
00:12:05.929 --> 00:12:09.090
a companies are generally looking for when
hiring first time data scientists. But before
183
00:12:09.210 --> 00:12:13.970
that, I suppose. Yeah,
a preliminary question is, are a lot
184
00:12:13.009 --> 00:12:18.279
of companies trying to hire a first
time data scientists, because a lot of
185
00:12:18.320 --> 00:12:22.559
the job list things like one ten
years experience of distributed computing and all of
186
00:12:22.639 --> 00:12:24.840
that jazz right. So, like
where is it? Has it had to
187
00:12:24.840 --> 00:12:28.000
use the term entry level? But
for a first time job? Are there?
188
00:12:28.039 --> 00:12:31.230
Are there a lot of jobs out
there? Yeah, I think they're.
189
00:12:31.309 --> 00:12:35.909
There are, although from so the
perspective I think will we could take
190
00:12:35.950 --> 00:12:39.789
this interview is that I'm sitting on
the other side of the table where I'm
191
00:12:39.789 --> 00:12:41.750
doing a lot of the working with
a team it devoted to do a lot
192
00:12:41.750 --> 00:12:45.980
of hiring for a data science team
and I talked to a lot of my
193
00:12:46.059 --> 00:12:48.700
friends who are doing similar roles at
other companies. You know, so in
194
00:12:48.740 --> 00:12:52.019
the interview they're on that they're on
the hiring side. So definitely are a
195
00:12:52.100 --> 00:12:56.899
lot of junior roles and I wouldn't
put into someone's head that they're like junior
196
00:12:56.899 --> 00:13:00.090
rules are dying or everyone needs experience
and that kind of stuff. You know,
197
00:13:00.250 --> 00:13:07.049
there is something I definitely see that
as a team you can absorb infinite
198
00:13:07.090 --> 00:13:09.649
number of senior hires. So say
you have a team of six people,
199
00:13:09.929 --> 00:13:15.080
you can hire six new other senior
people and be pretty okay that everything's just
200
00:13:15.120 --> 00:13:18.919
going to work because their senior right
like it'll do fine. It is much
201
00:13:18.919 --> 00:13:22.799
more of a risk from the organization's
perspective to if you have six people on
202
00:13:22.879 --> 00:13:26.590
your team and then you hire six
junior but because there's a like those junior
203
00:13:26.629 --> 00:13:31.389
people need a lot more support and
if you're unable to give that support,
204
00:13:33.350 --> 00:13:35.470
it is not good for the junior
people or for you. And definitely take
205
00:13:35.509 --> 00:13:37.750
this from the perspective that junior person
like you. Do not want to be
206
00:13:37.830 --> 00:13:43.940
in a place where there is one
senior data scientists and there's six junior people
207
00:13:43.940 --> 00:13:48.779
who are all hired at the same
time, because you are not gonna Learn
208
00:13:48.899 --> 00:13:50.860
what you need to learn. Like
you want to be the one junior person
209
00:13:52.340 --> 00:13:54.700
on a team of, you know, six, sixteenior people. I'd would
210
00:13:54.700 --> 00:13:58.090
be the ideal situation, and you
can just sit down and take all the
211
00:13:58.169 --> 00:14:01.090
time of theirs you want and you
can work with them very closely and you're
212
00:14:01.090 --> 00:14:03.330
learning would be massive. That would
be incredible. There's early junior jobs out
213
00:14:03.330 --> 00:14:07.649
there, but there is something from
an organizations prospective where you know you might
214
00:14:07.649 --> 00:14:09.879
have a small start up and you
only have three data scientists or two data
215
00:14:09.919 --> 00:14:16.720
scientists at this company because data science
is a specialty job and they you just
216
00:14:16.799 --> 00:14:20.120
can't. You can't have a posting
that says you want to hire five junior
217
00:14:20.200 --> 00:14:22.159
people like I wouldn't. Don't.
Don't apply that job. I think I
218
00:14:22.200 --> 00:14:26.309
would be very horrible. Yeah,
instead, you want somewhere. I think
219
00:14:26.309 --> 00:14:28.789
if I was looking, if I
was junior. So one thing we should
220
00:14:28.789 --> 00:14:31.950
point out that when I was junior
and data science, it was a very
221
00:14:31.990 --> 00:14:35.629
different field. So you should not
take my how I got into the field
222
00:14:35.629 --> 00:14:39.820
as an example of how you should
get into the field, because when I
223
00:14:39.940 --> 00:14:45.059
started there wasn't really the concept of
data science and you you was sort of
224
00:14:45.100 --> 00:14:46.940
a bunch of people are interested in
the same topic and just sort of like
225
00:14:48.100 --> 00:14:50.220
got together. Yeah, I'm doing
it, doing it now. It's a
226
00:14:50.220 --> 00:14:54.570
little it's different, obviously, but
I think I can see some things around.
227
00:14:54.450 --> 00:14:58.409
If you are, you know,
if you're looking for that first job,
228
00:14:58.450 --> 00:15:05.450
I would definitely look for places in
mid to larger companies and not in
229
00:15:05.090 --> 00:15:09.600
smaller, smaller, scrappier startups.
I think one of the things that I've
230
00:15:09.720 --> 00:15:15.279
found a few times during the hiring
process is that people who got their first
231
00:15:15.279 --> 00:15:18.480
job at, say, you know, facebook, and they're a junior,
232
00:15:18.519 --> 00:15:22.230
junior data scientist at facebook, there's
so much more support that they got in
233
00:15:22.350 --> 00:15:28.830
that time for learning, for you
know, experimentation, for using things at
234
00:15:28.870 --> 00:15:31.750
scale, right, like learning how
to work at facebook scale, but doing
235
00:15:31.830 --> 00:15:37.700
so as a very junior person.
They have such great experience that then they
236
00:15:37.779 --> 00:15:39.820
move on to say, you know, say apply for a job at devoted
237
00:15:39.860 --> 00:15:45.259
or something like that, and they
have all that experience and and we can
238
00:15:45.460 --> 00:15:48.100
use that experience, like that's great, that, you're awesome, like cool,
239
00:15:48.139 --> 00:15:50.889
you have this great experience. This
is hard experience to get because you
240
00:15:52.009 --> 00:15:54.330
know, it's hard for a boot
camp to, to say, replicate fifty
241
00:15:54.409 --> 00:15:58.450
million messages the day that you have
to process or something like that. There's
242
00:15:58.490 --> 00:16:00.649
there's that's a that's a weird boot
camp project, but that is what a
243
00:16:00.649 --> 00:16:03.009
lot of companies end up doing.
So going to somewhere like facebook or going
244
00:16:03.090 --> 00:16:06.399
to I'm just like picking on Facebook, and this isn't it a for facebook.
245
00:16:06.399 --> 00:16:08.200
FACEBOOK is in a playing room,
but any kind of larger company that
246
00:16:08.440 --> 00:16:12.320
can absorb can absorb you as a
junior higher and out like a allow you
247
00:16:12.360 --> 00:16:15.879
to learn and allow you to learn
from from really senior engineers and then moving
248
00:16:15.919 --> 00:16:18.549
on to something else is, I
think, a great perspective. PUSHUL and
249
00:16:18.629 --> 00:16:22.830
those companies, I think, have
the infrastructure all set up as well,
250
00:16:22.870 --> 00:16:26.269
so you not like battling with your
your data likes and aflow and all of
251
00:16:26.350 --> 00:16:30.269
that run. Yeah, well,
and there's absolutely a trap that I think
252
00:16:30.269 --> 00:16:33.179
junior data scientists fall into, even
midleveld to scientists fall into, where they
253
00:16:33.259 --> 00:16:37.220
join a small, scrappy start up, not as a founder, right,
254
00:16:37.259 --> 00:16:40.740
but just, you know, the
start up as six people or something like
255
00:16:40.820 --> 00:16:44.620
that, and they're the first day
to hire and there's no data infrastructure in
256
00:16:44.740 --> 00:16:48.649
place at all and there's no safetiness, there's no, you know, ability
257
00:16:48.690 --> 00:16:52.009
to get data from the database in
a way that's useful and you have to
258
00:16:52.049 --> 00:16:56.169
sit down and build all that,
which in certain times can be okay.
259
00:16:56.450 --> 00:17:00.049
But I think in a really scrappy
start up where they're like struggling to make
260
00:17:00.090 --> 00:17:03.440
payroll or the you know, they're
like grinding away or something, it can
261
00:17:03.480 --> 00:17:07.799
be a very, very hard experience
and probably not the best experience as opposed
262
00:17:07.839 --> 00:17:11.519
to something where you had, you
know, you go work at some really
263
00:17:11.599 --> 00:17:14.000
hard problems at a larger company,
but you do so in a way that
264
00:17:14.440 --> 00:17:17.309
you can leave work at the end
of the day and you know knowing that
265
00:17:17.349 --> 00:17:19.509
everything's fine, and you come back
the next day and all that infrastructures in
266
00:17:19.549 --> 00:17:22.430
place and off talk some lectures and
you can go over to some person who's
267
00:17:22.430 --> 00:17:26.950
done data engineering for ten years and
say, Hey, why does it work
268
00:17:26.990 --> 00:17:27.630
like this, and they'll be like, oh, it's because of x,
269
00:17:27.630 --> 00:17:30.019
Y and Z and that kind of
stuff. As I think there's just there's
270
00:17:30.019 --> 00:17:33.539
so much learning that can happen at
larger companies. I've never worked at larger
271
00:17:33.579 --> 00:17:37.740
company, so this is me talking
from the outside but I do think that,
272
00:17:38.099 --> 00:17:41.940
as someone has worked at a few
smaller companies or small organizations, that
273
00:17:41.980 --> 00:17:48.009
if you those organizations do better if
you're senior, just because you can sort
274
00:17:48.049 --> 00:17:51.690
of be left alone and figure everything
out by yourself because you've done this before,
275
00:17:52.009 --> 00:17:53.450
as opposed to actually, I don't
know what I'm doing, I really
276
00:17:53.490 --> 00:17:56.930
need someone to tell me how to
do it right and so you learn things
277
00:17:56.930 --> 00:17:59.079
the right way. I think there's
there's a lot of things where like,
278
00:17:59.119 --> 00:18:03.440
particularly where a data scientist is doing
more software engineering stuff, where it's really
279
00:18:03.440 --> 00:18:06.240
nice to sit down for someone to
be like Hey, explain to me object
280
00:18:06.319 --> 00:18:08.319
oriented programming, because I don't understand
what in the world's happening, or explained
281
00:18:08.319 --> 00:18:11.720
to me like how testing works,
like unit tests, integration test or you
282
00:18:11.799 --> 00:18:15.950
know patterns, like Software Engineering Patterns, like factory of factories and that kind
283
00:18:15.990 --> 00:18:18.589
of stuff. The tell me about
that. Like all those are totally simple
284
00:18:18.630 --> 00:18:22.190
concepts that anyone listen to this podcast
could know. Just need to like have
285
00:18:22.430 --> 00:18:26.309
someone tell you about him or know
that. You should read about them like
286
00:18:26.750 --> 00:18:27.940
understand. Oh, actually, I
this comes up a lot. I should
287
00:18:27.940 --> 00:18:32.180
read about this. That kind of
stuff, which is great in companies with
288
00:18:32.339 --> 00:18:36.740
more support and is terrible and really, really small, scrappy places. So,
289
00:18:37.059 --> 00:18:40.779
in general, what a company's looking
for when hiring first on Ditoscientists,
290
00:18:41.019 --> 00:18:41.890
do you think? And if you
don't want to answer that, you can
291
00:18:41.930 --> 00:18:45.890
tell me, like what you in
particular are looking for when haring first on
292
00:18:45.009 --> 00:18:48.569
ditoscientists. Sure, it's also I'll
try to stab at the one, but
293
00:18:48.609 --> 00:18:51.210
then we'll definitely hit on the other
one. So Great. It depends on
294
00:18:51.250 --> 00:18:52.809
the company. I should say organization, because I work for a lot enough
295
00:18:52.809 --> 00:18:56.799
parbets, but we'll just use company
as like a forget for that. Places
296
00:18:56.880 --> 00:19:03.279
that are very small and scrappy tend
to look for senior people that they can
297
00:19:03.599 --> 00:19:07.279
run the whole like I think you
would call them like a full sack data
298
00:19:07.319 --> 00:19:10.750
scientist. I'm not entirely enamored with
that phrase, but whatever, like a
299
00:19:10.869 --> 00:19:14.109
generalist, a generalist so that you
could you could just have them join the
300
00:19:14.470 --> 00:19:18.470
software engineering team or the product team
or something like that and you give them
301
00:19:18.630 --> 00:19:22.150
pseudo access on your you know,
on your server, and they will just
302
00:19:22.230 --> 00:19:25.339
build everything they need to build the
build on the pipelining, the build all
303
00:19:25.420 --> 00:19:27.740
the you know, saving backups,
all the data. They'll build the tables
304
00:19:27.779 --> 00:19:30.460
that they need to build and they'll
run the analysis and they'll run it in
305
00:19:30.500 --> 00:19:33.420
a way that runs every single you
know, every single day, but only
306
00:19:33.460 --> 00:19:37.700
on certain times, and they'll install
air flow by themselves and they'll do all
307
00:19:37.819 --> 00:19:41.049
this kind of stuff that just they
can. They can do. That's awesome
308
00:19:41.089 --> 00:19:45.809
for that person. That's not really
a junior role no more. I think
309
00:19:45.849 --> 00:19:48.609
if you go to the other end
of the spectrum and you go for large
310
00:19:48.690 --> 00:19:52.289
companies, they do a mix.
Well, they will hire people from more
311
00:19:52.410 --> 00:19:56.920
senior positions to do things like,
say, someone very specialized, say someone
312
00:19:56.960 --> 00:20:02.200
who just got their PhD in ai
or something like that. Like they might
313
00:20:02.240 --> 00:20:06.480
have them join the team that's working
on an algorithm and that junior person can
314
00:20:06.519 --> 00:20:08.630
work on a part of the algorithm
or work on some testing part and grind
315
00:20:08.710 --> 00:20:14.150
to that kind of stuff. Or
I think what's very common is more generalist
316
00:20:14.150 --> 00:20:18.230
data scientists who are junior join larger
companies and they do probably what I would
317
00:20:18.230 --> 00:20:23.220
it would call like advanced analyzes or
advanced analytics. Right, so you have
318
00:20:23.420 --> 00:20:27.579
like a new product and you want
to understand how that products is actually doing
319
00:20:27.619 --> 00:20:30.539
at scale, right, because it's, you know, deploy globally. On
320
00:20:30.619 --> 00:20:34.059
all android devices or something like that, and you want to see how that
321
00:20:34.140 --> 00:20:37.089
product is working and how people are
using it. There is some very,
322
00:20:37.130 --> 00:20:41.089
very, very complicated analyzes that need
to be done for that to be true,
323
00:20:41.809 --> 00:20:45.410
and that is a great sort of
thing for a junior person to kind
324
00:20:45.450 --> 00:20:49.519
of tear off and start working on. And it doesn't affect the production code,
325
00:20:49.759 --> 00:20:52.640
but it does affect that business and
in that respect, with I suppose
326
00:20:52.680 --> 00:20:56.440
we're talking about the data analyst breed
of Dita scientists, right, yeah,
327
00:20:56.440 --> 00:20:59.920
and so like again, I don't
know why I keep on talking aboutacebook,
328
00:20:59.920 --> 00:21:02.119
but I know it facebook. A
lot of people who have the title of
329
00:21:02.160 --> 00:21:04.750
data scientists do more analysis, you
know, which is totally reasonable and is
330
00:21:04.789 --> 00:21:08.509
super hard, and give them massive
credit and it is. That kind of
331
00:21:08.589 --> 00:21:11.390
stuff is, you know, is
a real roles, a real job,
332
00:21:11.430 --> 00:21:15.109
and I think people who come from
academia, you know, like the people
333
00:21:15.150 --> 00:21:19.140
who I know come from sort of
political sciences, social scientists, they go
334
00:21:19.500 --> 00:21:22.859
into those kind of roles and have
a great time because they are working on
335
00:21:23.140 --> 00:21:29.539
hard analyzes, like hard analytical problems. It's not you're not making dashboards to
336
00:21:29.579 --> 00:21:33.809
show that someone's clicking on something.
You're making really complicated analysies to figure out,
337
00:21:33.049 --> 00:21:37.529
you know, the churn model that
applies Bayesi into when we know we
338
00:21:37.769 --> 00:21:40.970
all that kind of stuff, which
is cool. I think in the middle,
339
00:21:41.250 --> 00:21:42.890
right to the the have the teeny
companies and you have a large companies.
340
00:21:44.490 --> 00:21:48.359
You have companies like devoted that sort
of sit in the middle where there
341
00:21:48.640 --> 00:21:56.000
is a mix between trying to hire
senior people who can craft the foundational data
342
00:21:56.079 --> 00:22:00.119
science if its structure right. So
devoted is building a health insurance companies text
343
00:22:00.240 --> 00:22:03.150
AC from scratch. You know,
if, like it was at one point
344
00:22:03.190 --> 00:22:07.990
an empty github repo. Now there's
now there's a lot of code in there,
345
00:22:07.069 --> 00:22:10.029
but yeah, at some point we're
like, okay, we are health
346
00:22:10.069 --> 00:22:12.630
insurance company. This is our codebase, like, let's write the first line
347
00:22:12.630 --> 00:22:18.140
of code. That's where we are. And in those kind of environments you
348
00:22:18.339 --> 00:22:21.779
want a mix of people who are
senior who can build the architecture for how
349
00:22:22.019 --> 00:22:26.660
things work and how data is moved
around and how, say, tasks or
350
00:22:26.660 --> 00:22:30.009
run every day or how there's,
you know, things around testing and if
351
00:22:30.250 --> 00:22:33.609
companies do testing and all that kind
of stuff. And in addition, you
352
00:22:33.849 --> 00:22:40.450
want people who have more junior experience
that you can come in you can provide
353
00:22:40.490 --> 00:22:42.130
some lift them, like teaching.
Wise, like you could do more teaching
354
00:22:42.210 --> 00:22:45.440
with them, and you can have
them support the role where you might tear
355
00:22:45.480 --> 00:22:48.680
off a piece of a project and
say, Hey, can you make this
356
00:22:48.839 --> 00:22:51.640
for me? Like so, I
don't need so, I don't need to
357
00:22:51.680 --> 00:22:53.400
make it because it was definitely so. One of the people who's on our
358
00:22:53.480 --> 00:22:57.400
team currently is Kit Rodolpha, who
is the former chief data scientists of the
359
00:22:57.400 --> 00:23:03.230
Hillary campaign, and he's done some
amazing complicated work and he's also done some
360
00:23:03.349 --> 00:23:07.589
amazing work that was well below his
level of expertise because there was just no
361
00:23:07.750 --> 00:23:12.019
one else there to do it.
And so there's I think typically midlevel companies
362
00:23:12.099 --> 00:23:15.339
like ours tend to have both.
Some are longer range. And so for
363
00:23:15.420 --> 00:23:18.700
you, for your size of company, what would a for a junior ditis
364
00:23:18.740 --> 00:23:22.619
on his wood skills? Would I
oh, what would be good qualities?
365
00:23:22.259 --> 00:23:26.579
What would they need to do a
demonstrate fee to be like how this person
366
00:23:26.700 --> 00:23:30.650
would be a good feed here.
Yeah, I think for us at devoted
367
00:23:30.210 --> 00:23:36.490
we are a health insurance company,
but if you looked at the internal workings
368
00:23:36.650 --> 00:23:40.690
of how the Tech Team Works,
we operate for more like a Silicon Valley
369
00:23:40.849 --> 00:23:47.559
Tech Company and concepts of move fast
and break things or ask for forgiveness rather
370
00:23:47.599 --> 00:23:49.119
than permission are real things that we
are doing. I think a lot of
371
00:23:49.160 --> 00:23:52.319
the people have devoted are building things
that is the largest and hardest thing that
372
00:23:52.400 --> 00:23:56.829
they've built ever in their career,
and that's what we want you to be
373
00:23:56.950 --> 00:24:00.269
that we want you to be the
person who is building the doing the best
374
00:24:00.349 --> 00:24:03.349
work of career, doing the hardest
thing, that is pushing yourself right to
375
00:24:03.430 --> 00:24:07.230
the limit of what you think you
can do. And we have a very
376
00:24:07.309 --> 00:24:11.539
strong culture that says, Hey,
we want you to run right up until
377
00:24:11.539 --> 00:24:14.980
the point that you break the thing, break the thing and then come and
378
00:24:15.259 --> 00:24:18.700
and be able to say, Hey, I totally broke this, can someone
379
00:24:18.140 --> 00:24:22.460
fix this, Dear God, and
then we'll go back and we'll work with
380
00:24:22.579 --> 00:24:25.569
them to fix it and, and
I say this is them, but like
381
00:24:25.690 --> 00:24:29.650
I have in fact broken many things
at devoted and to have a culture that
382
00:24:29.730 --> 00:24:32.849
says, Hey, you can break
these things, it's okay, don't worry
383
00:24:32.890 --> 00:24:34.410
about it. You should definitely admit
that it broken, because we should know
384
00:24:34.529 --> 00:24:37.210
that, but run right up until
you break it. Then we'll talk about
385
00:24:37.210 --> 00:24:41.119
why it was broken and will then
run again, like don't stop running,
386
00:24:41.119 --> 00:24:45.119
keep on running. But that fast
paced, the fast face doesn't translate into
387
00:24:45.119 --> 00:24:48.720
a huge amount of work hours because, you know, devoted, like I
388
00:24:48.799 --> 00:24:51.920
have a family and no one's no
one had devoted, I think, is
389
00:24:51.960 --> 00:24:55.069
grinding the midnight oil left, right
and center. There's definitely people are working
390
00:24:55.829 --> 00:24:59.390
to five ish, but just you
know, and what you're doing. I
391
00:24:59.470 --> 00:25:06.390
think there's a lot of pushing the
boundaries of what we can do skill wise
392
00:25:06.430 --> 00:25:11.420
right to the end and a lot
of learning and trying to build the things
393
00:25:11.460 --> 00:25:15.900
that are the limit of our capability. Is Pretty Common. I think that
394
00:25:15.059 --> 00:25:19.299
would be a skill regardless of your
expert you know, level of experience,
395
00:25:19.339 --> 00:25:22.410
whether you have one year of experience
or no. Love you know, no
396
00:25:22.569 --> 00:25:26.250
years of experience or ten years of
experience. We really do look for people
397
00:25:26.690 --> 00:25:32.250
who are very happy to take on
a task that they've never done before,
398
00:25:32.450 --> 00:25:33.930
but they're pretty you know, they
kind of understand how they might go about
399
00:25:33.930 --> 00:25:38.039
it and go for it, like
go and run it and we have safety
400
00:25:38.039 --> 00:25:42.240
guards in place that nothing's nothing's bad
going to happen, that they can push
401
00:25:42.319 --> 00:25:45.960
themselves and then once they had that, once they've built that thing, they
402
00:25:45.960 --> 00:25:48.279
go okay, cool, I understand
it now. It's great, I've got
403
00:25:48.400 --> 00:25:52.069
it. Like let's do something else, like let's go harder, let's do
404
00:25:52.230 --> 00:25:55.390
this, build the next thing.
I think that's really it's hard to instill
405
00:25:55.430 --> 00:26:00.109
because it is not experience per se. It's more, you know, aptitude,
406
00:26:00.150 --> 00:26:06.099
and attitude is probably a closer one
where I don't necessarily we don't necessarily
407
00:26:06.140 --> 00:26:11.819
say we're only looking for people with
x years. We tend to have a
408
00:26:11.940 --> 00:26:14.460
willingness to say, Hey, you're
actually, you know, like we were
409
00:26:14.460 --> 00:26:17.059
kind of looking for someone more senior, but you are more junior, but
410
00:26:17.339 --> 00:26:21.650
you really are kicking butt and we
can see that when we talk to you
411
00:26:22.089 --> 00:26:26.609
that you have a lot of aptitude
for your level, like for your levels
412
00:26:26.650 --> 00:26:29.930
of experience, and we love your
attitude for running with it, like let's
413
00:26:29.930 --> 00:26:32.569
go, like, come on board. We've definitely, you know, we've
414
00:26:32.609 --> 00:26:36.319
had a few hires like that and
I think it is worked out, but
415
00:26:36.400 --> 00:26:38.599
it is it is something. It's
not uniquely devoted, but it is something
416
00:26:38.640 --> 00:26:42.160
that is probably pretty uncommon for our
industry. Yeah, and that speaks to
417
00:26:42.440 --> 00:26:45.960
certain skills that you've mentioned in there
are being like an active problem solver,
418
00:26:47.230 --> 00:26:56.230
communication, having a passion and drive. Will jump right back into our interview
419
00:26:56.230 --> 00:27:04.859
with Chris Alban after a short segment. Now it's time for a segment called
420
00:27:06.059 --> 00:27:10.539
guidelines for online experiments. So I'm
here with Emily Robinson, a data signed
421
00:27:10.539 --> 00:27:12.019
us on the growth theme here at
data camp. Hey, emily, Hi
422
00:27:12.099 --> 00:27:17.170
Hugo. So, Emily, you've
written what I consider a fantastic post on
423
00:27:17.250 --> 00:27:21.450
guidelines for AB testing and you've also
given a related talk several times, and
424
00:27:21.730 --> 00:27:25.329
I'm really interested to hear your thoughts
on what kind of rules of thumb and
425
00:27:25.410 --> 00:27:27.930
guidelines for AB testing actually are.
But maybe first you can start by just
426
00:27:29.049 --> 00:27:33.480
reminding us or what an online experiment
or an AB test actually is. Sure,
427
00:27:33.519 --> 00:27:36.920
Hugo. So, the idea of
an online experiment is that you want
428
00:27:36.920 --> 00:27:40.480
to measure the impact of a change
that you make on your website, and
429
00:27:40.799 --> 00:27:45.150
how you do this is that you
randomly assigned people to either the control so
430
00:27:45.309 --> 00:27:48.069
the old experience, or the treatment
group, the new experience, and by
431
00:27:48.109 --> 00:27:55.069
randomly assigning people you can then make
sure that there's no differences between the two
432
00:27:55.150 --> 00:27:59.180
groups, so, for example,
that people the countries are randomized, that
433
00:27:59.539 --> 00:28:03.619
new and returning users etc. And
this is really helpful because another approach might
434
00:28:03.660 --> 00:28:07.700
just be to launch the change and
watch the graph move and say, Oh,
435
00:28:07.740 --> 00:28:11.740
okay, we launched this change on
Tuesday. DID REGISTRATION RATE GO UP?
436
00:28:11.339 --> 00:28:15.890
Well, the problem is registration rate
or your other metrics are not a
437
00:28:15.049 --> 00:28:18.769
fixed value each day. They have
their own variance and you also have other
438
00:28:18.809 --> 00:28:22.609
stuff going on in the world at
the same time, like, say,
439
00:28:22.690 --> 00:28:26.809
maybe there is a holiday or another
project was launched, and that means that
440
00:28:26.930 --> 00:28:30.720
it's very hard to tell if one
if there was any movement at all and
441
00:28:30.920 --> 00:28:34.559
to if that movement really was because
of your change. And online experiments allow
442
00:28:34.559 --> 00:28:38.319
you to make that inference by giving
you a control group telling you what would
443
00:28:38.319 --> 00:28:42.069
have happened without that change. Okay, so that's a great explanation of what
444
00:28:42.190 --> 00:28:47.069
an online experiment or an abatist actually
use before we get into the guideline.
445
00:28:47.109 --> 00:28:49.230
So there are a couple of,
I suppose, technical terms that would be
446
00:28:49.230 --> 00:28:52.630
good to Damis, define, and
I'm thinking really about test statistic and the
447
00:28:52.750 --> 00:28:56.420
scary pay value, right. So
maybe you could just tell us how you
448
00:28:56.500 --> 00:28:59.819
use these in online experiments and then
we can jump into you guidelines. Sure.
449
00:29:00.180 --> 00:29:03.900
So let's say you have your control
in your treatment group and you're interested
450
00:29:03.980 --> 00:29:08.140
in registration rate on your website and
it's five percent in the control and five
451
00:29:08.259 --> 00:29:12.009
point zero five percent in the treatment. Does that mean that your treatment was
452
00:29:12.049 --> 00:29:17.210
a success and you increase registration rate? Well, the problem is there is
453
00:29:17.289 --> 00:29:21.410
some randomness again, in that there's
variability and it could have just been chance
454
00:29:21.569 --> 00:29:23.289
that this happened. You know,
for example, if you plut a coin
455
00:29:23.440 --> 00:29:27.079
two times and it turns up heads, you wouldn't say that's necessarily an unfair
456
00:29:27.160 --> 00:29:32.920
coin. That could just happen by
chance. So what I do is we
457
00:29:33.039 --> 00:29:37.400
run what's called a proportion test for
a specifically where we're looking for the percent
458
00:29:37.480 --> 00:29:41.390
of people who did a certain action. A proportion test then we compare the
459
00:29:41.910 --> 00:29:47.069
proportion in the control versus the treatment, and not just five versus five point
460
00:29:47.069 --> 00:29:51.549
zero five, but specifically. You
know, is that five thousand of a
461
00:29:51.630 --> 00:29:56.380
hundred thousand or five hundred out of
a thousand? Because that matters and that
462
00:29:56.500 --> 00:30:00.299
gives it. That test gives us
back a P value, and the definition
463
00:30:00.500 --> 00:30:03.420
of a pe value is, if
you're know, hypothesis was true, which
464
00:30:03.460 --> 00:30:07.490
in this case is that there's no
difference between the control and the treatment group.
465
00:30:08.089 --> 00:30:12.450
How likely is it that you would
have seen a difference this extreme or
466
00:30:12.569 --> 00:30:15.849
more right. So the idea then
is it's really it has to do with
467
00:30:15.890 --> 00:30:21.250
how surprised you should be about seeing
your result right exactly. So statistics can
468
00:30:21.329 --> 00:30:23.759
never tell us, oh, we're
absolutely sure that there was a, quote
469
00:30:23.759 --> 00:30:27.480
unquote, real difference, but it
can't tell you a P value of point
470
00:30:27.559 --> 00:30:30.599
your one would say, for example, okay, if there was no difference,
471
00:30:30.799 --> 00:30:34.480
only one percent of a time would
you see the results that you got
472
00:30:34.559 --> 00:30:37.750
or results that were more extreme.
So you know, maybe you're that one
473
00:30:37.829 --> 00:30:41.509
percent. Maybe there was no difference, but generally there's a rule of thumb
474
00:30:41.549 --> 00:30:45.829
that if your p value is less
than point zero, five, so is
475
00:30:45.829 --> 00:30:48.950
a lesson five percent chance of getting
a false positive. If there was no
476
00:30:49.099 --> 00:30:52.099
difference, then we'll go ahead and
say, okay, we are going to
477
00:30:52.500 --> 00:30:59.180
reject then hypothesis and assume that there
was a real difference. Right, okay.
478
00:30:59.420 --> 00:31:03.019
So that's a great introduction to online
experimentation. You quote a great saying
479
00:31:03.019 --> 00:31:06.450
in your blood post in your talked
up generating numbers. So getting data for
480
00:31:06.490 --> 00:31:08.890
these type of stuff, generating numbers
is easy. Generating numbers, you should
481
00:31:08.890 --> 00:31:12.890
trust, is hard, and you've
written that there are many ways ab tasting
482
00:31:12.970 --> 00:31:17.089
can go wrong, but most of
them won't be obvious. So I did.
483
00:31:17.130 --> 00:31:21.279
That's a brief introduction to, or
motivation for, your guidelines, and
484
00:31:21.359 --> 00:31:23.480
we here to start doing about best
practices for AB testing. So what's your
485
00:31:23.559 --> 00:31:27.759
first guideline? The first one is
to have one key metric per experiment.
486
00:31:29.119 --> 00:31:33.230
So what that means is define what
your success metric is. That doesn't mean
487
00:31:33.309 --> 00:31:37.190
that you can't monitor multiple metrics to
make sure you don't accidentally take them.
488
00:31:37.549 --> 00:31:41.349
For example, here at data camp, where subscription business, and so while
489
00:31:41.390 --> 00:31:45.910
we might be targeting registration rate for
a certain experiment, if suddenly subscription rate
490
00:31:47.029 --> 00:31:49.339
went to zero in the treatment,
that would be pretty bad. But the
491
00:31:49.460 --> 00:31:53.220
reason we only want one metric is
that if we're looking at many metrics,
492
00:31:53.380 --> 00:31:59.420
you'll end up with an increased false
positive rate unless you apply corrections. What
493
00:31:59.579 --> 00:32:02.690
that means is if you say are
looking at ten metrics, the probability that
494
00:32:02.930 --> 00:32:07.529
one of them has a p value
of below zero five is, even if
495
00:32:07.529 --> 00:32:13.049
there's no difference, if your treatment
made no change and no impact your metrics
496
00:32:13.569 --> 00:32:16.720
is way above five percent. In
fact I think it's above thirty percent.
497
00:32:17.000 --> 00:32:21.599
Fantastic. So they're kind of lads
to my next question. How long should
498
00:32:21.640 --> 00:32:23.920
you run an experiment in order to
see an effect? This is a tricky
499
00:32:23.960 --> 00:32:30.309
question because is here you're looking to
avoid false and negative so thinking that there
500
00:32:30.509 --> 00:32:35.630
is no difference when there actually is
one, and when you say see and
501
00:32:35.710 --> 00:32:38.910
effect, it really matters what the
magnitude of effect is. So essentially,
502
00:32:38.910 --> 00:32:43.990
the longer you run it, the
smaller a difference you can detect. And
503
00:32:44.460 --> 00:32:49.819
how you can measure this or estimate
this is doing a power calculation. So
504
00:32:49.940 --> 00:32:52.539
you take your key metrics, to
say your registration rate. You figure out,
505
00:32:52.619 --> 00:32:57.500
all right, over a week we
have tenzero people and their registration rate
506
00:32:57.579 --> 00:33:00.130
is ten percent. You seem that's
going to be roughly the same the week
507
00:33:00.130 --> 00:33:04.890
you're running the experiment for the Control
Group. And then a power calculation can
508
00:33:04.930 --> 00:33:07.130
say, all right, with that
much data, with that registration rate,
509
00:33:07.569 --> 00:33:12.730
how big an effect can you detect? Where there's an eighty percent chance of
510
00:33:12.849 --> 00:33:15.599
if there's an effect of this size, you'll detect it. So, for
511
00:33:15.720 --> 00:33:19.079
example, your power calculation might say, Oh, in a week you can
512
00:33:19.160 --> 00:33:23.039
detect a five percent increase, and
that means if your increase is only two
513
00:33:23.119 --> 00:33:28.069
percent, you're probably not going to
detect it and even if it is really
514
00:33:28.150 --> 00:33:30.670
five percent, there's still a twenty
percent chance will miss it. But you
515
00:33:30.789 --> 00:33:36.069
have to make these tradeoffs because,
again, statistics can never tell you something
516
00:33:36.150 --> 00:33:39.630
definitively and you may decide, Hey, I'm okay missing a two percent change
517
00:33:40.150 --> 00:33:44.819
because we need to move fast and
a two percent change actually wouldn't even be
518
00:33:44.940 --> 00:33:46.819
that impactful. Right. And I
suppose one of the points is you want
519
00:33:46.819 --> 00:33:51.500
to do this power analysis during the
experimental design phase, right. You don't
520
00:33:51.500 --> 00:33:53.099
want to do a post talk at
power analysis, for example, right in
521
00:33:53.180 --> 00:33:58.690
the this guy scenario exactly, and
there's a couple reasons for this. One
522
00:33:58.890 --> 00:34:01.329
is you want to do it beforehand, because sometimes it may tell you that
523
00:34:01.410 --> 00:34:06.009
you shouldn't bother running the experiment.
So, for example, let's say you're
524
00:34:06.089 --> 00:34:07.929
making a change. You're like,
all right, I believe this change could
525
00:34:07.929 --> 00:34:13.840
have a ten percent increase in registration
rate and you run a power capitulation and
526
00:34:13.960 --> 00:34:17.840
it says in three weeks you could
only detect a thirty percent increase because maybe
527
00:34:17.840 --> 00:34:21.719
you just don't have them any visitors
on this part of the site. And
528
00:34:21.880 --> 00:34:24.030
given that, you may decide that
you don't want to run this test.
529
00:34:24.150 --> 00:34:28.309
After all perfect. So, emily, that's what we've got time for now,
530
00:34:28.389 --> 00:34:30.150
but I will really look forward to
coming back and hearing more about your
531
00:34:30.150 --> 00:34:35.070
guidelines for online experimentation. Thanks you
go. Looking forward to chatting more about
532
00:34:35.070 --> 00:34:45.460
it. Come to get straight back
into our chat with Chris. How about
533
00:34:45.500 --> 00:34:50.260
in terms of hard skills, whether
it be domain expertise or background in programming,
534
00:34:50.300 --> 00:34:52.650
whether it's Python, Ora or knowledge
of you know, how the math
535
00:34:52.730 --> 00:34:58.050
behind machine learning models are actually works
to like data analytic skills. Yeah,
536
00:34:58.050 --> 00:35:01.409
I've noticed something with hiring where typically, when we have someone in the hiring
537
00:35:01.449 --> 00:35:07.679
process, I will say to them
up front that there are places like,
538
00:35:07.760 --> 00:35:13.920
if you just got your master's degree
in machine learning and you believe that how
539
00:35:14.079 --> 00:35:16.199
you are going to advance your career
is to do machine learning, become an
540
00:35:16.199 --> 00:35:19.679
expert in machine learning, that's what
you're going to do. That is a
541
00:35:19.760 --> 00:35:24.670
completely valid jump my career track.
You should absolutely do that also. That
542
00:35:24.909 --> 00:35:29.949
isn't a place like devoted. Devoted
tends to be more generalist builders who are
543
00:35:30.070 --> 00:35:32.630
trying to you know, we will
use machine learning here because it's useful and
544
00:35:32.829 --> 00:35:36.420
then we'll use, you know,
fuzzy matching here because it's useful. And
545
00:35:36.460 --> 00:35:38.300
then we'll, you know, like
use some simple things somewhere else, because
546
00:35:38.340 --> 00:35:45.099
it's it gets the job done.
We tend to focus on the ability of
547
00:35:45.179 --> 00:35:51.449
people to build and solve the needs
of our internal users more than, say,
548
00:35:51.530 --> 00:35:55.010
someone's really nuanced view of machine learning
algorithms, just because, you know,
549
00:35:55.090 --> 00:35:59.730
we are a small team in a
new company. If you wanted to
550
00:35:59.969 --> 00:36:02.010
just do machine learning, there are
definitely companies that are big enough to that
551
00:36:02.250 --> 00:36:06.480
have like that have the infrastructure to
have you just do that right. Someone
552
00:36:06.519 --> 00:36:08.440
else will worry about, you know, some of the analytics or some or
553
00:36:08.440 --> 00:36:12.800
some of the data processing. You
can just sit down all day and read
554
00:36:13.039 --> 00:36:15.639
Machine Learning Journal articles and then implement
them. That's solely okay, and I
555
00:36:15.920 --> 00:36:19.150
you know, if I think,
I never worked at Google brain, but
556
00:36:19.190 --> 00:36:22.389
I imagine google brain has a strong
feeling around that, which is cool.
557
00:36:22.429 --> 00:36:25.710
Would that be your advice to first
time job seekers? Learn a bunch of
558
00:36:25.789 --> 00:36:30.349
general things in order to build stuff, as opposed to go deep into machine
559
00:36:30.389 --> 00:36:32.739
learning models and all of that?
If I was sitting in somewhat in front
560
00:36:32.780 --> 00:36:37.340
of someone who is taking or looking
at a junior job or something like that,
561
00:36:37.019 --> 00:36:40.300
I would say that you probably want
to take one of two tracks.
562
00:36:42.099 --> 00:36:47.170
One, if you have the educational
experience on paper and the credentials to go
563
00:36:47.610 --> 00:36:52.929
for, say, deep ai,
that just go deep on ai like you
564
00:36:52.050 --> 00:36:54.809
have. You have the master's degree
in it or you have a PhD in
565
00:36:54.889 --> 00:36:59.369
it or something like that. Go
for that track like that's an amazing it's
566
00:36:59.409 --> 00:37:04.280
incredibly high paying, it's super in
demand and there are places where you won't
567
00:37:04.280 --> 00:37:07.039
have to learn anything else but ML. That can be a career and I
568
00:37:07.039 --> 00:37:09.239
think there's probably going to be a
full career in that. There's absolutely no
569
00:37:09.360 --> 00:37:12.880
problem with that, because those are
very, very hard and but I I
570
00:37:12.920 --> 00:37:15.190
would almost exclusively say neural networks at
this point. Yeah, there's a lot
571
00:37:15.190 --> 00:37:20.389
of companies that are doing things that
are self driving cars or things like that
572
00:37:20.550 --> 00:37:22.510
where you just you need a lot
of people with that kind of knowledge to
573
00:37:22.590 --> 00:37:27.150
work on those problems because of those
are incredibly hard problems. But it is
574
00:37:27.469 --> 00:37:31.219
definitely along the lines of neural networks. I deep learning to do those and
575
00:37:31.420 --> 00:37:36.780
for this truck that is a certain
mathematical overhead which perhaps the other track,
576
00:37:36.820 --> 00:37:38.380
we haven't got to that yet,
might not have. But, for example,
577
00:37:38.860 --> 00:37:45.369
knowing enough about multi vary calculus to
understand back prop and the venishing guidient
578
00:37:45.409 --> 00:37:46.690
problem in all this stuff. Right. Oh, no, absolutely, and
579
00:37:46.969 --> 00:37:52.849
you I would expect that the interviews
for those would be very math heavy and
580
00:37:52.289 --> 00:37:55.690
very, very heavy, such that
it would almost feel like a I don't
581
00:37:55.690 --> 00:37:59.199
know, like a dissertation defense.
Yeah, in that area, for that
582
00:37:59.320 --> 00:38:02.440
truck, as you said, you'd
expect some sort of graduate work to have
583
00:38:02.559 --> 00:38:07.639
been done in math or something related. Yeah, I would expect like ninety
584
00:38:07.679 --> 00:38:09.599
five percent of people to have some
kind of like very obvious that they're going
585
00:38:09.639 --> 00:38:13.150
down that track. I know,
I'm sure there's some people who have like
586
00:38:13.309 --> 00:38:15.630
gone around that and those people are
awesome and amazing, but I think typically
587
00:38:16.190 --> 00:38:22.110
you would expect someone to be from
that perspective. The other one, if
588
00:38:22.150 --> 00:38:25.590
you don't have that like very obvious
sort of machine learning deep learning focus,
589
00:38:25.630 --> 00:38:30.780
which I don't. So this is
more my perspective. There's another field that
590
00:38:30.900 --> 00:38:35.780
is way more sort of doing data
science more generally at a company. Right,
591
00:38:35.780 --> 00:38:37.260
so instead of just being like all
I do is machine learning all day,
592
00:38:37.380 --> 00:38:40.769
there's so many other problems that need
to be solved using data science,
593
00:38:40.809 --> 00:38:45.849
whether you do like Baysi and analyzes
or you do some kind of you know,
594
00:38:45.969 --> 00:38:47.690
like random force, or even if
you use some deep learning stuff,
595
00:38:47.730 --> 00:38:52.849
but you're not doing cutting edge deem
learning stuff all day. There's some far
596
00:38:52.050 --> 00:38:55.960
more jobs, although like the the
hype and the focus is on this sort
597
00:38:57.000 --> 00:39:00.440
of AI jobs, there's far more
jobs, like fifty times as many jobs
598
00:39:00.559 --> 00:39:07.639
of people who are semi generalist data
scientists at companies solving those companies needs to
599
00:39:07.679 --> 00:39:12.789
understand what's happening in their data,
whether that's predicting when their drone should fly
600
00:39:12.909 --> 00:39:16.429
over and water the crops or predicting
when a customer I might return or,
601
00:39:16.710 --> 00:39:21.389
you know, for us, like
predicting when someone might have a particular illness
602
00:39:21.429 --> 00:39:24.460
so we could do an intervention and
prevent that illness from happening. Those kind
603
00:39:24.539 --> 00:39:29.820
of analyzes fit better for people who
sort of have a more general experience,
604
00:39:29.820 --> 00:39:32.019
because there isn't there isn't a very
type for that right. There's no Bayesian
605
00:39:32.539 --> 00:39:36.820
analyze PhD, and if you have
that you get to do Baysi analyzes and
606
00:39:36.820 --> 00:39:38.210
if you don't, you don't get
to do that. It's much more general
607
00:39:38.250 --> 00:39:42.690
so that the people who apply for
for us in those kind of roles can
608
00:39:42.730 --> 00:39:45.489
come from any perspective and can be
everything from you know, a music major,
609
00:39:45.530 --> 00:39:50.130
or can be someone with a PhD
and data science and one of the
610
00:39:50.130 --> 00:39:52.239
new programs, or can come from
a boot camp, from a PhD in
611
00:39:52.320 --> 00:39:57.440
some other crazy field, or could
not have a PhD. It's more about
612
00:39:57.519 --> 00:40:00.239
just if that kind of person fits
what we want and what we need.
613
00:40:00.599 --> 00:40:01.920
But it is I mean that.
I think that's where most people do it.
614
00:40:02.000 --> 00:40:06.159
I I say this only because I
think there's a lot of people who
615
00:40:06.159 --> 00:40:10.429
think if I know enough machine learning
and I go deep enough on machine learning,
616
00:40:10.429 --> 00:40:15.670
I'm like that's the guaranteed job,
which isn't true, because you could
617
00:40:15.670 --> 00:40:19.829
self teach a lot of machine learning
and then you apply a Google brain and
618
00:40:19.869 --> 00:40:22.980
they could be like hey, you're
just not even close because you didn't get
619
00:40:22.980 --> 00:40:25.179
a PhD in this and haven't spent, you know, like a huge amount
620
00:40:25.179 --> 00:40:28.980
of time doing that, I think. But then you could go to another
621
00:40:28.980 --> 00:40:30.380
company, take that same person and
go to another company say hey, I
622
00:40:30.420 --> 00:40:34.059
know a lot of machine learning.
They are awesome. This is like like,
623
00:40:34.380 --> 00:40:36.449
you're not going to just be doing
machine learning, but we have need
624
00:40:36.530 --> 00:40:39.489
for people who know machine learning because
our products use natural language processing and that
625
00:40:39.530 --> 00:40:43.130
kind of stuff. Come on,
join the team. You'll be great,
626
00:40:43.289 --> 00:40:45.409
and so that kind of I think
that second job has a lot more generaloius
627
00:40:45.449 --> 00:40:50.000
things where you end up working.
You end up sort of having to be
628
00:40:50.800 --> 00:40:53.440
you know, a lot more selfware
engineering skills, a lot more, you
629
00:40:53.519 --> 00:40:58.039
know, skills working with sort of
the software side of skills of working with
630
00:40:58.199 --> 00:41:01.199
cut like internal customers. So says
someone on the sales team wants some kind
631
00:41:01.199 --> 00:41:06.269
of particular analysis. That analysis is
very difficult to do. You go back
632
00:41:06.309 --> 00:41:07.110
and you do it, but then
you have to go and present it to
633
00:41:07.190 --> 00:41:09.630
a way and then work with them
to tweak the analysis in the way that
634
00:41:09.710 --> 00:41:12.750
they want. You need to be
able to work with them such as that
635
00:41:12.750 --> 00:41:15.550
they're happy with what you're delivering with
them. That kind of stuff, I
636
00:41:15.590 --> 00:41:17.940
think, is more common. I
think that's what most people do. It's
637
00:41:17.980 --> 00:41:22.619
also it's different than just saying,
Hey, I'm just going to like know
638
00:41:22.860 --> 00:41:24.860
everything about machine learning and that I
won't need to care about any other topic.
639
00:41:24.980 --> 00:41:28.619
And so, for this secondtrol,
which I think is kind of the
640
00:41:28.619 --> 00:41:31.260
lawn share of what we what we're
discussing today, there's a chicken and egg
641
00:41:31.260 --> 00:41:35.250
problem in the senset. For a
first time, dito scientist applying for their
642
00:41:35.329 --> 00:41:38.210
first job? How can they demonstrate
that they have kind of this general array
643
00:41:38.289 --> 00:41:42.730
of skills? Would it be project
Bise, like rotting a blog themselves to
644
00:41:42.849 --> 00:41:45.769
demonstrate it? Because it seems like
you need the experience in order to get
645
00:41:45.849 --> 00:41:50.360
the first job. Essentially, yeah, for us, when someone applies some
646
00:41:50.360 --> 00:41:53.679
of the best things that they can
apply with our projects that they've done or,
647
00:41:54.280 --> 00:41:57.039
you know, something at like,
I say, a boot camp or
648
00:41:57.159 --> 00:42:01.150
to maybe they're maybe they're dissertation research
or something like that, where we can
649
00:42:01.230 --> 00:42:04.989
take a look and say, oh, cool, like you've done some interesting
650
00:42:04.989 --> 00:42:07.309
stuff, you worked with some data
some interesting ways, and then for us
651
00:42:08.030 --> 00:42:13.829
we have designed a take home that
is very without giving out it any kind
652
00:42:13.829 --> 00:42:16.019
of secret to the TAKEOM is very
open. So there's many, many ways.
653
00:42:16.059 --> 00:42:19.659
I mean there's an infinite amount of
ways essentially that someone could solve it
654
00:42:19.780 --> 00:42:22.659
and how they solve it really says
a lot about them. But those kind
655
00:42:22.739 --> 00:42:27.099
of skills of being able to demonstrate, hey, like I can write software,
656
00:42:27.099 --> 00:42:30.090
like I can write you a test. I can sit down and do
657
00:42:30.210 --> 00:42:32.929
basie and I'm totally comfortable with it. Here's my blog post around how the
658
00:42:34.050 --> 00:42:37.449
certain type of Baysie analysis works in
the setting, or here's this really sweet
659
00:42:37.489 --> 00:42:40.090
data visualization that I did. That
kind of stuff is like a nice demonstration.
660
00:42:40.130 --> 00:42:45.400
I don't think it's required, but
I think if you come from them,
661
00:42:45.440 --> 00:42:47.960
say, a field that isn't known
for making a budget quantitative people,
662
00:42:49.000 --> 00:42:52.760
and I come from political science,
I think people don't think about political science
663
00:42:52.800 --> 00:42:55.119
and think role math nerds. The
more things that you can do around that
664
00:42:55.239 --> 00:43:00.030
where you can sort of demonstrate that
you have that kind of experience is better,
665
00:43:00.230 --> 00:43:06.190
because otherwise you don't hit people's biases
around what a social scientist is right,
666
00:43:06.190 --> 00:43:07.429
because someone can say, Oh,
you do political science, you must
667
00:43:07.429 --> 00:43:12.429
really love Kant and Rousseau and that's
all you're talking about all day. And
668
00:43:12.550 --> 00:43:15.539
if that's not true, like you
need to demonstrate that. It's probably not
669
00:43:15.619 --> 00:43:16.699
fair that you need to demonstrate that, but that is just what it is,
670
00:43:16.739 --> 00:43:21.619
that you need to demonstrate that.
And so things like boot camps,
671
00:43:21.980 --> 00:43:23.780
things like projects that you can run
your own blog post, a great because
672
00:43:23.780 --> 00:43:27.019
it's really easy to access them,
like you can just click, you know,
673
00:43:27.099 --> 00:43:29.849
like someone has a link in their
resume and they don't click the link.
674
00:43:29.889 --> 00:43:30.650
You say, Oh cool, this
is like a really nice you know,
675
00:43:30.690 --> 00:43:35.329
I love the way that they're thinking
about this particular problem of feature importance
676
00:43:35.409 --> 00:43:37.170
in random force or something like that
is a really nice way and I think
677
00:43:37.690 --> 00:43:43.480
it is helpful. We don't require
someone has side projects, but we are
678
00:43:43.639 --> 00:43:49.159
trying to do filtering of thousands of
candidates and so it's nice to find people
679
00:43:49.159 --> 00:43:52.480
who have who can show you right
up front that they have those kind of
680
00:43:52.559 --> 00:43:58.110
skills and can do so in a
way that is more than just a resume
681
00:43:58.269 --> 00:44:01.070
line, because, as someone who's
looked at a lot of resumes, everyone
682
00:44:01.190 --> 00:44:06.349
says that they do every skill under
the sun. Like everyone lists every skill.
683
00:44:06.389 --> 00:44:10.820
So everyone is everyone says Python and
sequel and are and machine learning and
684
00:44:12.019 --> 00:44:15.619
random forests, and everyone says every
skill, which is like. So it's
685
00:44:15.619 --> 00:44:19.260
not a very strong signal. But
if you also have, say, blog
686
00:44:19.340 --> 00:44:22.699
post about some nuance point about random
forest or some nuance point of Baysi analysis,
687
00:44:23.179 --> 00:44:25.289
that is a real thing. That's
a costly signal to me that you
688
00:44:25.369 --> 00:44:29.090
actually do know that rather than just
typing in the world in your rsume,
689
00:44:29.570 --> 00:44:31.409
and it helps, I think it
helps for me to get a handle on
690
00:44:31.690 --> 00:44:36.289
who that person is. It helps
to guide interview process in the future.
691
00:44:36.329 --> 00:44:37.760
That, like other people do,
and I don't do that. I can
692
00:44:37.840 --> 00:44:42.039
sort of say hey, like,
here's this article from this person's blog that
693
00:44:42.199 --> 00:44:45.719
probably will be a subject of an
interview, which is totally fine and probably
694
00:44:45.719 --> 00:44:46.800
helps the candid a little bit because
they can sort of stay in the area
695
00:44:46.800 --> 00:44:52.440
that they know. But it is
demonstrations of that are very helpful. Do
696
00:44:52.559 --> 00:44:54.309
I think everyone needs to do side
projects left and right? Otherwise there are
697
00:44:54.309 --> 00:44:57.710
a crappy data scientists? No,
of course, not like this. Isn't
698
00:44:58.030 --> 00:45:00.349
you don't need to live and breathe
data science to get a job in the
699
00:45:00.389 --> 00:45:04.829
data science but it is helpful to
someone who's hiring to sort of see the
700
00:45:04.869 --> 00:45:07.980
things that you've worked on and see
the things that you've done more than just
701
00:45:07.059 --> 00:45:12.019
adding that keyword to your rhythme.
And I think the other thing that running
702
00:45:12.059 --> 00:45:15.539
blogs demonstrates his the ability to take
a project through to the end and actually
703
00:45:15.579 --> 00:45:19.820
do a ride up and, on
top of that, demonstrates communication skills,
704
00:45:19.860 --> 00:45:22.449
which, of course, incredibly important
in this line of work. We had
705
00:45:22.530 --> 00:45:24.289
one candidate who I really, I
really liked. They ended up taking another
706
00:45:24.289 --> 00:45:29.329
job somewhere else, but I really
liked and they sat down and they actually
707
00:45:29.369 --> 00:45:31.889
had a github project that they were
working on. I don't think it was
708
00:45:32.010 --> 00:45:37.639
like a full python package yet,
but they had basically built an open source
709
00:45:37.679 --> 00:45:39.800
library very close to one. I
don't think they'd released it, but they
710
00:45:39.840 --> 00:45:44.400
had like a little opens ource library
for some project and they had testing in
711
00:45:44.480 --> 00:45:47.719
there and they had documentation and they
had the odjectory and python in there and
712
00:45:47.760 --> 00:45:52.510
they were importing like relative imports of
modules so they could do the tests and
713
00:45:52.550 --> 00:45:53.349
all the kind of stuff, and
it was cool, like you could look
714
00:45:53.349 --> 00:45:55.429
at that and be like, okay, I kind of like this person has
715
00:45:55.510 --> 00:46:00.070
this level right, this person has
this this amount of knowledge, because they've
716
00:46:00.070 --> 00:46:02.110
clearly written it in their a gidthub
account and I can see it. It's
717
00:46:02.110 --> 00:46:06.460
a nice signal for me. You
could do that anyway that there's no particular
718
00:46:06.460 --> 00:46:10.500
way that I like absolutely want,
but I think the resumes that our send
719
00:46:10.539 --> 00:46:15.219
the least signal to me or other
people who are hiring on devoted are the
720
00:46:15.340 --> 00:46:21.250
ones that have just here's the five
skills that I have and then kind of
721
00:46:21.289 --> 00:46:23.010
don't do any kind of explaining because
there's a lot of like cheap signals,
722
00:46:23.050 --> 00:46:27.130
like you could, I mean I
could say that I do deep learning and
723
00:46:27.210 --> 00:46:30.289
then you're like Oh really, let's
talk about that. And it turns out
724
00:46:30.289 --> 00:46:31.610
that I have not known enough about
deep learning, run and the other thing
725
00:46:31.650 --> 00:46:35.239
you mentioned in there. There are
a couple of other points, which github
726
00:46:35.280 --> 00:46:37.119
repository, I'm sure, is incredibly
helpful, I don't talk about. You
727
00:46:37.119 --> 00:46:42.480
mentioned testing and a through line through
this as being the importance of at least
728
00:46:42.559 --> 00:46:45.039
bicic software engineering stools in terms of
ditosions, and I find that that's something
729
00:46:45.119 --> 00:46:50.510
that's missing with a lot of kind
of early career data analysts and ditosionists,
730
00:46:50.550 --> 00:46:54.190
whether it be using day Buggas or
unit testing or versioning, these types of
731
00:46:54.269 --> 00:46:57.670
things, people need to work on
a bit more. I think that's key.
732
00:46:57.670 --> 00:47:00.900
If if there's anything, if there's
anything that goes through our data science
733
00:47:00.980 --> 00:47:05.500
team, it devoted when we are
looking to hire someone or when we are
734
00:47:05.539 --> 00:47:07.739
working on our own projects, is
that the stuff that we work on our
735
00:47:07.860 --> 00:47:13.099
products. We are building full product
x for people, whether they are only
736
00:47:13.139 --> 00:47:16.250
a few lines of code or,
you know, say they're something simpler like
737
00:47:16.690 --> 00:47:21.690
moving some kind of data from a
Google spreadsheet to redshift, just something really
738
00:47:21.730 --> 00:47:28.329
simple like that, or something more
complicated. We are building full products products
739
00:47:28.369 --> 00:47:30.199
for people, and thus they need
to have as much of that as we
740
00:47:30.280 --> 00:47:34.079
can have. We want testing in
there. We want things to be,
741
00:47:34.159 --> 00:47:37.719
say, lnted, we want things
to follow some kind of object oriented notion
742
00:47:37.840 --> 00:47:42.679
or, say, functional python with
the dock strings in there, the describable
743
00:47:42.719 --> 00:47:45.829
the documents are doing, say,
static typing, if we could do that.
744
00:47:45.909 --> 00:47:47.230
We're seemingly on the verge of doing
that, but not doing they get
745
00:47:47.750 --> 00:47:51.469
but that kind of stuff like.
We think of that as a software product.
746
00:47:51.469 --> 00:47:53.389
We make software products for people,
whether that's, you know, whether
747
00:47:53.389 --> 00:47:58.269
it's actually like a reusable tool or
analyzes. That's just what we do and
748
00:47:58.940 --> 00:48:04.019
I wish I thought about that more
from the start. And I think as
749
00:48:04.099 --> 00:48:07.420
data science matures, I think there's
definitely going to be this movement in the
750
00:48:07.539 --> 00:48:13.250
direction of data scientists sitting on software
engineering teams and being a member of a
751
00:48:13.329 --> 00:48:17.329
software engineering team just with a certain
specialty set of skills, and part of
752
00:48:17.369 --> 00:48:21.289
that means that you need to be
able to work with them and write code
753
00:48:21.329 --> 00:48:24.050
that they can use and write code
that they are fine to incorporate in their
754
00:48:24.090 --> 00:48:28.639
stuff, and that just means more
software engineering skills for sure. And so
755
00:48:29.039 --> 00:48:31.639
something that we've thrown around a bit
is this idea of graduate school and dissertations,
756
00:48:31.760 --> 00:48:35.480
and a question I get a lot
is from people who are thinking of
757
00:48:35.559 --> 00:48:37.599
going to Grad school and they're actually
wondering whether to go to Grud's. If
758
00:48:37.639 --> 00:48:40.000
I want to, will can ditosions
with it go to red school or with
759
00:48:40.119 --> 00:48:44.510
it to get like a diet analyst
job that I can get at that point
760
00:48:44.630 --> 00:48:47.309
and then try to progress into Ditosians
demonstrating that they have developed a bunch of
761
00:48:47.349 --> 00:48:51.590
analytical tools. Do you have any
thoughts on that from your side of the
762
00:48:51.630 --> 00:48:54.860
hiring tyble? Sure, I would
never recommend someone get a PhD. I
763
00:48:54.940 --> 00:48:58.900
mean masters might be different. Bust
will sort of. I would never recommend
764
00:48:58.900 --> 00:49:02.219
someone get a PhD because they wanted
to get a better job. Like a
765
00:49:02.340 --> 00:49:07.420
PhD's as a long and difficult process
and you got to really want to do
766
00:49:07.539 --> 00:49:09.929
a PhD in order to do a
Pahda. My PhD, like a huge
767
00:49:09.969 --> 00:49:14.369
amount of people quit, maybe like
half or more than half of people quit
768
00:49:14.809 --> 00:49:16.769
along the way and got nothing right, because if you a Pahd can be
769
00:49:17.010 --> 00:49:20.730
six years long. If you quit
after a year three, you don't get
770
00:49:20.730 --> 00:49:22.289
anything. I think you maybe they
might give you a master's is like a
771
00:49:22.409 --> 00:49:25.320
sort of a constellation price or something
that. But it is a very,
772
00:49:25.400 --> 00:49:29.559
very tough activity and it's very,
very hard and most people don't complete it
773
00:49:29.679 --> 00:49:31.880
and people have a lot of stress
around it. It is difficult and the
774
00:49:31.960 --> 00:49:36.719
final year in particular, I just
remember my bag. The final year was
775
00:49:36.880 --> 00:49:40.030
absolutely brutal. I don't want to
say I hided the work I was doing,
776
00:49:40.150 --> 00:49:44.550
but there was certainly, at the
end, a bunch of negative sentiments
777
00:49:44.590 --> 00:49:46.909
that involved, you know, the
stress and the suffering and also the sleeping
778
00:49:46.949 --> 00:49:50.469
on to my disk for the final
six months secual. Oh yeah, now
779
00:49:50.550 --> 00:49:52.579
completely. I hated my dissertation by
the end. I mean I yeah,
780
00:49:52.780 --> 00:49:55.139
the phrase that I kept on saying
over my mind is that the only good
781
00:49:55.139 --> 00:50:00.340
dissertation is a done dissertation. Yeah, just grinding through and like. But
782
00:50:00.619 --> 00:50:06.739
it is worth it because I got
to spend five years studying a topic that
783
00:50:06.860 --> 00:50:09.329
I really, really cared about and
I got to go as I mean imagine
784
00:50:09.730 --> 00:50:12.809
being able to go. I mean
you don't need to imagine, but like
785
00:50:13.130 --> 00:50:16.250
if you're a staph, imagine being
given five years to go as deep into
786
00:50:16.250 --> 00:50:20.969
a topic as you could possibly go. Like there is no level of deepness.
787
00:50:21.050 --> 00:50:22.800
That is okay, keep on going
deeper over and over and over now,
788
00:50:23.320 --> 00:50:29.039
and that is super interesting and super
cool and it's a it totally changes
789
00:50:29.079 --> 00:50:32.079
your thinking and it totally changes you
as a person and it's not a good
790
00:50:32.119 --> 00:50:35.789
way to get it, like as
a stepping stone to getting another job.
791
00:50:36.110 --> 00:50:39.750
It really just is it. You
could have much better spent that time doing
792
00:50:39.869 --> 00:50:45.070
other things that are directly related to
getting an interview then going off on some
793
00:50:45.230 --> 00:50:52.260
crazy quest to study some amphibian in
the whatever, in the South Polynesian islands
794
00:50:52.380 --> 00:50:54.739
or something like that. Like there's
definitely better ways of just getting a pay
795
00:50:54.780 --> 00:50:58.420
raise and them, you know,
more advanced suf so is it a viable
796
00:50:58.460 --> 00:51:00.579
option to enter as a data analyst
and try to progress to Ditas on?
797
00:51:00.780 --> 00:51:04.260
So? So I think there's probably
two tracks. One, if you can
798
00:51:04.329 --> 00:51:09.610
find the right place, you can
absolutely go from data analysts into data science.
799
00:51:09.690 --> 00:51:13.809
And I think it more it's about
trying to find the place where there
800
00:51:13.849 --> 00:51:16.130
isn't a firm division between the two. Right. So, like if I
801
00:51:16.690 --> 00:51:20.960
joined a larger company as a data
analyst, as a junior day Lanas,
802
00:51:21.039 --> 00:51:24.119
I would try to find chances where
I could do more software engineering stuff.
803
00:51:24.320 --> 00:51:28.079
I would start to work on more
complicated project. I would try to see
804
00:51:28.159 --> 00:51:29.880
like okay, cool, like yeah, sure, I can make this this
805
00:51:30.440 --> 00:51:32.750
quick analysie, but can I do
it better using Bayesian or somebody that?
806
00:51:32.829 --> 00:51:36.909
Like you'd but you sort of have
to be self motivated to gain more of
807
00:51:36.989 --> 00:51:39.269
those skills. Another option is like
a master's degree, which could be one
808
00:51:39.269 --> 00:51:44.710
or two years and can expose you
to a lot of that kind of stuff
809
00:51:45.429 --> 00:51:49.420
in a relatively quick amount of time
and then you get out and then you
810
00:51:49.500 --> 00:51:51.900
can you know, I think you
can do like a step up, like
811
00:51:51.980 --> 00:51:55.659
we don't care around about education,
like you're education. We don't care about
812
00:51:55.659 --> 00:52:00.500
degrees at devoted there's no requirement for
some kind of degree. But there is
813
00:52:00.579 --> 00:52:02.530
lots of skills that people would learn, say in a data science master's program
814
00:52:02.570 --> 00:52:07.010
or any kind of quantitative master's program
Masters and mathematics or somebody, that they
815
00:52:07.050 --> 00:52:12.010
could really take advantage of and it
can be a big step up for people
816
00:52:12.210 --> 00:52:15.320
to do that and I think it's
probably relatively cheap. I don't really know
817
00:52:15.440 --> 00:52:17.480
exactly, but I would either.
You know. So you can either do
818
00:52:17.559 --> 00:52:21.880
the self learning path, which is
you have to be heal, scrappy,
819
00:52:21.880 --> 00:52:23.480
you have to kind of find opportunities
where they are and move up through that
820
00:52:23.639 --> 00:52:28.519
and probably have to switch jobs a
few times because the people hired you as
821
00:52:28.519 --> 00:52:30.829
a basic data analyst and all of
a sudden you're doing basi and left,
822
00:52:30.869 --> 00:52:32.349
right and center, and then you
want to be paid like you're doing basing
823
00:52:32.750 --> 00:52:36.349
Baysian work all the time, and
then you go find another job that focus
824
00:52:36.469 --> 00:52:38.309
on basing and then you move up
from there and then the other one is
825
00:52:39.030 --> 00:52:43.590
getting that master's degree and then you
know, going back and trying to find
826
00:52:43.590 --> 00:52:45.099
another job after that, saying that
you have you know, you've got this
827
00:52:45.219 --> 00:52:51.260
experience of that, you know more
about the formal training around some of this
828
00:52:51.340 --> 00:52:52.539
kind of stuff, and it doesn't
help you with a lot of business cases,
829
00:52:52.619 --> 00:52:57.300
but it does help you that you
can say you understand the problems around
830
00:52:57.420 --> 00:53:00.210
baysing analysis or the problems around some
kind of machine learning model that then you
831
00:53:00.250 --> 00:53:02.690
can apply in a real way.
And if someone comes to you and knows
832
00:53:02.769 --> 00:53:06.369
these they these types of things,
but doesn't necessarily know how it works in
833
00:53:06.369 --> 00:53:08.690
the health space, I presume a
good strategy there is to say hi,
834
00:53:08.769 --> 00:53:10.929
I know all these techniques, I
don't know a lot about health, but
835
00:53:12.409 --> 00:53:15.639
I would love to learn about this
stuff. To demonstrate like a passion for
836
00:53:15.719 --> 00:53:19.480
the domain expertise in essentially, this
is the first health insurance company I've ever
837
00:53:19.559 --> 00:53:22.800
worked for and we do a lot
of learning around that, around that area
838
00:53:22.880 --> 00:53:25.719
where. So I remember, I
think my second day at devoted, they're
839
00:53:25.760 --> 00:53:28.309
like hey, you know, you
should come to this meeting, and I
840
00:53:28.349 --> 00:53:30.750
came to the meeting and it's just
for data scientist sitting in a room with
841
00:53:30.909 --> 00:53:35.190
a doctor explaining how medical coding works. So like, you know, like
842
00:53:35.309 --> 00:53:37.949
what is the code for someone who
breaks their hip and how does that relate
843
00:53:37.989 --> 00:53:39.309
to the other code and how people
are build on nothing, and it's just
844
00:53:39.349 --> 00:53:43.219
a doctor is sitting around, you
know, telling us all how all these
845
00:53:43.300 --> 00:53:46.659
things work and it was so educational
and it was so new that, you
846
00:53:46.739 --> 00:53:49.780
know, it's a big part of
it, absolutely, and I really like
847
00:53:49.980 --> 00:53:52.099
the way that this conversation is going
in terms of providing a variety of different
848
00:53:52.139 --> 00:53:58.369
paths, from the machine learning to
the first time data scientists demonstrating what they've
849
00:53:58.369 --> 00:54:00.650
done through project, so through quantititive
research, through to the self learning approach.
850
00:54:00.809 --> 00:54:04.289
And for anyone out there who wants
to take the self learning approach,
851
00:54:04.289 --> 00:54:10.480
I've actually I've heard that data campcom
is an incredible place. Yeah, it
852
00:54:10.599 --> 00:54:14.159
was actually Chris who told me that
before we started recording. He was like,
853
00:54:14.199 --> 00:54:15.159
Guy, that's it. I was
just like data camp, you got
854
00:54:15.239 --> 00:54:17.079
to use day again. Oh No, no one's paying for me for this.
855
00:54:17.599 --> 00:54:21.639
I'm a I'm ambivalent. Yeah,
this is a sponsored yeah, and
856
00:54:21.719 --> 00:54:27.269
this is not sponsored by Facebook,
EI, though, or basing analysis exactly.
857
00:54:27.309 --> 00:54:30.269
Yeah, Gelman, Gilman isn't paying
us. Hey, that's a good
858
00:54:30.510 --> 00:54:35.469
that's a good idea actually. So
I'm wondering if there are any like any
859
00:54:35.550 --> 00:54:38.219
advice, things that you love people
doing when they come into interviews. All
860
00:54:38.260 --> 00:54:42.059
that, you think of the worst
things that people could do. I'm just
861
00:54:42.179 --> 00:54:45.019
any general tidbits of advice the first
time in interview ways from your side of
862
00:54:45.019 --> 00:54:49.940
the tyble. That's a good question, I think for us. I'm trying
863
00:54:49.980 --> 00:54:52.300
to say us, because I don't
want to say it's just me, because
864
00:54:52.300 --> 00:54:53.449
that's biased, like that would be
biased if I was just like well,
865
00:54:53.489 --> 00:54:55.730
maybe, actually, well, those
we hire as a team. So it's
866
00:54:55.769 --> 00:54:58.929
not just me, obviously siring,
but on the persons. Any way,
867
00:54:58.929 --> 00:55:00.849
I will say what I what I
tend to have like think rather than like
868
00:55:00.969 --> 00:55:04.050
us as a team, because I
don't know what they are biased towards.
869
00:55:04.489 --> 00:55:09.000
For me, I'm a big fan
of people who are excited about learning new
870
00:55:09.039 --> 00:55:14.000
things and excited about data, because
I genuinely enjoy what I do. I
871
00:55:14.519 --> 00:55:19.039
enjoy learning a new technique. I
enjoy sitting around with, you know,
872
00:55:19.159 --> 00:55:22.710
a new book around some kind of
analysis or an old book around some analysis
873
00:55:22.750 --> 00:55:24.909
that I don't know very well,
and I really, really enjoy that.
874
00:55:25.869 --> 00:55:30.989
And I think when you're coming in
as a junior person and you can sort
875
00:55:30.989 --> 00:55:32.789
of say, Hey, I don't
know how a lot of this stuff works,
876
00:55:34.070 --> 00:55:37.019
but I am really, really interested
in what it is and I know
877
00:55:37.139 --> 00:55:38.780
where I want to be in five
years and that involves a huge amount of
878
00:55:38.820 --> 00:55:43.139
learning, that is exactly what we
want. Like we are not hiring you
879
00:55:43.219 --> 00:55:46.300
because we're junior. We're hiring you
because we think that you could be senior
880
00:55:46.659 --> 00:55:50.969
with some training, with some mentorship, with some projects and that kind of
881
00:55:50.969 --> 00:55:53.130
stuff. Like we do not have
want you to be junior forever. We
882
00:55:53.250 --> 00:55:57.969
want you to be senior and like
we acknowledge that the more we train you,
883
00:55:58.050 --> 00:56:00.210
the more you might leave, and
that's totally okay with us. Like,
884
00:56:00.570 --> 00:56:02.730
come joined of a did be a
junior person for a while, enjoy
885
00:56:02.809 --> 00:56:06.880
yourself, like learn a huge amount
of stuff, you really excited what you
886
00:56:06.920 --> 00:56:08.079
do and then and then go find
a better job somewhere else. is a
887
00:56:08.119 --> 00:56:12.519
completely reasonable like thing that we don't
have expectations that you that you don't do
888
00:56:12.639 --> 00:56:15.559
that, but it is I want
someone to be excited about it. That
889
00:56:15.760 --> 00:56:20.829
excitement can come out in various ways. So people who have lots of side
890
00:56:20.869 --> 00:56:22.750
projects obviously like that's a signal that
people used to be like I love it
891
00:56:22.829 --> 00:56:25.269
in my spare time, but other
people don't have a lot of spare time.
892
00:56:25.750 --> 00:56:29.190
So you know that's not like.
That can be one signal, but
893
00:56:29.230 --> 00:56:30.989
it's not the only thing that we
care about. If you just come in
894
00:56:30.110 --> 00:56:34.860
and are really, really excited about
it and I can tell that you have
895
00:56:34.980 --> 00:56:37.500
spent a lot of time thinking about
it and do you have a lot of
896
00:56:37.579 --> 00:56:39.940
knowledge around it, and like you
like an a lot of knowledge around it
897
00:56:40.099 --> 00:56:43.699
for what I would expect someone at
your level to have. Like that's a
898
00:56:43.699 --> 00:56:45.300
nice sign that you just you really
care about this one thing. You might
899
00:56:45.340 --> 00:56:49.050
not have knowledge around everything, but
you might have this one thing that's like
900
00:56:49.090 --> 00:56:51.769
I just I think random forests is
super cool. I spent a lot of
901
00:56:51.809 --> 00:56:53.530
time reading about them. Like I
don't know deep learning, I don't know
902
00:56:53.610 --> 00:56:58.769
selfare engineering, I don't know anything
except for like I'm just like super interested
903
00:56:58.769 --> 00:57:00.250
in this one thing. Like that's
kind of cool. I think that that
904
00:57:00.440 --> 00:57:06.000
excitement bleeds off on to me,
possibly on to other people who interviewing,
905
00:57:06.039 --> 00:57:09.400
but definitely definitely for me, because
if you can't get the job based on
906
00:57:09.519 --> 00:57:13.519
your experience, which, when you
are more experienced, you could just get
907
00:57:13.519 --> 00:57:17.190
the job because you're experienced, you
kind of need another strategy and one of
908
00:57:17.230 --> 00:57:22.670
the strategies is just saying that you
have the right attitude and you're excited to
909
00:57:22.750 --> 00:57:24.349
do it and you know you can
learn a lot and you could be a
910
00:57:24.429 --> 00:57:27.590
good member of the team that people
want to work with, and that's a
911
00:57:27.630 --> 00:57:30.340
great way of doing it. If
if you don't have the ten years under
912
00:57:30.340 --> 00:57:31.940
your belt or something, so sure
you need something that differentiates you right,
913
00:57:32.139 --> 00:57:36.980
differentiating fact on. I understand that
one of the hard parts for junior data
914
00:57:37.019 --> 00:57:43.500
scientists is that their resumes often look
a lot alike because they go what person
915
00:57:43.579 --> 00:57:45.849
went to a boot camp, someone
else went to a different boot camp.
916
00:57:45.969 --> 00:57:50.010
Someone else, you know, like
was a math major, someone else like
917
00:57:50.250 --> 00:57:53.050
did this mathematic project, someone else
wrote one research paper. Like you know,
918
00:57:53.130 --> 00:57:57.889
there's a lot of like signals that
are pretty much the same, and
919
00:57:58.050 --> 00:58:00.320
so the way that people can distinguish
themselves of I think, at least in
920
00:58:00.400 --> 00:58:05.920
my mind, is to having some
enjoyment for what you do, because we
921
00:58:06.039 --> 00:58:08.000
want you to enjoy it and therefore
learn more at it. You know,
922
00:58:08.079 --> 00:58:10.400
learn more about doing it. You
don't need to grind them, you know,
923
00:58:10.440 --> 00:58:13.909
you don't need to burn the midnight
oil to do it. You could
924
00:58:13.909 --> 00:58:15.230
totally work down to five with the
rest of us, but we want you
925
00:58:15.269 --> 00:58:21.789
to be interested in it. And
if we every higher, now that I'm
926
00:58:21.789 --> 00:58:24.230
sitting on this side of table,
every higher is a bet. And for
927
00:58:24.389 --> 00:58:28.699
the junior person, the best bet
that we could make is that we are
928
00:58:28.739 --> 00:58:32.940
hiring you and you don't know everything
we wish you knew, but that in
929
00:58:34.099 --> 00:58:37.179
two years you could know a like, a big portion of what we wish
930
00:58:37.219 --> 00:58:39.739
you knew. As a senior person
right, like. I mean, you
931
00:58:39.739 --> 00:58:42.730
wouldn't be senior tears, but you
get my point right. Like you,
932
00:58:42.969 --> 00:58:46.250
you can be so much more capable
and so much more of a resource for
933
00:58:46.369 --> 00:58:49.929
the team. So we come in
and hire you when your junior and then
934
00:58:49.929 --> 00:58:52.570
all of a sudden you're mid level
and your kick and butt, and then
935
00:58:52.610 --> 00:58:54.210
we try to retain you because we
want you to stay there because you're so
936
00:58:54.250 --> 00:58:58.840
awesome that that part is like a
big thing. Yeah, and I suppose
937
00:58:58.880 --> 00:59:02.000
one thing I'd like to guide your
opinion on is the difference between what you
938
00:59:02.079 --> 00:59:06.760
learn self learning or even in research
in terms of, you know, using
939
00:59:06.840 --> 00:59:09.070
by Jin inference, machine learning,
whatever it might be, the difference between
940
00:59:09.389 --> 00:59:13.550
the types of tools and techniques you
use there, which might be importing cs
941
00:59:13.590 --> 00:59:17.429
phase and then doing machine learning in
production in a company, and you might
942
00:59:17.510 --> 00:59:22.670
have all these things that you haven't
actually been exposed to before by doing cago
943
00:59:22.710 --> 00:59:25.219
competitions, for example. I'll give
another example, like one of the things
944
00:59:25.500 --> 00:59:31.099
they is very hard to learn is
data engineering. So data scientists are different
945
00:59:31.099 --> 00:59:35.340
than data engineers, of course,
but there's a reason that there isn't a
946
00:59:35.340 --> 00:59:38.250
bunch of data engineering book camps because
to do day engineering you basically need a
947
00:59:38.369 --> 00:59:42.730
production system to learn that skill,
like you need to have millions of data
948
00:59:42.769 --> 00:59:45.090
points flowing around and all this kind
of stuff where you did actually do stuff
949
00:59:45.130 --> 00:59:47.210
with that. And so if you
don't learn it on the job, like
950
00:59:47.289 --> 00:59:52.329
there's relatively few areas that you could
learn that on your own, and I
951
00:59:52.409 --> 00:59:55.719
think there's some parallels to data science. For there's just some some things that
952
00:59:55.840 --> 01:00:00.039
are hard to do, as you
know, a selfdirector project. I would
953
01:00:00.079 --> 01:00:07.469
recommend that people do side projects or
do learning projects that take advantage of those.
954
01:00:07.510 --> 01:00:12.909
So like instead of, say,
instead of importing a CSD with the
955
01:00:12.949 --> 01:00:15.590
data like loaded into a database and
then pull it down and then pull it
956
01:00:15.630 --> 01:00:19.309
down once an hour or something like
that, just to like get more experience
957
01:00:19.429 --> 01:00:22.139
with that. And there's no I
mean these tools are either free or cheap.
958
01:00:22.820 --> 01:00:25.340
So if you have like a small
amount of data, so, for
959
01:00:25.420 --> 01:00:30.380
example, like there's Amazon Athena,
which is like a way of kind of
960
01:00:30.500 --> 01:00:32.460
search, like it's basically, I'd
like a way to do quarries on big
961
01:00:32.460 --> 01:00:37.650
data. You can totally upload just
a little bit of data to that and
962
01:00:37.730 --> 01:00:39.489
then you pay by quarry. So
you could pay like thirty cents to do
963
01:00:39.530 --> 01:00:43.690
a quarry and so you do like
thirty quarries and you kind of understand what's
964
01:00:43.730 --> 01:00:45.530
happening and then you kind of use
it for a project and then you drop
965
01:00:45.610 --> 01:00:47.809
that project and you never pay for
it again, but you have that experience
966
01:00:47.809 --> 01:00:50.920
into your belt and then so when
someone comes to you and says, Hey,
967
01:00:51.239 --> 01:00:53.159
I saw you, you know have
this aws experience communice. Yeah,
968
01:00:53.159 --> 01:00:55.880
because I, you know, I
did this kind of project we use Athena.
969
01:00:57.480 --> 01:01:00.440
I, you know, had to
work out that I am security rolls
970
01:01:00.519 --> 01:01:02.760
of me and then how I put
it onto a server and how I etcetera,
971
01:01:02.800 --> 01:01:05.630
etc. Etc. And all that
kind of stuff you could do,
972
01:01:06.309 --> 01:01:08.670
because with things like a ws you
just pay for usage. Just you,
973
01:01:09.030 --> 01:01:12.469
you know, just have a small
amount of data when you do it,
974
01:01:12.550 --> 01:01:15.230
but you do have that experience.
That is really useful. That that's that's
975
01:01:15.230 --> 01:01:16.670
a good signal that, like,
Hey, you, you are really interested
976
01:01:16.670 --> 01:01:22.300
in the kind of stuff. If
I release you onto our system that costs
977
01:01:22.579 --> 01:01:25.139
tens of thousands of dollars a month, like you can sit down and do
978
01:01:25.420 --> 01:01:29.420
some really interesting stuff and you would
be interested that. I think that's great.
979
01:01:29.460 --> 01:01:32.289
I I would recommend that if you
really wanted to stand out, you
980
01:01:32.289 --> 01:01:37.730
should try to identify the skills that
are harder for people to get right,
981
01:01:37.769 --> 01:01:44.730
because it's very easy to download a
CSV and then do some group by statements
982
01:01:44.849 --> 01:01:47.519
and pandas or something like that.
And that's why does fiftyzero tutorials that do
983
01:01:47.599 --> 01:01:51.719
that, and that's why my home
page has a bunch of tutorials on doing
984
01:01:51.800 --> 01:01:53.360
that. And that is useless,
rough. But to stand out if you
985
01:01:53.440 --> 01:01:58.679
had some kind of varience in something
that was harder for to set up and
986
01:01:58.880 --> 01:02:00.989
something that was harder to do.
I think that's a nice way of sort
987
01:02:01.030 --> 01:02:02.989
of saying like no, no,
no, I'm super serious about this,
988
01:02:04.110 --> 01:02:07.750
like I try to do this thing
that was more difficult and it's more in
989
01:02:07.869 --> 01:02:09.789
line with like what a real businesses
would do, just add a much,
990
01:02:09.829 --> 01:02:13.429
much smaller scale, but I'm still
using the same technologies. I mean,
991
01:02:13.429 --> 01:02:15.670
that's awesome, that's great. So, Chris, I want to ask you
992
01:02:15.139 --> 01:02:19.500
what one of your favorite Dietosaians.
He techniques of methodolog Jesus. But before
993
01:02:19.539 --> 01:02:23.260
that you've said random forest enough.
My first question is random forests or support
994
01:02:23.260 --> 01:02:29.139
vector machines? Always random. For
as who who uses support vector machines,
995
01:02:29.179 --> 01:02:31.409
I don't think, like, I
don't think there's ever. I cannot think
996
01:02:31.409 --> 01:02:37.969
of a single case of someone who
has used a support vector machine in production.
997
01:02:38.210 --> 01:02:43.809
So we're victor. Machines are mathematically
cool. They're just super cool.
998
01:02:43.889 --> 01:02:46.280
Like when you explain how they work, it's just super interesting. It's a
999
01:02:46.320 --> 01:02:51.599
very clever technique, but random forests
are like, if you need to get
1000
01:02:51.679 --> 01:02:57.159
something done, a random forest like
just seemingly works out of the box in
1001
01:02:57.320 --> 01:03:01.190
so many ways and so many cases
very, very, very easily that it
1002
01:03:01.349 --> 01:03:05.349
is definitely the one of the best
starting points. I think a random forest
1003
01:03:05.469 --> 01:03:08.590
is definitely one of the best starting
points for a lot of a lot of
1004
01:03:08.710 --> 01:03:12.469
modeling, and then I would kind
of go from there and decide if he
1005
01:03:12.550 --> 01:03:15.579
wanted to, you know, make
their random forest more complex or add more
1006
01:03:15.619 --> 01:03:19.380
feature engineering or change your model up
or something like that. But I would
1007
01:03:19.420 --> 01:03:22.099
definitely I'm a big random force fan
for sure, and I actually I was
1008
01:03:22.219 --> 01:03:24.260
roofing off, I I was a
couple of weeks ago someone tweeted out,
1009
01:03:24.699 --> 01:03:29.090
does anybody use support vector machines for
anything? And you were and you were
1010
01:03:29.369 --> 01:03:31.449
you just replied, know, yeah, I think like, I don't know
1011
01:03:31.530 --> 01:03:35.849
who uses that. I'd I cannot
think of a single time the someone which
1012
01:03:36.130 --> 01:03:43.320
choose a support vector machine over over
anything. I don't know, I don't
1013
01:03:43.320 --> 01:03:45.199
know. I don't see that that
one moment where it's like the killer,
1014
01:03:45.239 --> 01:03:50.000
killer model to use. Someone's going
to listen to this and send me some
1015
01:03:50.320 --> 01:03:52.400
kind of article that it, you
know, like they can't one. They
1016
01:03:52.480 --> 01:03:54.949
cured some kind of cancer using some
kind of supportmentcas you to say, but
1017
01:03:55.110 --> 01:03:59.030
right I can't to them. I
cannot think of an example off the top
1018
01:03:59.070 --> 01:04:02.309
of my head. I always considered
a supporvector machine more as sort of a
1019
01:04:02.389 --> 01:04:05.750
teaching function for like, Hey,
here's one of the techniques that people tried.
1020
01:04:06.590 --> 01:04:11.300
It's mathematically genius. I think it
explains a lot of the concepts of,
1021
01:04:11.579 --> 01:04:14.179
you know, like a decision line
between groups and that kind of stuff
1022
01:04:14.219 --> 01:04:16.139
really well for sure. And what
happens when people are on the wrong side
1023
01:04:16.139 --> 01:04:18.059
of the DEC like when a data
point is on the wrong side of the
1024
01:04:18.139 --> 01:04:20.579
decision mine, all that kind of
stuff. I think that's totally, totally
1025
01:04:20.579 --> 01:04:27.010
useful. Also, I've I've never
seen someone use it in production. So
1026
01:04:27.449 --> 01:04:29.969
do you have a funal, cool
co action for our listeners out there?
1027
01:04:30.090 --> 01:04:35.090
Sure, I mean one. You
should completely apply for the jobs that devoted.
1028
01:04:35.170 --> 01:04:38.530
This is not actually an attempt for
me to get this is not a
1029
01:04:38.679 --> 01:04:42.199
tempt for me to get people to
apply for jobs voted. But Hey,
1030
01:04:42.280 --> 01:04:45.559
we are a awesome company. Come
work with DJ Patil and myself and a
1031
01:04:45.559 --> 01:04:48.840
number of great people. That is
it'll be fun, it'll be great and
1032
01:04:49.000 --> 01:04:53.110
I hope you apply. Incredible.
It will total link in the show notes
1033
01:04:53.110 --> 01:04:55.389
as well too. Awesome, see
look at this. Finally, Guy,
1034
01:04:55.469 --> 01:04:57.349
can I expense? I mean,
I'm not really paying any money for this,
1035
01:04:57.429 --> 01:04:59.309
but I feel like I should expend, like I'm I don't know,
1036
01:04:59.349 --> 01:05:01.389
like a lunch because of this or
something. Absolutely, Let's do that.
1037
01:05:02.030 --> 01:05:04.309
This would be my other, more
general call to action, I think,
1038
01:05:04.309 --> 01:05:09.340
for someone is interested in data science, think very carefully around the skills that
1039
01:05:09.420 --> 01:05:13.860
you're trying to develop and the skills
that other people who are applying or trying
1040
01:05:13.900 --> 01:05:17.780
to develop and sit down and and
think about ways that you can apply those
1041
01:05:17.820 --> 01:05:23.409
skills in a way that both demonstrates
that you have those skills because and or
1042
01:05:23.489 --> 01:05:27.690
that you're learning those skills or something
bad, but has some kind of real
1043
01:05:27.809 --> 01:05:31.449
impact in society around us. Right
there is there is so much free data
1044
01:05:31.530 --> 01:05:38.119
out there. There's so many really, really, really interesting projects that you
1045
01:05:38.199 --> 01:05:42.840
can do by yourself, on your
own, with basically no money, that
1046
01:05:42.960 --> 01:05:45.599
you can sit down and say hey, like I have. You know,
1047
01:05:45.679 --> 01:05:47.559
I've used random for us in this
really strong way, but I've also used
1048
01:05:47.599 --> 01:05:53.230
it in a way that I whatever
detected some in crazy cool thing, detected
1049
01:05:53.510 --> 01:05:57.630
a parking tickets in New York or
detected like some kind of I don't know,
1050
01:05:57.829 --> 01:06:00.349
like some kind of thing around like
police, police shootings, or some
1051
01:06:00.389 --> 01:06:02.590
kind of thing around crime or some
kind of anything like that. Like,
1052
01:06:02.750 --> 01:06:05.739
I think there's so many really interesting
things that you can do with data science.
1053
01:06:05.820 --> 01:06:11.059
This is a applied field, and
so learn things in data science,
1054
01:06:11.179 --> 01:06:14.099
that's a big part of it,
but then also apply it in your home
1055
01:06:14.179 --> 01:06:17.769
life and in the projects that interest
you. That is the ultimate thing that
1056
01:06:17.809 --> 01:06:20.889
I want to talk about in every
single interview. It's just the cool things
1057
01:06:20.969 --> 01:06:25.010
that they've done and use data for, and I want to see that spark
1058
01:06:25.090 --> 01:06:28.050
of excitement in the things that they've
done, because then I want to work
1059
01:06:28.090 --> 01:06:30.530
with them, because if they're say
excited about it, I'm excited about it
1060
01:06:30.570 --> 01:06:31.730
and it makes you want to go
to work every day. So and this
1061
01:06:31.849 --> 01:06:34.400
has been a great conversation, Chris. So thank you so much for coming
1062
01:06:34.440 --> 01:06:38.519
on the show. Thank you,
thank you for having me such a pleasure.
1063
01:06:43.800 --> 01:06:46.429
Thanks for joining the conversation with Chris
Alban about getting your first data science
1064
01:06:46.469 --> 01:06:50.269
job. Chris made it clear that
one thing you need to do is provide
1065
01:06:50.309 --> 01:06:55.869
a differentiating factor in your resume and
interviews. When most resumes state the same
1066
01:06:55.909 --> 01:06:59.949
skill set, such as our or
python sequel machine learning, how will you
1067
01:07:00.110 --> 01:07:02.219
stand apart from the pack? One
Way, although not the only way,
1068
01:07:02.539 --> 01:07:05.820
is to have a small portfolio of
projects you've worked on, whether in a
1069
01:07:05.860 --> 01:07:10.900
github repository or in a blog.
Another way to stand apart is to have
1070
01:07:10.940 --> 01:07:15.340
a bit of experience with projects and
techniques that aren't so commonplace. For example,
1071
01:07:15.579 --> 01:07:18.250
instead of importing your data as a
CSV loaded into a database and then
1072
01:07:18.289 --> 01:07:21.090
pull it down and then pull it
down at regular intervals, or get up
1073
01:07:21.090 --> 01:07:26.369
to speed with using Amazon web services, for example. Yet another way is
1074
01:07:26.530 --> 01:07:30.440
just to be super duper excited about
the organizations you're applying to and the techniques
1075
01:07:30.599 --> 01:07:35.519
you like to use to answer questions. Chris also gave the great advice of
1076
01:07:35.639 --> 01:07:41.639
looking for positions in mid to larger
organizations and not in smaller, scrappier startups.
1077
01:07:41.880 --> 01:07:45.909
In midto larger orgs you'll be able
to learn a lot develop your career
1078
01:07:45.309 --> 01:07:49.030
and won't need to be too concerned
with all the data infrast structure challenges at
1079
01:07:49.070 --> 01:07:54.510
that point. More generally, look
for orgs that will foster your career aspirations
1080
01:07:54.550 --> 01:07:58.739
and will invest in you. I'd
suggest making that a first order principle of
1081
01:07:58.820 --> 01:08:01.820
your job search. Next week I
have the great pleasure of speaking with no
1082
01:08:01.940 --> 01:08:08.219
Ami Dershey, a senior inventive scientist
at a TNT labs within the data science
1083
01:08:08.380 --> 01:08:13.250
and Ai Research Organization, doing lots
of science with lots of data. Will
1084
01:08:13.250 --> 01:08:15.850
be talking about her work at atnt
labs, research, the mission of which
1085
01:08:15.970 --> 01:08:21.529
is to look beyond today's technology solutions
to invent disruptive technologies that meet future needs.
1086
01:08:21.970 --> 01:08:28.079
ATNT labs works on a multitude of
projects, from product development at atnt
1087
01:08:28.560 --> 01:08:32.640
to how to combat bias and Fannessy
issues in targeted advertising and creating drones for
1088
01:08:32.800 --> 01:08:40.239
Cell Tower Inspection. Research that leverages
AIML and video analytics. Will be talking
1089
01:08:40.279 --> 01:08:45.069
about some of the work no AMI
does, from characterizing human mobility from cellular
1090
01:08:45.229 --> 01:08:50.069
network data to characterizing their mobile network
to analyze how it's topology compares to other
1091
01:08:50.229 --> 01:08:57.100
real social networks reported and understanding TV
viewership and how engage people are in different
1092
01:08:57.180 --> 01:09:00.899
shows. All this and, as
always, more next week. I'm your
1093
01:09:01.060 --> 01:09:04.699
host, Hugo bound Anderson. You
can follow data camp on twitter at data
1094
01:09:04.739 --> 01:09:09.539
camp and me at Hugo bound.
You can find all our episodes and show
1095
01:09:09.619 --> 01:09:13.409
notes at Data Campcom community podcast
1
00:00:00.240 --> 00:00:04.519
This week I'll be speaking with Chris
Alban about getting your first data science job.
2
00:00:04.559 --> 00:00:08.669
Chris is a data scientist at devoted
health, where he uses data science
3
00:00:08.710 --> 00:00:12.990
and machine learning to help fix America's
healthcare system. Chris is also doing a
4
00:00:13.070 --> 00:00:17.429
lot of hiring a devoted and that's
why he's so excited today to talk about
5
00:00:17.510 --> 00:00:21.460
how to get your first data science
job. You may know Chris as Co
6
00:00:21.620 --> 00:00:26.140
host to the podcast partially derivative from
his educational resources, such as is blog
7
00:00:26.300 --> 00:00:30.059
and machine learning flash cards, or
is one of the funniest data scientists on
8
00:00:30.100 --> 00:00:35.140
twitter. Welcome to data framed,
the weekly data camp podcast exploring what data
9
00:00:35.179 --> 00:00:38.810
science looks like on the ground for
working data scientists and what problems that can
10
00:00:38.890 --> 00:00:42.689
solve. I'm your host, Hugo
bound Anderson. You can follow data camp
11
00:00:42.770 --> 00:00:46.090
on twitter, that data camp and
me at Hugo bound. You can find
12
00:00:46.130 --> 00:00:54.240
all our episodes and show notes at
Data Campcom community podcast. This is data
13
00:00:54.359 --> 00:01:17.909
framed. Hi there, Chris,
and welcome to data framed. Hey,
14
00:01:17.950 --> 00:01:19.459
how's it going? It's great,
man. How are you? I'm good.
15
00:01:19.540 --> 00:01:22.900
This is like one of the first
podcasts I've done in a while.
16
00:01:23.060 --> 00:01:26.459
This is like I stopped my podcast
and now I've gone on a few ones,
17
00:01:26.540 --> 00:01:30.219
but then I had a kid and
I did a move and that kind
18
00:01:30.219 --> 00:01:33.129
of stuff, and now I'm back
back on the podcasting circuit. This is
19
00:01:33.209 --> 00:01:36.170
it fantastic. How long has it
been? Who? It's been like?
20
00:01:36.609 --> 00:01:38.370
You know. So my kid is
two months old, so I think I
21
00:01:38.489 --> 00:01:42.209
did my last podcast like two months
before that. So it's been four months
22
00:01:42.450 --> 00:01:46.569
without run them or congratulations on the
new member of your family. Thank you,
23
00:01:46.650 --> 00:01:49.239
thank you. She is doing great
all as well. Awesome, and
24
00:01:49.319 --> 00:01:52.760
congrats on the new job and the
move. There's been a lot of change.
25
00:01:52.799 --> 00:01:55.959
Right, it's been a lot of
change. There's definitely been a lot
26
00:01:55.959 --> 00:01:59.599
of change. I feel like it's
one of those things where I have a
27
00:01:59.640 --> 00:02:02.670
very hard time saying no to DJ. So when he does think that you
28
00:02:02.750 --> 00:02:06.870
should come do something, get to
think really heard about whether or not you
29
00:02:07.390 --> 00:02:08.750
you want to say no to him, which I've said no to him in
30
00:02:08.789 --> 00:02:12.189
the past, that I could not
say no to him this time. Thus
31
00:02:12.349 --> 00:02:15.860
I'm with DJ on a crazy adventure. So this is DJ Patil. This
32
00:02:15.020 --> 00:02:21.219
is DJ DJ Patil, former chief
data scientists of the US, former head
33
00:02:21.219 --> 00:02:23.580
of data a linkedin. I don't
know his whole resume, but a wellknown,
34
00:02:23.780 --> 00:02:27.900
wellknown person in the in the data
science world, very well known,
35
00:02:27.979 --> 00:02:30.969
very well respected, doing a lot
of interesting work at devoted and otherwise as
36
00:02:31.009 --> 00:02:35.849
well. I mean he's recent series
of articles with Michael Katie's and Hillary mayson
37
00:02:36.289 --> 00:02:39.289
around kind of forming a conversation around
that designs. I thinks he's really interesting
38
00:02:39.289 --> 00:02:44.360
as well. Yeah, I think
he's doing a lot of really important thinking
39
00:02:44.360 --> 00:02:46.680
in a space that we're still figuring
out. I mean this was something that
40
00:02:46.879 --> 00:02:51.360
I think a lot of us have
been talking about sort of off and on
41
00:02:51.439 --> 00:02:53.879
for a long time of like hey, we don't actually you know, we
42
00:02:53.120 --> 00:02:55.360
have lots of goals for that what
this field can do, but what does
43
00:02:55.360 --> 00:02:59.469
it actually mean in practice? And
the more we get to do that and
44
00:02:59.590 --> 00:03:01.150
the more that people sort of carved
out space in their work day to say,
45
00:03:01.189 --> 00:03:04.750
Hey, I'm going to think about
this topic, I'm going to produce
46
00:03:04.830 --> 00:03:07.990
something that other people can read and
agree with and disagree with and uses a
47
00:03:08.069 --> 00:03:12.900
point of discussion or build something off
of, is great and I hope he
48
00:03:13.020 --> 00:03:15.180
does it more and I hope Hillary
does it more and I think more people
49
00:03:15.659 --> 00:03:20.819
should spend a little bit more time
thinking about that. And it's part of
50
00:03:20.860 --> 00:03:23.979
it. That was a nice to
go work for him, because it is
51
00:03:23.060 --> 00:03:25.699
sort of Nice to go work for
someone who's thinking about that kind of stuff
52
00:03:25.819 --> 00:03:30.370
and really is big on ethics and
it's big on trying to use technology for
53
00:03:30.490 --> 00:03:35.449
for social good. Because my I
mean my for people on this packets,
54
00:03:35.490 --> 00:03:39.250
like my background is not in business, like I have started a startup,
55
00:03:39.689 --> 00:03:46.080
but my main background is nonprofits,
humanitarian nonprofits. I work on data on
56
00:03:46.120 --> 00:03:50.759
humanitarying round not profits. Did that
for most of my career, and so
57
00:03:51.319 --> 00:03:54.520
to work for a team that's led
by someone who spends a lot of time
58
00:03:54.639 --> 00:04:00.870
thinking about how to build companies with
a soul was very refreshablesome. So maybe
59
00:04:00.909 --> 00:04:04.270
you can sort up by telling us
just about devoted in general and the work
60
00:04:04.310 --> 00:04:10.939
you're doing. Sure. So devoted
a health insurance company that was started by
61
00:04:11.939 --> 00:04:15.740
Todd and Ed Park, who Todd
Park was the former CTO of the United
62
00:04:15.740 --> 00:04:21.060
States and Ed Park was the CEO
of another health insurance company or another healthcare
63
00:04:21.100 --> 00:04:26.850
company before this. And what devoted
tries to do is tries to friendly,
64
00:04:26.930 --> 00:04:30.730
like create a health insurance company that
you would want your own family members to
65
00:04:30.810 --> 00:04:32.170
be at. It is it is
a company. It is a startup.
66
00:04:32.649 --> 00:04:35.810
EXAC operates exactly like the startup,
is funded exactly like the startup. I
67
00:04:35.850 --> 00:04:39.879
think there's there's areas that would set
it apart from the start up, but
68
00:04:39.879 --> 00:04:42.279
I think overall you would look at
it me like, yes, this is
69
00:04:42.360 --> 00:04:46.920
a startup. However, devoted is
on emission. We are trying to make
70
00:04:46.040 --> 00:04:50.879
healthcare that works, that works for
people, particularly right now senior citizens.
71
00:04:50.920 --> 00:04:55.829
So we work on Medicare. If
you don't know, Medicare is the health
72
00:04:55.949 --> 00:04:59.990
insurance there's government health insurance program for
senior citizens. That is what devoted is
73
00:05:00.110 --> 00:05:02.389
focused on. That sort of what
we consider ourselves as a Medicare company.
74
00:05:03.509 --> 00:05:06.699
But on the daily basis, if
you work inside devoted, you can see
75
00:05:06.699 --> 00:05:12.500
that we are trying to build something
that would be a company with the soul,
76
00:05:12.540 --> 00:05:15.939
a company that's trying to do something
real and trying to do something that
77
00:05:15.060 --> 00:05:18.540
matters in people's lives and do right
by people and is hyper compliant with the
78
00:05:18.579 --> 00:05:24.689
law and do more than just simply
profit. Awesome. That was with draw
79
00:05:24.810 --> 00:05:27.769
drew me to it. I think
that's what draws a lot of people to
80
00:05:27.889 --> 00:05:30.810
devoted. That's great and, as
will discuss, you're working a lot on
81
00:05:30.970 --> 00:05:32.329
hiring at the moment, and what
we're going to talk about today, among
82
00:05:32.370 --> 00:05:38.160
other things, is people getting their
first Dietosians jobs and advice for such people,
83
00:05:38.519 --> 00:05:40.879
how it works, what it looks
like from your your side of the
84
00:05:40.920 --> 00:05:46.079
conversation as well, and I think
particularly this point, where junior dietosaians parts
85
00:05:46.079 --> 00:05:49.000
aren't necessarily fleshed out with sign to
see a bunch of specialization in the industry.
86
00:05:49.189 --> 00:05:53.629
They such things will be incredibly important
going forward. But before we dive
87
00:05:53.670 --> 00:05:56.509
in to that, I just want
to get a bit more background about you
88
00:05:56.750 --> 00:05:59.269
and if you could tell me,
like you said you'll, your background isn't
89
00:05:59.269 --> 00:06:01.990
necessarily in ditosase. I'm wondering how
you got involved in Dicosians in the first
90
00:06:02.029 --> 00:06:08.220
place. My backgrounds in quantitative political
science, so political science studying of politics.
91
00:06:08.259 --> 00:06:11.939
I studied civil wars, but it's
political science from completely from the perspective
92
00:06:11.980 --> 00:06:16.860
of statistics, of quantitative research,
of experimentation, rather than qualitative work,
93
00:06:17.180 --> 00:06:21.730
interviewing people, looking at historic documents, that kind of thing. And when
94
00:06:21.810 --> 00:06:27.970
I was getting my PhD, I
kept on having drinks with these people in
95
00:06:28.009 --> 00:06:30.129
San Francisco, where I was sort
of living at the time. And they
96
00:06:30.129 --> 00:06:32.800
were working at places like Linkedin.
So there was DJ and there's number of
97
00:06:32.800 --> 00:06:38.240
other people and they were doing such
cool applied stuff, so many, you
98
00:06:38.319 --> 00:06:43.120
know, amazing projects, amazing uses
of data in a very, very applied
99
00:06:43.160 --> 00:06:46.680
way. And there was about then
that I kind of decided that if I
100
00:06:46.959 --> 00:06:51.110
really wanted to have a real impact
and really wanted to do something that would
101
00:06:51.509 --> 00:06:56.589
matter, that I needed to not
be an academia. I needed to go
102
00:06:56.670 --> 00:07:00.110
out and actually apply the skills in
a way that was beyond research. And
103
00:07:00.189 --> 00:07:03.259
don't get me wrong, I love
research. I every single person who applies
104
00:07:03.339 --> 00:07:05.579
for devoted who has any kind of
PhD. I just want to talk to
105
00:07:05.620 --> 00:07:10.459
about their phcl day. I think
you know that that's a a bias of
106
00:07:10.540 --> 00:07:13.459
mine. But there's so many ways
that you could apply it. And then
107
00:07:13.500 --> 00:07:16.569
at the time there was some chances
for me to go work for some Joint
108
00:07:16.649 --> 00:07:20.009
Canyon US nonprofits, Canyon nonprofits,
that kind of stuff, and I spent
109
00:07:20.089 --> 00:07:25.089
a number of years working, being
sort of the first day to hire over
110
00:07:25.129 --> 00:07:28.610
there at say, Ushihiti, which
is a Canyon nonprofit that works on election
111
00:07:28.649 --> 00:07:32.360
monitoring and disaster relief. Went to
go work for brick, which is a
112
00:07:32.439 --> 00:07:39.160
canyon, startup that works on providing
free Wi fi to lower inco people in
113
00:07:39.160 --> 00:07:43.759
Kenya. CARLINE SE MEST did a
lot of workaround election monitoring. But,
114
00:07:44.079 --> 00:07:46.029
you know, taking data, real
data about, say, say, an
115
00:07:46.069 --> 00:07:50.949
election in a country with an authoritarian
leadership, and actually seem like is this
116
00:07:50.990 --> 00:07:55.069
day of true? Are People filing
in fake you know our peel filing and
117
00:07:55.149 --> 00:07:58.550
fake elections? Are People filing in
fake reports, like what is actually happening
118
00:07:58.660 --> 00:08:03.019
on the ground, with real data, with people who are in places where
119
00:08:03.060 --> 00:08:05.899
they can get arrested if things go
wrong? You know, that was a
120
00:08:05.980 --> 00:08:09.259
big eye opener to be around issues
of safety, around its use of ethics,
121
00:08:09.620 --> 00:08:13.290
rund issues of like will like,
for example, if I go to
122
00:08:13.370 --> 00:08:18.089
a place that has election monitoring and
we're running election monitoring campaign with the local
123
00:08:18.170 --> 00:08:22.410
Ngoh, if the copsbuts down the
door for some reason because something happened,
124
00:08:22.889 --> 00:08:26.730
I would be arrested and then flown
back to the US. They would be
125
00:08:26.810 --> 00:08:31.839
arrested and God knows what would happen. And that mean that our threat models
126
00:08:31.879 --> 00:08:35.360
are very different and sort of understanding
that your threat model isn't their threat model,
127
00:08:35.840 --> 00:08:37.960
I think has a lot of applications
for data science, you know in
128
00:08:39.039 --> 00:08:41.159
the US and everywhere. That,
I think, is it was. It
129
00:08:41.240 --> 00:08:43.190
was an important lesson for me.
Absolutely, and I've a question around.
130
00:08:43.710 --> 00:08:46.389
Just seems like on these time this
type of work, you'll learn a lot
131
00:08:46.470 --> 00:08:50.750
on the job, both in terms
of domain expertise but also in terms of
132
00:08:50.789 --> 00:08:54.590
data scientific techniques. I'm just wondering, before you got your first day to
133
00:08:54.629 --> 00:08:58.139
science job, did you know how
to program in Python, or did you
134
00:08:58.220 --> 00:09:01.340
have it like an idea of what
the landscape look like from your time in
135
00:09:01.539 --> 00:09:05.100
in research? Where I was when
I was finishing at my phd was I
136
00:09:05.299 --> 00:09:11.570
had done some web work. So
I'm not going to say web development because
137
00:09:11.570 --> 00:09:13.490
it wasn't a lot of javascript that
I stuff, but I done like html
138
00:09:13.769 --> 00:09:16.730
and CSS, which aren't really programming, but just you know that kind of
139
00:09:16.769 --> 00:09:20.690
stuff, like making some of the
pages with a little bit of Javascript in
140
00:09:20.730 --> 00:09:26.080
there and that kind of stuff.
And I done all of my research in
141
00:09:26.279 --> 00:09:31.399
Stata and are but I was a
very good at I don't think. I
142
00:09:31.600 --> 00:09:33.759
think my my code was very poor
at that moment, but I did see
143
00:09:35.039 --> 00:09:37.789
through my conversation to other people that
there was this world of coding wasn't just
144
00:09:37.909 --> 00:09:41.950
a little script that you run and
like leave your laptop, but you know
145
00:09:41.990 --> 00:09:43.190
if it needs to run for more
than like six hours, you just leave
146
00:09:43.190 --> 00:09:46.950
your laptop open for six hours.
That I realized that there was much more
147
00:09:48.470 --> 00:09:52.419
into the world of software engineering and
I started to move in that direction because
148
00:09:54.019 --> 00:09:56.860
when you want to build stuff with
other engineers, it's nice to use the
149
00:09:56.899 --> 00:09:58.940
tools that they use and to think
about things that they use and to take
150
00:10:00.019 --> 00:10:03.299
code that they can insert into their
into their projects, and so I really
151
00:10:03.299 --> 00:10:09.289
started to push more along the lines
of developing more software engineering skills, more
152
00:10:09.409 --> 00:10:13.090
languages that are pretty common in software
engineering, such as Python. But I
153
00:10:13.169 --> 00:10:18.169
definitely when I left my PhD,
I was not the wizard programmer that I'm
154
00:10:18.169 --> 00:10:20.679
not today, but I could.
I love that. So I usually ask
155
00:10:20.759 --> 00:10:24.519
a question around what your colleagues think
you do as opposed to what you actually
156
00:10:24.559 --> 00:10:28.559
do. That might be slightly different
in what you do. Devoted people might
157
00:10:28.559 --> 00:10:31.399
have more of an idea. I
think historically, though, in terms of
158
00:10:31.399 --> 00:10:35.830
the setups you've worked in doing analytics
and diato signs, is there a mismatch
159
00:10:35.950 --> 00:10:39.230
between what people think you do and
what you actually do? I think people
160
00:10:41.269 --> 00:10:45.710
so we'll take brick brick is a
Kenyan startup that does free Wi fi for
161
00:10:45.789 --> 00:10:48.950
low and could people it. The
whole team is Kenyan. They're based out
162
00:10:48.990 --> 00:10:52.899
of Kenya and I was the first
day to hire at brick. I think
163
00:10:52.940 --> 00:10:56.940
a lot of them thought that I
was doing wizard mathematics, like behind you
164
00:10:56.980 --> 00:11:00.940
know, there was a white board
and my office and I'm writing equations and
165
00:11:01.860 --> 00:11:05.409
solving riddles of mathematics to get them
some kind of thing, where in fact
166
00:11:05.490 --> 00:11:09.690
what I was doing was a lot
of software engineering stuff, a lot of
167
00:11:09.809 --> 00:11:13.570
building functions, running things, crown
jobs, like using a lot of python
168
00:11:15.250 --> 00:11:18.679
or a lot of pandas some psycit
learned that kind of stuff. But I
169
00:11:18.840 --> 00:11:22.200
was I was mostly using established tools, but it would provide them with actually
170
00:11:22.200 --> 00:11:26.360
something that was useful in there in
their work. But I think from their
171
00:11:26.399 --> 00:11:28.519
perspective I was like wizardry right,
because I would do something like imputation,
172
00:11:28.720 --> 00:11:33.669
where you take missing values in your
data and you impute the value. You
173
00:11:33.789 --> 00:11:37.629
like fake what the value would probably
be, and it was like wizardry right,
174
00:11:37.710 --> 00:11:39.669
like wow, I can't believe that
happened, like you were predicting this
175
00:11:39.789 --> 00:11:45.590
stuff, and so I think it
feels very normal to the data scientists and
176
00:11:45.750 --> 00:11:48.379
to me, but I think it
felt very it was. It was very
177
00:11:48.419 --> 00:11:50.580
exciting for a team that didn't have
that to start to have that kind of
178
00:11:50.659 --> 00:11:54.500
option for stuff, and I think
it worked well intasty. So, as
179
00:11:54.539 --> 00:11:58.740
we've said, we here to talk
about getting your first Dita Science job and
180
00:11:58.220 --> 00:12:01.850
you're working on hiring dita scientists,
devoted and thinking a lot about people getting
181
00:12:01.850 --> 00:12:05.889
the first data science job today and
I want to kind of figure out what
182
00:12:05.929 --> 00:12:09.090
a companies are generally looking for when
hiring first time data scientists. But before
183
00:12:09.210 --> 00:12:13.970
that, I suppose. Yeah,
a preliminary question is, are a lot
184
00:12:13.009 --> 00:12:18.279
of companies trying to hire a first
time data scientists, because a lot of
185
00:12:18.320 --> 00:12:22.559
the job list things like one ten
years experience of distributed computing and all of
186
00:12:22.639 --> 00:12:24.840
that jazz right. So, like
where is it? Has it had to
187
00:12:24.840 --> 00:12:28.000
use the term entry level? But
for a first time job? Are there?
188
00:12:28.039 --> 00:12:31.230
Are there a lot of jobs out
there? Yeah, I think they're.
189
00:12:31.309 --> 00:12:35.909
There are, although from so the
perspective I think will we could take
190
00:12:35.950 --> 00:12:39.789
this interview is that I'm sitting on
the other side of the table where I'm
191
00:12:39.789 --> 00:12:41.750
doing a lot of the working with
a team it devoted to do a lot
192
00:12:41.750 --> 00:12:45.980
of hiring for a data science team
and I talked to a lot of my
193
00:12:46.059 --> 00:12:48.700
friends who are doing similar roles at
other companies. You know, so in
194
00:12:48.740 --> 00:12:52.019
the interview they're on that they're on
the hiring side. So definitely are a
195
00:12:52.100 --> 00:12:56.899
lot of junior roles and I wouldn't
put into someone's head that they're like junior
196
00:12:56.899 --> 00:13:00.090
rules are dying or everyone needs experience
and that kind of stuff. You know,
197
00:13:00.250 --> 00:13:07.049
there is something I definitely see that
as a team you can absorb infinite
198
00:13:07.090 --> 00:13:09.649
number of senior hires. So say
you have a team of six people,
199
00:13:09.929 --> 00:13:15.080
you can hire six new other senior
people and be pretty okay that everything's just
200
00:13:15.120 --> 00:13:18.919
going to work because their senior right
like it'll do fine. It is much
201
00:13:18.919 --> 00:13:22.799
more of a risk from the organization's
perspective to if you have six people on
202
00:13:22.879 --> 00:13:26.590
your team and then you hire six
junior but because there's a like those junior
203
00:13:26.629 --> 00:13:31.389
people need a lot more support and
if you're unable to give that support,
204
00:13:33.350 --> 00:13:35.470
it is not good for the junior
people or for you. And definitely take
205
00:13:35.509 --> 00:13:37.750
this from the perspective that junior person
like you. Do not want to be
206
00:13:37.830 --> 00:13:43.940
in a place where there is one
senior data scientists and there's six junior people
207
00:13:43.940 --> 00:13:48.779
who are all hired at the same
time, because you are not gonna Learn
208
00:13:48.899 --> 00:13:50.860
what you need to learn. Like
you want to be the one junior person
209
00:13:52.340 --> 00:13:54.700
on a team of, you know, six, sixteenior people. I'd would
210
00:13:54.700 --> 00:13:58.090
be the ideal situation, and you
can just sit down and take all the
211
00:13:58.169 --> 00:14:01.090
time of theirs you want and you
can work with them very closely and you're
212
00:14:01.090 --> 00:14:03.330
learning would be massive. That would
be incredible. There's early junior jobs out
213
00:14:03.330 --> 00:14:07.649
there, but there is something from
an organizations prospective where you know you might
214
00:14:07.649 --> 00:14:09.879
have a small start up and you
only have three data scientists or two data
215
00:14:09.919 --> 00:14:16.720
scientists at this company because data science
is a specialty job and they you just
216
00:14:16.799 --> 00:14:20.120
can't. You can't have a posting
that says you want to hire five junior
217
00:14:20.200 --> 00:14:22.159
people like I wouldn't. Don't.
Don't apply that job. I think I
218
00:14:22.200 --> 00:14:26.309
would be very horrible. Yeah,
instead, you want somewhere. I think
219
00:14:26.309 --> 00:14:28.789
if I was looking, if I
was junior. So one thing we should
220
00:14:28.789 --> 00:14:31.950
point out that when I was junior
and data science, it was a very
221
00:14:31.990 --> 00:14:35.629
different field. So you should not
take my how I got into the field
222
00:14:35.629 --> 00:14:39.820
as an example of how you should
get into the field, because when I
223
00:14:39.940 --> 00:14:45.059
started there wasn't really the concept of
data science and you you was sort of
224
00:14:45.100 --> 00:14:46.940
a bunch of people are interested in
the same topic and just sort of like
225
00:14:48.100 --> 00:14:50.220
got together. Yeah, I'm doing
it, doing it now. It's a
226
00:14:50.220 --> 00:14:54.570
little it's different, obviously, but
I think I can see some things around.
227
00:14:54.450 --> 00:14:58.409
If you are, you know,
if you're looking for that first job,
228
00:14:58.450 --> 00:15:05.450
I would definitely look for places in
mid to larger companies and not in
229
00:15:05.090 --> 00:15:09.600
smaller, smaller, scrappier startups.
I think one of the things that I've
230
00:15:09.720 --> 00:15:15.279
found a few times during the hiring
process is that people who got their first
231
00:15:15.279 --> 00:15:18.480
job at, say, you know, facebook, and they're a junior,
232
00:15:18.519 --> 00:15:22.230
junior data scientist at facebook, there's
so much more support that they got in
233
00:15:22.350 --> 00:15:28.830
that time for learning, for you
know, experimentation, for using things at
234
00:15:28.870 --> 00:15:31.750
scale, right, like learning how
to work at facebook scale, but doing
235
00:15:31.830 --> 00:15:37.700
so as a very junior person.
They have such great experience that then they
236
00:15:37.779 --> 00:15:39.820
move on to say, you know, say apply for a job at devoted
237
00:15:39.860 --> 00:15:45.259
or something like that, and they
have all that experience and and we can
238
00:15:45.460 --> 00:15:48.100
use that experience, like that's great, that, you're awesome, like cool,
239
00:15:48.139 --> 00:15:50.889
you have this great experience. This
is hard experience to get because you
240
00:15:52.009 --> 00:15:54.330
know, it's hard for a boot
camp to, to say, replicate fifty
241
00:15:54.409 --> 00:15:58.450
million messages the day that you have
to process or something like that. There's
242
00:15:58.490 --> 00:16:00.649
there's that's a that's a weird boot
camp project, but that is what a
243
00:16:00.649 --> 00:16:03.009
lot of companies end up doing.
So going to somewhere like facebook or going
244
00:16:03.090 --> 00:16:06.399
to I'm just like picking on Facebook, and this isn't it a for facebook.
245
00:16:06.399 --> 00:16:08.200
FACEBOOK is in a playing room,
but any kind of larger company that
246
00:16:08.440 --> 00:16:12.320
can absorb can absorb you as a
junior higher and out like a allow you
247
00:16:12.360 --> 00:16:15.879
to learn and allow you to learn
from from really senior engineers and then moving
248
00:16:15.919 --> 00:16:18.549
on to something else is, I
think, a great perspective. PUSHUL and
249
00:16:18.629 --> 00:16:22.830
those companies, I think, have
the infrastructure all set up as well,
250
00:16:22.870 --> 00:16:26.269
so you not like battling with your
your data likes and aflow and all of
251
00:16:26.350 --> 00:16:30.269
that run. Yeah, well,
and there's absolutely a trap that I think
252
00:16:30.269 --> 00:16:33.179
junior data scientists fall into, even
midleveld to scientists fall into, where they
253
00:16:33.259 --> 00:16:37.220
join a small, scrappy start up, not as a founder, right,
254
00:16:37.259 --> 00:16:40.740
but just, you know, the
start up as six people or something like
255
00:16:40.820 --> 00:16:44.620
that, and they're the first day
to hire and there's no data infrastructure in
256
00:16:44.740 --> 00:16:48.649
place at all and there's no safetiness, there's no, you know, ability
257
00:16:48.690 --> 00:16:52.009
to get data from the database in
a way that's useful and you have to
258
00:16:52.049 --> 00:16:56.169
sit down and build all that,
which in certain times can be okay.
259
00:16:56.450 --> 00:17:00.049
But I think in a really scrappy
start up where they're like struggling to make
260
00:17:00.090 --> 00:17:03.440
payroll or the you know, they're
like grinding away or something, it can
261
00:17:03.480 --> 00:17:07.799
be a very, very hard experience
and probably not the best experience as opposed
262
00:17:07.839 --> 00:17:11.519
to something where you had, you
know, you go work at some really
263
00:17:11.599 --> 00:17:14.000
hard problems at a larger company,
but you do so in a way that
264
00:17:14.440 --> 00:17:17.309
you can leave work at the end
of the day and you know knowing that
265
00:17:17.349 --> 00:17:19.509
everything's fine, and you come back
the next day and all that infrastructures in
266
00:17:19.549 --> 00:17:22.430
place and off talk some lectures and
you can go over to some person who's
267
00:17:22.430 --> 00:17:26.950
done data engineering for ten years and
say, Hey, why does it work
268
00:17:26.990 --> 00:17:27.630
like this, and they'll be like, oh, it's because of x,
269
00:17:27.630 --> 00:17:30.019
Y and Z and that kind of
stuff. As I think there's just there's
270
00:17:30.019 --> 00:17:33.539
so much learning that can happen at
larger companies. I've never worked at larger
271
00:17:33.579 --> 00:17:37.740
company, so this is me talking
from the outside but I do think that,
272
00:17:38.099 --> 00:17:41.940
as someone has worked at a few
smaller companies or small organizations, that
273
00:17:41.980 --> 00:17:48.009
if you those organizations do better if
you're senior, just because you can sort
274
00:17:48.049 --> 00:17:51.690
of be left alone and figure everything
out by yourself because you've done this before,
275
00:17:52.009 --> 00:17:53.450
as opposed to actually, I don't
know what I'm doing, I really
276
00:17:53.490 --> 00:17:56.930
need someone to tell me how to
do it right and so you learn things
277
00:17:56.930 --> 00:17:59.079
the right way. I think there's
there's a lot of things where like,
278
00:17:59.119 --> 00:18:03.440
particularly where a data scientist is doing
more software engineering stuff, where it's really
279
00:18:03.440 --> 00:18:06.240
nice to sit down for someone to
be like Hey, explain to me object
280
00:18:06.319 --> 00:18:08.319
oriented programming, because I don't understand
what in the world's happening, or explained
281
00:18:08.319 --> 00:18:11.720
to me like how testing works,
like unit tests, integration test or you
282
00:18:11.799 --> 00:18:15.950
know patterns, like Software Engineering Patterns, like factory of factories and that kind
283
00:18:15.990 --> 00:18:18.589
of stuff. The tell me about
that. Like all those are totally simple
284
00:18:18.630 --> 00:18:22.190
concepts that anyone listen to this podcast
could know. Just need to like have
285
00:18:22.430 --> 00:18:26.309
someone tell you about him or know
that. You should read about them like
286
00:18:26.750 --> 00:18:27.940
understand. Oh, actually, I
this comes up a lot. I should
287
00:18:27.940 --> 00:18:32.180
read about this. That kind of
stuff, which is great in companies with
288
00:18:32.339 --> 00:18:36.740
more support and is terrible and really, really small, scrappy places. So,
289
00:18:37.059 --> 00:18:40.779
in general, what a company's looking
for when hiring first on Ditoscientists,
290
00:18:41.019 --> 00:18:41.890
do you think? And if you
don't want to answer that, you can
291
00:18:41.930 --> 00:18:45.890
tell me, like what you in
particular are looking for when haring first on
292
00:18:45.009 --> 00:18:48.569
ditoscientists. Sure, it's also I'll
try to stab at the one, but
293
00:18:48.609 --> 00:18:51.210
then we'll definitely hit on the other
one. So Great. It depends on
294
00:18:51.250 --> 00:18:52.809
the company. I should say organization, because I work for a lot enough
295
00:18:52.809 --> 00:18:56.799
parbets, but we'll just use company
as like a forget for that. Places
296
00:18:56.880 --> 00:19:03.279
that are very small and scrappy tend
to look for senior people that they can
297
00:19:03.599 --> 00:19:07.279
run the whole like I think you
would call them like a full sack data
298
00:19:07.319 --> 00:19:10.750
scientist. I'm not entirely enamored with
that phrase, but whatever, like a
299
00:19:10.869 --> 00:19:14.109
generalist, a generalist so that you
could you could just have them join the
300
00:19:14.470 --> 00:19:18.470
software engineering team or the product team
or something like that and you give them
301
00:19:18.630 --> 00:19:22.150
pseudo access on your you know,
on your server, and they will just
302
00:19:22.230 --> 00:19:25.339
build everything they need to build the
build on the pipelining, the build all
303
00:19:25.420 --> 00:19:27.740
the you know, saving backups,
all the data. They'll build the tables
304
00:19:27.779 --> 00:19:30.460
that they need to build and they'll
run the analysis and they'll run it in
305
00:19:30.500 --> 00:19:33.420
a way that runs every single you
know, every single day, but only
306
00:19:33.460 --> 00:19:37.700
on certain times, and they'll install
air flow by themselves and they'll do all
307
00:19:37.819 --> 00:19:41.049
this kind of stuff that just they
can. They can do. That's awesome
308
00:19:41.089 --> 00:19:45.809
for that person. That's not really
a junior role no more. I think
309
00:19:45.849 --> 00:19:48.609
if you go to the other end
of the spectrum and you go for large
310
00:19:48.690 --> 00:19:52.289
companies, they do a mix.
Well, they will hire people from more
311
00:19:52.410 --> 00:19:56.920
senior positions to do things like,
say, someone very specialized, say someone
312
00:19:56.960 --> 00:20:02.200
who just got their PhD in ai
or something like that. Like they might
313
00:20:02.240 --> 00:20:06.480
have them join the team that's working
on an algorithm and that junior person can
314
00:20:06.519 --> 00:20:08.630
work on a part of the algorithm
or work on some testing part and grind
315
00:20:08.710 --> 00:20:14.150
to that kind of stuff. Or
I think what's very common is more generalist
316
00:20:14.150 --> 00:20:18.230
data scientists who are junior join larger
companies and they do probably what I would
317
00:20:18.230 --> 00:20:23.220
it would call like advanced analyzes or
advanced analytics. Right, so you have
318
00:20:23.420 --> 00:20:27.579
like a new product and you want
to understand how that products is actually doing
319
00:20:27.619 --> 00:20:30.539
at scale, right, because it's, you know, deploy globally. On
320
00:20:30.619 --> 00:20:34.059
all android devices or something like that, and you want to see how that
321
00:20:34.140 --> 00:20:37.089
product is working and how people are
using it. There is some very,
322
00:20:37.130 --> 00:20:41.089
very, very complicated analyzes that need
to be done for that to be true,
323
00:20:41.809 --> 00:20:45.410
and that is a great sort of
thing for a junior person to kind
324
00:20:45.450 --> 00:20:49.519
of tear off and start working on. And it doesn't affect the production code,
325
00:20:49.759 --> 00:20:52.640
but it does affect that business and
in that respect, with I suppose
326
00:20:52.680 --> 00:20:56.440
we're talking about the data analyst breed
of Dita scientists, right, yeah,
327
00:20:56.440 --> 00:20:59.920
and so like again, I don't
know why I keep on talking aboutacebook,
328
00:20:59.920 --> 00:21:02.119
but I know it facebook. A
lot of people who have the title of
329
00:21:02.160 --> 00:21:04.750
data scientists do more analysis, you
know, which is totally reasonable and is
330
00:21:04.789 --> 00:21:08.509
super hard, and give them massive
credit and it is. That kind of
331
00:21:08.589 --> 00:21:11.390
stuff is, you know, is
a real roles, a real job,
332
00:21:11.430 --> 00:21:15.109
and I think people who come from
academia, you know, like the people
333
00:21:15.150 --> 00:21:19.140
who I know come from sort of
political sciences, social scientists, they go
334
00:21:19.500 --> 00:21:22.859
into those kind of roles and have
a great time because they are working on
335
00:21:23.140 --> 00:21:29.539
hard analyzes, like hard analytical problems. It's not you're not making dashboards to
336
00:21:29.579 --> 00:21:33.809
show that someone's clicking on something.
You're making really complicated analysies to figure out,
337
00:21:33.049 --> 00:21:37.529
you know, the churn model that
applies Bayesi into when we know we
338
00:21:37.769 --> 00:21:40.970
all that kind of stuff, which
is cool. I think in the middle,
339
00:21:41.250 --> 00:21:42.890
right to the the have the teeny
companies and you have a large companies.
340
00:21:44.490 --> 00:21:48.359
You have companies like devoted that sort
of sit in the middle where there
341
00:21:48.640 --> 00:21:56.000
is a mix between trying to hire
senior people who can craft the foundational data
342
00:21:56.079 --> 00:22:00.119
science if its structure right. So
devoted is building a health insurance companies text
343
00:22:00.240 --> 00:22:03.150
AC from scratch. You know,
if, like it was at one point
344
00:22:03.190 --> 00:22:07.990
an empty github repo. Now there's
now there's a lot of code in there,
345
00:22:07.069 --> 00:22:10.029
but yeah, at some point we're
like, okay, we are health
346
00:22:10.069 --> 00:22:12.630
insurance company. This is our codebase, like, let's write the first line
347
00:22:12.630 --> 00:22:18.140
of code. That's where we are. And in those kind of environments you
348
00:22:18.339 --> 00:22:21.779
want a mix of people who are
senior who can build the architecture for how
349
00:22:22.019 --> 00:22:26.660
things work and how data is moved
around and how, say, tasks or
350
00:22:26.660 --> 00:22:30.009
run every day or how there's,
you know, things around testing and if
351
00:22:30.250 --> 00:22:33.609
companies do testing and all that kind
of stuff. And in addition, you
352
00:22:33.849 --> 00:22:40.450
want people who have more junior experience
that you can come in you can provide
353
00:22:40.490 --> 00:22:42.130
some lift them, like teaching.
Wise, like you could do more teaching
354
00:22:42.210 --> 00:22:45.440
with them, and you can have
them support the role where you might tear
355
00:22:45.480 --> 00:22:48.680
off a piece of a project and
say, Hey, can you make this
356
00:22:48.839 --> 00:22:51.640
for me? Like so, I
don't need so, I don't need to
357
00:22:51.680 --> 00:22:53.400
make it because it was definitely so. One of the people who's on our
358
00:22:53.480 --> 00:22:57.400
team currently is Kit Rodolpha, who
is the former chief data scientists of the
359
00:22:57.400 --> 00:23:03.230
Hillary campaign, and he's done some
amazing complicated work and he's also done some
360
00:23:03.349 --> 00:23:07.589
amazing work that was well below his
level of expertise because there was just no
361
00:23:07.750 --> 00:23:12.019
one else there to do it.
And so there's I think typically midlevel companies
362
00:23:12.099 --> 00:23:15.339
like ours tend to have both.
Some are longer range. And so for
363
00:23:15.420 --> 00:23:18.700
you, for your size of company, what would a for a junior ditis
364
00:23:18.740 --> 00:23:22.619
on his wood skills? Would I
oh, what would be good qualities?
365
00:23:22.259 --> 00:23:26.579
What would they need to do a
demonstrate fee to be like how this person
366
00:23:26.700 --> 00:23:30.650
would be a good feed here.
Yeah, I think for us at devoted
367
00:23:30.210 --> 00:23:36.490
we are a health insurance company,
but if you looked at the internal workings
368
00:23:36.650 --> 00:23:40.690
of how the Tech Team Works,
we operate for more like a Silicon Valley
369
00:23:40.849 --> 00:23:47.559
Tech Company and concepts of move fast
and break things or ask for forgiveness rather
370
00:23:47.599 --> 00:23:49.119
than permission are real things that we
are doing. I think a lot of
371
00:23:49.160 --> 00:23:52.319
the people have devoted are building things
that is the largest and hardest thing that
372
00:23:52.400 --> 00:23:56.829
they've built ever in their career,
and that's what we want you to be
373
00:23:56.950 --> 00:24:00.269
that we want you to be the
person who is building the doing the best
374
00:24:00.349 --> 00:24:03.349
work of career, doing the hardest
thing, that is pushing yourself right to
375
00:24:03.430 --> 00:24:07.230
the limit of what you think you
can do. And we have a very
376
00:24:07.309 --> 00:24:11.539
strong culture that says, Hey,
we want you to run right up until
377
00:24:11.539 --> 00:24:14.980
the point that you break the thing, break the thing and then come and
378
00:24:15.259 --> 00:24:18.700
and be able to say, Hey, I totally broke this, can someone
379
00:24:18.140 --> 00:24:22.460
fix this, Dear God, and
then we'll go back and we'll work with
380
00:24:22.579 --> 00:24:25.569
them to fix it and, and
I say this is them, but like
381
00:24:25.690 --> 00:24:29.650
I have in fact broken many things
at devoted and to have a culture that
382
00:24:29.730 --> 00:24:32.849
says, Hey, you can break
these things, it's okay, don't worry
383
00:24:32.890 --> 00:24:34.410
about it. You should definitely admit
that it broken, because we should know
384
00:24:34.529 --> 00:24:37.210
that, but run right up until
you break it. Then we'll talk about
385
00:24:37.210 --> 00:24:41.119
why it was broken and will then
run again, like don't stop running,
386
00:24:41.119 --> 00:24:45.119
keep on running. But that fast
paced, the fast face doesn't translate into
387
00:24:45.119 --> 00:24:48.720
a huge amount of work hours because, you know, devoted, like I
388
00:24:48.799 --> 00:24:51.920
have a family and no one's no
one had devoted, I think, is
389
00:24:51.960 --> 00:24:55.069
grinding the midnight oil left, right
and center. There's definitely people are working
390
00:24:55.829 --> 00:24:59.390
to five ish, but just you
know, and what you're doing. I
391
00:24:59.470 --> 00:25:06.390
think there's a lot of pushing the
boundaries of what we can do skill wise
392
00:25:06.430 --> 00:25:11.420
right to the end and a lot
of learning and trying to build the things
393
00:25:11.460 --> 00:25:15.900
that are the limit of our capability. Is Pretty Common. I think that
394
00:25:15.059 --> 00:25:19.299
would be a skill regardless of your
expert you know, level of experience,
395
00:25:19.339 --> 00:25:22.410
whether you have one year of experience
or no. Love you know, no
396
00:25:22.569 --> 00:25:26.250
years of experience or ten years of
experience. We really do look for people
397
00:25:26.690 --> 00:25:32.250
who are very happy to take on
a task that they've never done before,
398
00:25:32.450 --> 00:25:33.930
but they're pretty you know, they
kind of understand how they might go about
399
00:25:33.930 --> 00:25:38.039
it and go for it, like
go and run it and we have safety
400
00:25:38.039 --> 00:25:42.240
guards in place that nothing's nothing's bad
going to happen, that they can push
401
00:25:42.319 --> 00:25:45.960
themselves and then once they had that, once they've built that thing, they
402
00:25:45.960 --> 00:25:48.279
go okay, cool, I understand
it now. It's great, I've got
403
00:25:48.400 --> 00:25:52.069
it. Like let's do something else, like let's go harder, let's do
404
00:25:52.230 --> 00:25:55.390
this, build the next thing.
I think that's really it's hard to instill
405
00:25:55.430 --> 00:26:00.109
because it is not experience per se. It's more, you know, aptitude,
406
00:26:00.150 --> 00:26:06.099
and attitude is probably a closer one
where I don't necessarily we don't necessarily
407
00:26:06.140 --> 00:26:11.819
say we're only looking for people with
x years. We tend to have a
408
00:26:11.940 --> 00:26:14.460
willingness to say, Hey, you're
actually, you know, like we were
409
00:26:14.460 --> 00:26:17.059
kind of looking for someone more senior, but you are more junior, but
410
00:26:17.339 --> 00:26:21.650
you really are kicking butt and we
can see that when we talk to you
411
00:26:22.089 --> 00:26:26.609
that you have a lot of aptitude
for your level, like for your levels
412
00:26:26.650 --> 00:26:29.930
of experience, and we love your
attitude for running with it, like let's
413
00:26:29.930 --> 00:26:32.569
go, like, come on board. We've definitely, you know, we've
414
00:26:32.609 --> 00:26:36.319
had a few hires like that and
I think it is worked out, but
415
00:26:36.400 --> 00:26:38.599
it is it is something. It's
not uniquely devoted, but it is something
416
00:26:38.640 --> 00:26:42.160
that is probably pretty uncommon for our
industry. Yeah, and that speaks to
417
00:26:42.440 --> 00:26:45.960
certain skills that you've mentioned in there
are being like an active problem solver,
418
00:26:47.230 --> 00:26:56.230
communication, having a passion and drive. Will jump right back into our interview
419
00:26:56.230 --> 00:27:04.859
with Chris Alban after a short segment. Now it's time for a segment called
420
00:27:06.059 --> 00:27:10.539
guidelines for online experiments. So I'm
here with Emily Robinson, a data signed
421
00:27:10.539 --> 00:27:12.019
us on the growth theme here at
data camp. Hey, emily, Hi
422
00:27:12.099 --> 00:27:17.170
Hugo. So, Emily, you've
written what I consider a fantastic post on
423
00:27:17.250 --> 00:27:21.450
guidelines for AB testing and you've also
given a related talk several times, and
424
00:27:21.730 --> 00:27:25.329
I'm really interested to hear your thoughts
on what kind of rules of thumb and
425
00:27:25.410 --> 00:27:27.930
guidelines for AB testing actually are.
But maybe first you can start by just
426
00:27:29.049 --> 00:27:33.480
reminding us or what an online experiment
or an AB test actually is. Sure,
427
00:27:33.519 --> 00:27:36.920
Hugo. So, the idea of
an online experiment is that you want
428
00:27:36.920 --> 00:27:40.480
to measure the impact of a change
that you make on your website, and
429
00:27:40.799 --> 00:27:45.150
how you do this is that you
randomly assigned people to either the control so
430
00:27:45.309 --> 00:27:48.069
the old experience, or the treatment
group, the new experience, and by
431
00:27:48.109 --> 00:27:55.069
randomly assigning people you can then make
sure that there's no differences between the two
432
00:27:55.150 --> 00:27:59.180
groups, so, for example,
that people the countries are randomized, that
433
00:27:59.539 --> 00:28:03.619
new and returning users etc. And
this is really helpful because another approach might
434
00:28:03.660 --> 00:28:07.700
just be to launch the change and
watch the graph move and say, Oh,
435
00:28:07.740 --> 00:28:11.740
okay, we launched this change on
Tuesday. DID REGISTRATION RATE GO UP?
436
00:28:11.339 --> 00:28:15.890
Well, the problem is registration rate
or your other metrics are not a
437
00:28:15.049 --> 00:28:18.769
fixed value each day. They have
their own variance and you also have other
438
00:28:18.809 --> 00:28:22.609
stuff going on in the world at
the same time, like, say,
439
00:28:22.690 --> 00:28:26.809
maybe there is a holiday or another
project was launched, and that means that
440
00:28:26.930 --> 00:28:30.720
it's very hard to tell if one
if there was any movement at all and
441
00:28:30.920 --> 00:28:34.559
to if that movement really was because
of your change. And online experiments allow
442
00:28:34.559 --> 00:28:38.319
you to make that inference by giving
you a control group telling you what would
443
00:28:38.319 --> 00:28:42.069
have happened without that change. Okay, so that's a great explanation of what
444
00:28:42.190 --> 00:28:47.069
an online experiment or an abatist actually
use before we get into the guideline.
445
00:28:47.109 --> 00:28:49.230
So there are a couple of,
I suppose, technical terms that would be
446
00:28:49.230 --> 00:28:52.630
good to Damis, define, and
I'm thinking really about test statistic and the
447
00:28:52.750 --> 00:28:56.420
scary pay value, right. So
maybe you could just tell us how you
448
00:28:56.500 --> 00:28:59.819
use these in online experiments and then
we can jump into you guidelines. Sure.
449
00:29:00.180 --> 00:29:03.900
So let's say you have your control
in your treatment group and you're interested
450
00:29:03.980 --> 00:29:08.140
in registration rate on your website and
it's five percent in the control and five
451
00:29:08.259 --> 00:29:12.009
point zero five percent in the treatment. Does that mean that your treatment was
452
00:29:12.049 --> 00:29:17.210
a success and you increase registration rate? Well, the problem is there is
453
00:29:17.289 --> 00:29:21.410
some randomness again, in that there's
variability and it could have just been chance
454
00:29:21.569 --> 00:29:23.289
that this happened. You know,
for example, if you plut a coin
455
00:29:23.440 --> 00:29:27.079
two times and it turns up heads, you wouldn't say that's necessarily an unfair
456
00:29:27.160 --> 00:29:32.920
coin. That could just happen by
chance. So what I do is we
457
00:29:33.039 --> 00:29:37.400
run what's called a proportion test for
a specifically where we're looking for the percent
458
00:29:37.480 --> 00:29:41.390
of people who did a certain action. A proportion test then we compare the
459
00:29:41.910 --> 00:29:47.069
proportion in the control versus the treatment, and not just five versus five point
460
00:29:47.069 --> 00:29:51.549
zero five, but specifically. You
know, is that five thousand of a
461
00:29:51.630 --> 00:29:56.380
hundred thousand or five hundred out of
a thousand? Because that matters and that
462
00:29:56.500 --> 00:30:00.299
gives it. That test gives us
back a P value, and the definition
463
00:30:00.500 --> 00:30:03.420
of a pe value is, if
you're know, hypothesis was true, which
464
00:30:03.460 --> 00:30:07.490
in this case is that there's no
difference between the control and the treatment group.
465
00:30:08.089 --> 00:30:12.450
How likely is it that you would
have seen a difference this extreme or
466
00:30:12.569 --> 00:30:15.849
more right. So the idea then
is it's really it has to do with
467
00:30:15.890 --> 00:30:21.250
how surprised you should be about seeing
your result right exactly. So statistics can
468
00:30:21.329 --> 00:30:23.759
never tell us, oh, we're
absolutely sure that there was a, quote
469
00:30:23.759 --> 00:30:27.480
unquote, real difference, but it
can't tell you a P value of point
470
00:30:27.559 --> 00:30:30.599
your one would say, for example, okay, if there was no difference,
471
00:30:30.799 --> 00:30:34.480
only one percent of a time would
you see the results that you got
472
00:30:34.559 --> 00:30:37.750
or results that were more extreme.
So you know, maybe you're that one
473
00:30:37.829 --> 00:30:41.509
percent. Maybe there was no difference, but generally there's a rule of thumb
474
00:30:41.549 --> 00:30:45.829
that if your p value is less
than point zero, five, so is
475
00:30:45.829 --> 00:30:48.950
a lesson five percent chance of getting
a false positive. If there was no
476
00:30:49.099 --> 00:30:52.099
difference, then we'll go ahead and
say, okay, we are going to
477
00:30:52.500 --> 00:30:59.180
reject then hypothesis and assume that there
was a real difference. Right, okay.
478
00:30:59.420 --> 00:31:03.019
So that's a great introduction to online
experimentation. You quote a great saying
479
00:31:03.019 --> 00:31:06.450
in your blood post in your talked
up generating numbers. So getting data for
480
00:31:06.490 --> 00:31:08.890
these type of stuff, generating numbers
is easy. Generating numbers, you should
481
00:31:08.890 --> 00:31:12.890
trust, is hard, and you've
written that there are many ways ab tasting
482
00:31:12.970 --> 00:31:17.089
can go wrong, but most of
them won't be obvious. So I did.
483
00:31:17.130 --> 00:31:21.279
That's a brief introduction to, or
motivation for, your guidelines, and
484
00:31:21.359 --> 00:31:23.480
we here to start doing about best
practices for AB testing. So what's your
485
00:31:23.559 --> 00:31:27.759
first guideline? The first one is
to have one key metric per experiment.
486
00:31:29.119 --> 00:31:33.230
So what that means is define what
your success metric is. That doesn't mean
487
00:31:33.309 --> 00:31:37.190
that you can't monitor multiple metrics to
make sure you don't accidentally take them.
488
00:31:37.549 --> 00:31:41.349
For example, here at data camp, where subscription business, and so while
489
00:31:41.390 --> 00:31:45.910
we might be targeting registration rate for
a certain experiment, if suddenly subscription rate
490
00:31:47.029 --> 00:31:49.339
went to zero in the treatment,
that would be pretty bad. But the
491
00:31:49.460 --> 00:31:53.220
reason we only want one metric is
that if we're looking at many metrics,
492
00:31:53.380 --> 00:31:59.420
you'll end up with an increased false
positive rate unless you apply corrections. What
493
00:31:59.579 --> 00:32:02.690
that means is if you say are
looking at ten metrics, the probability that
494
00:32:02.930 --> 00:32:07.529
one of them has a p value
of below zero five is, even if
495
00:32:07.529 --> 00:32:13.049
there's no difference, if your treatment
made no change and no impact your metrics
496
00:32:13.569 --> 00:32:16.720
is way above five percent. In
fact I think it's above thirty percent.
497
00:32:17.000 --> 00:32:21.599
Fantastic. So they're kind of lads
to my next question. How long should
498
00:32:21.640 --> 00:32:23.920
you run an experiment in order to
see an effect? This is a tricky
499
00:32:23.960 --> 00:32:30.309
question because is here you're looking to
avoid false and negative so thinking that there
500
00:32:30.509 --> 00:32:35.630
is no difference when there actually is
one, and when you say see and
501
00:32:35.710 --> 00:32:38.910
effect, it really matters what the
magnitude of effect is. So essentially,
502
00:32:38.910 --> 00:32:43.990
the longer you run it, the
smaller a difference you can detect. And
503
00:32:44.460 --> 00:32:49.819
how you can measure this or estimate
this is doing a power calculation. So
504
00:32:49.940 --> 00:32:52.539
you take your key metrics, to
say your registration rate. You figure out,
505
00:32:52.619 --> 00:32:57.500
all right, over a week we
have tenzero people and their registration rate
506
00:32:57.579 --> 00:33:00.130
is ten percent. You seem that's
going to be roughly the same the week
507
00:33:00.130 --> 00:33:04.890
you're running the experiment for the Control
Group. And then a power calculation can
508
00:33:04.930 --> 00:33:07.130
say, all right, with that
much data, with that registration rate,
509
00:33:07.569 --> 00:33:12.730
how big an effect can you detect? Where there's an eighty percent chance of
510
00:33:12.849 --> 00:33:15.599
if there's an effect of this size, you'll detect it. So, for
511
00:33:15.720 --> 00:33:19.079
example, your power calculation might say, Oh, in a week you can
512
00:33:19.160 --> 00:33:23.039
detect a five percent increase, and
that means if your increase is only two
513
00:33:23.119 --> 00:33:28.069
percent, you're probably not going to
detect it and even if it is really
514
00:33:28.150 --> 00:33:30.670
five percent, there's still a twenty
percent chance will miss it. But you
515
00:33:30.789 --> 00:33:36.069
have to make these tradeoffs because,
again, statistics can never tell you something
516
00:33:36.150 --> 00:33:39.630
definitively and you may decide, Hey, I'm okay missing a two percent change
517
00:33:40.150 --> 00:33:44.819
because we need to move fast and
a two percent change actually wouldn't even be
518
00:33:44.940 --> 00:33:46.819
that impactful. Right. And I
suppose one of the points is you want
519
00:33:46.819 --> 00:33:51.500
to do this power analysis during the
experimental design phase, right. You don't
520
00:33:51.500 --> 00:33:53.099
want to do a post talk at
power analysis, for example, right in
521
00:33:53.180 --> 00:33:58.690
the this guy scenario exactly, and
there's a couple reasons for this. One
522
00:33:58.890 --> 00:34:01.329
is you want to do it beforehand, because sometimes it may tell you that
523
00:34:01.410 --> 00:34:06.009
you shouldn't bother running the experiment.
So, for example, let's say you're
524
00:34:06.089 --> 00:34:07.929
making a change. You're like,
all right, I believe this change could
525
00:34:07.929 --> 00:34:13.840
have a ten percent increase in registration
rate and you run a power capitulation and
526
00:34:13.960 --> 00:34:17.840
it says in three weeks you could
only detect a thirty percent increase because maybe
527
00:34:17.840 --> 00:34:21.719
you just don't have them any visitors
on this part of the site. And
528
00:34:21.880 --> 00:34:24.030
given that, you may decide that
you don't want to run this test.
529
00:34:24.150 --> 00:34:28.309
After all perfect. So, emily, that's what we've got time for now,
530
00:34:28.389 --> 00:34:30.150
but I will really look forward to
coming back and hearing more about your
531
00:34:30.150 --> 00:34:35.070
guidelines for online experimentation. Thanks you
go. Looking forward to chatting more about
532
00:34:35.070 --> 00:34:45.460
it. Come to get straight back
into our chat with Chris. How about
533
00:34:45.500 --> 00:34:50.260
in terms of hard skills, whether
it be domain expertise or background in programming,
534
00:34:50.300 --> 00:34:52.650
whether it's Python, Ora or knowledge
of you know, how the math
535
00:34:52.730 --> 00:34:58.050
behind machine learning models are actually works
to like data analytic skills. Yeah,
536
00:34:58.050 --> 00:35:01.409
I've noticed something with hiring where typically, when we have someone in the hiring
537
00:35:01.449 --> 00:35:07.679
process, I will say to them
up front that there are places like,
538
00:35:07.760 --> 00:35:13.920
if you just got your master's degree
in machine learning and you believe that how
539
00:35:14.079 --> 00:35:16.199
you are going to advance your career
is to do machine learning, become an
540
00:35:16.199 --> 00:35:19.679
expert in machine learning, that's what
you're going to do. That is a
541
00:35:19.760 --> 00:35:24.670
completely valid jump my career track.
You should absolutely do that also. That
542
00:35:24.909 --> 00:35:29.949
isn't a place like devoted. Devoted
tends to be more generalist builders who are
543
00:35:30.070 --> 00:35:32.630
trying to you know, we will
use machine learning here because it's useful and
544
00:35:32.829 --> 00:35:36.420
then we'll use, you know,
fuzzy matching here because it's useful. And
545
00:35:36.460 --> 00:35:38.300
then we'll, you know, like
use some simple things somewhere else, because
546
00:35:38.340 --> 00:35:45.099
it's it gets the job done.
We tend to focus on the ability of
547
00:35:45.179 --> 00:35:51.449
people to build and solve the needs
of our internal users more than, say,
548
00:35:51.530 --> 00:35:55.010
someone's really nuanced view of machine learning
algorithms, just because, you know,
549
00:35:55.090 --> 00:35:59.730
we are a small team in a
new company. If you wanted to
550
00:35:59.969 --> 00:36:02.010
just do machine learning, there are
definitely companies that are big enough to that
551
00:36:02.250 --> 00:36:06.480
have like that have the infrastructure to
have you just do that right. Someone
552
00:36:06.519 --> 00:36:08.440
else will worry about, you know, some of the analytics or some or
553
00:36:08.440 --> 00:36:12.800
some of the data processing. You
can just sit down all day and read
554
00:36:13.039 --> 00:36:15.639
Machine Learning Journal articles and then implement
them. That's solely okay, and I
555
00:36:15.920 --> 00:36:19.150
you know, if I think,
I never worked at Google brain, but
556
00:36:19.190 --> 00:36:22.389
I imagine google brain has a strong
feeling around that, which is cool.
557
00:36:22.429 --> 00:36:25.710
Would that be your advice to first
time job seekers? Learn a bunch of
558
00:36:25.789 --> 00:36:30.349
general things in order to build stuff, as opposed to go deep into machine
559
00:36:30.389 --> 00:36:32.739
learning models and all of that?
If I was sitting in somewhat in front
560
00:36:32.780 --> 00:36:37.340
of someone who is taking or looking
at a junior job or something like that,
561
00:36:37.019 --> 00:36:40.300
I would say that you probably want
to take one of two tracks.
562
00:36:42.099 --> 00:36:47.170
One, if you have the educational
experience on paper and the credentials to go
563
00:36:47.610 --> 00:36:52.929
for, say, deep ai,
that just go deep on ai like you
564
00:36:52.050 --> 00:36:54.809
have. You have the master's degree
in it or you have a PhD in
565
00:36:54.889 --> 00:36:59.369
it or something like that. Go
for that track like that's an amazing it's
566
00:36:59.409 --> 00:37:04.280
incredibly high paying, it's super in
demand and there are places where you won't
567
00:37:04.280 --> 00:37:07.039
have to learn anything else but ML. That can be a career and I
568
00:37:07.039 --> 00:37:09.239
think there's probably going to be a
full career in that. There's absolutely no
569
00:37:09.360 --> 00:37:12.880
problem with that, because those are
very, very hard and but I I
570
00:37:12.920 --> 00:37:15.190
would almost exclusively say neural networks at
this point. Yeah, there's a lot
571
00:37:15.190 --> 00:37:20.389
of companies that are doing things that
are self driving cars or things like that
572
00:37:20.550 --> 00:37:22.510
where you just you need a lot
of people with that kind of knowledge to
573
00:37:22.590 --> 00:37:27.150
work on those problems because of those
are incredibly hard problems. But it is
574
00:37:27.469 --> 00:37:31.219
definitely along the lines of neural networks. I deep learning to do those and
575
00:37:31.420 --> 00:37:36.780
for this truck that is a certain
mathematical overhead which perhaps the other track,
576
00:37:36.820 --> 00:37:38.380
we haven't got to that yet,
might not have. But, for example,
577
00:37:38.860 --> 00:37:45.369
knowing enough about multi vary calculus to
understand back prop and the venishing guidient
578
00:37:45.409 --> 00:37:46.690
problem in all this stuff. Right. Oh, no, absolutely, and
579
00:37:46.969 --> 00:37:52.849
you I would expect that the interviews
for those would be very math heavy and
580
00:37:52.289 --> 00:37:55.690
very, very heavy, such that
it would almost feel like a I don't
581
00:37:55.690 --> 00:37:59.199
know, like a dissertation defense.
Yeah, in that area, for that
582
00:37:59.320 --> 00:38:02.440
truck, as you said, you'd
expect some sort of graduate work to have
583
00:38:02.559 --> 00:38:07.639
been done in math or something related. Yeah, I would expect like ninety
584
00:38:07.679 --> 00:38:09.599
five percent of people to have some
kind of like very obvious that they're going
585
00:38:09.639 --> 00:38:13.150
down that track. I know,
I'm sure there's some people who have like
586
00:38:13.309 --> 00:38:15.630
gone around that and those people are
awesome and amazing, but I think typically
587
00:38:16.190 --> 00:38:22.110
you would expect someone to be from
that perspective. The other one, if
588
00:38:22.150 --> 00:38:25.590
you don't have that like very obvious
sort of machine learning deep learning focus,
589
00:38:25.630 --> 00:38:30.780
which I don't. So this is
more my perspective. There's another field that
590
00:38:30.900 --> 00:38:35.780
is way more sort of doing data
science more generally at a company. Right,
591
00:38:35.780 --> 00:38:37.260
so instead of just being like all
I do is machine learning all day,
592
00:38:37.380 --> 00:38:40.769
there's so many other problems that need
to be solved using data science,
593
00:38:40.809 --> 00:38:45.849
whether you do like Baysi and analyzes
or you do some kind of you know,
594
00:38:45.969 --> 00:38:47.690
like random force, or even if
you use some deep learning stuff,
595
00:38:47.730 --> 00:38:52.849
but you're not doing cutting edge deem
learning stuff all day. There's some far
596
00:38:52.050 --> 00:38:55.960
more jobs, although like the the
hype and the focus is on this sort
597
00:38:57.000 --> 00:39:00.440
of AI jobs, there's far more
jobs, like fifty times as many jobs
598
00:39:00.559 --> 00:39:07.639
of people who are semi generalist data
scientists at companies solving those companies needs to
599
00:39:07.679 --> 00:39:12.789
understand what's happening in their data,
whether that's predicting when their drone should fly
600
00:39:12.909 --> 00:39:16.429
over and water the crops or predicting
when a customer I might return or,
601
00:39:16.710 --> 00:39:21.389
you know, for us, like
predicting when someone might have a particular illness
602
00:39:21.429 --> 00:39:24.460
so we could do an intervention and
prevent that illness from happening. Those kind
603
00:39:24.539 --> 00:39:29.820
of analyzes fit better for people who
sort of have a more general experience,
604
00:39:29.820 --> 00:39:32.019
because there isn't there isn't a very
type for that right. There's no Bayesian
605
00:39:32.539 --> 00:39:36.820
analyze PhD, and if you have
that you get to do Baysi analyzes and
606
00:39:36.820 --> 00:39:38.210
if you don't, you don't get
to do that. It's much more general
607
00:39:38.250 --> 00:39:42.690
so that the people who apply for
for us in those kind of roles can
608
00:39:42.730 --> 00:39:45.489
come from any perspective and can be
everything from you know, a music major,
609
00:39:45.530 --> 00:39:50.130
or can be someone with a PhD
and data science and one of the
610
00:39:50.130 --> 00:39:52.239
new programs, or can come from
a boot camp, from a PhD in
611
00:39:52.320 --> 00:39:57.440
some other crazy field, or could
not have a PhD. It's more about
612
00:39:57.519 --> 00:40:00.239
just if that kind of person fits
what we want and what we need.
613
00:40:00.599 --> 00:40:01.920
But it is I mean that.
I think that's where most people do it.
614
00:40:02.000 --> 00:40:06.159
I I say this only because I
think there's a lot of people who
615
00:40:06.159 --> 00:40:10.429
think if I know enough machine learning
and I go deep enough on machine learning,
616
00:40:10.429 --> 00:40:15.670
I'm like that's the guaranteed job,
which isn't true, because you could
617
00:40:15.670 --> 00:40:19.829
self teach a lot of machine learning
and then you apply a Google brain and
618
00:40:19.869 --> 00:40:22.980
they could be like hey, you're
just not even close because you didn't get
619
00:40:22.980 --> 00:40:25.179
a PhD in this and haven't spent, you know, like a huge amount
620
00:40:25.179 --> 00:40:28.980
of time doing that, I think. But then you could go to another
621
00:40:28.980 --> 00:40:30.380
company, take that same person and
go to another company say hey, I
622
00:40:30.420 --> 00:40:34.059
know a lot of machine learning.
They are awesome. This is like like,
623
00:40:34.380 --> 00:40:36.449
you're not going to just be doing
machine learning, but we have need
624
00:40:36.530 --> 00:40:39.489
for people who know machine learning because
our products use natural language processing and that
625
00:40:39.530 --> 00:40:43.130
kind of stuff. Come on,
join the team. You'll be great,
626
00:40:43.289 --> 00:40:45.409
and so that kind of I think
that second job has a lot more generaloius
627
00:40:45.449 --> 00:40:50.000
things where you end up working.
You end up sort of having to be
628
00:40:50.800 --> 00:40:53.440
you know, a lot more selfware
engineering skills, a lot more, you
629
00:40:53.519 --> 00:40:58.039
know, skills working with sort of
the software side of skills of working with
630
00:40:58.199 --> 00:41:01.199
cut like internal customers. So says
someone on the sales team wants some kind
631
00:41:01.199 --> 00:41:06.269
of particular analysis. That analysis is
very difficult to do. You go back
632
00:41:06.309 --> 00:41:07.110
and you do it, but then
you have to go and present it to
633
00:41:07.190 --> 00:41:09.630
a way and then work with them
to tweak the analysis in the way that
634
00:41:09.710 --> 00:41:12.750
they want. You need to be
able to work with them such as that
635
00:41:12.750 --> 00:41:15.550
they're happy with what you're delivering with
them. That kind of stuff, I
636
00:41:15.590 --> 00:41:17.940
think, is more common. I
think that's what most people do. It's
637
00:41:17.980 --> 00:41:22.619
also it's different than just saying,
Hey, I'm just going to like know
638
00:41:22.860 --> 00:41:24.860
everything about machine learning and that I
won't need to care about any other topic.
639
00:41:24.980 --> 00:41:28.619
And so, for this secondtrol,
which I think is kind of the
640
00:41:28.619 --> 00:41:31.260
lawn share of what we what we're
discussing today, there's a chicken and egg
641
00:41:31.260 --> 00:41:35.250
problem in the senset. For a
first time, dito scientist applying for their
642
00:41:35.329 --> 00:41:38.210
first job? How can they demonstrate
that they have kind of this general array
643
00:41:38.289 --> 00:41:42.730
of skills? Would it be project
Bise, like rotting a blog themselves to
644
00:41:42.849 --> 00:41:45.769
demonstrate it? Because it seems like
you need the experience in order to get
645
00:41:45.849 --> 00:41:50.360
the first job. Essentially, yeah, for us, when someone applies some
646
00:41:50.360 --> 00:41:53.679
of the best things that they can
apply with our projects that they've done or,
647
00:41:54.280 --> 00:41:57.039
you know, something at like,
I say, a boot camp or
648
00:41:57.159 --> 00:42:01.150
to maybe they're maybe they're dissertation research
or something like that, where we can
649
00:42:01.230 --> 00:42:04.989
take a look and say, oh, cool, like you've done some interesting
650
00:42:04.989 --> 00:42:07.309
stuff, you worked with some data
some interesting ways, and then for us
651
00:42:08.030 --> 00:42:13.829
we have designed a take home that
is very without giving out it any kind
652
00:42:13.829 --> 00:42:16.019
of secret to the TAKEOM is very
open. So there's many, many ways.
653
00:42:16.059 --> 00:42:19.659
I mean there's an infinite amount of
ways essentially that someone could solve it
654
00:42:19.780 --> 00:42:22.659
and how they solve it really says
a lot about them. But those kind
655
00:42:22.739 --> 00:42:27.099
of skills of being able to demonstrate, hey, like I can write software,
656
00:42:27.099 --> 00:42:30.090
like I can write you a test. I can sit down and do
657
00:42:30.210 --> 00:42:32.929
basie and I'm totally comfortable with it. Here's my blog post around how the
658
00:42:34.050 --> 00:42:37.449
certain type of Baysie analysis works in
the setting, or here's this really sweet
659
00:42:37.489 --> 00:42:40.090
data visualization that I did. That
kind of stuff is like a nice demonstration.
660
00:42:40.130 --> 00:42:45.400
I don't think it's required, but
I think if you come from them,
661
00:42:45.440 --> 00:42:47.960
say, a field that isn't known
for making a budget quantitative people,
662
00:42:49.000 --> 00:42:52.760
and I come from political science,
I think people don't think about political science
663
00:42:52.800 --> 00:42:55.119
and think role math nerds. The
more things that you can do around that
664
00:42:55.239 --> 00:43:00.030
where you can sort of demonstrate that
you have that kind of experience is better,
665
00:43:00.230 --> 00:43:06.190
because otherwise you don't hit people's biases
around what a social scientist is right,
666
00:43:06.190 --> 00:43:07.429
because someone can say, Oh,
you do political science, you must
667
00:43:07.429 --> 00:43:12.429
really love Kant and Rousseau and that's
all you're talking about all day. And
668
00:43:12.550 --> 00:43:15.539
if that's not true, like you
need to demonstrate that. It's probably not
669
00:43:15.619 --> 00:43:16.699
fair that you need to demonstrate that, but that is just what it is,
670
00:43:16.739 --> 00:43:21.619
that you need to demonstrate that.
And so things like boot camps,
671
00:43:21.980 --> 00:43:23.780
things like projects that you can run
your own blog post, a great because
672
00:43:23.780 --> 00:43:27.019
it's really easy to access them,
like you can just click, you know,
673
00:43:27.099 --> 00:43:29.849
like someone has a link in their
resume and they don't click the link.
674
00:43:29.889 --> 00:43:30.650
You say, Oh cool, this
is like a really nice you know,
675
00:43:30.690 --> 00:43:35.329
I love the way that they're thinking
about this particular problem of feature importance
676
00:43:35.409 --> 00:43:37.170
in random force or something like that
is a really nice way and I think
677
00:43:37.690 --> 00:43:43.480
it is helpful. We don't require
someone has side projects, but we are
678
00:43:43.639 --> 00:43:49.159
trying to do filtering of thousands of
candidates and so it's nice to find people
679
00:43:49.159 --> 00:43:52.480
who have who can show you right
up front that they have those kind of
680
00:43:52.559 --> 00:43:58.110
skills and can do so in a
way that is more than just a resume
681
00:43:58.269 --> 00:44:01.070
line, because, as someone who's
looked at a lot of resumes, everyone
682
00:44:01.190 --> 00:44:06.349
says that they do every skill under
the sun. Like everyone lists every skill.
683
00:44:06.389 --> 00:44:10.820
So everyone is everyone says Python and
sequel and are and machine learning and
684
00:44:12.019 --> 00:44:15.619
random forests, and everyone says every
skill, which is like. So it's
685
00:44:15.619 --> 00:44:19.260
not a very strong signal. But
if you also have, say, blog
686
00:44:19.340 --> 00:44:22.699
post about some nuance point about random
forest or some nuance point of Baysi analysis,
687
00:44:23.179 --> 00:44:25.289
that is a real thing. That's
a costly signal to me that you
688
00:44:25.369 --> 00:44:29.090
actually do know that rather than just
typing in the world in your rsume,
689
00:44:29.570 --> 00:44:31.409
and it helps, I think it
helps for me to get a handle on
690
00:44:31.690 --> 00:44:36.289
who that person is. It helps
to guide interview process in the future.
691
00:44:36.329 --> 00:44:37.760
That, like other people do,
and I don't do that. I can
692
00:44:37.840 --> 00:44:42.039
sort of say hey, like,
here's this article from this person's blog that
693
00:44:42.199 --> 00:44:45.719
probably will be a subject of an
interview, which is totally fine and probably
694
00:44:45.719 --> 00:44:46.800
helps the candid a little bit because
they can sort of stay in the area
695
00:44:46.800 --> 00:44:52.440
that they know. But it is
demonstrations of that are very helpful. Do
696
00:44:52.559 --> 00:44:54.309
I think everyone needs to do side
projects left and right? Otherwise there are
697
00:44:54.309 --> 00:44:57.710
a crappy data scientists? No,
of course, not like this. Isn't
698
00:44:58.030 --> 00:45:00.349
you don't need to live and breathe
data science to get a job in the
699
00:45:00.389 --> 00:45:04.829
data science but it is helpful to
someone who's hiring to sort of see the
700
00:45:04.869 --> 00:45:07.980
things that you've worked on and see
the things that you've done more than just
701
00:45:07.059 --> 00:45:12.019
adding that keyword to your rhythme.
And I think the other thing that running
702
00:45:12.059 --> 00:45:15.539
blogs demonstrates his the ability to take
a project through to the end and actually
703
00:45:15.579 --> 00:45:19.820
do a ride up and, on
top of that, demonstrates communication skills,
704
00:45:19.860 --> 00:45:22.449
which, of course, incredibly important
in this line of work. We had
705
00:45:22.530 --> 00:45:24.289
one candidate who I really, I
really liked. They ended up taking another
706
00:45:24.289 --> 00:45:29.329
job somewhere else, but I really
liked and they sat down and they actually
707
00:45:29.369 --> 00:45:31.889
had a github project that they were
working on. I don't think it was
708
00:45:32.010 --> 00:45:37.639
like a full python package yet,
but they had basically built an open source
709
00:45:37.679 --> 00:45:39.800
library very close to one. I
don't think they'd released it, but they
710
00:45:39.840 --> 00:45:44.400
had like a little opens ource library
for some project and they had testing in
711
00:45:44.480 --> 00:45:47.719
there and they had documentation and they
had the odjectory and python in there and
712
00:45:47.760 --> 00:45:52.510
they were importing like relative imports of
modules so they could do the tests and
713
00:45:52.550 --> 00:45:53.349
all the kind of stuff, and
it was cool, like you could look
714
00:45:53.349 --> 00:45:55.429
at that and be like, okay, I kind of like this person has
715
00:45:55.510 --> 00:46:00.070
this level right, this person has
this this amount of knowledge, because they've
716
00:46:00.070 --> 00:46:02.110
clearly written it in their a gidthub
account and I can see it. It's
717
00:46:02.110 --> 00:46:06.460
a nice signal for me. You
could do that anyway that there's no particular
718
00:46:06.460 --> 00:46:10.500
way that I like absolutely want,
but I think the resumes that our send
719
00:46:10.539 --> 00:46:15.219
the least signal to me or other
people who are hiring on devoted are the
720
00:46:15.340 --> 00:46:21.250
ones that have just here's the five
skills that I have and then kind of
721
00:46:21.289 --> 00:46:23.010
don't do any kind of explaining because
there's a lot of like cheap signals,
722
00:46:23.050 --> 00:46:27.130
like you could, I mean I
could say that I do deep learning and
723
00:46:27.210 --> 00:46:30.289
then you're like Oh really, let's
talk about that. And it turns out
724
00:46:30.289 --> 00:46:31.610
that I have not known enough about
deep learning, run and the other thing
725
00:46:31.650 --> 00:46:35.239
you mentioned in there. There are
a couple of other points, which github
726
00:46:35.280 --> 00:46:37.119
repository, I'm sure, is incredibly
helpful, I don't talk about. You
727
00:46:37.119 --> 00:46:42.480
mentioned testing and a through line through
this as being the importance of at least
728
00:46:42.559 --> 00:46:45.039
bicic software engineering stools in terms of
ditosions, and I find that that's something
729
00:46:45.119 --> 00:46:50.510
that's missing with a lot of kind
of early career data analysts and ditosionists,
730
00:46:50.550 --> 00:46:54.190
whether it be using day Buggas or
unit testing or versioning, these types of
731
00:46:54.269 --> 00:46:57.670
things, people need to work on
a bit more. I think that's key.
732
00:46:57.670 --> 00:47:00.900
If if there's anything, if there's
anything that goes through our data science
733
00:47:00.980 --> 00:47:05.500
team, it devoted when we are
looking to hire someone or when we are
734
00:47:05.539 --> 00:47:07.739
working on our own projects, is
that the stuff that we work on our
735
00:47:07.860 --> 00:47:13.099
products. We are building full product
x for people, whether they are only
736
00:47:13.139 --> 00:47:16.250
a few lines of code or,
you know, say they're something simpler like
737
00:47:16.690 --> 00:47:21.690
moving some kind of data from a
Google spreadsheet to redshift, just something really
738
00:47:21.730 --> 00:47:28.329
simple like that, or something more
complicated. We are building full products products
739
00:47:28.369 --> 00:47:30.199
for people, and thus they need
to have as much of that as we
740
00:47:30.280 --> 00:47:34.079
can have. We want testing in
there. We want things to be,
741
00:47:34.159 --> 00:47:37.719
say, lnted, we want things
to follow some kind of object oriented notion
742
00:47:37.840 --> 00:47:42.679
or, say, functional python with
the dock strings in there, the describable
743
00:47:42.719 --> 00:47:45.829
the documents are doing, say,
static typing, if we could do that.
744
00:47:45.909 --> 00:47:47.230
We're seemingly on the verge of doing
that, but not doing they get
745
00:47:47.750 --> 00:47:51.469
but that kind of stuff like.
We think of that as a software product.
746
00:47:51.469 --> 00:47:53.389
We make software products for people,
whether that's, you know, whether
747
00:47:53.389 --> 00:47:58.269
it's actually like a reusable tool or
analyzes. That's just what we do and
748
00:47:58.940 --> 00:48:04.019
I wish I thought about that more
from the start. And I think as
749
00:48:04.099 --> 00:48:07.420
data science matures, I think there's
definitely going to be this movement in the
750
00:48:07.539 --> 00:48:13.250
direction of data scientists sitting on software
engineering teams and being a member of a
751
00:48:13.329 --> 00:48:17.329
software engineering team just with a certain
specialty set of skills, and part of
752
00:48:17.369 --> 00:48:21.289
that means that you need to be
able to work with them and write code
753
00:48:21.329 --> 00:48:24.050
that they can use and write code
that they are fine to incorporate in their
754
00:48:24.090 --> 00:48:28.639
stuff, and that just means more
software engineering skills for sure. And so
755
00:48:29.039 --> 00:48:31.639
something that we've thrown around a bit
is this idea of graduate school and dissertations,
756
00:48:31.760 --> 00:48:35.480
and a question I get a lot
is from people who are thinking of
757
00:48:35.559 --> 00:48:37.599
going to Grad school and they're actually
wondering whether to go to Grud's. If
758
00:48:37.639 --> 00:48:40.000
I want to, will can ditosions
with it go to red school or with
759
00:48:40.119 --> 00:48:44.510
it to get like a diet analyst
job that I can get at that point
760
00:48:44.630 --> 00:48:47.309
and then try to progress into Ditosians
demonstrating that they have developed a bunch of
761
00:48:47.349 --> 00:48:51.590
analytical tools. Do you have any
thoughts on that from your side of the
762
00:48:51.630 --> 00:48:54.860
hiring tyble? Sure, I would
never recommend someone get a PhD. I
763
00:48:54.940 --> 00:48:58.900
mean masters might be different. Bust
will sort of. I would never recommend
764
00:48:58.900 --> 00:49:02.219
someone get a PhD because they wanted
to get a better job. Like a
765
00:49:02.340 --> 00:49:07.420
PhD's as a long and difficult process
and you got to really want to do
766
00:49:07.539 --> 00:49:09.929
a PhD in order to do a
Pahda. My PhD, like a huge
767
00:49:09.969 --> 00:49:14.369
amount of people quit, maybe like
half or more than half of people quit
768
00:49:14.809 --> 00:49:16.769
along the way and got nothing right, because if you a Pahd can be
769
00:49:17.010 --> 00:49:20.730
six years long. If you quit
after a year three, you don't get
770
00:49:20.730 --> 00:49:22.289
anything. I think you maybe they
might give you a master's is like a
771
00:49:22.409 --> 00:49:25.320
sort of a constellation price or something
that. But it is a very,
772
00:49:25.400 --> 00:49:29.559
very tough activity and it's very,
very hard and most people don't complete it
773
00:49:29.679 --> 00:49:31.880
and people have a lot of stress
around it. It is difficult and the
774
00:49:31.960 --> 00:49:36.719
final year in particular, I just
remember my bag. The final year was
775
00:49:36.880 --> 00:49:40.030
absolutely brutal. I don't want to
say I hided the work I was doing,
776
00:49:40.150 --> 00:49:44.550
but there was certainly, at the
end, a bunch of negative sentiments
777
00:49:44.590 --> 00:49:46.909
that involved, you know, the
stress and the suffering and also the sleeping
778
00:49:46.949 --> 00:49:50.469
on to my disk for the final
six months secual. Oh yeah, now
779
00:49:50.550 --> 00:49:52.579
completely. I hated my dissertation by
the end. I mean I yeah,
780
00:49:52.780 --> 00:49:55.139
the phrase that I kept on saying
over my mind is that the only good
781
00:49:55.139 --> 00:50:00.340
dissertation is a done dissertation. Yeah, just grinding through and like. But
782
00:50:00.619 --> 00:50:06.739
it is worth it because I got
to spend five years studying a topic that
783
00:50:06.860 --> 00:50:09.329
I really, really cared about and
I got to go as I mean imagine
784
00:50:09.730 --> 00:50:12.809
being able to go. I mean
you don't need to imagine, but like
785
00:50:13.130 --> 00:50:16.250
if you're a staph, imagine being
given five years to go as deep into
786
00:50:16.250 --> 00:50:20.969
a topic as you could possibly go. Like there is no level of deepness.
787
00:50:21.050 --> 00:50:22.800
That is okay, keep on going
deeper over and over and over now,
788
00:50:23.320 --> 00:50:29.039
and that is super interesting and super
cool and it's a it totally changes
789
00:50:29.079 --> 00:50:32.079
your thinking and it totally changes you
as a person and it's not a good
790
00:50:32.119 --> 00:50:35.789
way to get it, like as
a stepping stone to getting another job.
791
00:50:36.110 --> 00:50:39.750
It really just is it. You
could have much better spent that time doing
792
00:50:39.869 --> 00:50:45.070
other things that are directly related to
getting an interview then going off on some
793
00:50:45.230 --> 00:50:52.260
crazy quest to study some amphibian in
the whatever, in the South Polynesian islands
794
00:50:52.380 --> 00:50:54.739
or something like that. Like there's
definitely better ways of just getting a pay
795
00:50:54.780 --> 00:50:58.420
raise and them, you know,
more advanced suf so is it a viable
796
00:50:58.460 --> 00:51:00.579
option to enter as a data analyst
and try to progress to Ditas on?
797
00:51:00.780 --> 00:51:04.260
So? So I think there's probably
two tracks. One, if you can
798
00:51:04.329 --> 00:51:09.610
find the right place, you can
absolutely go from data analysts into data science.
799
00:51:09.690 --> 00:51:13.809
And I think it more it's about
trying to find the place where there
800
00:51:13.849 --> 00:51:16.130
isn't a firm division between the two. Right. So, like if I
801
00:51:16.690 --> 00:51:20.960
joined a larger company as a data
analyst, as a junior day Lanas,
802
00:51:21.039 --> 00:51:24.119
I would try to find chances where
I could do more software engineering stuff.
803
00:51:24.320 --> 00:51:28.079
I would start to work on more
complicated project. I would try to see
804
00:51:28.159 --> 00:51:29.880
like okay, cool, like yeah, sure, I can make this this
805
00:51:30.440 --> 00:51:32.750
quick analysie, but can I do
it better using Bayesian or somebody that?
806
00:51:32.829 --> 00:51:36.909
Like you'd but you sort of have
to be self motivated to gain more of
807
00:51:36.989 --> 00:51:39.269
those skills. Another option is like
a master's degree, which could be one
808
00:51:39.269 --> 00:51:44.710
or two years and can expose you
to a lot of that kind of stuff
809
00:51:45.429 --> 00:51:49.420
in a relatively quick amount of time
and then you get out and then you
810
00:51:49.500 --> 00:51:51.900
can you know, I think you
can do like a step up, like
811
00:51:51.980 --> 00:51:55.659
we don't care around about education,
like you're education. We don't care about
812
00:51:55.659 --> 00:52:00.500
degrees at devoted there's no requirement for
some kind of degree. But there is
813
00:52:00.579 --> 00:52:02.530
lots of skills that people would learn, say in a data science master's program
814
00:52:02.570 --> 00:52:07.010
or any kind of quantitative master's program
Masters and mathematics or somebody, that they
815
00:52:07.050 --> 00:52:12.010
could really take advantage of and it
can be a big step up for people
816
00:52:12.210 --> 00:52:15.320
to do that and I think it's
probably relatively cheap. I don't really know
817
00:52:15.440 --> 00:52:17.480
exactly, but I would either.
You know. So you can either do
818
00:52:17.559 --> 00:52:21.880
the self learning path, which is
you have to be heal, scrappy,
819
00:52:21.880 --> 00:52:23.480
you have to kind of find opportunities
where they are and move up through that
820
00:52:23.639 --> 00:52:28.519
and probably have to switch jobs a
few times because the people hired you as
821
00:52:28.519 --> 00:52:30.829
a basic data analyst and all of
a sudden you're doing basi and left,
822
00:52:30.869 --> 00:52:32.349
right and center, and then you
want to be paid like you're doing basing
823
00:52:32.750 --> 00:52:36.349
Baysian work all the time, and
then you go find another job that focus
824
00:52:36.469 --> 00:52:38.309
on basing and then you move up
from there and then the other one is
825
00:52:39.030 --> 00:52:43.590
getting that master's degree and then you
know, going back and trying to find
826
00:52:43.590 --> 00:52:45.099
another job after that, saying that
you have you know, you've got this
827
00:52:45.219 --> 00:52:51.260
experience of that, you know more
about the formal training around some of this
828
00:52:51.340 --> 00:52:52.539
kind of stuff, and it doesn't
help you with a lot of business cases,
829
00:52:52.619 --> 00:52:57.300
but it does help you that you
can say you understand the problems around
830
00:52:57.420 --> 00:53:00.210
baysing analysis or the problems around some
kind of machine learning model that then you
831
00:53:00.250 --> 00:53:02.690
can apply in a real way.
And if someone comes to you and knows
832
00:53:02.769 --> 00:53:06.369
these they these types of things,
but doesn't necessarily know how it works in
833
00:53:06.369 --> 00:53:08.690
the health space, I presume a
good strategy there is to say hi,
834
00:53:08.769 --> 00:53:10.929
I know all these techniques, I
don't know a lot about health, but
835
00:53:12.409 --> 00:53:15.639
I would love to learn about this
stuff. To demonstrate like a passion for
836
00:53:15.719 --> 00:53:19.480
the domain expertise in essentially, this
is the first health insurance company I've ever
837
00:53:19.559 --> 00:53:22.800
worked for and we do a lot
of learning around that, around that area
838
00:53:22.880 --> 00:53:25.719
where. So I remember, I
think my second day at devoted, they're
839
00:53:25.760 --> 00:53:28.309
like hey, you know, you
should come to this meeting, and I
840
00:53:28.349 --> 00:53:30.750
came to the meeting and it's just
for data scientist sitting in a room with
841
00:53:30.909 --> 00:53:35.190
a doctor explaining how medical coding works. So like, you know, like
842
00:53:35.309 --> 00:53:37.949
what is the code for someone who
breaks their hip and how does that relate
843
00:53:37.989 --> 00:53:39.309
to the other code and how people
are build on nothing, and it's just
844
00:53:39.349 --> 00:53:43.219
a doctor is sitting around, you
know, telling us all how all these
845
00:53:43.300 --> 00:53:46.659
things work and it was so educational
and it was so new that, you
846
00:53:46.739 --> 00:53:49.780
know, it's a big part of
it, absolutely, and I really like
847
00:53:49.980 --> 00:53:52.099
the way that this conversation is going
in terms of providing a variety of different
848
00:53:52.139 --> 00:53:58.369
paths, from the machine learning to
the first time data scientists demonstrating what they've
849
00:53:58.369 --> 00:54:00.650
done through project, so through quantititive
research, through to the self learning approach.
850
00:54:00.809 --> 00:54:04.289
And for anyone out there who wants
to take the self learning approach,
851
00:54:04.289 --> 00:54:10.480
I've actually I've heard that data campcom
is an incredible place. Yeah, it
852
00:54:10.599 --> 00:54:14.159
was actually Chris who told me that
before we started recording. He was like,
853
00:54:14.199 --> 00:54:15.159
Guy, that's it. I was
just like data camp, you got
854
00:54:15.239 --> 00:54:17.079
to use day again. Oh No, no one's paying for me for this.
855
00:54:17.599 --> 00:54:21.639
I'm a I'm ambivalent. Yeah,
this is a sponsored yeah, and
856
00:54:21.719 --> 00:54:27.269
this is not sponsored by Facebook,
EI, though, or basing analysis exactly.
857
00:54:27.309 --> 00:54:30.269
Yeah, Gelman, Gilman isn't paying
us. Hey, that's a good
858
00:54:30.510 --> 00:54:35.469
that's a good idea actually. So
I'm wondering if there are any like any
859
00:54:35.550 --> 00:54:38.219
advice, things that you love people
doing when they come into interviews. All
860
00:54:38.260 --> 00:54:42.059
that, you think of the worst
things that people could do. I'm just
861
00:54:42.179 --> 00:54:45.019
any general tidbits of advice the first
time in interview ways from your side of
862
00:54:45.019 --> 00:54:49.940
the tyble. That's a good question, I think for us. I'm trying
863
00:54:49.980 --> 00:54:52.300
to say us, because I don't
want to say it's just me, because
864
00:54:52.300 --> 00:54:53.449
that's biased, like that would be
biased if I was just like well,
865
00:54:53.489 --> 00:54:55.730
maybe, actually, well, those
we hire as a team. So it's
866
00:54:55.769 --> 00:54:58.929
not just me, obviously siring,
but on the persons. Any way,
867
00:54:58.929 --> 00:55:00.849
I will say what I what I
tend to have like think rather than like
868
00:55:00.969 --> 00:55:04.050
us as a team, because I
don't know what they are biased towards.
869
00:55:04.489 --> 00:55:09.000
For me, I'm a big fan
of people who are excited about learning new
870
00:55:09.039 --> 00:55:14.000
things and excited about data, because
I genuinely enjoy what I do. I
871
00:55:14.519 --> 00:55:19.039
enjoy learning a new technique. I
enjoy sitting around with, you know,
872
00:55:19.159 --> 00:55:22.710
a new book around some kind of
analysis or an old book around some analysis
873
00:55:22.750 --> 00:55:24.909
that I don't know very well,
and I really, really enjoy that.
874
00:55:25.869 --> 00:55:30.989
And I think when you're coming in
as a junior person and you can sort
875
00:55:30.989 --> 00:55:32.789
of say, Hey, I don't
know how a lot of this stuff works,
876
00:55:34.070 --> 00:55:37.019
but I am really, really interested
in what it is and I know
877
00:55:37.139 --> 00:55:38.780
where I want to be in five
years and that involves a huge amount of
878
00:55:38.820 --> 00:55:43.139
learning, that is exactly what we
want. Like we are not hiring you
879
00:55:43.219 --> 00:55:46.300
because we're junior. We're hiring you
because we think that you could be senior
880
00:55:46.659 --> 00:55:50.969
with some training, with some mentorship, with some projects and that kind of
881
00:55:50.969 --> 00:55:53.130
stuff. Like we do not have
want you to be junior forever. We
882
00:55:53.250 --> 00:55:57.969
want you to be senior and like
we acknowledge that the more we train you,
883
00:55:58.050 --> 00:56:00.210
the more you might leave, and
that's totally okay with us. Like,
884
00:56:00.570 --> 00:56:02.730
come joined of a did be a
junior person for a while, enjoy
885
00:56:02.809 --> 00:56:06.880
yourself, like learn a huge amount
of stuff, you really excited what you
886
00:56:06.920 --> 00:56:08.079
do and then and then go find
a better job somewhere else. is a
887
00:56:08.119 --> 00:56:12.519
completely reasonable like thing that we don't
have expectations that you that you don't do
888
00:56:12.639 --> 00:56:15.559
that, but it is I want
someone to be excited about it. That
889
00:56:15.760 --> 00:56:20.829
excitement can come out in various ways. So people who have lots of side
890
00:56:20.869 --> 00:56:22.750
projects obviously like that's a signal that
people used to be like I love it
891
00:56:22.829 --> 00:56:25.269
in my spare time, but other
people don't have a lot of spare time.
892
00:56:25.750 --> 00:56:29.190
So you know that's not like.
That can be one signal, but
893
00:56:29.230 --> 00:56:30.989
it's not the only thing that we
care about. If you just come in
894
00:56:30.110 --> 00:56:34.860
and are really, really excited about
it and I can tell that you have
895
00:56:34.980 --> 00:56:37.500
spent a lot of time thinking about
it and do you have a lot of
896
00:56:37.579 --> 00:56:39.940
knowledge around it, and like you
like an a lot of knowledge around it
897
00:56:40.099 --> 00:56:43.699
for what I would expect someone at
your level to have. Like that's a
898
00:56:43.699 --> 00:56:45.300
nice sign that you just you really
care about this one thing. You might
899
00:56:45.340 --> 00:56:49.050
not have knowledge around everything, but
you might have this one thing that's like
900
00:56:49.090 --> 00:56:51.769
I just I think random forests is
super cool. I spent a lot of
901
00:56:51.809 --> 00:56:53.530
time reading about them. Like I
don't know deep learning, I don't know
902
00:56:53.610 --> 00:56:58.769
selfare engineering, I don't know anything
except for like I'm just like super interested
903
00:56:58.769 --> 00:57:00.250
in this one thing. Like that's
kind of cool. I think that that
904
00:57:00.440 --> 00:57:06.000
excitement bleeds off on to me,
possibly on to other people who interviewing,
905
00:57:06.039 --> 00:57:09.400
but definitely definitely for me, because
if you can't get the job based on
906
00:57:09.519 --> 00:57:13.519
your experience, which, when you
are more experienced, you could just get
907
00:57:13.519 --> 00:57:17.190
the job because you're experienced, you
kind of need another strategy and one of
908
00:57:17.230 --> 00:57:22.670
the strategies is just saying that you
have the right attitude and you're excited to
909
00:57:22.750 --> 00:57:24.349
do it and you know you can
learn a lot and you could be a
910
00:57:24.429 --> 00:57:27.590
good member of the team that people
want to work with, and that's a
911
00:57:27.630 --> 00:57:30.340
great way of doing it. If
if you don't have the ten years under
912
00:57:30.340 --> 00:57:31.940
your belt or something, so sure
you need something that differentiates you right,
913
00:57:32.139 --> 00:57:36.980
differentiating fact on. I understand that
one of the hard parts for junior data
914
00:57:37.019 --> 00:57:43.500
scientists is that their resumes often look
a lot alike because they go what person
915
00:57:43.579 --> 00:57:45.849
went to a boot camp, someone
else went to a different boot camp.
916
00:57:45.969 --> 00:57:50.010
Someone else, you know, like
was a math major, someone else like
917
00:57:50.250 --> 00:57:53.050
did this mathematic project, someone else
wrote one research paper. Like you know,
918
00:57:53.130 --> 00:57:57.889
there's a lot of like signals that
are pretty much the same, and
919
00:57:58.050 --> 00:58:00.320
so the way that people can distinguish
themselves of I think, at least in
920
00:58:00.400 --> 00:58:05.920
my mind, is to having some
enjoyment for what you do, because we
921
00:58:06.039 --> 00:58:08.000
want you to enjoy it and therefore
learn more at it. You know,
922
00:58:08.079 --> 00:58:10.400
learn more about doing it. You
don't need to grind them, you know,
923
00:58:10.440 --> 00:58:13.909
you don't need to burn the midnight
oil to do it. You could
924
00:58:13.909 --> 00:58:15.230
totally work down to five with the
rest of us, but we want you
925
00:58:15.269 --> 00:58:21.789
to be interested in it. And
if we every higher, now that I'm
926
00:58:21.789 --> 00:58:24.230
sitting on this side of table,
every higher is a bet. And for
927
00:58:24.389 --> 00:58:28.699
the junior person, the best bet
that we could make is that we are
928
00:58:28.739 --> 00:58:32.940
hiring you and you don't know everything
we wish you knew, but that in
929
00:58:34.099 --> 00:58:37.179
two years you could know a like, a big portion of what we wish
930
00:58:37.219 --> 00:58:39.739
you knew. As a senior person
right, like. I mean, you
931
00:58:39.739 --> 00:58:42.730
wouldn't be senior tears, but you
get my point right. Like you,
932
00:58:42.969 --> 00:58:46.250
you can be so much more capable
and so much more of a resource for
933
00:58:46.369 --> 00:58:49.929
the team. So we come in
and hire you when your junior and then
934
00:58:49.929 --> 00:58:52.570
all of a sudden you're mid level
and your kick and butt, and then
935
00:58:52.610 --> 00:58:54.210
we try to retain you because we
want you to stay there because you're so
936
00:58:54.250 --> 00:58:58.840
awesome that that part is like a
big thing. Yeah, and I suppose
937
00:58:58.880 --> 00:59:02.000
one thing I'd like to guide your
opinion on is the difference between what you
938
00:59:02.079 --> 00:59:06.760
learn self learning or even in research
in terms of, you know, using
939
00:59:06.840 --> 00:59:09.070
by Jin inference, machine learning,
whatever it might be, the difference between
940
00:59:09.389 --> 00:59:13.550
the types of tools and techniques you
use there, which might be importing cs
941
00:59:13.590 --> 00:59:17.429
phase and then doing machine learning in
production in a company, and you might
942
00:59:17.510 --> 00:59:22.670
have all these things that you haven't
actually been exposed to before by doing cago
943
00:59:22.710 --> 00:59:25.219
competitions, for example. I'll give
another example, like one of the things
944
00:59:25.500 --> 00:59:31.099
they is very hard to learn is
data engineering. So data scientists are different
945
00:59:31.099 --> 00:59:35.340
than data engineers, of course,
but there's a reason that there isn't a
946
00:59:35.340 --> 00:59:38.250
bunch of data engineering book camps because
to do day engineering you basically need a
947
00:59:38.369 --> 00:59:42.730
production system to learn that skill,
like you need to have millions of data
948
00:59:42.769 --> 00:59:45.090
points flowing around and all this kind
of stuff where you did actually do stuff
949
00:59:45.130 --> 00:59:47.210
with that. And so if you
don't learn it on the job, like
950
00:59:47.289 --> 00:59:52.329
there's relatively few areas that you could
learn that on your own, and I
951
00:59:52.409 --> 00:59:55.719
think there's some parallels to data science. For there's just some some things that
952
00:59:55.840 --> 01:00:00.039
are hard to do, as you
know, a selfdirector project. I would
953
01:00:00.079 --> 01:00:07.469
recommend that people do side projects or
do learning projects that take advantage of those.
954
01:00:07.510 --> 01:00:12.909
So like instead of, say,
instead of importing a CSD with the
955
01:00:12.949 --> 01:00:15.590
data like loaded into a database and
then pull it down and then pull it
956
01:00:15.630 --> 01:00:19.309
down once an hour or something like
that, just to like get more experience
957
01:00:19.429 --> 01:00:22.139
with that. And there's no I
mean these tools are either free or cheap.
958
01:00:22.820 --> 01:00:25.340
So if you have like a small
amount of data, so, for
959
01:00:25.420 --> 01:00:30.380
example, like there's Amazon Athena,
which is like a way of kind of
960
01:00:30.500 --> 01:00:32.460
search, like it's basically, I'd
like a way to do quarries on big
961
01:00:32.460 --> 01:00:37.650
data. You can totally upload just
a little bit of data to that and
962
01:00:37.730 --> 01:00:39.489
then you pay by quarry. So
you could pay like thirty cents to do
963
01:00:39.530 --> 01:00:43.690
a quarry and so you do like
thirty quarries and you kind of understand what's
964
01:00:43.730 --> 01:00:45.530
happening and then you kind of use
it for a project and then you drop
965
01:00:45.610 --> 01:00:47.809
that project and you never pay for
it again, but you have that experience
966
01:00:47.809 --> 01:00:50.920
into your belt and then so when
someone comes to you and says, Hey,
967
01:00:51.239 --> 01:00:53.159
I saw you, you know have
this aws experience communice. Yeah,
968
01:00:53.159 --> 01:00:55.880
because I, you know, I
did this kind of project we use Athena.
969
01:00:57.480 --> 01:01:00.440
I, you know, had to
work out that I am security rolls
970
01:01:00.519 --> 01:01:02.760
of me and then how I put
it onto a server and how I etcetera,
971
01:01:02.800 --> 01:01:05.630
etc. Etc. And all that
kind of stuff you could do,
972
01:01:06.309 --> 01:01:08.670
because with things like a ws you
just pay for usage. Just you,
973
01:01:09.030 --> 01:01:12.469
you know, just have a small
amount of data when you do it,
974
01:01:12.550 --> 01:01:15.230
but you do have that experience.
That is really useful. That that's that's
975
01:01:15.230 --> 01:01:16.670
a good signal that, like,
Hey, you, you are really interested
976
01:01:16.670 --> 01:01:22.300
in the kind of stuff. If
I release you onto our system that costs
977
01:01:22.579 --> 01:01:25.139
tens of thousands of dollars a month, like you can sit down and do
978
01:01:25.420 --> 01:01:29.420
some really interesting stuff and you would
be interested that. I think that's great.
979
01:01:29.460 --> 01:01:32.289
I I would recommend that if you
really wanted to stand out, you
980
01:01:32.289 --> 01:01:37.730
should try to identify the skills that
are harder for people to get right,
981
01:01:37.769 --> 01:01:44.730
because it's very easy to download a
CSV and then do some group by statements
982
01:01:44.849 --> 01:01:47.519
and pandas or something like that.
And that's why does fiftyzero tutorials that do
983
01:01:47.599 --> 01:01:51.719
that, and that's why my home
page has a bunch of tutorials on doing
984
01:01:51.800 --> 01:01:53.360
that. And that is useless,
rough. But to stand out if you
985
01:01:53.440 --> 01:01:58.679
had some kind of varience in something
that was harder for to set up and
986
01:01:58.880 --> 01:02:00.989
something that was harder to do.
I think that's a nice way of sort
987
01:02:01.030 --> 01:02:02.989
of saying like no, no,
no, I'm super serious about this,
988
01:02:04.110 --> 01:02:07.750
like I try to do this thing
that was more difficult and it's more in
989
01:02:07.869 --> 01:02:09.789
line with like what a real businesses
would do, just add a much,
990
01:02:09.829 --> 01:02:13.429
much smaller scale, but I'm still
using the same technologies. I mean,
991
01:02:13.429 --> 01:02:15.670
that's awesome, that's great. So, Chris, I want to ask you
992
01:02:15.139 --> 01:02:19.500
what one of your favorite Dietosaians.
He techniques of methodolog Jesus. But before
993
01:02:19.539 --> 01:02:23.260
that you've said random forest enough.
My first question is random forests or support
994
01:02:23.260 --> 01:02:29.139
vector machines? Always random. For
as who who uses support vector machines,
995
01:02:29.179 --> 01:02:31.409
I don't think, like, I
don't think there's ever. I cannot think
996
01:02:31.409 --> 01:02:37.969
of a single case of someone who
has used a support vector machine in production.
997
01:02:38.210 --> 01:02:43.809
So we're victor. Machines are mathematically
cool. They're just super cool.
998
01:02:43.889 --> 01:02:46.280
Like when you explain how they work, it's just super interesting. It's a
999
01:02:46.320 --> 01:02:51.599
very clever technique, but random forests
are like, if you need to get
1000
01:02:51.679 --> 01:02:57.159
something done, a random forest like
just seemingly works out of the box in
1001
01:02:57.320 --> 01:03:01.190
so many ways and so many cases
very, very, very easily that it
1002
01:03:01.349 --> 01:03:05.349
is definitely the one of the best
starting points. I think a random forest
1003
01:03:05.469 --> 01:03:08.590
is definitely one of the best starting
points for a lot of a lot of
1004
01:03:08.710 --> 01:03:12.469
modeling, and then I would kind
of go from there and decide if he
1005
01:03:12.550 --> 01:03:15.579
wanted to, you know, make
their random forest more complex or add more
1006
01:03:15.619 --> 01:03:19.380
feature engineering or change your model up
or something like that. But I would
1007
01:03:19.420 --> 01:03:22.099
definitely I'm a big random force fan
for sure, and I actually I was
1008
01:03:22.219 --> 01:03:24.260
roofing off, I I was a
couple of weeks ago someone tweeted out,
1009
01:03:24.699 --> 01:03:29.090
does anybody use support vector machines for
anything? And you were and you were
1010
01:03:29.369 --> 01:03:31.449
you just replied, know, yeah, I think like, I don't know
1011
01:03:31.530 --> 01:03:35.849
who uses that. I'd I cannot
think of a single time the someone which
1012
01:03:36.130 --> 01:03:43.320
choose a support vector machine over over
anything. I don't know, I don't
1013
01:03:43.320 --> 01:03:45.199
know. I don't see that that
one moment where it's like the killer,
1014
01:03:45.239 --> 01:03:50.000
killer model to use. Someone's going
to listen to this and send me some
1015
01:03:50.320 --> 01:03:52.400
kind of article that it, you
know, like they can't one. They
1016
01:03:52.480 --> 01:03:54.949
cured some kind of cancer using some
kind of supportmentcas you to say, but
1017
01:03:55.110 --> 01:03:59.030
right I can't to them. I
cannot think of an example off the top
1018
01:03:59.070 --> 01:04:02.309
of my head. I always considered
a supporvector machine more as sort of a
1019
01:04:02.389 --> 01:04:05.750
teaching function for like, Hey,
here's one of the techniques that people tried.
1020
01:04:06.590 --> 01:04:11.300
It's mathematically genius. I think it
explains a lot of the concepts of,
1021
01:04:11.579 --> 01:04:14.179
you know, like a decision line
between groups and that kind of stuff
1022
01:04:14.219 --> 01:04:16.139
really well for sure. And what
happens when people are on the wrong side
1023
01:04:16.139 --> 01:04:18.059
of the DEC like when a data
point is on the wrong side of the
1024
01:04:18.139 --> 01:04:20.579
decision mine, all that kind of
stuff. I think that's totally, totally
1025
01:04:20.579 --> 01:04:27.010
useful. Also, I've I've never
seen someone use it in production. So
1026
01:04:27.449 --> 01:04:29.969
do you have a funal, cool
co action for our listeners out there?
1027
01:04:30.090 --> 01:04:35.090
Sure, I mean one. You
should completely apply for the jobs that devoted.
1028
01:04:35.170 --> 01:04:38.530
This is not actually an attempt for
me to get this is not a
1029
01:04:38.679 --> 01:04:42.199
tempt for me to get people to
apply for jobs voted. But Hey,
1030
01:04:42.280 --> 01:04:45.559
we are a awesome company. Come
work with DJ Patil and myself and a
1031
01:04:45.559 --> 01:04:48.840
number of great people. That is
it'll be fun, it'll be great and
1032
01:04:49.000 --> 01:04:53.110
I hope you apply. Incredible.
It will total link in the show notes
1033
01:04:53.110 --> 01:04:55.389
as well too. Awesome, see
look at this. Finally, Guy,
1034
01:04:55.469 --> 01:04:57.349
can I expense? I mean,
I'm not really paying any money for this,
1035
01:04:57.429 --> 01:04:59.309
but I feel like I should expend, like I'm I don't know,
1036
01:04:59.349 --> 01:05:01.389
like a lunch because of this or
something. Absolutely, Let's do that.
1037
01:05:02.030 --> 01:05:04.309
This would be my other, more
general call to action, I think,
1038
01:05:04.309 --> 01:05:09.340
for someone is interested in data science, think very carefully around the skills that
1039
01:05:09.420 --> 01:05:13.860
you're trying to develop and the skills
that other people who are applying or trying
1040
01:05:13.900 --> 01:05:17.780
to develop and sit down and and
think about ways that you can apply those
1041
01:05:17.820 --> 01:05:23.409
skills in a way that both demonstrates
that you have those skills because and or
1042
01:05:23.489 --> 01:05:27.690
that you're learning those skills or something
bad, but has some kind of real
1043
01:05:27.809 --> 01:05:31.449
impact in society around us. Right
there is there is so much free data
1044
01:05:31.530 --> 01:05:38.119
out there. There's so many really, really, really interesting projects that you
1045
01:05:38.199 --> 01:05:42.840
can do by yourself, on your
own, with basically no money, that
1046
01:05:42.960 --> 01:05:45.599
you can sit down and say hey, like I have. You know,
1047
01:05:45.679 --> 01:05:47.559
I've used random for us in this
really strong way, but I've also used
1048
01:05:47.599 --> 01:05:53.230
it in a way that I whatever
detected some in crazy cool thing, detected
1049
01:05:53.510 --> 01:05:57.630
a parking tickets in New York or
detected like some kind of I don't know,
1050
01:05:57.829 --> 01:06:00.349
like some kind of thing around like
police, police shootings, or some
1051
01:06:00.389 --> 01:06:02.590
kind of thing around crime or some
kind of anything like that. Like,
1052
01:06:02.750 --> 01:06:05.739
I think there's so many really interesting
things that you can do with data science.
1053
01:06:05.820 --> 01:06:11.059
This is a applied field, and
so learn things in data science,
1054
01:06:11.179 --> 01:06:14.099
that's a big part of it,
but then also apply it in your home
1055
01:06:14.179 --> 01:06:17.769
life and in the projects that interest
you. That is the ultimate thing that
1056
01:06:17.809 --> 01:06:20.889
I want to talk about in every
single interview. It's just the cool things
1057
01:06:20.969 --> 01:06:25.010
that they've done and use data for, and I want to see that spark
1058
01:06:25.090 --> 01:06:28.050
of excitement in the things that they've
done, because then I want to work
1059
01:06:28.090 --> 01:06:30.530
with them, because if they're say
excited about it, I'm excited about it
1060
01:06:30.570 --> 01:06:31.730
and it makes you want to go
to work every day. So and this
1061
01:06:31.849 --> 01:06:34.400
has been a great conversation, Chris. So thank you so much for coming
1062
01:06:34.440 --> 01:06:38.519
on the show. Thank you,
thank you for having me such a pleasure.
1063
01:06:43.800 --> 01:06:46.429
Thanks for joining the conversation with Chris
Alban about getting your first data science
1064
01:06:46.469 --> 01:06:50.269
job. Chris made it clear that
one thing you need to do is provide
1065
01:06:50.309 --> 01:06:55.869
a differentiating factor in your resume and
interviews. When most resumes state the same
1066
01:06:55.909 --> 01:06:59.949
skill set, such as our or
python sequel machine learning, how will you
1067
01:07:00.110 --> 01:07:02.219
stand apart from the pack? One
Way, although not the only way,
1068
01:07:02.539 --> 01:07:05.820
is to have a small portfolio of
projects you've worked on, whether in a
1069
01:07:05.860 --> 01:07:10.900
github repository or in a blog.
Another way to stand apart is to have
1070
01:07:10.940 --> 01:07:15.340
a bit of experience with projects and
techniques that aren't so commonplace. For example,
1071
01:07:15.579 --> 01:07:18.250
instead of importing your data as a
CSV loaded into a database and then
1072
01:07:18.289 --> 01:07:21.090
pull it down and then pull it
down at regular intervals, or get up
1073
01:07:21.090 --> 01:07:26.369
to speed with using Amazon web services, for example. Yet another way is
1074
01:07:26.530 --> 01:07:30.440
just to be super duper excited about
the organizations you're applying to and the techniques
1075
01:07:30.599 --> 01:07:35.519
you like to use to answer questions. Chris also gave the great advice of
1076
01:07:35.639 --> 01:07:41.639
looking for positions in mid to larger
organizations and not in smaller, scrappier startups.
1077
01:07:41.880 --> 01:07:45.909
In midto larger orgs you'll be able
to learn a lot develop your career
1078
01:07:45.309 --> 01:07:49.030
and won't need to be too concerned
with all the data infrast structure challenges at
1079
01:07:49.070 --> 01:07:54.510
that point. More generally, look
for orgs that will foster your career aspirations
1080
01:07:54.550 --> 01:07:58.739
and will invest in you. I'd
suggest making that a first order principle of
1081
01:07:58.820 --> 01:08:01.820
your job search. Next week I
have the great pleasure of speaking with no
1082
01:08:01.940 --> 01:08:08.219
Ami Dershey, a senior inventive scientist
at a TNT labs within the data science
1083
01:08:08.380 --> 01:08:13.250
and Ai Research Organization, doing lots
of science with lots of data. Will
1084
01:08:13.250 --> 01:08:15.850
be talking about her work at atnt
labs, research, the mission of which
1085
01:08:15.970 --> 01:08:21.529
is to look beyond today's technology solutions
to invent disruptive technologies that meet future needs.
1086
01:08:21.970 --> 01:08:28.079
ATNT labs works on a multitude of
projects, from product development at atnt
1087
01:08:28.560 --> 01:08:32.640
to how to combat bias and Fannessy
issues in targeted advertising and creating drones for
1088
01:08:32.800 --> 01:08:40.239
Cell Tower Inspection. Research that leverages
AIML and video analytics. Will be talking
1089
01:08:40.279 --> 01:08:45.069
about some of the work no AMI
does, from characterizing human mobility from cellular
1090
01:08:45.229 --> 01:08:50.069
network data to characterizing their mobile network
to analyze how it's topology compares to other
1091
01:08:50.229 --> 01:08:57.100
real social networks reported and understanding TV
viewership and how engage people are in different
1092
01:08:57.180 --> 01:09:00.899
shows. All this and, as
always, more next week. I'm your
1093
01:09:01.060 --> 01:09:04.699
host, Hugo bound Anderson. You
can follow data camp on twitter at data
1094
01:09:04.739 --> 01:09:09.539
camp and me at Hugo bound.
You can find all our episodes and show
1095
01:09:09.619 --> 01:09:13.409
notes at Data Campcom community podcast
