TRANSCRIPT 🔗
1
00:00:00,000 --> 00:00:04,800
You can speak to GPT-4 like you would speak to another human.
2
00:00:04,800 --> 00:00:06,440
We know how to communicate.
3
00:00:06,440 --> 00:00:07,760
I don't speak protein.
4
00:00:07,760 --> 00:00:10,080
I don't know anyone else that does.
5
00:00:10,080 --> 00:00:11,840
Do I spell it out in amino acid?
6
00:00:15,600 --> 00:00:17,640
How did the best machine learning practitioners
7
00:00:17,640 --> 00:00:19,600
get involved in the field?
8
00:00:19,600 --> 00:00:22,080
What challenges have they faced?
9
00:00:22,080 --> 00:00:24,240
What has helped them flourish?
10
00:00:24,240 --> 00:00:26,020
Let's ask them.
11
00:00:26,020 --> 00:00:28,460
Welcome to Learning from Machine Learning.
12
00:00:28,460 --> 00:00:31,240
I'm your host, Seth Levine.
13
00:00:31,240 --> 00:00:34,120
Hello, and welcome to Learning from Machine Learning.
14
00:00:34,120 --> 00:00:37,520
On this episode, we have a very special guest, Dr. Michelle
15
00:00:37,520 --> 00:00:38,760
Gill.
16
00:00:38,760 --> 00:00:42,120
She's the tech lead and applied research manager at NVIDIA.
17
00:00:42,120 --> 00:00:45,880
She works on projects like BioNemo and BioFoundation
18
00:00:45,880 --> 00:00:48,800
models, frameworks, and inference service
19
00:00:48,800 --> 00:00:51,360
for AI-assisted drug discovery.
20
00:00:51,360 --> 00:00:55,920
Recently, she gave the keynote at the most recent PiData NYC.
21
00:00:55,920 --> 00:00:57,800
It was one of my favorite talks, and I'm
22
00:00:57,800 --> 00:01:00,920
thrilled to have you on the podcast.
23
00:01:00,920 --> 00:01:02,380
Thank you for the invitation, Seth.
24
00:01:02,380 --> 00:01:03,560
It is awesome to be here.
25
00:01:03,560 --> 00:01:05,120
I'm really excited.
26
00:01:05,120 --> 00:01:06,280
Me too.
27
00:01:06,280 --> 00:01:09,240
So you are a little unique for the podcast
28
00:01:09,240 --> 00:01:11,320
with my background in NLP.
29
00:01:11,320 --> 00:01:13,640
So yeah, why don't you kick us off?
30
00:01:13,640 --> 00:01:16,000
What's your background, and what initially attracted you
31
00:01:16,000 --> 00:01:17,240
to machine learning?
32
00:01:17,240 --> 00:01:17,800
Yep.
33
00:01:17,800 --> 00:01:21,920
So proud that I don't fit many molds,
34
00:01:21,920 --> 00:01:23,800
where it is a badge of honor.
35
00:01:23,800 --> 00:01:27,440
I did my PhD in structural biology and biophysics
36
00:01:27,440 --> 00:01:28,480
at Yale.
37
00:01:28,480 --> 00:01:31,240
And then for postdoc, I studied enzyme dynamics
38
00:01:31,240 --> 00:01:33,440
with a technique called nuclear magnetic resonance
39
00:01:33,440 --> 00:01:37,400
spectroscopy, NMR for short.
40
00:01:37,400 --> 00:01:40,260
I was particularly unique in that I didn't really
41
00:01:40,260 --> 00:01:41,560
do computational work.
42
00:01:41,560 --> 00:01:44,100
For my scientific career, I was what
43
00:01:44,100 --> 00:01:45,920
we would call a wet lab scientist.
44
00:01:45,920 --> 00:01:49,040
I actually did experiments, expressed proteins,
45
00:01:49,040 --> 00:01:51,200
and studied them.
46
00:01:51,200 --> 00:01:56,520
When I became a scientist at the NIH, it was around 2014.
47
00:01:56,520 --> 00:02:00,880
That's when AlexNet won in the ImageNet competition.
48
00:02:00,880 --> 00:02:03,560
And it became very clear to me that there was probably
49
00:02:03,560 --> 00:02:07,240
going to be something to this machine learning thing
50
00:02:07,240 --> 00:02:10,920
that many people were very excited about at the time,
51
00:02:10,920 --> 00:02:13,800
and that it was going to, while it had already
52
00:02:13,800 --> 00:02:15,360
had some impact in various fields,
53
00:02:15,360 --> 00:02:19,440
it was going to have tremendous impact across many fields.
54
00:02:19,440 --> 00:02:23,840
And certainly, I wanted to stay very close to science,
55
00:02:23,840 --> 00:02:25,440
but I wanted to learn more about this
56
00:02:25,440 --> 00:02:29,080
and basically hopefully do this for my career.
57
00:02:29,080 --> 00:02:31,200
Like I said at that time, machine learning and science
58
00:02:31,200 --> 00:02:36,760
wasn't exactly a thing yet, certainly not to the level
59
00:02:36,760 --> 00:02:38,040
that it is now.
60
00:02:38,040 --> 00:02:40,560
But I managed to stay pretty close,
61
00:02:40,560 --> 00:02:43,520
and now it's actually pretty easy to work
62
00:02:43,520 --> 00:02:46,200
at those intersections.
63
00:02:46,200 --> 00:02:46,960
Very nice.
64
00:02:46,960 --> 00:02:50,120
Yeah, I think AlexNet was a moment for a lot of people,
65
00:02:50,120 --> 00:02:52,240
both in and out of the field, being
66
00:02:52,240 --> 00:02:55,280
able to see something that could come in and just
67
00:02:55,280 --> 00:02:58,200
like have error rates on such a hard problem.
68
00:02:58,200 --> 00:03:00,640
And it started to become more tangible
69
00:03:00,640 --> 00:03:03,160
that this sort of stuff was going to be usable
70
00:03:03,160 --> 00:03:07,720
and was going to be able to be applied to other fields.
71
00:03:07,720 --> 00:03:10,840
OK, yeah, so back in 2014, during that time,
72
00:03:10,840 --> 00:03:12,400
you were working in the lab.
73
00:03:12,400 --> 00:03:14,520
But now fast forwarding, now you are
74
00:03:14,520 --> 00:03:16,400
doing some computational stuff, right?
75
00:03:16,400 --> 00:03:20,400
So being able to see that difference between the hands
76
00:03:20,400 --> 00:03:22,840
on work and how fast things are moving now,
77
00:03:22,840 --> 00:03:24,840
can you speak to that?
78
00:03:24,840 --> 00:03:27,760
So I think it impacts the way, so certainly,
79
00:03:27,760 --> 00:03:30,160
having worked in the lab, it impacts the way
80
00:03:30,160 --> 00:03:34,920
I think about data and perceptions of,
81
00:03:34,920 --> 00:03:36,800
it helps you deeply understand the data where
82
00:03:36,800 --> 00:03:38,760
there might be errors.
83
00:03:38,760 --> 00:03:42,680
But certainly, the field now of machine learning and science
84
Load full transcript
00:03:42,680 --> 00:03:43,960
is moving so fast.
85
00:03:43,960 --> 00:03:47,320
When I gave my Pi data talk, I had a slide that was,
86
00:03:47,320 --> 00:03:48,800
it was a joke, but not really, which
87
00:03:48,800 --> 00:03:51,120
is like, by the time you finish reading this,
88
00:03:51,120 --> 00:03:52,960
there will be a new protein design paper
89
00:03:52,960 --> 00:03:55,520
out completely revolutionizing the field.
90
00:03:55,520 --> 00:03:57,680
And it's kind of true.
91
00:03:57,680 --> 00:03:59,360
Actually, one of my side projects
92
00:03:59,360 --> 00:04:03,120
is using some retrieval augmented generation models
93
00:04:03,120 --> 00:04:06,800
to identify and recommend new literature
94
00:04:06,800 --> 00:04:10,080
as it comes out for me, because it's a nontrivial thing
95
00:04:10,080 --> 00:04:13,720
to follow the literature these days.
96
00:04:13,720 --> 00:04:16,840
So yeah, it's changing incredibly fast.
97
00:04:16,840 --> 00:04:21,520
I think there's so many potential areas where it's going to,
98
00:04:21,520 --> 00:04:23,440
it's starting to and will change the field.
99
00:04:23,440 --> 00:04:27,320
I think we don't always know exactly the right ways it's
100
00:04:27,320 --> 00:04:30,160
going to fit in, but there are many, many people
101
00:04:30,160 --> 00:04:32,000
trying to figure that out.
102
00:04:32,000 --> 00:04:33,200
Yeah.
103
00:04:33,200 --> 00:04:38,000
Machine learning, AI, NLP, the field's moving so fast.
104
00:04:38,000 --> 00:04:40,920
The joke is that by the time it gets peer reviewed,
105
00:04:40,920 --> 00:04:43,360
it will be outdated.
106
00:04:43,360 --> 00:04:46,280
So that's why there's nice things like archive,
107
00:04:46,280 --> 00:04:51,040
where people can just kind of get it directly to their readers.
108
00:04:51,040 --> 00:04:53,240
There are some pros and cons to it, though.
109
00:04:53,240 --> 00:04:54,360
Of course.
110
00:04:54,360 --> 00:04:55,640
There certainly are.
111
00:04:55,640 --> 00:04:58,360
You always have to evaluate what's there
112
00:04:58,360 --> 00:05:01,320
and understand where you have to learn
113
00:05:01,320 --> 00:05:03,920
to read between the lines what the strengths
114
00:05:03,920 --> 00:05:06,680
and actual weaknesses of a particular piece of research
115
00:05:06,680 --> 00:05:07,180
are.
116
00:05:07,180 --> 00:05:08,960
You don't always know.
117
00:05:08,960 --> 00:05:11,560
But I also think that is a model.
118
00:05:11,560 --> 00:05:16,040
The open access is a model that has allowed machine learning
119
00:05:16,040 --> 00:05:18,720
to gain a lot of traction.
120
00:05:18,720 --> 00:05:20,200
Certainly, the field of science is
121
00:05:20,200 --> 00:05:21,640
undergoing some transitions.
122
00:05:21,640 --> 00:05:23,720
And not all of scientific literature
123
00:05:23,720 --> 00:05:27,200
has been open access, but that's a lot changing.
124
00:05:27,200 --> 00:05:30,080
I think that's been very helpful to the growth and acceptance
125
00:05:30,080 --> 00:05:31,320
of machine learning.
126
00:05:31,320 --> 00:05:33,080
Yeah, absolutely.
127
00:05:33,080 --> 00:05:35,360
So in this field, the limited amount of stuff
128
00:05:35,360 --> 00:05:37,560
that I know about it, I'm very interested in it.
129
00:05:37,560 --> 00:05:43,080
I wish I knew more, but I can't go by without talking
130
00:05:43,080 --> 00:05:45,280
a little bit about Alpha Fold.
131
00:05:45,280 --> 00:05:48,440
What were your initial reactions to it?
132
00:05:48,440 --> 00:05:50,720
Could you tell us some more about it?
133
00:05:50,720 --> 00:05:51,220
Sure.
134
00:05:51,220 --> 00:05:54,080
So Alpha Fold is a model.
135
00:05:54,080 --> 00:05:58,920
It's a family of models called Equivariant Models.
136
00:05:58,920 --> 00:06:02,320
And the purpose is to predict the three-dimensional structure
137
00:06:02,320 --> 00:06:04,360
of a protein.
138
00:06:04,360 --> 00:06:08,240
So biology, we can talk maybe a bit about this in a moment.
139
00:06:08,240 --> 00:06:12,040
But we often can be represented with text,
140
00:06:12,040 --> 00:06:14,000
which is not a coincidence because that's
141
00:06:14,000 --> 00:06:15,680
how we communicate with each other.
142
00:06:15,680 --> 00:06:19,040
So it's not surprising that we have developed ways
143
00:06:19,040 --> 00:06:23,120
to represent various biological moieties
144
00:06:23,120 --> 00:06:25,320
in different fashions with text.
145
00:06:25,320 --> 00:06:27,720
And certainly, it's a valid representation.
146
00:06:27,720 --> 00:06:31,440
Those models have power and use and advantages.
147
00:06:31,440 --> 00:06:33,880
But to describe a three-dimensional protein
148
00:06:33,880 --> 00:06:39,040
structure, as you might guess, often need three dimensions.
149
00:06:39,040 --> 00:06:41,880
There's some other ways to do it.
150
00:06:41,880 --> 00:06:44,360
So Alpha Fold takes in a sequence.
151
00:06:44,360 --> 00:06:46,720
It's called amino acids.
152
00:06:46,720 --> 00:06:49,640
That's the building block of a protein.
153
00:06:49,640 --> 00:06:51,680
It takes that amino acid sequence.
154
00:06:51,680 --> 00:06:53,880
There's one-letter abbreviations for each
155
00:06:53,880 --> 00:06:56,640
of the 20 naturally occurring amino acids.
156
00:06:56,640 --> 00:06:59,440
And then it predicts the three-dimensional coordinates
157
00:06:59,440 --> 00:07:02,600
of that protein.
158
00:07:02,600 --> 00:07:06,600
And so in 2018, Alpha Fold won CASP,
159
00:07:06,600 --> 00:07:09,200
which is the Critical Assessment of Protein Structure.
160
00:07:09,200 --> 00:07:12,160
It's a protein structure prediction competition.
161
00:07:12,160 --> 00:07:15,640
And there's a great paper written by a colleague of mine,
162
00:07:15,640 --> 00:07:17,320
Jeff Hoke.
163
00:07:17,320 --> 00:07:20,400
And he went in and examined all the metrics
164
00:07:20,400 --> 00:07:23,440
and all the different competition types of metrics
165
00:07:23,440 --> 00:07:25,080
that were
measured from CASP.
166
00:07:25,080 --> 00:07:27,320
And Alpha Fold didn't just win.
167
00:07:27,320 --> 00:07:28,600
It dominated.
168
00:07:28,600 --> 00:07:30,560
It really, really changed the way
169
00:07:30,560 --> 00:07:33,080
we think about machine learning.
170
00:07:33,080 --> 00:07:37,680
And that was the Alex Net moment for science, in my opinion.
171
00:07:37,680 --> 00:07:40,520
So I started doing machine learning in 2014,
172
00:07:40,520 --> 00:07:44,080
but not too far down the road.
173
00:07:44,080 --> 00:07:47,320
We had our own moment of intense realization
174
00:07:47,320 --> 00:07:49,800
that this was going to change everything.
175
00:07:49,800 --> 00:07:51,800
And there are Alpha Fold.
176
00:07:51,800 --> 00:07:53,800
There are certainly challenges with Alpha Fold.
177
00:07:53,800 --> 00:07:56,680
I'm not saying it's a panacea.
178
00:07:56,680 --> 00:08:00,840
Drug discovery and science just in general are challenging.
179
00:08:00,840 --> 00:08:02,000
And it has limitations.
180
00:08:02,000 --> 00:08:04,080
But of course, we iterate on those.
181
00:08:04,080 --> 00:08:06,160
That's how the field progresses.
182
00:08:06,160 --> 00:08:06,960
Right.
183
00:08:06,960 --> 00:08:10,280
So some of the work that you're doing at NVIDIA.
184
00:08:10,280 --> 00:08:13,080
So can you tell us about BioNemo?
185
00:08:13,080 --> 00:08:13,600
Sure.
186
00:08:13,600 --> 00:08:15,800
So BioNemo, as you briefly alluded to,
187
00:08:15,800 --> 00:08:19,040
it's an inference service and a framework.
188
00:08:19,040 --> 00:08:23,000
The inference service is models from the community,
189
00:08:23,000 --> 00:08:26,760
including Alpha Fold, Open Fold, all these derivatives
190
00:08:26,760 --> 00:08:29,120
of the folders.
191
00:08:29,120 --> 00:08:33,720
There's models for docking small molecules in proteins
192
00:08:33,720 --> 00:08:37,560
and finding the binding pocket of it.
193
00:08:37,560 --> 00:08:42,080
There are models for protein representation learning.
194
00:08:42,080 --> 00:08:44,320
There's a model called ESM, which
195
00:08:44,320 --> 00:08:47,800
was developed by a group at Meta at FAIR.
196
00:08:47,800 --> 00:08:51,600
They're now their own group.
197
00:08:51,600 --> 00:08:55,280
And there are other models as well for small molecules.
198
00:08:55,280 --> 00:08:58,280
And so we basically accelerated those checkpoints
199
00:08:58,280 --> 00:08:59,640
as much as possible.
200
00:08:59,640 --> 00:09:03,160
We did our NVIDIA thing with them, put them behind an API.
201
00:09:03,160 --> 00:09:05,040
And so they are accessible to users.
202
00:09:05,040 --> 00:09:07,920
There's a Python client.
203
00:09:07,920 --> 00:09:09,880
Individuals, companies developing software
204
00:09:09,880 --> 00:09:12,000
could put those API calls into their software
205
00:09:12,000 --> 00:09:13,400
if they don't want to host their models.
206
00:09:13,400 --> 00:09:14,680
There's load balancing.
207
00:09:14,680 --> 00:09:19,240
There's all these nice features at scalable,
208
00:09:19,240 --> 00:09:21,000
so you don't have to deal with that.
209
00:09:21,000 --> 00:09:25,040
There's also a framework that's more for ML researchers
210
00:09:25,040 --> 00:09:27,920
to train and develop their own models.
211
00:09:27,920 --> 00:09:28,420
Very cool.
212
00:09:28,420 --> 00:09:32,040
So the users are both, say, research institutes
213
00:09:32,040 --> 00:09:33,720
and companies as well?
214
00:09:33,720 --> 00:09:38,520
Yeah, I would say the users range from data scientists
215
00:09:38,520 --> 00:09:42,560
to even bench scientists who want to use an API
216
00:09:42,560 --> 00:09:45,800
to researchers who want to build their own stuff.
217
00:09:45,800 --> 00:09:46,800
Right.
218
00:09:46,800 --> 00:09:50,560
So yeah, being at NVIDIA, I wouldn't
219
00:09:50,560 --> 00:09:54,720
say the first thing that I would think of is AI drug discovery.
220
00:09:54,720 --> 00:09:55,920
Well, I guess the AI part.
221
00:09:55,920 --> 00:09:58,720
But I wouldn't put NVIDIA and drug discovery together.
222
00:09:58,720 --> 00:10:01,000
But yeah, so can you tell us why is NVIDIA
223
00:10:01,000 --> 00:10:03,640
interested in doing this type of work?
224
00:10:03,640 --> 00:10:04,680
Yeah, great question.
225
00:10:04,680 --> 00:10:06,120
And I get that a lot.
226
00:10:06,120 --> 00:10:08,600
NVIDIA's objective is not to become a drug discovery
227
00:10:08,600 --> 00:10:09,120
company.
228
00:10:09,120 --> 00:10:13,240
We want to enable those doing drug discovery or developing
229
00:10:13,240 --> 00:10:17,120
software for drug discovery with the best of what
230
00:10:17,120 --> 00:10:19,760
we can offer on GPUs.
231
00:10:19,760 --> 00:10:21,500
NVIDIA is in kind of a unique position
232
00:10:21,500 --> 00:10:25,920
because we do make the hardware and a lot of the software
233
00:10:25,920 --> 00:10:29,400
along the stack building up to BioNemo we touch.
234
00:10:29,400 --> 00:10:33,320
So we can surface optimizations and the best
235
00:10:33,320 --> 00:10:36,640
of what the hardware has to offer in BioNemo.
236
00:10:36,640 --> 00:10:41,520
And in turn, what users need and ask for and pain points
237
00:10:41,520 --> 00:10:42,880
can get filtered back to us.
238
00:10:42,880 --> 00:10:45,640
And that can influence hardware, software,
239
00:10:45,640 --> 00:10:47,000
you never know down the road.
240
00:10:47,000 --> 00:10:49,480
So that's really our objective.
241
00:10:49,480 --> 00:10:52,120
Right, that makes a lot of sense.
242
00:10:52,120 --> 00:10:54,240
So there's probably like some, you
243
00:10:54,240 --> 00:10:55,680
talked about it in your Pi data talk,
244
00:10:55,680 --> 00:10:59,880
like there's this cyclic nature of supply and demand, right,
245
00:10:59,880 --> 00:11:02,720
between the hardware and developing
246
00:11:02,720 --> 00:11:03,960
these sorts of solutions.
247
00:11:03,960 --> 00:11:06,640
So you have this architecture that you
248
00:11:06,640 -->
00:11:07,920
can run things in parallel.
249
00:11:07,920 --> 00:11:09,720
And you can do all of these amazing things.
250
00:11:09,720 --> 00:11:11,640
You can test out all of this stuff.
251
00:11:11,640 --> 00:11:14,480
And you have these sorts of machine learning models
252
00:11:14,480 --> 00:11:18,200
that can create tons of different whatever
253
00:11:18,200 --> 00:11:20,800
tasks you're trying to do.
254
00:11:20,800 --> 00:11:25,480
So yeah, it's a very interesting relationship.
255
00:11:25,480 --> 00:11:29,880
So just talking about, well, let's back up for a second,
256
00:11:29,880 --> 00:11:31,040
AI drug discovery.
257
00:11:31,040 --> 00:11:33,480
Why is that so important?
258
00:11:33,480 --> 00:11:38,800
Why is it something that people should care about?
259
00:11:38,800 --> 00:11:40,120
Why should we put it in?
260
00:11:40,120 --> 00:11:42,120
Why should people be thinking about it?
261
00:11:42,120 --> 00:11:47,800
Yeah, the drug discovery process is long and expensive.
262
00:11:47,800 --> 00:11:51,320
For a particular class of drug, small molecules,
263
00:11:51,320 --> 00:11:56,640
it's something like close to $3 billion in 10-ish years.
264
00:11:56,640 --> 00:11:58,960
And there's tons of failures along the way.
265
00:11:58,960 --> 00:12:02,120
It's a very manual process.
266
00:12:02,120 --> 00:12:05,000
And there's certainly a lot of repetitiveness.
267
00:12:05,000 --> 00:12:07,480
So that's a place where you can start to think that machine
268
00:12:07,480 --> 00:12:09,360
learning might come in.
269
00:12:09,360 --> 00:12:12,000
For example, small molecule development,
270
00:12:12,000 --> 00:12:18,600
once we have a candidate, a lead that binds to a protein,
271
00:12:18,600 --> 00:12:20,160
then it's optimized.
272
00:12:20,160 --> 00:12:21,520
And that's a very, you know, there's
273
00:12:21,520 --> 00:12:24,440
the molecule synthesized by a chemist.
274
00:12:24,440 --> 00:12:26,080
It's tested in an assay.
275
00:12:26,080 --> 00:12:27,360
It's evaluated.
276
00:12:27,360 --> 00:12:31,320
You know, this design-make-test cycle goes over and over.
277
00:12:31,320 --> 00:12:35,280
If we can start to help superpower those chemists
278
00:12:35,280 --> 00:12:38,560
by enabling some of that to be done in silico,
279
00:12:38,560 --> 00:12:42,080
then our objective is to make this process faster,
280
00:12:42,080 --> 00:12:43,960
make their lives better.
281
00:12:43,960 --> 00:12:45,560
We can give scientists the power that they
282
00:12:45,560 --> 00:12:50,080
need to work more efficiently.
283
00:12:50,080 --> 00:12:51,520
It must be pretty tricky.
284
00:12:51,520 --> 00:12:55,080
I mean, evaluation is always a tough thing
285
00:12:55,080 --> 00:12:58,320
for any machine learning, really, I guess, any problem.
286
00:12:58,320 --> 00:13:02,720
But when do you know, OK, like, you know, in silicon,
287
00:13:02,720 --> 00:13:06,200
like, this molecule, this protein
288
00:13:06,200 --> 00:13:10,000
has reached
the point where it's time.
289
00:13:10,000 --> 00:13:12,480
Like, let's actually synthesize this thing
290
00:13:12,480 --> 00:13:16,560
and let's start testing it in a real lab, not a, you know,
291
00:13:16,560 --> 00:13:18,320
in a physical lab.
292
00:13:18,320 --> 00:13:21,160
Yeah, I can't answer that because it's certainly
293
00:13:21,160 --> 00:13:28,480
different for every application, every chemist, every model.
294
00:13:28,480 --> 00:13:31,520
So, yeah, as for my objective, I think
295
00:13:31,520 --> 00:13:33,120
to use these effectively, you would
296
00:13:33,120 --> 00:13:35,960
have to get a sense of how well does the model perform
297
00:13:35,960 --> 00:13:37,160
in this assay.
298
00:13:37,160 --> 00:13:38,880
And certainly, that's something.
299
00:13:38,880 --> 00:13:42,960
Developing generalized models, like foundation models,
300
00:13:42,960 --> 00:13:48,320
for example, that can then be specialized by users,
301
00:13:48,320 --> 00:13:50,600
that's why it's important for NVIDIA.
302
00:13:50,600 --> 00:13:52,200
We can train these powerful models,
303
00:13:52,200 --> 00:13:55,800
but users will specialize them to their particular drug
304
00:13:55,800 --> 00:13:58,640
program, their particular target.
305
00:13:58,640 --> 00:14:01,600
You know, we understand that certainly there's
306
00:14:01,600 --> 00:14:04,400
a lot of specifics that have to go into this,
307
00:14:04,400 --> 00:14:06,000
even after the model's created.
308
00:14:06,000 --> 00:14:07,480
Right, that makes a lot of sense.
309
00:14:07,480 --> 00:14:10,160
So you're more focusing on the tools
310
00:14:10,160 --> 00:14:12,560
and enabling people to do this type of work.
311
00:14:12,560 --> 00:14:14,320
Exactly.
312
00:14:14,320 --> 00:14:17,240
So talking about, and you alluded to it
313
00:14:17,240 --> 00:14:21,080
before, some of the overlap or how you can use MLP to help you
314
00:14:21,080 --> 00:14:22,000
with biology.
315
00:14:22,000 --> 00:14:23,800
Where do you see that there's overlap
316
00:14:23,800 --> 00:14:27,040
and where are there some differences?
317
00:14:27,040 --> 00:14:30,880
Yeah, so there are, yeah, like I said,
318
00:14:30,880 --> 00:14:35,440
natural language is an excellent way to represent biology.
319
00:14:35,440 --> 00:14:40,120
The different moieties in biology, DNA, RNA, proteins,
320
00:14:40,120 --> 00:14:43,640
small molecules, they all have their own language
321
00:14:43,640 --> 00:14:44,840
that can be used.
322
00:14:44,840 --> 00:14:47,960
And that enables us to beg, borrow, and steal
323
00:14:47,960 --> 00:14:52,200
from all of these NLP breakthroughs that have happened.
324
00:14:52,200 --> 00:14:56,400
And that has really helped jumpstart that field.
325
00:14:56,400 --> 00:14:59,000
It's also, there's considerations
326
00:14:59,000 --> 00:15:01,040
from a biological standpoint, which
327
00:15:01,040 --> 00:15:06,360
is that there's a lot of NLP data, of text data available.
328
00:15:06,360 --> 00:15:09,040
For example, with proteins, we have many more protein
329
00:15:09,040 --> 00:15:11,440
sequences than we do protein structures,
330
00:15:11,440 --> 00:15:12,720
than three-dimensional structures,
331
00:15:12,720 --> 00:15:14,320
like we discussed earlier.
332
00:15:14,320 --> 00:15:18,040
So in some contexts, you have a lot more data.
333
00:15:18,040 --> 00:15:20,760
So that's very powerful.
334
00:15:20,760 --> 00:15:27,120
I think differences in NLP are certainly vocabulary.
335
00:15:27,120 --> 00:15:29,920
Many times, biological vocabularies
336
00:15:29,920 --> 00:15:34,760
are like tokenized three letters at a time,
337
00:15:34,760 --> 00:15:39,120
or every amino acid, in which case there's 20 amino acids.
338
00:15:39,120 --> 00:15:44,160
So the vocabulary size is certainly less than 50.
339
00:15:44,160 --> 00:15:47,120
And the distribution of the tokens is very different.
340
00:15:47,120 --> 00:15:51,760
So those are ways that biology doesn't necessarily always
341
00:15:51,760 --> 00:15:54,480
follow the same trends as NLP.
342
00:15:54,480 --> 00:15:56,760
We still try to do experiments empirically.
343
00:15:56,760 --> 00:16:01,160
Sure, x, y, and z features improve the model for NLP,
344
00:16:01,160 --> 00:16:04,840
but we need to do the test for biology as well.
345
00:16:04,840 --> 00:16:06,560
Right.
346
00:16:06,560 --> 00:16:08,880
Yeah, it's so interesting to hear
347
00:16:08,880 --> 00:16:11,960
you mention tokenization, because that's
348
00:16:11,960 --> 00:16:14,400
something in natural language processing
349
00:16:14,400 --> 00:16:16,120
that you're always thinking about.
350
00:16:16,120 --> 00:16:18,480
And it's like, what level of analysis
351
00:16:18,480 --> 00:16:20,360
do you want to be thinking of your problem?
352
00:16:20,360 --> 00:16:22,000
How do you want to break down your problem?
353
00:16:22,000 --> 00:16:24,400
And then for any sort of problem,
354
00:16:24,400 --> 00:16:27,600
that's how you're going to have to figure out
355
00:16:27,600 --> 00:16:30,960
what should be the proper way of tokenizing these things.
356
00:16:30,960 --> 00:16:33,840
And that's an open question in biology, I would say.
357
00:16:33,840 --> 00:16:36,360
Yeah, there certainly have been explorations
358
00:16:36,360 --> 00:16:39,000
to do more other types of tokenization.
359
00:16:39,000 --> 00:16:45,280
But I don't think anything has had sufficient sticking power
360
00:16:45,280 --> 00:16:46,440
yet.
361
00:16:46,440 --> 00:16:47,600
That's interesting.
362
00:16:47,600 --> 00:16:53,120
Yeah, there's always new things coming along in NLP.
363
00:16:55,720 --> 00:17:00,800
For a while, there was subword tokenization.
364
00:17:00,800 --> 00:17:03,080
And then when you're doing sentence tokenization,
365
00:17:03,080 --> 00:17:06,440
well, how should you do the sentence tokenization now
366
00:17:06,440 --> 00:17:08,320
with retrieval augmented generation?
367
00:17:08,320 --> 00:17:11,240
It's like, how
should you do the chunking properly?
368
00:17:11,240 --> 00:17:13,480
Because it's always about capturing
369
00:17:13,480 --> 00:17:16,640
the amount of meaning and getting the right context
370
00:17:16,640 --> 00:17:18,520
that you have.
371
00:17:18,520 --> 00:17:21,240
And it's interesting, because you think with language,
372
00:17:21,240 --> 00:17:22,480
oh, we understand language.
373
00:17:22,480 --> 00:17:23,720
We should be able to do this.
374
00:17:23,720 --> 00:17:24,800
It's hard there.
375
00:17:24,800 --> 00:17:30,120
So I can't imagine in biochem, it
376
00:17:30,120 --> 00:17:32,280
must be extremely difficult to understand
377
00:17:32,280 --> 00:17:40,000
what's the smallest level of meaning for these systems.
378
00:17:40,000 --> 00:17:42,360
Right, and you don't know, too.
379
00:17:42,360 --> 00:17:44,880
It may be different for different questions.
380
00:17:44,880 --> 00:17:46,320
This is a fundamental challenge.
381
00:17:46,320 --> 00:17:50,200
If you want to do things with in-context learning,
382
00:17:50,200 --> 00:17:55,000
if you can speak to GPT-4 like you would speak to another
383
00:17:55,000 --> 00:17:57,120
human, we know how to communicate.
384
00:17:57,120 --> 00:17:58,400
I don't speak protein.
385
00:17:58,400 --> 00:18:00,720
I don't know anyone else that does.
386
00:18:00,720 --> 00:18:02,400
Do I spell it out in amino acids?
387
00:18:02,400 --> 00:18:03,400
I don't know.
388
00:18:03,400 --> 00:18:04,440
How do I tell it?
389
00:18:04,440 --> 00:18:08,320
So that has led to a lot of ideas
390
00:18:08,320 --> 00:18:12,080
that the multimodal models might be a way to go if you actually
391
00:18:12,080 --> 00:18:15,760
want to speak to a model that will design a protein for you
392
00:18:15,760 --> 00:18:17,520
the way you would speak about how you're
393
00:18:17,520 --> 00:18:19,760
going to do an experiment.
394
00:18:19,760 --> 00:18:21,560
So certainly, those are active areas.
395
00:18:21,560 --> 00:18:26,720
And that's a difference as well, I guess, with biology versus NLP.
396
00:18:26,720 --> 00:18:29,840
But yeah, back to the discussion about tokenization,
397
00:18:29,840 --> 00:18:30,840
we don't always know.
398
00:18:30,840 --> 00:18:34,080
Proteins have a whole hierarchy of structure.
399
00:18:34,080 --> 00:18:35,640
There's amino acid sequence.
400
00:18:35,640 --> 00:18:38,080
There's something called secondary structure.
401
00:18:38,080 --> 00:18:40,040
And then there's the tertiary structure, which
402
00:18:40,040 --> 00:18:42,200
is sort of the 3D coordinates.
403
00:18:42,200 --> 00:18:46,280
Maybe you need to break it along secondary structure entities.
404
00:18:46,280 --> 00:18:49,960
And many sort of sentence piece type tokenizers
405
00:18:49,960 --> 00:18:51,400
will actually do that a little bit.
406
00:18:51,400 --> 00:18:54,120
I've done some experimentation.
407
00:18:54,120 --> 00:18:55,440
But not always.
408
00:18:55,440 --> 00:18:57,400
And so yeah, what's the right way?
409
00:18:57,400 --> 00:19:00,400
Or do you need to break it along three dimensional domains
410
00:19:00,400 --> 00:19:01,480
for it to be useful?
411
00:19:01,480 --> 00:19:02,000
Maybe.
412
00:19:02,000 --> 00:19:03,120
I don't know.
413
00:19:03,120 --> 00:19:05,920
But then it's not always the same amino acid.
414
00:19:05,920 --> 00:19:08,160
And some of the amino acids are kind of interchangeable.
415
00:19:08,160 --> 00:19:12,320
So then you need to be able to account for that too.
416
00:19:12,320 --> 00:19:12,880
Right.
417
00:19:12,880 --> 00:19:15,840
So I can't even imagine.
418
00:19:15,840 --> 00:19:17,600
It's so tricky.
419
00:19:17,600 --> 00:19:19,280
I mean, I'm just going to keep coming back
420
00:19:19,280 --> 00:19:21,020
to natural language processing, thinking
421
00:19:21,020 --> 00:19:22,600
about different domains.
422
00:19:22,600 --> 00:19:26,560
And you think about a task, say, like sentiment analysis.
423
00:19:26,560 --> 00:19:30,720
And then you think about it in product reviews.
424
00:19:30,720 --> 00:19:32,960
Or you think about it in news articles.
425
00:19:32,960 --> 00:19:34,520
Or you think about it in dialogue.
426
00:19:34,520 --> 00:19:37,600
And those three, yes, they're all sentiment analysis.
427
00:19:37,600 --> 00:19:41,240
But those three are all extremely different tasks.
428
00:19:41,240 --> 00:19:43,840
And you have to do specific things for all of those things.
429
00:19:43,840 --> 00:19:47,160
So I'm sure there's parallels for the work that you're doing,
430
00:19:47,160 --> 00:19:51,620
trying to understand what is the specific context
431
00:19:51,620 --> 00:19:53,460
and what's the specific methodology that
432
00:19:53,460 --> 00:19:56,720
would work for this type of problem.
433
00:19:56,720 --> 00:19:58,380
So it's probably a lot of experimentation
434
00:19:58,380 --> 00:19:59,880
that needs to get done there.
435
00:19:59,880 --> 00:20:02,560
And another thing that I would say that it's very hard,
436
00:20:02,560 --> 00:20:05,840
you as we as speakers of the English language
437
00:20:05,840 --> 00:20:10,400
have some gauge whether or not the prediction of that sentiment
438
00:20:10,400 --> 00:20:12,920
is close, way off.
439
00:20:12,920 --> 00:20:14,880
We can kind of sniff test that.
440
00:20:14,880 --> 00:20:16,760
It's very hard with proteins.
441
00:20:16,760 --> 00:20:18,720
Same thing, I don't speak protein.
442
00:20:18,720 --> 00:20:21,160
So we don't know.
443
00:20:21,160 --> 00:20:23,080
We have to compare it to existing assays.
444
00:20:23,080 --> 00:20:26,680
We have to be in a situation where one can run assays,
445
00:20:26,680 --> 00:20:30,000
or these lab in the loop ideas, which is a very smart way
446
00:20:30,000 --> 00:20:32,240
of doing things.
447
00:20:32,240 --> 00:20:34,920
But yeah, that's another challenging part with biology,
448
00:20:34,920 --> 00:20:38,160
that we don't have a great sense of how correct a prediction is.
449
00:20:38,160 --> 00:20:42,960
And there are many predictive tasks, as you alluded to,
450
00:20:42,960 --> 00:20:47,780
that are useful, that are useful in the drug design process.
451
00:20:47,780 --> 00:20:49,960
Certainly, the value of a foundation model
452
00:20:49,960 --> 00:20:52,800
would be the very rich predictive embeddings.
453
00:20:52,800 --> 00:20:55,040
that it produces.
454
00:20:55,040 --> 00:20:55,920
Very cool.
455
00:20:55,920 --> 00:20:58,040
So let's go into it.
456
00:20:58,040 --> 00:21:02,400
What does it take to create, or try to create, or begin
457
00:21:02,400 --> 00:21:06,400
to even think about bio foundation models?
458
00:21:06,400 --> 00:21:08,720
Yeah, well, certainly first and foremost, I
459
00:21:08,720 --> 00:21:10,960
think a data strategy.
460
00:21:10,960 --> 00:21:14,280
The hard lessons learned, I think,
461
00:21:14,280 --> 00:21:17,000
by many data scientists and researchers
462
00:21:17,000 --> 00:21:19,760
is that data is the most important thing.
463
00:21:19,760 --> 00:21:21,360
And I certainly think that's starting
464
00:21:21,360 --> 00:21:26,640
to be true because model architectures, obviously,
465
00:21:26,640 --> 00:21:27,960
they will progress and change.
466
00:21:27,960 --> 00:21:29,280
And there are improvements.
467
00:21:29,280 --> 00:21:31,480
But there are a lot of model architectures
468
00:21:31,480 --> 00:21:33,160
that are starting to be very good.
469
00:21:33,160 --> 00:21:34,600
They're easy to use.
470
00:21:34,600 --> 00:21:38,080
So data can be a huge source of advantage.
471
00:21:38,080 --> 00:21:40,320
So I'm part of a team right now that's
472
00:21:40,320 --> 00:21:44,520
starting to think about building these bio foundation models.
473
00:21:44,520 --> 00:21:49,880
And it's a matter of picking a problem where we think NVIDIA
474
00:21:49,880 --> 00:21:53,440
can uniquely succeed.
475
00:21:53,440 --> 00:21:55,440
NVIDIA is not a drug discovery company,
476
00:21:55,440 --> 00:21:57,920
so we don't generate our own data.
477
00:21:57,920 --> 00:22:00,320
That doesn't mean we can't get it from places.
478
00:22:00,320 --> 00:22:03,400
But we have to think very carefully about that.
479
00:22:03,400 --> 00:22:05,800
And we're also trying to think very carefully about,
480
00:22:05,800 --> 00:22:08,280
like I said, we do have a lot of compute.
481
00:22:08,280 --> 00:22:12,920
So that is a very significant advantage.
482
00:22:12,920 --> 00:22:14,880
But what is the data we want to use?
483
00:22:14,880 --> 00:22:16,240
What is the problem?
484
00:22:16,240 --> 00:22:19,120
We also want to really, we're thinking
485
00:22:19,120 --> 00:22:22,800
about what are grand challenge problems in biology
486
00:22:22,800 --> 00:22:26,200
and trying to make sure we position ourselves
487
00:22:26,200 --> 00:22:29,760
to work towards something that is a really fundamentally
488
00:22:29,760 --> 00:22:31,560
difficult problem.
489
00:22:31,560 --> 00:22:33,920
Yeah, that makes sense.
490
00:22:33,920 --> 00:22:36,800
What are some of the potential grand challenge problems?
491
00:22:36,800 --> 00:22:39,720
I mean, I don't know that I should say too much.
492
00:22:39,720 --> 00:22:42,200
But I think these are probably fairly obvious things,
493
00:22:42,200 --> 00:22:45,960
like how do you simulate protein-protein interactions?
494
00:22:45,960 --> 00:22:48,880
How do you simulate parts of a cell, functionality of a cell?
495
00:22:48,880 --> 00:22:52,840
The way cells work, there's not an absolute cell model.
496
00:22:52,840 --> 00:22:54,800
There are temporal parts to that.
497
00:22:54,800 --> 00:22:57,600
There are tissue-specific aspects.
498
00:22:57,600 --> 00:23:00,440
All these things are very hard.
499
00:23:00,440 --> 00:23:04,240
Yeah, that makes a lot of sense.
500
00:23:04,240 --> 00:23:04,860
OK, cool.
501
00:23:04,860 --> 00:23:07,720
So trying to create these sorts of things,
502
00:23:07,720 --> 00:23:12,120
you probably need a pretty multidisciplinary team.
503
00:23:12,120 --> 00:23:14,600
I know in your Pi Data talk, you spoke
504
00:23:14,600 --> 00:23:17,760
about building a team in your previous project.
505
00:23:17,760 --> 00:23:23,680
You went from two to about 30 or 40 people.
506
00:23:23,680 --> 00:23:28,040
Yeah, the broader product team is about 40,
507
00:23:28,040 --> 00:23:29,920
right close to 40 people right now.
508
00:23:29,920 --> 00:23:31,400
Yeah.
509
00:23:31,400 --> 00:23:32,240
How was that?
510
00:23:32,240 --> 00:23:35,720
What was that like building a team?
511
00:23:35,720 --> 00:23:38,240
I mean, there's so much that goes into it.
512
00:23:38,240 --> 00:23:40,800
There's all these machine learning problems that we face,
513
00:23:40,800 --> 00:23:45,000
but dealing with people is a whole other level.
514
00:23:45,000 --> 00:23:50,000
And people are always the most challenging problem.
515
00:23:50,000 --> 00:23:52,920
I think first and foremost, you've
516
00:23:52,920 --> 00:23:55,000
got to get the culture right.
517
00:23:55,000 --> 00:23:58,200
Team dynamics matters so, so much.
518
00:23:58,200 --> 00:24:03,320
And you have to cultivate an environment where people
519
00:24:03,320 --> 00:24:04,840
feel valued for their expertise.
520
00:24:04,840 --> 00:24:06,400
They feel like they're working on things
521
00:24:06,400 --> 00:24:10,600
that are important, that are of some level of enjoyment
522
00:24:10,600 --> 00:24:13,240
to them, but that also align.
523
00:24:13,240 --> 00:24:15,400
It's always like getting all the three buckets
524
00:24:15,400 --> 00:24:19,760
to align of things, your interest, product, impact.
525
00:24:19,760 --> 00:24:21,960
Getting those things to align is very hard,
526
00:24:21,960 --> 00:24:24,440
but try to strike a balance.
527
00:24:24,440 --> 00:24:28,320
So hiring, certainly, culture is really important.
528
00:24:28,320 --> 00:24:31,000
We certainly, not every role, but some roles,
529
00:24:31,000 --> 00:24:32,600
we do look
for some domain experience
530
00:24:32,600 --> 00:24:36,400
because there are a lot of details and nuances
531
00:24:36,400 --> 00:24:37,400
to scientific data.
532
00:24:37,400 --> 00:24:41,760
And it's even genomics data is different from protein data,
533
00:24:41,760 --> 00:24:45,440
is different from chem informatics data,
534
00:24:45,440 --> 00:24:50,240
and then usually some deep learning experience.
535
00:24:50,240 --> 00:24:53,760
One thing that is very true is that the field
536
00:24:53,760 --> 00:24:55,160
is changing so fast.
537
00:24:55,160 --> 00:24:57,360
The things that we, as a team, have had to do
538
00:24:57,360 --> 00:24:59,600
have changed and evolved pretty quickly.
539
00:24:59,600 --> 00:25:01,160
So it's also figuring out how you
540
00:25:01,160 --> 00:25:05,520
can hire for people that can adapt and learn new things.
541
00:25:05,520 --> 00:25:09,120
And I mean, layer on top of that the speed at which the field is
542
00:25:09,120 --> 00:25:11,320
changing, which we've discussed, but also
543
00:25:11,320 --> 00:25:14,160
NVIDIA comes out with new GPUs pretty regularly.
544
00:25:14,160 --> 00:25:17,360
In fact, I think it's a very regular cycle,
545
00:25:17,360 --> 00:25:20,600
and maybe even increasing in frequency from what it has been.
546
00:25:20,600 --> 00:25:24,680
So we want to surface the best of those GPUs.
547
00:25:24,680 --> 00:25:27,880
So that means constantly updating what you're doing
548
00:25:27,880 --> 00:25:30,560
or the way you have things implemented
549
00:25:30,560 --> 00:25:31,760
in your software stack.
550
00:25:31,760 --> 00:25:34,120
So benchmarking, lots of benchmarking,
551
00:25:34,120 --> 00:25:38,040
and not just predictive power, but inference and training
552
00:25:38,040 --> 00:25:39,000
time.
553
00:25:39,000 --> 00:25:41,160
So we do a lot of that.
554
00:25:41,160 --> 00:25:43,960
So those are very different, I think,
555
00:25:43,960 --> 00:25:51,000
from what data scientist roles might be like at other companies.
556
00:25:51,000 --> 00:25:52,760
Yeah.
557
00:25:52,760 --> 00:25:55,400
You really need to have a strong background.
558
00:25:55,400 --> 00:25:57,960
I think it's always so helpful.
559
00:25:57,960 --> 00:26:00,960
Well, like in your case, you worked in the lab
560
00:26:00,960 --> 00:26:03,520
before now working on these sorts of problems.
561
00:26:03,520 --> 00:26:05,400
So even though I know you were saying
562
00:26:05,400 --> 00:26:06,960
it's hard to know if it makes sense,
563
00:26:06,960 --> 00:26:10,240
but you do have a general sense.
564
00:26:10,240 --> 00:26:13,800
Is that number even remotely right?
565
00:26:13,800 --> 00:26:14,480
Right.
566
00:26:14,480 --> 00:26:17,480
Is it even in the right ballpark?
567
00:26:17,480 --> 00:26:18,600
Yeah.
568
00:26:18,600 --> 00:26:21,160
Which is so important for all of these problems.
569
00:26:24,200 --> 00:26:25,360
Yeah, there's a lot of things.
570
00:26:25,360 --> 00:26:27,040
I guess one thing
I'm curious about,
571
00:26:27,040 --> 00:26:29,920
so yeah, you've played different roles, right?
572
00:26:29,920 --> 00:26:35,720
So you've been an AI researcher, and that was more hands on.
573
00:26:35,720 --> 00:26:37,160
And now you've kind of transitioned
574
00:26:37,160 --> 00:26:38,800
into a manager role.
575
00:26:38,800 --> 00:26:41,400
Are there anything that I'm missing in there?
576
00:26:41,400 --> 00:26:41,880
No.
577
00:26:41,880 --> 00:26:43,000
Are those the two?
578
00:26:43,000 --> 00:26:44,120
Oh, those are the two?
579
00:26:44,120 --> 00:26:44,640
Yeah.
580
00:26:44,640 --> 00:26:45,840
Those are the two?
581
00:26:45,840 --> 00:26:46,480
I was a teacher.
582
00:26:46,480 --> 00:26:51,480
I taught a data science boot camp after I left the NIH.
583
00:26:51,480 --> 00:26:54,080
I was basically learning more machine learning
584
00:26:54,080 --> 00:26:55,640
as I was teaching it.
585
00:26:55,640 --> 00:26:58,000
OK.
586
00:26:58,000 --> 00:26:59,480
That's the best way to learn.
587
00:26:59,480 --> 00:27:00,360
Yeah, it is.
588
00:27:00,360 --> 00:27:03,040
By teaching it, because then you're
589
00:27:03,040 --> 00:27:04,720
forced to really know all of it.
590
00:27:04,720 --> 00:27:06,200
And then someone will ask you some question,
591
00:27:06,200 --> 00:27:09,360
and you'll be like, I never really thought of it like that.
592
00:27:09,360 --> 00:27:11,200
Or when you're forced to explain it to someone,
593
00:27:11,200 --> 00:27:14,960
you're like, oh, I see this gap in my knowledge.
594
00:27:14,960 --> 00:27:16,280
Yeah.
595
00:27:16,280 --> 00:27:17,120
Humility.
596
00:27:17,120 --> 00:27:18,960
It's good to have a little bit of humility.
597
00:27:18,960 --> 00:27:22,760
That's also very important on a multidisciplinary team,
598
00:27:22,760 --> 00:27:25,320
is to be open to learning from each other.
599
00:27:25,320 --> 00:27:26,080
Yeah.
600
00:27:26,080 --> 00:27:30,200
Any other traits that you're looking for in a team?
601
00:27:30,200 --> 00:27:36,160
I think, obviously, technical excellence and experience
602
00:27:36,160 --> 00:27:37,760
are important.
603
00:27:37,760 --> 00:27:42,000
But I culture and attitude are the really big ones,
604
00:27:42,000 --> 00:27:46,080
because I think they can make a break.
605
00:27:46,080 --> 00:27:47,680
Yeah.
606
00:27:47,680 --> 00:27:50,680
A coworker of mine just shared this piece.
607
00:27:50,680 --> 00:27:54,720
It was about looking for people who are hungry, humble,
608
00:27:54,720 --> 00:27:55,840
and smart.
609
00:27:55,840 --> 00:27:57,760
But not smart in the traditional way.
610
00:27:57,760 --> 00:28:01,720
Smart in like, you know when you're
611
00:28:01,720 --> 00:28:03,440
in a meeting what you're saying, how
612
00:28:03,440 --> 00:28:06,040
it's going to influence the people in the room kind of thing.
613
00:28:06,040 --> 00:28:06,920
Emotional.
614
00:28:06,920 --> 00:28:07,760
Yeah.
615
00:28:07,760 --> 00:28:08,360
EQ.
616
00:28:08,360 -->
00:28:09,000
High EQ.
617
00:28:09,000 --> 00:28:09,960
Yeah.
618
00:28:09,960 --> 00:28:12,120
Some high EQ.
619
00:28:12,120 --> 00:28:16,920
Very hard to find people that check all of those boxes.
620
00:28:16,920 --> 00:28:17,960
It is.
621
00:28:17,960 --> 00:28:21,960
But that's the fun part of everything.
622
00:28:21,960 --> 00:28:22,480
Yeah.
623
00:28:22,480 --> 00:28:25,760
I think I was just going to say, yeah, becoming a manager,
624
00:28:25,760 --> 00:28:31,040
it's really fun to see these extremely talented people grow
625
00:28:31,040 --> 00:28:34,520
and develop in their role and start to lead things.
626
00:28:34,520 --> 00:28:37,640
And that part is really fun about a manager.
627
00:28:37,640 --> 00:28:41,720
It's really fun seeing them succeed and cheering them on.
628
00:28:41,720 --> 00:28:44,720
Yeah, absolutely.
629
00:28:44,720 --> 00:28:48,080
I've spoken to people who have done a similar transition
630
00:28:48,080 --> 00:28:52,440
from sort of like a IC.
631
00:28:52,440 --> 00:28:54,920
No one's really an IC, right?
632
00:28:54,920 --> 00:28:56,400
You're always working in a team.
633
00:28:56,400 --> 00:28:58,320
A team, yeah.
634
00:28:58,320 --> 00:29:01,840
But sort of that IC role to the manager role
635
00:29:01,840 --> 00:29:06,280
and this idea of, oh, I'm a 5x, 10x engineer.
636
00:29:06,280 --> 00:29:09,400
But when you go into managing, you
637
00:29:09,400 --> 00:29:11,520
can actually unblock so many people
638
00:29:11,520 --> 00:29:15,800
that you can have a bigger impact on your company,
639
00:29:15,800 --> 00:29:16,280
actually.
640
00:29:16,280 --> 00:29:20,520
So yeah, it's such an interesting thing
641
00:29:20,520 --> 00:29:25,440
because people want to be very, sometimes you
642
00:29:25,440 --> 00:29:27,400
want to be very focused on what you're doing.
643
00:29:27,400 --> 00:29:29,120
And then you don't want to be distracted
644
00:29:29,120 --> 00:29:30,360
by all of these other things.
645
00:29:30,360 --> 00:29:31,920
But I guess it's just each person
646
00:29:31,920 --> 00:29:37,840
needs to sort of find their own balance between that.
647
00:29:37,840 --> 00:29:41,200
So like, NVIDIA is a very ground up company,
648
00:29:41,200 --> 00:29:44,240
which it's one of the things I love about it.
649
00:29:44,240 --> 00:29:48,960
But then it's helping your mentees
650
00:29:48,960 --> 00:29:52,840
to understand how to prioritize things.
651
00:29:52,840 --> 00:29:54,720
Because it gets to be kind of nuanced
652
00:29:54,720 --> 00:29:57,120
or how to help with the situation
653
00:29:57,120 --> 00:30:00,200
but not get completely sucked into something that probably
654
00:30:00,200 --> 00:30:03,160
isn't a top priority for you.
655
00:30:03,160 --> 00:30:06,200
It's like the nuances.
656
00:30:06,200 --> 00:30:07,680
And I think there's some of it too,
657
00:30:07,680 --> 00:30:10,440
just acknowledging that the work that we do is hard
658
00:30:10,440 --> 00:30:12,320
and helping them understand that it's expected.
659
00:30:12,320 --> 00:30:14,800
I expect that this is going to take a while to learn
660
00:30:14,800 --> 00:30:15,680
how to do it.
661
00:30:15,680 --> 00:30:18,880
So I don't love, I kind of made a face at the 5x, 10x engineer.
662
00:30:18,880 --> 00:30:20,280
I don't love that stuff.
663
00:30:20,280 --> 00:30:22,160
I don't know why.
664
00:30:22,160 --> 00:30:23,240
Because what does it mean?
665
00:30:23,240 --> 00:30:24,520
What does it really mean?
666
00:30:24,520 --> 00:30:24,880
Right.
667
00:30:24,880 --> 00:30:25,560
Yeah.
668
00:30:25,560 --> 00:30:26,160
I know.
669
00:30:26,160 --> 00:30:27,440
But people say it.
670
00:30:27,440 --> 00:30:28,840
Yeah.
671
00:30:28,840 --> 00:30:30,080
But I like them.
672
00:30:30,080 --> 00:30:32,280
But yeah, just acknowledging that what we do is hard
673
00:30:32,280 --> 00:30:33,960
and that that's OK.
674
00:30:33,960 --> 00:30:35,600
You're going to have to work at this.
675
00:30:35,600 --> 00:30:37,320
Yeah, for sure.
676
00:30:37,320 --> 00:30:42,720
I think working with other people where I'm at,
677
00:30:42,720 --> 00:30:46,000
we have so many initiatives and projects that are taking place.
678
00:30:46,000 --> 00:30:49,680
And you can't get so in the details on everything.
679
00:30:49,680 --> 00:30:53,040
But having a team that can be a sounding board
680
00:30:53,040 --> 00:30:57,520
and having someone that you can say, OK, this is what I'm up to.
681
00:30:57,520 --> 00:30:59,400
These are the next things I was thinking of.
682
00:30:59,400 --> 00:31:01,200
And then someone can kind of say, well, I
683
00:31:01,200 --> 00:31:02,960
did a project that was similar to this.
684
00:31:02,960 --> 00:31:06,400
And ABC, you'll probably go down this rabbit hole.
685
00:31:06,400 --> 00:31:08,040
So maybe do DEF.
686
00:31:08,040 --> 00:31:08,600
You know?
687
00:31:08,600 --> 00:31:09,600
Yeah.
688
00:31:09,600 --> 00:31:14,760
And it's really nice to, yeah, working in a team,
689
00:31:14,760 --> 00:31:16,280
it's incredible.
690
00:31:16,280 --> 00:31:19,920
I think at one point in my life, I always thought,
691
00:31:19,920 --> 00:31:22,680
oh, I can go very far myself.
692
00:31:22,680 --> 00:31:25,800
But being a part of a startup is where I realize, wow,
693
00:31:25,800 --> 00:31:28,080
you can do unbelievable things.
694
00:31:28,080 --> 00:31:28,760
As a team.
695
00:31:28,760 --> 00:31:29,520
Yeah.
696
00:31:29,520 --> 00:31:31,600
Yeah.
697
00:31:31,600 --> 00:31:32,440
For sure.
698
00:31:32,440 --> 00:31:32,960
Yeah.
699
00:31:32,960 --> 00:31:37,960
So not really transitioning, but just the next question,
700
00:31:37,960 --> 00:31:39,440
just talking about machine learning,
701
00:31:39,440 --> 00:31:41,880
and we can still talk about the same stuff.
702
00:31:41,880 --> 00:31:44,360
What's an important question that you believe
703
00:31:44,360 --> 00:31:47,720
remains unanswered in machine learning?
704
00:31:47,720 --> 00:31:48,200
Yeah.
705
00:31:48,200 --> 00:31:50,360
So I alluded to this a little bit when
706
00:31:50,360 --> 00:31:52,720
I was talking about my postdoc.
707
00:31:52,720 --> 00:31:55,440
So biological and chemical modalities.
708
00:31:55,440 --> 00:31:58,160
So number one, they aren't taxed, even though we
709
00:31:58,160 --> 00:31:59,240
represent them that way.
710
00:31:59,240 --> 00:32:02,320
They're three-dimensional, but they're also dynamic.
711
00:32:02,320 --> 00:32:08,320
And that motion is very fundamental to the roles
712
00:32:08,320 --> 00:32:09,680
that they play in biology.
713
00:32:09,680 --> 00:32:11,880
To protein-protein interactions,
714
00:32:11,880 --> 00:32:18,160
like proteins bind together, to the binding of a compound
715
00:32:18,160 --> 00:32:20,160
or a drug, a ligand.
716
00:32:20,160 --> 00:32:22,360
A ligand or a drug, usually a drug
717
00:32:22,360 --> 00:32:25,120
is just a ligand that binds in a specific way
718
00:32:25,120 --> 00:32:28,680
and causes an enzyme.
719
00:32:28,680 --> 00:32:30,600
Those are really fundamental to biology,
720
00:32:30,600 --> 00:32:33,760
and we don't have good representations for them yet.
721
00:32:33,760 --> 00:32:39,160
Not even alpha-fold is trained on the structural data,
722
00:32:39,160 --> 00:32:41,840
our x-ray crystallographic structures
723
00:32:41,840 --> 00:32:45,920
from a repository called the Protein Data Bank, which
724
00:32:45,920 --> 00:32:49,280
is if you publish a structure in a scientific journal,
725
00:32:49,280 --> 00:32:52,280
you have to deposit the coordinates there.
726
00:32:52,280 --> 00:32:53,840
But those are static.
727
00:32:53,840 --> 00:32:57,520
And so they're static, and they're also,
728
00:32:57,520 --> 00:32:59,720
like to crystallize a protein, it
729
00:32:59,720 --> 00:33:01,520
has to pack into this ordered lattice.
730
00:33:01,520 --> 00:33:03,680
So it's whatever conformation of a protein
731
00:33:03,680 --> 00:33:05,480
packed into this lattice.
732
00:33:05,480 --> 00:33:10,560
Many people don't think about that, but they're not moving.
733
00:33:10,560 --> 00:33:14,840
So there's very, and sometimes protein motions are small,
734
00:33:14,840 --> 00:33:16,280
but sometimes they're really big.
735
00:33:16,280 --> 00:33:20,160
They're really very, very major conformational changes.
736
00:33:20,160 --> 00:33:23,560
So we don't have a good way of describing that
737
00:33:23,560 --> 00:33:25,200
in machine learning models yet.
738
00:33:25,200 --> 00:33:26,600
We're starting to think about it.
739
00:33:26,600 --> 00:33:29,200
There's ways you can collect data.
740
00:33:29,200 --> 00:33:32,840
For example, molecular dynamic simulation data
741
00:33:32,840 --> 00:33:35,240
can be used for that to some degree,
742
00:33:35,240 --> 00:33:37,360
but we just don't have good models yet for it.
743
00:33:37,360 --> 00:33:40,280
Yeah, and so dynamic here, just to make sure,
744
00:33:40,280 --> 00:33:43,600
you're talking like changing
through time, right?
745
00:33:43,600 --> 00:33:46,080
Basically, it's not stationary.
746
00:33:46,080 --> 00:33:48,200
It's actually changing.
747
00:33:48,200 --> 00:33:51,040
Yeah, OK, yeah.
748
00:33:51,040 --> 00:33:54,000
So as much as I'm interested in machine learning
749
00:33:54,000 --> 00:33:55,640
and artificial intelligence, I'm very
750
00:33:55,640 --> 00:33:58,400
interested in natural intelligence also
751
00:33:58,400 --> 00:34:00,600
and how the brain works and everything.
752
00:34:00,600 --> 00:34:05,760
And when I was first, really took a deep dive into that,
753
00:34:05,760 --> 00:34:07,680
I found that to be the case also,
754
00:34:07,680 --> 00:34:10,880
is that people looked at things a lot in a static way
755
00:34:10,880 --> 00:34:13,240
and that understanding that it's like things are,
756
00:34:13,240 --> 00:34:17,840
it's dynamic, complex, adaptive, changing system,
757
00:34:17,840 --> 00:34:20,760
and that it's not in isolation also, which
758
00:34:20,760 --> 00:34:23,880
is what you're mentioning to how it interacts
759
00:34:23,880 --> 00:34:25,920
with the other.
760
00:34:25,920 --> 00:34:28,680
Yes, cells are changing.
761
00:34:28,680 --> 00:34:30,360
They're different in different tissues.
762
00:34:30,360 --> 00:34:33,880
So there's physical differences.
763
00:34:33,880 --> 00:34:36,440
There are temporal differences with cells,
764
00:34:36,440 --> 00:34:39,000
like in disease states, the cells
765
00:34:39,000 --> 00:34:41,240
interact with each other.
766
00:34:41,240 --> 00:34:46,720
So it's a really, really complex coupled system
767
00:34:46,720 --> 00:34:48,640
that is associated with biology.
768
00:34:48,640 --> 00:34:52,000
Yeah, so all of this amazing progress
769
00:34:52,000 --> 00:34:56,120
that's happening in your field with AI drug discovery
770
00:34:56,120 --> 00:34:58,320
and natural language processing, really just
771
00:34:58,320 --> 00:34:59,680
like across the board, right?
772
00:34:59,680 --> 00:35:03,360
There's like a new multimodal model that comes out every day.
773
00:35:03,360 --> 00:35:07,880
There's a new technique that comes out every day.
774
00:35:07,880 --> 00:35:13,640
How do you view the gap between the hype of this frenzied
775
00:35:13,640 --> 00:35:20,320
state that the field is in and the reality of AI?
776
00:35:20,320 --> 00:35:21,800
I think so.
777
00:35:21,800 --> 00:35:24,720
Certainly, you have to accept that it's a thing.
778
00:35:24,720 --> 00:35:29,280
And you have to evaluate papers very carefully.
779
00:35:29,280 --> 00:35:32,480
I think you have to speak to researchers in the field,
780
00:35:32,480 --> 00:35:34,280
going to conferences.
781
00:35:34,280 --> 00:35:36,480
I'm actually, after this, I'm going to go listen to this.
782
00:35:36,480 --> 00:35:40,640
There's a bunch of NeurIPS workshops.
783
00:35:40,640 --> 00:35:42,440
And actually, I was laughing because when
784
00:35:42,440 --> 00:35:46,320
I went to NeurIPS in 2018, there was
one biology workshop.
785
00:35:46,320 --> 00:35:48,640
And I think there's like five each day or something
786
00:35:48,640 --> 00:35:49,720
on Friday and Saturday.
787
00:35:49,720 --> 00:35:51,600
It's crazy.
788
00:35:51,600 --> 00:35:53,160
We're talking to researchers, talking
789
00:35:53,160 --> 00:35:55,080
to those in pharma companies that
790
00:35:55,080 --> 00:35:58,880
are in the field that can help you really understand
791
00:35:58,880 --> 00:36:01,120
the flaws in what you have developed, which
792
00:36:01,120 --> 00:36:03,280
is not always fun.
793
00:36:03,280 --> 00:36:05,120
But yeah, I think we have to be very careful.
794
00:36:05,120 --> 00:36:08,080
We have to make sure that we're doing good baselines.
795
00:36:08,080 --> 00:36:10,200
There was a paper published recently
796
00:36:10,200 --> 00:36:11,680
in the single cell genomics field
797
00:36:11,680 --> 00:36:15,520
where they evaluated two transformer models
798
00:36:15,520 --> 00:36:17,960
and found that linear regression did better
799
00:36:17,960 --> 00:36:19,160
than both of those models.
800
00:36:19,160 --> 00:36:19,880
That's taboo.
801
00:36:19,880 --> 00:36:23,400
You're not allowed to say that.
802
00:36:23,400 --> 00:36:28,000
So I think really making sure that we do the right baselines.
803
00:36:28,000 --> 00:36:29,240
100%.
804
00:36:29,240 --> 00:36:31,000
Man, the amount of times.
805
00:36:31,000 --> 00:36:33,680
I think that every single person that I've spoken to
806
00:36:33,680 --> 00:36:36,760
for this podcast has spoken about the importance
807
00:36:36,760 --> 00:36:39,400
of getting baselines.
808
00:36:39,400 --> 00:36:43,760
And in the work that I'm doing, it's so true too.
809
00:36:43,760 --> 00:36:46,040
I know I was briefly telling you about this before.
810
00:36:46,040 --> 00:36:49,520
But yeah, like natural language processing,
811
00:36:49,520 --> 00:36:51,440
for some of the simpler tasks, people
812
00:36:51,440 --> 00:36:57,480
are trying to throw the heaviest, monstrous of a model
813
00:36:57,480 --> 00:37:01,320
at problems that can be solved using simple embeddings
814
00:37:01,320 --> 00:37:04,280
and either logistic regression or support vector machine,
815
00:37:04,280 --> 00:37:06,960
or at least get a baseline with it, right?
816
00:37:06,960 --> 00:37:09,640
To at least just see, because maybe you'll find a problem
817
00:37:09,640 --> 00:37:12,240
in your other pipeline just based off
818
00:37:12,240 --> 00:37:16,320
of what you did with the other pipeline
819
00:37:16,320 --> 00:37:18,320
that you created to get a baseline.
820
00:37:18,320 --> 00:37:20,240
If nothing else, the baseline is something
821
00:37:20,240 --> 00:37:23,360
that you can put into an interface or your product
822
00:37:23,360 --> 00:37:27,160
to keep building so that you're not blocked by this model.
823
00:37:27,160 --> 00:37:29,960
Because my joke is that, especially deep learning,
824
00:37:29,960 --> 00:37:31,360
it's like an ideal gas.
825
00:37:31,360 --> 00:37:37,680
It will expand to fill whatever space and time you give it.
826
00:37:37,680 --> 00:37:39,080
But it's true.
827
00:37:39,080 --> 00:37:42,440
Even in chem informatics, chem informaticians
828
00:37:42,440 --> 00:37:44,880
will tell you that they've used machine learning
829
00:37:44,880 --> 00:37:46,080
for a very long time.
830
00:37:46,080 --> 00:37:50,920
And the random force models and some simple fingerprint
831
00:37:50,920 --> 00:37:54,840
features of small molecules are pretty hard to beat.
832
00:37:54,840 --> 00:37:57,960
And they're not wrong.
833
00:37:57,960 --> 00:37:59,360
Yeah, it's true.
834
00:37:59,360 --> 00:38:03,800
Yeah, if you ask some Kagglers, they'll
835
00:38:03,800 --> 00:38:06,400
say XGBoost is all you need.
836
00:38:06,400 --> 00:38:08,960
That's the joke there.
837
00:38:08,960 --> 00:38:12,320
But no, ensemble models, you can really
838
00:38:12,320 --> 00:38:15,360
do a lot of amazing things.
839
00:38:15,360 --> 00:38:18,440
And in a little bit of a different perspective,
840
00:38:18,440 --> 00:38:21,760
I'm thinking about your Pi Data talk.
841
00:38:21,760 --> 00:38:26,480
You spoke about creating a product where
842
00:38:26,480 --> 00:38:28,880
you were visualizing things that wasn't really
843
00:38:28,880 --> 00:38:33,360
heavy on machine learning and the importance of that.
844
00:38:33,360 --> 00:38:35,560
Do you want to speak to it a little bit?
845
00:38:35,560 --> 00:38:36,720
Sure.
846
00:38:36,720 --> 00:38:39,040
I guess, are you asking sort of generally,
847
00:38:39,040 --> 00:38:40,800
or are you asking about that particular part?
848
00:38:40,800 --> 00:38:43,680
Well, no, just the importance of visualizing things
849
00:38:43,680 --> 00:38:46,200
before you even jump into the machine learning parts.
850
00:38:46,200 --> 00:38:46,920
Yeah.
851
00:38:46,920 --> 00:38:48,160
Oh, yeah, I understood that.
852
00:38:48,160 --> 00:38:50,760
Yeah, understanding your data, like I said,
853
00:38:50,760 --> 00:38:52,280
data is the most important thing.
854
00:38:52,280 --> 00:38:55,520
And getting to know it on a very personal level
855
00:38:55,520 --> 00:38:57,840
is really useful.
856
00:38:57,840 --> 00:39:01,120
And it can give you ideas for different scenarios
857
00:39:01,120 --> 00:39:03,480
that you want to test with the data.
858
00:39:03,480 --> 00:39:04,840
How do you split your data?
859
00:39:04,840 --> 00:39:09,600
So for example, with protein structure prediction,
860
00:39:09,600 --> 00:39:13,000
the data are often split temporally
861
00:39:13,000 --> 00:39:16,200
based on the data at which they were entered
862
00:39:16,200 --> 00:39:18,320
into the protein data bank.
863
00:39:18,320 --> 00:39:22,280
So one hypothesis is that that enables you
864
00:39:22,280 --> 00:39:24,360
to predict future structures.
865
00:39:24,360 --> 00:39:26,440
And that's not a bad hypothesis.
866
00:39:26,440 --> 00:39:29,480
However, having been a structural
biologist,
867
00:39:29,480 --> 00:39:33,560
I can tell you that there is almost certainly bound
868
00:39:33,560 --> 00:39:36,720
to be a lot of redundancy in proteins
869
00:39:36,720 --> 00:39:37,920
that are deposited later.
870
00:39:37,920 --> 00:39:41,160
Because if you solve a crystal structure,
871
00:39:41,160 --> 00:39:43,240
even if it's something that's been solved before,
872
00:39:43,240 --> 00:39:44,800
but you do it as part of your paper,
873
00:39:44,800 --> 00:39:49,400
as like a check of something to ensure that you didn't disrupt,
874
00:39:49,400 --> 00:39:51,920
I don't know, the structure or something, which, by the way,
875
00:39:51,920 --> 00:39:53,120
that's the thing that is done.
876
00:39:53,120 --> 00:39:55,280
Then it gets deposited in the data bank.
877
00:39:55,280 --> 00:39:57,760
So if it ends up on the other side of that temporal split,
878
00:39:57,760 --> 00:40:01,640
there's actually a lot of redundancy in the data.
879
00:40:01,640 --> 00:40:03,360
So then you start to think, OK, what
880
00:40:03,360 --> 00:40:04,600
are better ways to do this?
881
00:40:04,600 --> 00:40:06,720
Maybe you cluster them.
882
00:40:06,720 --> 00:40:07,960
But how do you cluster them?
883
00:40:07,960 --> 00:40:10,200
You cluster them by amino acid sequence.
884
00:40:10,200 --> 00:40:13,840
Well, protein constructs can be crystallized.
885
00:40:13,840 --> 00:40:16,440
They can be changed when they're crystallized.
886
00:40:16,440 --> 00:40:18,880
Do you cluster it by three-dimensional similarity?
887
00:40:18,880 --> 00:40:19,800
And then how?
888
00:40:19,800 --> 00:40:21,600
And how do you do the alignment?
889
00:40:21,600 --> 00:40:24,440
I guess a very nuanced problem.
890
00:40:24,440 --> 00:40:26,720
So getting to know the data and visualizing it,
891
00:40:26,720 --> 00:40:29,720
where does the model do really well and why?
892
00:40:29,720 --> 00:40:32,640
Where does the model really fail and why?
893
00:40:32,640 --> 00:40:34,960
And sometimes it's useful to look at an aggregate plot.
894
00:40:34,960 --> 00:40:36,680
But there were situations, I was working
895
00:40:36,680 --> 00:40:42,360
with someone on the team, where there's like angular rotamers.
896
00:40:42,360 --> 00:40:44,400
It's hard to explain, but there's different angles
897
00:40:44,400 --> 00:40:45,600
that are predicted.
898
00:40:45,600 --> 00:40:47,720
And so we started going through the things,
899
00:40:47,720 --> 00:40:49,360
the ones that were really off, because we
900
00:40:49,360 --> 00:40:51,080
saw this periodicity.
901
00:40:51,080 --> 00:40:53,320
And so we started to understand that one of the,
902
00:40:53,320 --> 00:40:55,360
there was many issues, but I think one of the issues
903
00:40:55,360 --> 00:40:58,680
was probably the software that was measuring the angles
904
00:40:58,680 --> 00:41:02,520
didn't understand some aspects of chemistry.
905
00:41:02,520 --> 00:41:04,800
But when you start to see, the aggregate is useful,
906
00:41:04,800 --> 00:41:07,040
but so are the examples.
907
00:41:07,040 --> 00:41:08,000
Right.
908
00:41:08,000 --> 00:41:09,040
Yeah.
909
00:41:09,040 --> 00:41:12,320
I always, as much as in the work that I'm doing,
910
00:41:12,320 --> 00:41:14,800
we want to automate a lot of it, obviously.
911
00:41:14,800 --> 00:41:16,760
We want to streamline things.
912
00:41:16,760 --> 00:41:19,080
There's always this step in the beginning
913
00:41:19,080 --> 00:41:22,200
when you have a new data set, where you just
914
00:41:22,200 --> 00:41:23,920
visualize it in some way.
915
00:41:23,920 --> 00:41:26,800
Just think about how could you even possibly visualize it.
916
00:41:26,800 --> 00:41:29,280
The way that you're thinking about it is like all
917
00:41:29,280 --> 00:41:31,840
the different, either the states or the different features
918
00:41:31,840 --> 00:41:33,080
that you want to be looking at.
919
00:41:33,080 --> 00:41:35,920
But just even going through that exercise
920
00:41:35,920 --> 00:41:39,480
gives you this like inherent, not inherent,
921
00:41:39,480 --> 00:41:42,280
but like a little bit of a more intuitive understanding
922
00:41:42,280 --> 00:41:44,000
of what's taking place.
923
00:41:44,000 --> 00:41:46,320
And then it'll give you some nice hunches
924
00:41:46,320 --> 00:41:52,120
as you try to figure out even just the problem space itself.
925
00:41:52,120 --> 00:41:55,320
Yeah, that was really cool.
926
00:41:55,320 --> 00:41:58,880
I just have a real love of visualizations.
927
00:41:58,880 --> 00:42:03,320
So a lot of the eye candy in your presentation
928
00:42:03,320 --> 00:42:04,760
really attracted me also.
929
00:42:04,760 --> 00:42:06,800
Yeah.
930
00:42:06,800 --> 00:42:09,080
Probably there's some cool videos approaching structures
931
00:42:09,080 --> 00:42:09,800
that are in there.
932
00:42:09,800 --> 00:42:12,160
And that was really just for fun eye candy,
933
00:42:12,160 --> 00:42:14,560
because they're really cool to see.
934
00:42:14,560 --> 00:42:15,720
They are beautiful.
935
00:42:15,720 --> 00:42:17,400
Very beautiful.
936
00:42:17,400 --> 00:42:19,840
When I saw those when I was learning biochemistry
937
00:42:19,840 --> 00:42:21,920
and taking biochemistry in undergrad,
938
00:42:21,920 --> 00:42:23,520
that was the thing that made me really
939
00:42:23,520 --> 00:42:25,040
fall in love with structural biology.
940
00:42:25,040 --> 00:42:27,120
It was like, wow, this is really cool.
941
00:42:27,120 --> 00:42:27,640
Right.
942
00:42:27,640 --> 00:42:28,480
I want to do this.
943
00:42:28,480 --> 00:42:32,240
Yeah, a picture is there's you can't even
944
00:42:32,240 --> 00:42:33,480
put it into words, right?
945
00:42:33,480 --> 00:42:34,120
Yeah.
946
00:42:34,120 --> 00:42:35,040
It really.
947
00:42:35,040 --> 00:42:36,000
Powerful.
948
00:42:36,000 --> 00:42:38,800
Yeah, it can be very powerful.
949
00:42:38,800 --> 00:42:45,640
So now you've been working on drug discovery using AI
950
00:42:45,640 --> 00:42:49,120
and developing these sorts of tools.
951
00:42:49,120 --> 00:42:53,080
So how have you seen the field change
952
00:42:53,080 --> 00:42:57,040
since you started working in the industry?
953
00:42:57,040 --> 00:43:01,000
I think in general, when I started
954
00:43:01,000 --> 00:43:03,560
working in the industry, data science was still
955
00:43:03,560 --> 00:43:06,120
kind of a young new thing.
956
00:43:06,120 --> 00:43:10,600
And so there were a lot of generalists.
957
00:43:10,600 --> 00:43:14,520
And at least maybe this is biased by having been in my field
958
00:43:14,520 --> 00:43:15,020
too.
959
00:43:15,020 --> 00:43:19,320
I start to see a lot more individuals in the field who
960
00:43:19,320 --> 00:43:22,800
are coming to the field, but they have been trained
961
00:43:22,800 --> 00:43:23,520
in a domain.
962
00:43:23,520 --> 00:43:25,840
And then they picked up the data science
963
00:43:25,840 --> 00:43:28,200
as part of their domain specific education.
964
00:43:28,200 --> 00:43:30,520
And I think some of that just reflects
965
00:43:30,520 --> 00:43:32,080
the way universities are starting
966
00:43:32,080 --> 00:43:36,640
to integrate these computational skills
967
00:43:36,640 --> 00:43:41,080
into their curricula, which I think is very important.
968
00:43:41,080 --> 00:43:48,760
Teaching a compute literate, machine learning literate
969
00:43:48,760 --> 00:43:50,360
student is really important.
970
00:43:50,360 --> 00:43:51,600
So I start to see that.
971
00:43:51,600 --> 00:43:54,800
I think that is very useful.
972
00:43:54,800 --> 00:43:59,760
And demonstrating some level of deep understanding of data
973
00:43:59,760 --> 00:44:03,280
will benefit you, even if you change domains, in my opinion.
974
00:44:03,280 --> 00:44:05,360
Because then at least you have a frame of reference
975
00:44:05,360 --> 00:44:08,440
for the kind of ways that things really went
976
00:44:08,440 --> 00:44:10,200
wrong in the other domain.
977
00:44:10,200 --> 00:44:13,360
So I think that's a big one that I see changing.
978
00:44:16,040 --> 00:44:17,840
I think some of the machine learning tasks
979
00:44:17,840 --> 00:44:19,400
are getting to be a lot easier.
980
00:44:19,400 --> 00:44:20,240
We have libraries.
981
00:44:20,240 --> 00:44:21,320
We have AutoML.
982
00:44:21,320 --> 00:44:25,080
We have things that start to even hyperparameter tuning
983
00:44:25,080 --> 00:44:26,160
for deep learning models.
984
00:44:26,160 --> 00:44:27,800
It's getting a little easier.
985
00:44:27,800 --> 00:44:30,180
It's always hard if you're at the cutting edge of what's
986
00:44:30,180 --> 00:44:30,720
developed.
987
00:44:30,720 --> 00:44:35,160
But I think that stuff's all changing.
988
00:44:35,160 --> 00:44:36,960
Right.
989
00:44:36,960 --> 00:44:41,120
I think, yeah, at one point, there
990
00:44:41,120 --> 00:44:43,640
weren't that many models.
991
00:44:43,640 --> 00:44:46,240
There probably weren't that many resources.
992
00:44:46,240 --> 00:44:49,360
Now there's too many.
993
00:44:49,360 --> 00:44:52,240
There's so much out there.
994
00:44:52,240 --> 00:44:53,600
There's a lot of libraries.
995
00:44:53,600 --> 00:44:54,960
A lot of them are overlapping.
996
00:44:54,960 --> 00:44:57,400
There's a lot of vendors out there that
997
00:44:57,400 --> 00:45:00,800
are trying to do the same things.
998
00:45:00,800 --> 00:45:03,840
Yeah, there's this whole AutoML movement.
999
00:45:03,840 --> 00:45:09,000
But yeah, that's interesting the way
1000
00:45:09,000 --> 00:45:13,040
that people can learn about a particular topic.
1001
00:45:13,040 --> 00:45:15,960
You can start to apply these things more easily.
1002
00:45:15,960 --> 00:45:17,920
So maybe they can get a better understanding
1003
00:45:17,920 --> 00:45:20,520
of the pros and cons of it so they can understand
1004
00:45:20,520 --> 00:45:23,720
what they're learning there at a deeper level.
1005
00:45:23,720 --> 00:45:25,480
But then you can also take that and apply it
1006
00:45:25,480 --> 00:45:26,640
to other fields as well.
1007
00:45:26,640 --> 00:45:31,960
So yeah, that's going to be a very interesting trend.
1008
00:45:31,960 --> 00:45:33,880
Maybe a loaded question, but how do you
1009
00:45:33,880 --> 00:45:38,760
think machine learning will change in the next 10 years?
1010
00:45:38,760 --> 00:45:42,280
I guess for you, what impact do you
1011
00:45:42,280 --> 00:45:45,640
think it will have on scientific discovery?
1012
00:45:45,640 --> 00:45:51,440
I think it will profoundly benefit
1013
00:45:51,440 --> 00:45:54,560
the set of scientists who understand
1014
00:45:54,560 --> 00:45:57,120
how to incorporate it in their work
1015
00:45:57,120 --> 00:45:59,840
and how to critically evaluate it.
1016
00:45:59,840 --> 00:46:02,760
It's a question I get a lot from my former colleagues,
1017
00:46:02,760 --> 00:46:04,360
for example, in the Mars spectroscopy
1018
00:46:04,360 --> 00:46:07,120
and structural biology.
1019
00:46:07,120 --> 00:46:10,160
I think it will continue to become more
1020
00:46:10,160 --> 00:46:12,800
of a part of the field.
1021
00:46:12,800 --> 00:46:17,760
So thinking about structures from alpha-fold,
1022
00:46:17,760 --> 00:46:19,880
can and how should they be deposited
1023
00:46:19,880 --> 00:46:21,920
into the protein data bank?
1024
00:46:21,920 --> 00:46:24,960
That's a thing to think about.
1025
00:46:24,960 --> 00:46:26,720
It's not my job to solve, but certainly it's
1026
00:46:26,720 --> 00:46:28,680
an interesting thing to think about.
1027
00:46:28,680 --> 00:46:29,640
What does that mean?
1028
00:46:29,640 --> 00:46:30,280
Any different?
1029
00:46:30,280 --> 00:46:34,480
I mean, structures are computed from other data.
1030
00:46:34,480 --> 00:46:35,600
I'm not sure it's different.
1031
00:46:35,600 --> 00:46:39,320
Maybe it is, but how do you represent that?
1032
00:46:39,320 --> 00:46:42,600
I think the fields in general, like I sort of alluded
to,
1033
00:46:42,600 --> 00:46:47,160
needs to find better ways to represent biology.
1034
00:46:47,160 --> 00:46:50,600
And right now, there is certainly no one way
1035
00:46:50,600 --> 00:46:51,360
to represent it.
1036
00:46:51,360 --> 00:46:53,480
I think that's a very open question.
1037
00:46:53,480 --> 00:46:56,800
With small molecules, there is language called SMILES.
1038
00:46:56,800 --> 00:46:57,440
There's graphs.
1039
00:46:57,440 --> 00:46:59,920
Graph models do really well, by the way, with small molecules.
1040
00:46:59,920 --> 00:47:03,040
There's three-dimensional representations as well.
1041
00:47:03,040 --> 00:47:03,800
Same with protein.
1042
00:47:03,800 --> 00:47:04,920
There's a variety of ways.
1043
00:47:04,920 --> 00:47:09,720
So I think that will crystallize a bit more.
1044
00:47:09,720 --> 00:47:12,400
Trying to think of what else.
1045
00:47:12,400 --> 00:47:15,360
Yeah, I think just really big picture.
1046
00:47:15,360 --> 00:47:17,560
I think those who learn how to use these tools
1047
00:47:17,560 --> 00:47:21,000
and how to interrogate them will figure out useful ways.
1048
00:47:21,000 --> 00:47:25,600
I figure out ways to use GPT and fluorine
1049
00:47:25,600 --> 00:47:28,000
my everyday life all the time.
1050
00:47:28,000 --> 00:47:31,240
We have internal NLP models at NVIDIA.
1051
00:47:31,240 --> 00:47:33,760
We don't enter proprietary data, of course, in public models.
1052
00:47:33,760 --> 00:47:36,800
But using those, playing around with those from time to time
1053
00:47:36,800 --> 00:47:38,960
to try to do things.
1054
00:47:38,960 --> 00:47:41,000
But you have to kind of experiment.
1055
00:47:41,000 --> 00:47:42,840
Like, it's not straightforward.
1056
00:47:42,840 --> 00:47:44,960
So you actually have to devote time
1057
00:47:44,960 --> 00:47:48,800
to figuring out how to do that.
1058
00:47:48,800 --> 00:47:50,440
I think, at least.
1059
00:47:50,440 --> 00:47:51,520
No, definitely.
1060
00:47:51,520 --> 00:47:55,600
And for technologists or for innovators,
1061
00:47:55,600 --> 00:47:59,360
having these tools then allows you to do so much more.
1062
00:47:59,360 --> 00:48:02,960
I mean, that's why it's so incredible.
1063
00:48:02,960 --> 00:48:04,520
I don't know.
1064
00:48:04,520 --> 00:48:07,820
The music industry is benefiting from all of this stuff.
1065
00:48:07,820 --> 00:48:10,320
I mean, The Beatles released a new song, right?
1066
00:48:10,320 --> 00:48:11,200
Like, that's the crazy.
1067
00:48:11,200 --> 00:48:12,880
That's nuts.
1068
00:48:12,880 --> 00:48:16,960
Because they were able to create a way of segmenting
1069
00:48:16,960 --> 00:48:20,880
John Lennon's voice from the TV in this old track
1070
00:48:20,880 --> 00:48:21,480
that they had.
1071
00:48:21,480 --> 00:48:26,080
And honestly, I listen to that track too often.
1072
00:48:26,080 --> 00:48:29,640
It's haunting for some reason.
1073
00:48:29,640 --> 00:48:32,520
Yeah, I don't know what the analogous thing is there
1074
00:48:32,520 --> 00:48:34,560
for science.
1075
00:48:34,560 --> 00:48:36,120
We'll have to see.
1076
00:48:36,120 --> 00:48:40,160
I mean, there are models now where
1077
00:48:40,160 --> 00:48:42,760
dynamics can be predicted.
1078
00:48:42,760 --> 00:48:45,440
So it's literally a model that does this,
1079
00:48:45,440 --> 00:48:49,520
that predicts the next frames of the simulation.
1080
00:48:49,520 --> 00:48:51,400
They've got a ways to go yet before they're
1081
00:48:51,400 --> 00:48:53,740
simulating proteins and simulating proteins
1082
00:48:53,740 --> 00:48:55,120
on the full time scale.
1083
00:48:55,120 --> 00:48:58,440
But maybe that's the analogy of what you're thinking of.
1084
00:48:58,440 --> 00:49:00,000
I'm not really sure.
1085
00:49:00,000 --> 00:49:01,000
Yeah.
1086
00:49:01,000 --> 00:49:03,000
I'm just saying how cool it is.
1087
00:49:03,000 --> 00:49:04,680
Yeah, yeah, yeah.
1088
00:49:04,680 --> 00:49:07,200
And just what becomes possible.
1089
00:49:07,200 --> 00:49:13,280
And then I just think about this common story
1090
00:49:13,280 --> 00:49:16,800
that they say, it's in a book, Prediction Machines.
1091
00:49:16,800 --> 00:49:18,520
Like when new technology comes out,
1092
00:49:18,520 --> 00:49:21,320
they talk about accountants.
1093
00:49:21,320 --> 00:49:25,240
And when Excel came out, there was this new spreadsheet tool
1094
00:49:25,240 --> 00:49:25,800
basically.
1095
00:49:25,800 --> 00:49:28,560
And then people were saying, oh, all accountants
1096
00:49:28,560 --> 00:49:29,400
are going to lose.
1097
00:49:29,400 --> 00:49:31,000
There's not going to be any accountants.
1098
00:49:31,000 --> 00:49:34,880
But all it did was it actually created so much more,
1099
00:49:34,880 --> 00:49:36,360
like a richer.
1100
00:49:36,360 --> 00:49:40,240
People could then apply more human thinking to it.
1101
00:49:40,240 --> 00:49:43,200
And they were able to do more art of finance
1102
00:49:43,200 --> 00:49:46,520
and create more sophisticated models.
1103
00:49:46,520 --> 00:49:50,640
And yeah, in some sense, it allows
1104
00:49:50,640 --> 00:49:52,280
you to do a higher level work.
1105
00:49:52,280 --> 00:49:55,080
It does sometimes make things more complex
1106
00:49:55,080 --> 00:49:56,560
and conflate things as well.
1107
00:49:56,560 --> 00:50:01,080
So it'll be interesting to see over, say, the next decade
1108
00:50:01,080 --> 00:50:04,760
what's just kind of hype and what are actually
1109
00:50:04,760 --> 00:50:08,960
tools that are enabling us to do these incredible things when
1110
00:50:08,960 --> 00:50:15,840
it comes to drug discovery, being able to try and create
1111
00:50:15,840 --> 00:50:18,680
all of these different things that can help us
1112
00:50:18,680 --> 00:50:23,440
fight different diseases, help us deal with,
1113
00:50:23,440 --> 00:50:26,480
create new medicines.
1114
00:50:26,480 --> 00:50:29,720
I mean, it's so exciting from the outside looking in.
1115
00:50:29,720
--> 00:50:31,440
It's such exciting work.
1116
00:50:31,440 --> 00:50:36,120
And knowing what thinking about what
1117
00:50:36,120 --> 00:50:40,880
the process is of trying to get a drug from idea to market
1118
00:50:40,880 --> 00:50:43,480
and how long that is, and if there's
1119
00:50:43,480 --> 00:50:49,640
anything that can be done to expedite that in a safe way,
1120
00:50:49,640 --> 00:50:54,520
is really pretty awesome.
1121
00:50:54,520 --> 00:50:58,440
It's probably the coolest thing to be working on.
1122
00:50:58,440 --> 00:50:59,840
Thanks.
1123
00:50:59,840 --> 00:51:01,960
And I mean, it's tremendously important.
1124
00:51:01,960 --> 00:51:06,960
We certainly don't wish another pandemic on the world.
1125
00:51:06,960 --> 00:51:09,720
But I think it's pretty likely it's
1126
00:51:09,720 --> 00:51:11,880
going to happen again, unfortunately.
1127
00:51:11,880 --> 00:51:14,200
So how do we be ready and think about the next one?
1128
00:51:14,200 --> 00:51:17,840
And how do we have the tools in place
1129
00:51:17,840 --> 00:51:23,560
so that we don't have to scale up vaccine manufacturing as much
1130
00:51:23,560 --> 00:51:25,480
as we did this time, right?
1131
00:51:25,480 --> 00:51:28,680
So yeah.
1132
00:51:28,680 --> 00:51:29,920
Yeah.
1133
00:51:29,920 --> 00:51:32,360
Yeah, obviously, we don't want anything like that to happen.
1134
00:51:32,360 --> 00:51:34,280
But it's sort of inevitable that there will
1135
00:51:34,280 --> 00:51:35,880
be something along that level.
1136
00:51:35,880 --> 00:51:43,120
But knowing that there are teams and companies and research
1137
00:51:43,120 --> 00:51:47,800
institutes that have the tools that can enable them to quickly
1138
00:51:47,800 --> 00:51:52,560
combat that sort of stuff is a little reassuring.
1139
00:51:52,560 --> 00:51:54,840
There's still the human element.
1140
00:51:54,840 --> 00:51:57,480
People have to actually do.
1141
00:51:57,480 --> 00:51:58,560
I know.
1142
00:51:58,560 --> 00:52:01,640
But at least you can just give people the tools
1143
00:52:01,640 --> 00:52:07,240
to do the best they can.
1144
00:52:07,240 --> 00:52:10,920
So switching gears into the learning from machine learning
1145
00:52:10,920 --> 00:52:14,960
aspect, we'll get into some advice questions,
1146
00:52:14,960 --> 00:52:18,040
everyone's favorite type of question.
1147
00:52:18,040 --> 00:52:22,040
For people who are just starting out in the field thinking,
1148
00:52:22,040 --> 00:52:23,840
hey, I want to be a data scientist,
1149
00:52:23,840 --> 00:52:27,040
or that they're doing some biochem stuff,
1150
00:52:27,040 --> 00:52:31,320
what's advice that you would give to people that are just
1151
00:52:31,320 --> 00:52:33,280
starting out?
1152
00:52:33,280 --> 00:52:36,400
I think this one's probably always data, data, data.
1153
00:52:36,400 --> 00:52:39,920
It's always about the data.
1154
00:52:39,920 --> 00:52:42,600
I think the cool thing with machine learning,
1155
00:52:42,600 --> 00:52:43,760
it's a very open community.
1156
00:52:43,760 --> 00:52:47,800
There are folks who participate in Kaggle competitions
1157
00:52:47,800 --> 00:52:50,760
or something if you want to get to know a particular domain
1158
00:52:50,760 --> 00:52:51,280
better.
1159
00:52:51,280 --> 00:52:54,840
There are active discussions there on those competitions.
1160
00:52:54,840 --> 00:52:58,720
You can learn more about the field that way.
1161
00:52:58,720 --> 00:53:03,160
That's something that's relatively accessible to all people.
1162
00:53:03,160 --> 00:53:05,920
Yeah, I think that's the biggest thing is data.
1163
00:53:05,920 --> 00:53:08,880
I think in this field, you need to figure out
1164
00:53:08,880 --> 00:53:12,320
what works for you as a way to continue to learn.
1165
00:53:12,320 --> 00:53:15,120
And that isn't just reading papers,
1166
00:53:15,120 --> 00:53:19,640
or maybe it's testing a few new tools as they come out,
1167
00:53:19,640 --> 00:53:22,560
these new visualization and interpretation tools
1168
00:53:22,560 --> 00:53:24,800
as you were alluding to.
1169
00:53:24,800 --> 00:53:25,920
Maybe it's that.
1170
00:53:25,920 --> 00:53:28,440
Maybe it's continuing to refine your software development
1171
00:53:28,440 --> 00:53:29,800
skills.
1172
00:53:29,800 --> 00:53:31,000
Maybe it's reading papers.
1173
00:53:31,000 --> 00:53:32,720
It depends what you're doing.
1174
00:53:32,720 --> 00:53:36,360
But figuring out a way to do that is important.
1175
00:53:36,360 --> 00:53:38,040
I think it's harder as you get older.
1176
00:53:38,040 --> 00:53:40,560
I mean, I was completely self-taught with machine
1177
00:53:40,560 --> 00:53:42,520
learning.
1178
00:53:42,520 --> 00:53:45,240
But I used to get up at 5 AM on Saturday
1179
00:53:45,240 --> 00:53:48,800
and do all my machine learning courses and my homework
1180
00:53:48,800 --> 00:53:49,360
for the week.
1181
00:53:49,360 --> 00:53:51,280
And man, I can't even.
1182
00:53:51,280 --> 00:53:52,440
It was hard.
1183
00:53:52,440 --> 00:53:53,320
Yeah.
1184
00:53:53,320 --> 00:53:55,640
Yeah.
1185
00:53:55,640 --> 00:53:57,360
And Andrew Ng class.
1186
00:53:57,360 --> 00:53:57,880
Yeah.
1187
00:53:57,880 --> 00:53:58,400
Oh, yeah.
1188
00:53:58,400 --> 00:53:59,720
That stuff, Jeff Hinton's class.
1189
00:53:59,720 --> 00:54:00,800
Yeah.
1190
00:54:00,800 --> 00:54:04,200
That was how I did it.
1191
00:54:04,200 --> 00:54:06,880
And then, yeah, certainly I've kept learning since then.
1192
00:54:06,880 --> 00:54:09,600
The field has changed profoundly since then even.
1193
00:54:09,600 --> 00:54:10,320
Yeah.
1194
00:54:10,320 --> 00:54:12,960
That's a trait that I find, I guess,
1195
00:54:12,960 --> 00:54:16,680
in data scientists and software engineers,
1196
00:54:16,680 --> 00:54:20,640
the really good ones, it's just they're capable.
1197
00:54:20,640 --> 00:54:23,720
But then it's just this idea of just continuous learning.
1198
00:54:23,720 --> 00:54:28,080
I'm going
to continue to learn the newest things out there.
1199
00:54:28,080 --> 00:54:29,200
But then they have-
1200
00:54:29,200 --> 00:54:32,440
Tools, VS Code, whatever it is.
1201
00:54:32,440 --> 00:54:34,440
Master the tools.
1202
00:54:34,440 --> 00:54:36,200
But it's not mastering all your tools.
1203
00:54:36,200 --> 00:54:38,040
It's mastering the tools that matter.
1204
00:54:38,040 --> 00:54:39,880
And that's the hard part.
1205
00:54:39,880 --> 00:54:43,640
And it's the ability to master a tool.
1206
00:54:43,640 --> 00:54:47,400
And I think that's like school.
1207
00:54:47,400 --> 00:54:49,280
It's not like everything that you learn in school
1208
00:54:49,280 --> 00:54:50,880
you're actually applying in your job.
1209
00:54:50,880 --> 00:54:54,600
But for someone in a position like you,
1210
00:54:54,600 --> 00:55:00,480
you can consume research papers probably better than 99.9%
1211
00:55:00,480 --> 00:55:02,560
of the world at this point.
1212
00:55:02,560 --> 00:55:05,640
And being able to take the pieces of it
1213
00:55:05,640 --> 00:55:08,480
that are applicable for you.
1214
00:55:08,480 --> 00:55:14,400
But yeah, in this field, it's important to understand,
1215
00:55:14,400 --> 00:55:17,560
yeah, which tools are important and how quickly can I
1216
00:55:17,560 --> 00:55:19,720
get onboarded onto that tool.
1217
00:55:19,720 --> 00:55:21,000
Right, right.
1218
00:55:21,000 --> 00:55:24,120
Yeah, a little variation of the last one.
1219
00:55:24,120 --> 00:55:27,160
What advice would you give yourself, I guess,
1220
00:55:27,160 --> 00:55:29,040
earlier in your career?
1221
00:55:29,040 --> 00:55:32,440
Yeah, I was certainly when I was doing this transition,
1222
00:55:32,440 --> 00:55:34,600
I was very intimidated.
1223
00:55:34,600 --> 00:55:40,440
It's a big thing to sort of make a switch from something
1224
00:55:40,440 --> 00:55:44,360
I'd spent at that time, I don't know, 11, 12 years of my life
1225
00:55:44,360 --> 00:55:45,240
studying this.
1226
00:55:45,240 --> 00:55:47,600
And certainly, I still get to work
1227
00:55:47,600 --> 00:55:48,840
at the intersection of science.
1228
00:55:48,840 --> 00:55:51,480
But it's a very different type of job.
1229
00:55:51,480 --> 00:55:52,480
I love it.
1230
00:55:52,480 --> 00:55:57,280
But it was very intimidating and stressful at the time
1231
00:55:57,280 --> 00:55:58,960
to make that jump.
1232
00:55:58,960 --> 00:56:01,520
Right, right.
1233
00:56:01,520 --> 00:56:03,280
So what would it be?
1234
00:56:03,280 --> 00:56:06,440
Just you're going to make it through?
1235
00:56:06,440 --> 00:56:07,440
It's going to be OK?
1236
00:56:07,440 --> 00:56:08,920
Or don't be intimidated?
1237
00:56:08,920 --> 00:56:09,920
Don't be intimidated.
1238
00:56:09,920 --> 00:56:14,640
And I think keep your ear to the ground about things like this.
1239
00:56:14,640 --> 00:56:19,200
Don't be so focused on your, it's
1240
00:56:19,200 --> 00:56:21,440
important to focus on your domain, your world
1241
00:56:21,440 --> 00:56:26,120
as a structural biologist, as an NMR spectroscopist as I was.
1242
00:56:26,120 --> 00:56:29,560
But it's also important to be aware of other trends
1243
00:56:29,560 --> 00:56:31,800
and think about it.
1244
00:56:31,800 --> 00:56:35,280
Because I think the people that maybe are earlier on
1245
00:56:35,280 --> 00:56:38,320
in the field figured out that, oh, yeah, this is a thing.
1246
00:56:38,320 --> 00:56:40,000
I should go learn about this.
1247
00:56:40,000 --> 00:56:42,200
There's this whole revolution going on outside
1248
00:56:42,200 --> 00:56:44,440
of my purview of my day to day.
1249
00:56:44,440 --> 00:56:46,720
Maybe I should at least try to understand it.
1250
00:56:46,720 --> 00:56:48,400
Right.
1251
00:56:48,400 --> 00:56:53,120
Yeah, keep a pulse on the progress that's
1252
00:56:53,120 --> 00:56:55,760
taking place in other fields.
1253
00:56:55,760 --> 00:56:56,440
Yeah.
1254
00:56:56,440 --> 00:56:58,280
That's something that I learn.
1255
00:56:58,280 --> 00:57:00,520
Well, I mean, I guess I've always sort of tried to do it.
1256
00:57:00,520 --> 00:57:02,200
Just my interests have been so varied
1257
00:57:02,200 --> 00:57:03,440
that I've been able to do it.
1258
00:57:03,440 --> 00:57:09,800
But recently, I listened to Jeremy Howard from Vast AI,
1259
00:57:09,800 --> 00:57:14,400
just really hammering that point home.
1260
00:57:14,400 --> 00:57:16,520
If you're interested in natural language processing,
1261
00:57:16,520 --> 00:57:19,280
it's OK to learn something about computer vision.
1262
00:57:19,280 --> 00:57:20,760
If you're interested in computer vision,
1263
00:57:20,760 --> 00:57:22,440
you can do activity recognition.
1264
00:57:22,440 --> 00:57:25,320
Whatever it is, there'll be some thing
1265
00:57:25,320 --> 00:57:28,040
that you learn in signal processing that will help you.
1266
00:57:28,040 --> 00:57:30,440
Because when it comes down to it,
1267
00:57:30,440 --> 00:57:33,640
it's representing things numerically.
1268
00:57:33,640 --> 00:57:35,760
And it's doing manipulations to it.
1269
00:57:35,760 --> 00:57:37,400
It's pattern matching.
1270
00:57:37,400 --> 00:57:38,280
Right.
1271
00:57:38,280 --> 00:57:38,800
Yep.
1272
00:57:38,800 --> 00:57:40,840
So there's always a way.
1273
00:57:40,840 --> 00:57:41,600
Oh, yeah, sorry.
1274
00:57:41,600 --> 00:57:44,360
That's why I said that I think there
1275
00:57:44,360 --> 00:57:48,360
is a lot of value in having domain expertise,
1276
00:57:48,360 --> 00:57:50,960
and knowing and understanding something deeply what works
1277
00:57:50,960 --> 00:57:52,280
and what doesn't work.
1278
00:57:52,280 --> 00:57:54,920
And I don't actually believe that it prevents you
1279
00:57:54,920 --> 00:57:57,720
from switching domains later on if you want.
1280
00:57:57,720 --> 00:58:00,480
I think it is an asset because you
1281
00:58:00,480 --> 00:58:03,600
have dealt with some very fundamental problems
1282
00:58:03,600 -->
00:58:05,640
with data and modeling.
1283
00:58:05,640 --> 00:58:10,280
And you will see them apply, many of them in different ways,
1284
00:58:10,280 --> 00:58:12,040
perhaps, in that other domain.
1285
00:58:12,040 --> 00:58:15,440
But I think it teaches you what it takes to really interrogate
1286
00:58:15,440 --> 00:58:18,400
the data and build a good model.
1287
00:58:18,400 --> 00:58:20,960
Absolutely.
1288
00:58:20,960 --> 00:58:24,240
Something that's interesting, I know
1289
00:58:24,240 --> 00:58:30,120
that you work in the field with so many things going on.
1290
00:58:30,120 --> 00:58:33,960
But who are some people in the field that influence
1291
00:58:33,960 --> 00:58:37,320
you and your work?
1292
00:58:37,320 --> 00:58:40,280
I would say certainly my colleagues at NVIDIA.
1293
00:58:40,280 --> 00:58:47,160
They are tremendously talented and make me rethink and think
1294
00:58:47,160 --> 00:58:49,800
hard about problems every day and sometimes
1295
00:58:49,800 --> 00:58:50,760
help me when I'm stuck.
1296
00:58:50,760 --> 00:58:52,480
Sometimes I help them when I'm stuck.
1297
00:58:52,480 --> 00:58:55,800
I feel very, very fortunate to work
1298
00:58:55,800 --> 00:58:58,280
at such an incredible place.
1299
00:58:58,280 --> 00:59:00,880
I would say my postdoctoral advisor, Art Palmer,
1300
00:59:00,880 --> 00:59:05,280
and my grad school advisor, Patrick Loria and Scott Strobel,
1301
00:59:05,280 --> 00:59:08,520
they taught me to think very carefully and critically
1302
00:59:08,520 --> 00:59:12,840
about what I do and how to think about problems where they're
1303
00:59:12,840 --> 00:59:16,240
empirically, the problems that are very empirical.
1304
00:59:16,240 --> 00:59:18,120
And it actually has a lot of applications
1305
00:59:18,120 --> 00:59:20,400
in machine learning because it's a very empirical field
1306
00:59:20,400 --> 00:59:22,360
in many ways.
1307
00:59:22,360 --> 00:59:25,000
Certainly in the domain that I'm specifically in,
1308
00:59:25,000 --> 00:59:27,320
I think Alex Reibs from Evolution A Scale,
1309
00:59:27,320 --> 00:59:32,560
they've built some of the best, very, very well-known protein
1310
00:59:32,560 --> 00:59:34,920
representation models out there.
1311
00:59:34,920 --> 00:59:36,600
Deborah Marks, who's a professor at Harvard,
1312
00:59:36,600 --> 00:59:40,080
has done amazing things in machine learning
1313
00:59:40,080 --> 00:59:43,840
and development of some very fundamental techniques
1314
00:59:43,840 --> 00:59:49,040
with protein sequences, as well as does experiments
1315
00:59:49,040 --> 00:59:50,080
to back up the work.
1316
00:59:50,080 --> 00:59:54,400
So I think anyone who does both wet lab experiments
1317
00:59:54,400 --> 00:59:56,960
and machine learning, I think, gets a special place
1318
00:59:56,960 --> 00:59:59,280
of recognition from me, at least,
1319
00:59:59,280 --> 01:00:01,080
because I believe that's very hard to do.
1320
01:00:01,080 --> 01:00:04,240
And it really, really informs the work you do.
1321
01:00:04,240 --> 01:00:06,160
Yeah, absolutely.
1322
01:00:06,160 --> 01:00:08,640
Yeah, that makes a lot of sense.
1323
01:00:08,640 --> 01:00:11,840
And being on learning from machine learning,
1324
01:00:11,840 --> 01:00:14,360
I get to ask this question.
1325
01:00:14,360 --> 01:00:19,080
What has a career in machine learning taught you about life?
1326
01:00:19,080 --> 01:00:21,760
Yeah, I would say, so the importance of continuing,
1327
01:00:21,760 --> 01:00:25,720
of continually learning, whatever it means in your life,
1328
01:00:25,720 --> 01:00:30,160
whether it's, I learned how to snow ski two years ago.
1329
01:00:30,160 --> 01:00:34,760
And I wish I'd learned when I was young and made of rubber.
1330
01:00:34,760 --> 01:00:36,760
I'm not afraid to bite it, but I'm
1331
01:00:36,760 --> 01:00:40,040
a lot more afraid of tearing an ACL as an adult.
1332
01:00:40,040 --> 01:00:40,880
But it's fun.
1333
01:00:40,880 --> 01:00:44,080
And so this Christmas, we're going skiing.
1334
01:00:44,080 --> 01:00:46,600
And I'm excited to eventually get to the point
1335
01:00:46,600 --> 01:00:51,160
where I'm able to ski on blues pretty regularly.
1336
01:00:51,160 --> 01:00:52,880
My husband is an excellent skier.
1337
01:00:52,880 --> 01:00:55,360
So at some point, I want to get there.
1338
01:00:55,360 --> 01:00:56,760
That's my goal.
1339
01:00:56,760 --> 01:00:58,200
Nice.
1340
01:00:58,200 --> 01:01:01,960
I think how to debug complex empirical problems
1341
01:01:01,960 --> 01:01:04,440
and how to think rationally about, like, how do I try this?
1342
01:01:04,440 --> 01:01:06,920
And how do I debug this quickly?
1343
01:01:06,920 --> 01:01:11,800
Because it's not always as easy as you might think to solve it.
1344
01:01:11,800 --> 01:01:15,160
And then I think, I never really thought about classification
1345
01:01:15,160 --> 01:01:17,320
problems that much as a scientist.
1346
01:01:17,320 --> 01:01:20,280
So it really makes you think about,
1347
01:01:20,280 --> 01:01:23,320
there's many ways of being wrong.
1348
01:01:23,320 --> 01:01:25,120
And sometimes, some of those ways
1349
01:01:25,120 --> 01:01:27,880
matter a lot more than others.
1350
01:01:27,880 --> 01:01:30,960
So really understand what being wrong means
1351
01:01:30,960 --> 01:01:32,880
and what you need to optimize for.
1352
01:01:32,880 --> 01:01:35,120
It's not always accuracy, for example.
1353
01:01:35,120 --> 01:01:37,800
Sometimes false positives and false negatives,
1354
01:01:37,800 --> 01:01:40,600
they don't always carry the same weight.
1355
01:01:40,600 --> 01:01:42,080
Yeah, you beat me to it.
1356
01:01:42,080 --> 01:01:43,440
I was going to say, everyone wants
1357
01:01:43,440 --> 01:01:46,480
to know the accuracy of the model.
1358
01:01:46,480 --> 01:01:51,040
And I try to say, all errors are not equal.
1359
01:01:51,040 --> 01:01:52,760
They still want to just know the accuracy.
1360
01:01:52,760 --> 01:01:55,160
They don't care.
1361
01:01:55,160 --> 01:01:56,920
I try to explain it to them.
1362
01:01:56,920 --> 01:01:58,760
But then if you bring out a confusion matrix,
1363
01:01:58,760 --> 01:02:00,520
it gets a little too confusing.
1364
01:02:00,520 --> 01:02:03,920
But no, it's important the takeaway of just like,
1365
01:02:03,920 --> 01:02:05,600
all errors aren't the same.
1366
01:02:05,600 --> 01:02:10,240
And some matter much more than others.
1367
01:02:10,240 --> 01:02:13,680
That's a really good takeaway.
1368
01:02:13,680 --> 01:02:16,320
And then just about the debugging complex empirical
1369
01:02:16,320 --> 01:02:16,680
problems.
1370
01:02:16,680 --> 01:02:17,480
So what is it?
1371
01:02:17,480 --> 01:02:18,400
What makes it so hard?
1372
01:02:18,400 --> 01:02:20,120
Just all the moving parts?
1373
01:02:20,120 --> 01:02:21,640
Sometimes it could take a long.
1374
01:02:21,640 --> 01:02:25,120
If you have a problem, for example, with,
1375
01:02:25,120 --> 01:02:27,160
I'm just going to pick something, training
1376
01:02:27,160 --> 01:02:28,640
where it starts to throw nans.
1377
01:02:28,640 --> 01:02:31,160
But it doesn't happen until pretty far into it.
1378
01:02:31,160 --> 01:02:33,680
How can you figure out what the problem is
1379
01:02:33,680 --> 01:02:39,200
without minimizing the time it takes to solve the problem?
1380
01:02:39,200 --> 01:02:40,680
Are there things you can scale down?
1381
01:02:40,680 --> 01:02:44,320
Can you figure out if there's a piece of data point?
1382
01:02:44,320 --> 01:02:46,720
Can you figure out if, like just learning
1383
01:02:46,720 --> 01:02:51,680
how to debug that in an efficient way is,
1384
01:02:51,680 --> 01:02:52,880
it can be really tricky.
1385
01:02:52,880 --> 01:02:53,880
I don't know.
1386
01:02:53,880 --> 01:02:55,640
I certainly think it's hard.
1387
01:02:55,640 --> 01:02:56,720
No, absolutely.
1388
01:02:56,720 --> 01:02:57,240
Absolutely.
1389
01:02:57,240 --> 01:03:00,880
And being able to approach those problems
1390
01:03:00,880 --> 01:03:07,040
in a cool, calm, and collected way, you will get, well,
1391
01:03:07,040 --> 01:03:09,800
most of the time, you'll be able to figure it out.
1392
01:03:09,800 --> 01:03:12,560
Yeah.
1393
01:03:12,560 --> 01:03:14,360
Sometimes I get CUDA in it errors,
1394
01:03:14,360 --> 01:03:15,440
and then I don't like those.
1395
01:03:15,440 --> 01:03:15,920
Oh, no.
1396
01:03:15,920 --> 01:03:16,680
Because I'm like, oh, boy.
1397
01:03:16,680 --> 01:03:17,640
Oh, boy.
1398
01:03:17,640 --> 01:03:20,920
Yeah, well, then you have to go ask someone.
1399
01:03:20,920 --> 01:03:23,960
This is going to be a while.
1400
01:03:23,960 --> 01:03:24,440
Yeah.
1401
01:03:24,440 --> 01:03:25,480
CUDA alignment errors.
1402
01:03:25,480 --> 01:03:26,560
Those are not my friends.
1403
01:03:26,560 --> 01:03:28,160
No.
1404
01:03:28,160 --> 01:03:29,880
Or those out of memory errors.
1405
01:03:29,880 --> 01:03:30,880
And it's like, now?
1406
01:03:30,880 --> 01:03:32,800
This is when you're going to throw me that one?
1407
01:03:32,800 --> 01:03:33,320
OK, fine.
1408
01:03:33,320 --> 01:03:35,000
Yeah.
1409
01:03:35,000 --> 01:03:36,480
Yes.
1410
01:03:36,480 --> 01:03:39,480
Wow, Michelle, it's been such a pleasure.
1411
01:03:39,480 --> 01:03:43,520
This was so cool to dive into this area.
1412
01:03:43,520 --> 01:03:43,920
Thanks.
1413
01:03:43,920 --> 01:03:45,040
It was my pleasure, too.
1414
01:03:45,040 --> 01:03:46,200
Yeah.
1415
01:03:46,200 --> 01:03:49,040
Understanding how people even begin
1416
01:03:49,040 --> 01:03:55,240
to approach how to do drug discovery, leveraging AI,
1417
01:03:55,240 --> 01:03:57,920
it's such a fascinating field.
1418
01:03:57,920 --> 01:03:59,960
I think I'm going to use this as motivation
1419
01:03:59,960 --> 01:04:03,280
to continue to learn more about this.
1420
01:04:03,280 --> 01:04:06,960
For people that want to learn more about you and your work,
1421
01:04:06,960 --> 01:04:08,840
where would a good place to go?
1422
01:04:08,840 --> 01:04:11,320
Probably my website.
1423
01:04:11,320 --> 01:04:13,720
My papers, talks end up there.
1424
01:04:13,720 --> 01:04:15,160
I do need to update it.
1425
01:04:15,160 --> 01:04:16,520
That's a holiday project.
1426
01:04:16,520 --> 01:04:18,200
This is to go add a few more things,
1427
01:04:18,200 --> 01:04:20,880
including my Pi Data Talk.
1428
01:04:20,880 --> 01:04:23,320
But that's probably the best place.
1429
01:04:23,320 --> 01:04:25,920
I have Twitter and Mastodon.
1430
01:04:25,920 --> 01:04:28,560
Well, I guess X these days.
1431
01:04:28,560 --> 01:04:30,680
I don't tweet as much these days.
1432
01:04:30,680 --> 01:04:31,720
Maybe that will change.
1433
01:04:31,720 --> 01:04:33,560
I'm a modern scientist on Twitter.
1434
01:04:33,560 --> 01:04:36,200
But whatever accounts I'm using at the moment
1435
01:04:36,200 --> 01:04:38,400
will be linked to from my web page.
1436
01:04:38,400 --> 01:04:40,600
So it's michellelingale.com.
1437
01:04:40,600 --> 01:04:41,480
Cool.
1438
01:04:41,480 --> 01:04:44,440
And I can add some of those to the show notes as well.
1439
01:04:44,440 --> 01:04:47,720
And I encourage listeners to definitely check out
1440
01:04:47,720 --> 01:04:50,600
your most recent keynote at Pi Data NYC.
1441
01:04:50,600 --> 01:04:53,040
I'll add that as well.
1442
01:04:53,040 --> 01:04:55,000
What an amazing talk.
1443
01:04:55,000 --> 01:04:56,440
Michelle, thank you so much.
1444
01:04:56,440 --> 01:04:58,320
I really appreciate you giving me the time
1445
01:04:58,320 --> 01:05:00,200
and letting me pick your brain for a bit.
1446
01:05:00,200 --> 01:05:01,480
Yeah, it was all my pleasure.
1447
01:05:01,480 --> 01:05:02,200
Thank you.
1448
01:05:02,200 --> 01:05:03,400
Thanks for the invitation.
1449
01:05:03,400 --> 01:05:03,880
Thank you.
1450
01:05:03,880 --> 01:05:04,360
Thank you.
1451
01:05:04,360 --> 01:05:13,560
On this episode of Learning from Machine Learning,
1452
01:05:13,560 --> 01:05:16,440
Dr. Michelle Gill shared her incredible journey
1453
01:05:16,440 --> 01:05:19,520
from wet lab biochemist to driving cutting edge
1454
01:05:19,520 --> 01:05:21,400
AI at NVIDIA.
1455
01:05:21,400 --> 01:05:23,840
Her work helps address one of health care's biggest
1456
01:05:23,840 --> 01:05:26,400
challenges by enabling researchers
1457
01:05:26,400 --> 01:05:30,320
to do drug discovery, both faster and better.
1458
01:05:30,320 --> 01:05:32,920
Michelle discussed the critical need for better machine
1459
01:05:32,920 --> 01:05:36,360
learning representations for biological structures
1460
01:05:36,360 --> 01:05:40,520
and her insights on leading a multidisciplinary team.
1461
01:05:40,520 --> 01:05:41,840
Thank you for listening.
1462
01:05:41,840 --> 01:05:44,280
And remember to subscribe and share
1463
01:05:44,280 --> 01:05:46,160
with your friends and colleagues.
1464
01:05:46,160 --> 01:05:51,680
Until next time, keep on learning.