1
00:00:06,497 -->
00:00:07,907
Josh Rubin: So today we have, uh,
2
00:00:07,907 -->
00:00:10,547
a really special guest, um, Jeff
3
00:00:10,547 -->
00:00:12,137
Dalton, who's the Head of AI and,
4
00:00:12,137 -->
00:00:14,027
and Chief Scientist at Valence.
5
00:00:14,387 -->
00:00:16,067
He's also a professor of informatics
6
00:00:16,067 -->
00:00:17,237
at the University of Edinburgh.
7
00:00:18,137 -->
00:00:19,847
he's a world expert in,
8
00:00:19,847 -->
00:00:21,137
in conversational search.
9
00:00:21,497 -->
00:00:22,517
And has been working on this
10
00:00:22,517 -->
00:00:25,187
for, uh, at least a decade, um,
11
00:00:25,187 -->
00:00:26,597
in a lot of different contexts.
12
00:00:26,627 -->
00:00:29,597
Uh, he's leading, the development
13
00:00:29,597 -->
00:00:33,227
work behind, uh, Nadia at Valence,
14
00:00:33,227 -->
00:00:34,397
which is a chat assistant that's
15
00:00:34,397 -->
00:00:36,347
intended for coaching and improving
16
00:00:36,347 -->
00:00:37,637
people within enterprises and
17
00:00:37,637 -->
00:00:39,257
their enterprise specific contexts.
18
00:00:40,457 -->
00:00:42,617
so, uh, with that,
19
00:00:42,617 -->
00:00:43,877
welcome, welcome to Jeff.
20
00:00:44,747 -->
00:00:46,007
You wanna wanna pop up Jeff?
21
00:00:46,367 -->
00:00:46,817
Hi.
22
00:00:47,927 -->
00:00:49,967
So I, I thought I would start out by
23
00:00:49,967 -->
00:00:52,247
asking you a little bit about, you
24
00:00:52,247 -->
00:00:53,237
know, sort of your history you've
25
00:00:53,237 -->
00:00:54,587
been working on, like conversational
26
00:00:54,587 -->
00:00:57,347
search and conversation in the
27
00:00:57,377 -->
00:00:59,447
search and AI space for a long time.
28
00:00:59,927 -->
00:01:01,517
You know, obviously the last 15
29
00:01:01,517 -->
00:01:04,487
years have been like, radical in
30
00:01:04,487 -->
00:01:05,927
the amount of change year to year.
31
00:01:06,512 -->
00:01:06,842
Jeff Dalton: Indeed.
32
00:01:06,897 -->
00:01:07,127
Josh Rubin: I thought you might want
33
00:01:07,127 -->
00:01:08,117
to a little bit about kind of what you
34
00:01:08,117 -->
00:01:09,887
focused on over that period at various
35
00:01:09,887 -->
00:01:12,167
places that you've worked, um, you know,
36
00:01:12,167 -->
00:01:13,817
and, and how dramatic these changes
37
00:01:13,817 -->
00:01:15,557
are and, and what you've seen happen,
38
00:01:15,617 -->
00:01:16,817
how that's changed the landscape.
39
00:01:18,002 -->
00:01:18,452
Jeff Dalton: Yeah.
40
00:01:18,542 -->
00:01:19,202
Thank you Josh.
41
00:01:19,382 -->
00:01:20,942
Um, so a little bit about
42
00:01:20,942 -->
00:01:21,632
me and my background.
43
00:01:21,692 -->
00:01:22,802
Um, I, I actually started
44
00:01:22,862 -->
00:01:24,032
my career in search.
45
00:01:24,392 -->
00:01:26,132
So after I graduated, I worked at
46
00:01:26,162 -->
00:01:27,572
a startup search engine back then.
47
00:01:27,572 -->
00:01:28,982
This was almost 20 years ago now.
48
00:01:28,982 -->
00:01:30,032
So it tells you how old I am,
49
00:01:30,392 -->
00:01:31,562
but, um, then I went and I, and
50
00:01:31,562 -->
00:01:33,452
I studied it in a, in research.
51
00:01:33,772 -->
00:01:35,722
Um, and so my, my background
52
00:01:35,722 -->
00:01:36,952
straddles that intersection
53
00:01:36,952 -->
00:01:38,227
of research and industry.
54
00:01:38,852 -->
00:01:40,082
That, and I kind of bounced back and
55
00:01:40,082 -->
00:01:42,392
forth to making sure that we ha we're
56
00:01:42,392 -->
00:01:43,952
working on important things and we're,
57
00:01:43,952 -->
00:01:44,942
we're at the cutting edge of the
58
00:01:44,942 -->
00:01:47,072
frontier and we're also, uh, making that
59
00:01:47,072 -->
00:01:48,782
out and get, getting that to people.
60
00:01:49,082 -->
00:01:50,942
Um, so when I did my PhD was in search,
61
00:01:50,942 -->
00:01:52,022
we were working on knowledge graphs.
62
00:01:52,082 -->
00:01:53,132
How do we make knowledge graphs
63
00:01:53,132 -->
00:01:54,392
successful to search engines?
64
00:01:54,392 -->
00:01:55,142
How do we make that
65
00:01:55,142 -->
00:01:56,612
really useful for people?
66
00:01:57,092 -->
00:01:58,412
Then when I, I got that, I went and
67
00:01:58,412 -->
00:01:59,882
worked for Google, surprisingly enough, I
68
00:01:59,882 -->
00:02:01,592
worked on health search there at Google,
69
00:02:01,742 -->
00:02:03,362
and what we really realized was that we.
70
00:02:03,737 -->
00:02:04,787
The search box wasn't enough.
71
00:02:04,787 -->
00:02:05,987
We had to shift from, this
72
00:02:05,987 -->
00:02:07,217
is a conversation with
73
00:02:07,217 -->
00:02:08,207
the state about the user.
74
00:02:08,207 -->
00:02:10,427
So we worked to moving over to
75
00:02:10,427 -->
00:02:11,567
working on the Google Assistant,
76
00:02:11,897 -->
00:02:13,637
um, having conversational systems
77
00:02:13,637 -->
00:02:15,167
that, um, could start to do that.
78
00:02:15,287 -->
00:02:16,607
But they were really basic, right?
79
00:02:16,637 -->
00:02:17,927
We could, we, we could, you know,
80
00:02:17,927 -->
00:02:18,917
turn the lights off and on, play
81
00:02:18,917 -->
00:02:20,687
some songs, but what what we really
82
00:02:20,777 -->
00:02:22,427
needed was a health kind of virtual
83
00:02:22,457 -->
00:02:23,807
assistant that was conversational.
84
00:02:24,407 -->
00:02:25,097
So that was, that was
85
00:02:25,097 -->
00:02:26,597
motivating it my work.
86
00:02:26,837 -->
00:02:28,487
But going, how that relates to
87
00:02:28,487 -->
00:02:30,197
search was we wanted to have that
88
00:02:30,197 -->
00:02:31,457
conversational assistant be able to
89
00:02:31,457 -->
00:02:33,257
do work on hard research problems.
90
00:02:33,617 -->
00:02:34,907
So research your health, need
91
00:02:34,907 -->
00:02:36,917
research complex information needs.
92
00:02:37,157 -->
00:02:38,447
So that's one of what I'm going back and
93
00:02:38,447 -->
00:02:40,127
kind of have built a research lab around
94
00:02:40,127 -->
00:02:41,212
conversational search and assistance.
95
00:02:42,112 -->
00:02:44,092
So we worked on building evaluation
96
00:02:44,092 -->
00:02:46,042
benchmarks 'cause they didn't exist.
97
00:02:46,372 -->
00:02:47,662
Um, so, uh, for example, I built some
98
00:02:47,662 -->
00:02:49,432
of the first evaluation benchmarks for
99
00:02:49,432 -->
00:02:50,992
conversational search with partners
100
00:02:50,992 -->
00:02:53,032
at Microsoft and Carnegie Mellon.
101
00:02:53,272 -->
00:02:54,772
Uh, the TREC, conversational assistance
102
00:02:54,772 -->
00:02:57,894
track way back in 20 2018, 2019.
103
00:02:58,462 -->
00:03:00,292
Uh, ancient history now in these days.
104
00:03:00,682 -->
00:03:00,742
Um.
105
00:03:01,607 -->
00:03:03,527
What we started to realize was at the
106
00:03:03,527 -->
00:03:05,417
time it was really hard with the language
107
00:03:05,417 -->
00:03:06,917
and, and the evaluation tools that we
108
00:03:06,917 -->
00:03:08,417
had, kind of traditional NLP pipelines.
109
00:03:08,927 -->
00:03:10,487
And we really noticed a shift with GPT.
110
00:03:10,487 -->
00:03:12,227
Two things that we thought were going
111
00:03:12,227 -->
00:03:15,707
to take, like be hard, um, was uh,
112
00:03:15,737 -->
00:03:17,747
suddenly became really easy and we
113
00:03:17,747 -->
00:03:19,337
solved within six months or a year.
114
00:03:19,817 -->
00:03:22,127
So we continually up far some of
115
00:03:22,127 -->
00:03:23,537
the, those really hard benchmarks.
116
00:03:23,537 -->
00:03:25,967
Hard hand curated human data
117
00:03:26,147 -->
00:03:27,467
is still really fundamental.
118
00:03:27,857 -->
00:03:29,237
And so we continue to make actually
119
00:03:29,237 -->
00:03:30,167
really difficult, challenging
120
00:03:30,167 -->
00:03:32,327
benchmarks for that still hold up
121
00:03:32,327 -->
00:03:33,617
today even with state-of-the-art,
122
00:03:33,707 -->
00:03:34,817
uh, large language models.
123
00:03:35,117 -->
00:03:36,707
And now currently what the research
124
00:03:36,707 -->
00:03:38,327
is going, moving into personalization,
125
00:03:38,567 -->
00:03:41,087
um, tool use and agents and deep
126
00:03:41,087 -->
00:03:42,737
research agents, um, is kind of some
127
00:03:42,737 -->
00:03:44,177
of the kind of frontier of what some
128
00:03:44,177 -->
00:03:45,557
of what we're working on in the lab.
129
00:03:46,592 -->
00:03:46,672
Josh Rubin: Hmm.
130
00:03:47,147 -->
00:03:47,627
That's great.
131
00:03:47,627 -->
00:03:48,497
Great, great intro.
132
00:03:48,497 -->
00:03:49,787
I think before we dive into agents,
133
00:03:49,787 -->
00:03:53,357
one, I, I'm curious, just, uh, you
134
00:03:53,357 -->
00:03:55,277
know, some things have so dramatically
135
00:03:55,277 -->
00:03:58,127
changed that it might, uh, sort of
136
00:03:58,127 -->
00:03:59,357
beg the question of whether we just
137
00:03:59,357 -->
00:04:01,937
throw the old stuff out or, uh, you
138
00:04:01,937 -->
00:04:04,427
know, or there's also a philosophy that
139
00:04:04,427 -->
00:04:06,087
actually the old stuff really sort of.
140
00:04:06,842 -->
00:04:09,452
Methodologies, measurement techniques kind
141
00:04:09,452 -->
00:04:12,152
of undergird, um, best practice today.
142
00:04:12,152 -->
00:04:13,022
I'm kind of curious where
143
00:04:13,022 -->
00:04:13,772
you come down on that.
144
00:04:13,772 -->
00:04:15,062
Splits like some stuff
145
00:04:15,062 -->
00:04:18,362
must be, uh, easier.
146
00:04:18,422 -->
00:04:20,222
Uh, but you know, at least in
147
00:04:20,222 -->
00:04:21,872
my experience, I've found that,
148
00:04:22,802 -->
00:04:24,542
you know, simple is often still
149
00:04:24,542 -->
00:04:30,182
king and, uh, you know, uh, best
150
00:04:30,182 -->
00:04:31,622
practices don't, don't change a lot.
151
00:04:31,622 -->
00:04:32,492
Do you have a different, a
152
00:04:32,492 -->
00:04:33,512
different perspective on that?
153
00:04:34,912 -->
00:04:35,542
Jeff Dalton: I, I think I'll
154
00:04:35,542 -->
00:04:36,772
break that up into two pieces.
155
00:04:36,862 -->
00:04:38,422
First, it's just like a little bit of
156
00:04:38,512 -->
00:04:40,072
background, like on, on what I, what I
157
00:04:40,072 -->
00:04:41,872
think about it, and then what that, what
158
00:04:41,902 -->
00:04:44,272
also how, how we make that practical.
159
00:04:44,362 -->
00:04:46,132
Um, so stepping back for a
160
00:04:46,132 -->
00:04:48,652
minute, um, I, I've been at,
161
00:04:48,712 -->
00:04:50,147
we, there's a whole subfield of.
162
00:04:51,392 -->
00:04:52,472
Agents and agent systems.
163
00:04:52,472 -->
00:04:54,422
It's been around since the 19, 1970s.
164
00:04:54,422 -->
00:04:54,572
Right.
165
00:04:54,572 -->
00:04:56,462
It's really fundamental work.
166
00:04:56,522 -->
00:04:58,442
You know, we, everyone, you know, uses the
167
00:04:58,442 -->
00:05:00,182
word agent, but they often mean different
168
00:05:00,182 -->
00:05:01,802
things and the word becomes very blurry.
169
00:05:02,192 -->
00:05:04,472
So if we go back for, I want to
170
00:05:04,562 -->
00:05:05,882
really be real precise there.
171
00:05:05,882 -->
00:05:07,472
It's like, if you go back to the textbook,
172
00:05:07,472 -->
00:05:09,842
Peter Norvig textbook, um, and others
173
00:05:09,842 -->
00:05:12,122
kind of AI 1 0 1, uh, we have an, we
174
00:05:12,122 -->
00:05:14,852
have an agent that can plan, it can take
175
00:05:14,852 -->
00:05:17,192
actions, it can observe the environment.
176
00:05:17,882 -->
00:05:19,052
Maintain state, it has a
177
00:05:19,052 -->
00:05:20,132
representation of the world,
178
00:05:20,132 -->
00:05:22,712
state, and it can change over time.
179
00:05:23,027 -->
00:05:24,707
So those are kind of what the fundamentals
180
00:05:24,707 -->
00:05:26,057
of an agent are, and those haven't
181
00:05:26,057 -->
00:05:28,247
changed all since for 50 years.
182
00:05:28,247 -->
00:05:30,197
Really, fundamentally what's changed is
183
00:05:30,197 -->
00:05:31,997
how we implement them, which, which has
184
00:05:31,997 -->
00:05:34,007
changed is like the state representation,
185
00:05:34,277 -->
00:05:35,657
the fact that they, they're now operating
186
00:05:35,657 -->
00:05:37,697
in language rather than in small, kind
187
00:05:37,697 -->
00:05:40,097
of almost like finite state trees, right?
188
00:05:40,667 -->
00:05:42,647
So that's kind of, that's, we still
189
00:05:42,647 -->
00:05:43,727
need to have those fundamentals.
190
00:05:43,727 -->
00:05:44,957
We need to know that, that
191
00:05:44,957 -->
00:05:46,757
background in order to know
192
00:05:46,787 -->
00:05:48,587
where this 'cause those patterns.
193
00:05:49,742 -->
00:05:50,492
That we had with early
194
00:05:50,492 -->
00:05:51,662
systems are gonna continue.
195
00:05:51,662 -->
00:05:52,832
Those are gonna recur.
196
00:05:52,862 -->
00:05:54,062
So we need to know our history.
197
00:05:54,272 -->
00:05:55,562
We need to know the fundamentals
198
00:05:55,562 -->
00:05:57,032
in computer science so that when
199
00:05:57,032 -->
00:05:58,322
we start moving to multi-agent
200
00:05:58,322 -->
00:05:59,882
systems, there's a whole, we
201
00:05:59,882 -->
00:06:01,352
have lots of algorithms for this.
202
00:06:01,742 -->
00:06:03,872
We know there's a whole subfield of
203
00:06:03,872 -->
00:06:05,582
computer science that we need to make
204
00:06:05,582 -->
00:06:08,522
sure that we are using that history and
205
00:06:08,522 -->
00:06:10,262
use the learnings that we have from it.
206
00:06:11,132 -->
00:06:13,982
Um, that's, so that's part of part one.
207
00:06:14,012 -->
00:06:16,112
Part two is what does it actually mean?
208
00:06:16,412 -->
00:06:17,967
What, where, what, what hasn't changed?
209
00:06:18,977 -->
00:06:20,177
And what hasn't changed for me
210
00:06:20,177 -->
00:06:20,897
is the fact that we still have
211
00:06:20,897 -->
00:06:21,887
to have humans in the loop.
212
00:06:21,947 -->
00:06:22,517
We still have to have
213
00:06:22,517 -->
00:06:23,897
humans looking at the data.
214
00:06:24,077 -->
00:06:24,797
I always tell my grad
215
00:06:24,797 -->
00:06:25,967
students, look at the data.
216
00:06:25,997 -->
00:06:26,717
Look at the data.
217
00:06:26,747 -->
00:06:27,497
Look at the data.
218
00:06:27,767 -->
00:06:29,447
Now I tell my agents the same thing.
219
00:06:29,447 -->
00:06:30,047
Look at the data.
220
00:06:30,047 -->
00:06:30,467
Stop.
221
00:06:30,857 -->
00:06:31,232
Look at the data.
222
00:06:32,117 -->
00:06:33,077
Go line by line.
223
00:06:33,287 -->
00:06:34,247
Have you looked at it yourself?
224
00:06:34,277 -->
00:06:35,567
How, how are things working?
225
00:06:35,987 -->
00:06:36,917
Um, these agents, these
226
00:06:36,917 -->
00:06:38,057
systems are very complex.
227
00:06:38,477 -->
00:06:40,097
So you have to actually look
228
00:06:40,097 -->
00:06:41,357
at fundamental behavior.
229
00:06:41,447 -->
00:06:42,557
Can you do the task that
230
00:06:42,557 -->
00:06:43,517
you're trying to ask the agent?
231
00:06:43,877 -->
00:06:44,837
Um, how hard is it?
232
00:06:45,017 -->
00:06:46,517
Um, so trying to do some of those
233
00:06:46,517 -->
00:06:48,677
things yourself, starting really simple.
234
00:06:49,067 -->
00:06:51,857
Um, and also just having good human evals.
235
00:06:51,887 -->
00:06:53,627
So LLM as a judge is great.
236
00:06:53,657 -->
00:06:55,187
Everyone will is using them.
237
00:06:55,457 -->
00:06:56,537
It provides scale.
238
00:06:57,092 -->
00:06:58,502
But have you just, have
239
00:06:58,502 -->
00:06:59,582
you opened up the file?
240
00:06:59,582 -->
00:07:00,602
Have you, have you labeled
241
00:07:00,602 -->
00:07:01,412
some data yourself?
242
00:07:02,102 -->
00:07:03,662
How do you, how can you do it?
243
00:07:03,962 -->
00:07:05,072
So those are just really simple
244
00:07:05,072 -->
00:07:06,512
fundamentals that aren't going away,
245
00:07:06,512 -->
00:07:07,922
that are just becoming even more
246
00:07:07,922 -->
00:07:09,632
important as we, as we move forward
247
00:07:09,632 -->
00:07:10,787
with the complexity of these systems.
248
00:07:12,287 -->
00:07:12,677
Josh Rubin: Yeah.
249
00:07:12,707 -->
00:07:12,947
Yeah.
250
00:07:12,972 -->
00:07:14,267
I, I think it's, it's, it's
251
00:07:14,267 -->
00:07:16,157
really easy to sort of, um,
252
00:07:16,957 -->
00:07:17,247
Jeff Dalton: Yeah.
253
00:07:17,537 -->
00:07:18,797
Josh Rubin: I don't know, un understate
254
00:07:18,797 -->
00:07:20,177
the importance of like, look at
255
00:07:20,177 -->
00:07:21,152
10 examples with your own eyes.
256
00:07:21,587 -->
00:07:25,847
You know when people, you run tons
257
00:07:25,847 -->
00:07:27,827
of data, thousands of examples, but
258
00:07:28,307 -->
00:07:30,077
you know, what you learn with your
259
00:07:30,077 -->
00:07:32,567
human brain on, you know, 10 examples
260
00:07:32,567 -->
00:07:34,907
kind of gets you to 90% confidence if
261
00:07:34,907 -->
00:07:36,557
you can kinda spot what's going on.
262
00:07:36,947 -->
00:07:40,517
Oftentimes you can, uh, save a lot
263
00:07:40,517 -->
00:07:42,557
of effort, uh, by just applying
264
00:07:42,557 -->
00:07:44,567
your, your primate brain to a few
265
00:07:44,567 -->
00:07:45,977
examples that we've all been so.
266
00:07:47,372 -->
00:07:48,992
In, in our, our, you know, domain of
267
00:07:48,992 -->
00:07:53,552
expertise that evolution and, uh, our
268
00:07:53,822 -->
00:07:55,772
lived experience has, has provided us,
269
00:07:56,582 -->
00:07:56,992
Jeff Dalton: One of the, one of the
270
00:07:57,317 -->
00:07:58,907
common mistakes people really often
271
00:07:58,907 -->
00:07:59,987
make in grad students make, and
272
00:07:59,987 -->
00:08:01,967
I've made in the past is that we,
273
00:08:01,967 -->
00:08:03,227
we think, oh, I need lots of data.
274
00:08:03,227 -->
00:08:04,817
I need huge, huge amounts of data.
275
00:08:04,817 -->
00:08:05,627
I'm gonna run a massive
276
00:08:05,627 -->
00:08:07,127
experiment with, with thousands
277
00:08:07,127 -->
00:08:08,417
or tens of thousands of things.
278
00:08:08,417 -->
00:08:10,217
And you're like, well, yeah,
279
00:08:10,367 -->
00:08:11,357
sometimes you need that.
280
00:08:11,417 -->
00:08:13,547
Um, but sometimes, like,
281
00:08:13,637 -->
00:08:15,137
start small, start simple.
282
00:08:15,752 -->
00:08:17,882
Find, go find the, the cases
283
00:08:17,882 -->
00:08:18,602
that are gonna be hard.
284
00:08:18,602 -->
00:08:19,922
Find the cases that are gonna cause
285
00:08:19,922 -->
00:08:21,992
errors and make sure that those are
286
00:08:21,992 -->
00:08:23,492
the ones that you're looking at first.
287
00:08:23,672 -->
00:08:24,812
And that effect, you're almost like,
288
00:08:24,812 -->
00:08:26,522
it's like a test first data, first
289
00:08:26,702 -->
00:08:29,222
hard case, first kind of philosophy.
290
00:08:29,522 -->
00:08:31,202
Um, but start starting small and
291
00:08:31,202 -->
00:08:32,882
start simple that so that you can look
292
00:08:32,882 -->
00:08:34,112
at it and understand the behavior.
293
00:08:34,622 -->
00:08:34,952
Josh Rubin: Yeah.
294
00:08:34,952 -->
00:08:35,942
A hundred, a hundred percent.
295
00:08:36,032 -->
00:08:37,292
I, that's, I think those are, those
296
00:08:37,292 -->
00:08:39,722
are wise words to everybody out there.
297
00:08:40,232 -->
00:08:40,922
Jeff Dalton: Hard, hard lessons.
298
00:08:41,177 -->
00:08:41,897
Josh Rubin: Do you wanna talk a
299
00:08:41,897 -->
00:08:43,607
little bit about, um, I don't know
300
00:08:43,607 -->
00:08:46,337
how we think about agents today.
301
00:08:46,337 -->
00:08:47,327
I think you spoke a little sort of
302
00:08:47,327 -->
00:08:49,037
abstractly about like, sort of the
303
00:08:49,037 -->
00:08:50,987
definition of the agent, but you know, I
304
00:08:50,987 -->
00:08:52,817
think that there's, you know, people build
305
00:08:52,817 -->
00:08:55,667
things like, um, you know, there's, you
306
00:08:55,667 -->
00:08:58,937
know, uh, you know, we have RAG, we have
307
00:08:58,937 -->
00:09:00,797
MCP that's connecting everything together.
308
00:09:00,797 -->
00:09:03,287
We have different ways of managing state.
309
00:09:04,427 -->
00:09:05,717
I don't know, is there,
310
00:09:06,167 -->
00:09:06,692
how do you sort of.
311
00:09:07,457 -->
00:09:09,557
Characterize the landscape of technologies
312
00:09:09,557 -->
00:09:11,507
that people are sort of using right
313
00:09:11,507 -->
00:09:14,657
now to kind of construct agents?
314
00:09:14,657 -->
00:09:14,927
It does.
315
00:09:14,927 -->
00:09:16,667
I mean, it does go beyond the model.
316
00:09:16,667 -->
00:09:18,137
The models are changing in
317
00:09:18,137 -->
00:09:19,427
lots of important ways also.
318
00:09:19,487 -->
00:09:21,317
Um, but maybe you could paint a little
319
00:09:21,317 -->
00:09:22,667
picture around the, the, the agentic
320
00:09:22,667 -->
00:09:24,917
landscape and maybe how, I mean, I think
321
00:09:24,917 -->
00:09:28,337
that different guests that I've had have
322
00:09:28,337 -->
00:09:29,652
framed this slightly different ways.
323
00:09:29,752 -->
00:09:30,947
I'm sort of curious about how you
324
00:09:30,947 -->
00:09:33,137
think about the, technologies are
325
00:09:33,137 -->
00:09:34,577
involved and what parts are responsible
326
00:09:34,577 -->
00:09:35,777
for what and what changes have.
327
00:09:36,752 -->
00:09:40,802
Um, made any of this possible
328
00:09:40,802 -->
00:09:41,822
with large language models.
329
00:09:42,922 -->
00:09:43,312
Jeff Dalton: Yeah.
330
00:09:43,522 -->
00:09:46,042
So, uh, let's step back just for a minute
331
00:09:46,042 -->
00:09:47,692
to talk about also where we've come
332
00:09:47,692 -->
00:09:49,077
from and, and how things have evolved.
333
00:09:49,382 -->
00:09:49,742
Josh Rubin: please.
334
00:09:49,787 -->
00:09:50,492
Jeff Dalton: And, and things are
335
00:09:50,492 -->
00:09:51,992
changing just so rapidly as well.
336
00:09:52,202 -->
00:09:53,582
Um, but let's stepping back a little
337
00:09:53,582 -->
00:09:55,502
bit, um, we've, this things really
338
00:09:55,502 -->
00:09:57,512
started with deep learning, right?
339
00:09:57,512 -->
00:09:58,922
So starting with kind of non
340
00:09:58,922 -->
00:10:00,602
generative systems, things that
341
00:10:00,812 -->
00:10:02,642
classify kind of deep neural networks,
342
00:10:02,642 -->
00:10:03,992
and those were super successful.
343
00:10:04,082 -->
00:10:06,482
Um, and one with data at scale.
344
00:10:07,232 -->
00:10:07,802
And then we started
345
00:10:07,802 -->
00:10:09,482
moving into the, the GPT.
346
00:10:09,482 -->
00:10:10,262
We started moving into
347
00:10:10,262 -->
00:10:11,702
the generative AI world.
348
00:10:11,702 -->
00:10:12,932
Just the simple pre-trained
349
00:10:12,932 -->
00:10:15,002
foundation models that can
350
00:10:15,002 -->
00:10:16,382
generate language really well.
351
00:10:16,922 -->
00:10:17,852
Um, and that those were
352
00:10:17,852 -->
00:10:18,812
extremely powerful.
353
00:10:19,022 -->
00:10:20,252
And then we added, started, added
354
00:10:20,252 -->
00:10:21,572
instruction tuning that, so now
355
00:10:21,572 -->
00:10:22,742
they can follow instructions.
356
00:10:23,462 -->
00:10:24,662
And then we, then we realize,
357
00:10:24,692 -->
00:10:25,592
okay, but that's not enough.
358
00:10:25,592 -->
00:10:26,822
So now we're, we, we're going to
359
00:10:26,822 -->
00:10:28,472
need to add tools so they can call
360
00:10:28,472 -->
00:10:29,942
and use calculators and they can,
361
00:10:30,032 -->
00:10:31,472
and then we now have part, as part
362
00:10:31,472 -->
00:10:33,392
of our post training mechanism.
363
00:10:33,392 -->
00:10:34,802
After the model's been pre-trained, we can
364
00:10:34,802 -->
00:10:36,902
now, we can now learn to use these tools
365
00:10:36,902 -->
00:10:38,157
at the right time and at the right place.
366
00:10:39,527 -->
00:10:41,837
These models are now, um, they would
367
00:10:41,837 -->
00:10:43,397
sometimes call it distributed models.
368
00:10:43,397 -->
00:10:44,747
So there are architectures there that
369
00:10:44,747 -->
00:10:46,697
are like mixture of experts, and so
370
00:10:46,697 -->
00:10:48,497
under the hood, you're getting routed
371
00:10:48,497 -->
00:10:50,927
to an internal expert that can then work
372
00:10:50,927 -->
00:10:53,027
on answering, answering your question.
373
00:10:53,387 -->
00:10:54,467
So that's just a, there's a lot there
374
00:10:54,467 -->
00:10:55,877
that's been evolving and the found
375
00:10:55,877 -->
00:10:58,277
and the core foundation layer, um,
376
00:10:58,277 -->
00:10:59,597
and those actually starting to get
377
00:10:59,597 -->
00:11:00,527
a little bit blurry because some
378
00:11:00,527 -->
00:11:01,937
of those kind of core elements of
379
00:11:01,937 -->
00:11:02,957
like, oh, it can call tools now.
380
00:11:03,447 -->
00:11:04,437
Oh, it, it can remember,
381
00:11:04,677 -->
00:11:05,487
it can remember this.
382
00:11:05,487 -->
00:11:06,927
These things, we can add a,
383
00:11:06,927 -->
00:11:08,307
some additional adapter layer
384
00:11:08,307 -->
00:11:09,507
that has some additional memory.
385
00:11:10,137 -->
00:11:11,582
Um, we can have specialized components.
386
00:11:12,242 -->
00:11:13,442
Those starting to make, that those
387
00:11:13,502 -->
00:11:15,092
underlying foundation models start to
388
00:11:15,092 -->
00:11:16,712
feel a little bit closer to agents,
389
00:11:16,772 -->
00:11:18,302
even though they're not quite there.
390
00:11:18,572 -->
00:11:20,402
Um, and now with the, as we've
391
00:11:20,402 -->
00:11:21,662
seen with the latest models, like
392
00:11:21,662 -->
00:11:23,072
things like the, the next step was
393
00:11:23,072 -->
00:11:25,022
the reasoning models, but starting
394
00:11:25,022 -->
00:11:26,462
with a o series of models, right?
395
00:11:26,522 -->
00:11:28,472
So they now can have like reflective
396
00:11:28,532 -->
00:11:30,332
self-talk where they're talking to
397
00:11:30,362 -->
00:11:32,252
themselves effectively, not necessarily
398
00:11:32,252 -->
00:11:34,772
reasoning per se, but kind of reflective
399
00:11:35,012 -->
00:11:36,542
talk back and forth with each other.
400
00:11:37,082 -->
00:11:38,852
Um, and that's gonna continue to evolve.
401
00:11:39,062 -->
00:11:40,712
Um, I, I think we're going
402
00:11:40,712 -->
00:11:41,552
to continue to see some.
403
00:11:41,812 -->
00:11:43,852
Really significant advances over
404
00:11:43,852 -->
00:11:45,322
the next, over the next year or
405
00:11:45,322 -->
00:11:46,672
two on that, particularly with
406
00:11:46,672 -->
00:11:48,142
how they incorporate memory and
407
00:11:48,142 -->
00:11:49,402
other SY kind of system components.
408
00:11:50,627 -->
00:11:52,397
I think the shift that we're seeing
409
00:11:52,397 -->
00:11:54,197
is, is that we go from that foundation
410
00:11:54,197 -->
00:11:57,167
model to kind of first agents to
411
00:11:57,167 -->
00:11:58,997
now we're moving to agentic systems.
412
00:11:59,022 -->
00:12:01,037
So, so now we're, so we're in the land
413
00:12:01,037 -->
00:12:03,797
of Claude Code, which is really not just
414
00:12:03,947 -->
00:12:05,477
single agent, but it's a multi-agent
415
00:12:05,477 -->
00:12:07,487
system with multiple different kind
416
00:12:07,487 -->
00:12:09,107
of components that are, that are
417
00:12:09,197 -->
00:12:10,727
um, that are, that work together to
418
00:12:10,727 -->
00:12:12,317
help user accomplish complex tasks.
419
00:12:13,182 -->
00:12:13,752
And that's also what
420
00:12:13,752 -->
00:12:14,502
we're building at Valence.
421
00:12:14,502 -->
00:12:16,212
When we're building an AI coach.
422
00:12:16,392 -->
00:12:18,282
We're building an agentic system
423
00:12:18,372 -->
00:12:19,692
that is helping people for coaching.
424
00:12:21,317 -->
00:12:22,547
Josh Rubin: I think that this point
425
00:12:22,547 -->
00:12:26,267
about, um, when I was just thinking of
426
00:12:26,297 -->
00:12:28,667
the, I'm just responding kind of the
427
00:12:28,667 -->
00:12:31,187
cuff to like mixture of experts as like
428
00:12:31,187 -->
00:12:35,717
a, you know, what is the, you know.
429
00:12:36,887 -->
00:12:38,147
What's the distinction between the
430
00:12:38,147 -->
00:12:39,467
model and the scaffolding, right?
431
00:12:39,497 -->
00:12:40,457
Like by the time you're talking
432
00:12:40,457 -->
00:12:41,897
about models that have kind
433
00:12:41,897 -->
00:12:44,777
of, I don't know, internal
434
00:12:44,777 -->
00:12:47,207
disaggregation or sparsity to them.
435
00:12:47,567 -->
00:12:49,037
Uh, I think it's a really interesting
436
00:12:49,037 -->
00:12:52,487
time when there are, there is, um,
437
00:12:52,637 -->
00:12:54,107
orchestration happening at lots of
438
00:12:54,107 -->
00:12:56,087
different layers kind of within the model.
439
00:12:56,447 -->
00:12:58,097
And then, you know, every people are
440
00:12:58,097 -->
00:13:00,377
talking about harnesses and talking about,
441
00:13:00,407 -->
00:13:03,137
um, talking about orchestration and.
442
00:13:04,052 -->
00:13:06,092
Um, and then interconnecting these things.
443
00:13:06,152 -->
00:13:10,082
Uh, you know, I don't know that, well,
444
00:13:11,042 -->
00:13:12,482
it's all to say it's evolving quickly.
445
00:13:12,752 -->
00:13:16,292
Um, um, what do I want to,
446
00:13:16,292 -->
00:13:17,192
what else do I wanna say here?
447
00:13:17,732 -->
00:13:20,972
Um, let's talk, I don't know if you wanna
448
00:13:20,972 -->
00:13:23,432
jump into talking about, um, Nadia and
449
00:13:23,432 -->
00:13:25,922
what you're doing at Valence and why
450
00:13:26,192 -->
00:13:28,472
that's a particularly interesting problem
451
00:13:28,502 -->
00:13:30,212
given, like where you're coming from.
452
00:13:31,797 -->
00:13:32,397
Jeff Dalton: That'd be great.
453
00:13:32,622 -->
00:13:34,422
Josh Rubin: You want to jump into that?
454
00:13:34,422 -->
00:13:34,727
Who Nadia is.
455
00:13:36,197 -->
00:13:38,117
Jeff Dalton: So, um, I'm Head of AI at, at
456
00:13:38,117 -->
00:13:41,417
Valence we're building an AI, an AI coach.
457
00:13:42,017 -->
00:13:43,727
And what really is exciting and
458
00:13:43,727 -->
00:13:44,897
kind of, kind of some of the key
459
00:13:44,897 -->
00:13:47,057
differences there is that we do, it
460
00:13:47,057 -->
00:13:49,037
is an assistant, it can help people.
461
00:13:49,277 -->
00:13:50,357
Um, but the goal of what
462
00:13:50,357 -->
00:13:51,437
we're trying to build with.
463
00:13:51,667 -->
00:13:53,947
With Nadia is that we're trying to build
464
00:13:53,947 -->
00:13:55,867
a coach and the, the goal of the coach
465
00:13:55,927 -->
00:13:58,057
is to improve your performance over time.
466
00:13:58,657 -->
00:13:58,897
Fun.
467
00:13:58,957 -->
00:14:00,187
And that's fundamentally different from
468
00:14:00,187 -->
00:14:01,567
what an assistant is, which is just trying
469
00:14:01,567 -->
00:14:03,577
to help you draft, draft an email or, or,
470
00:14:03,637 -->
00:14:05,197
or do something that do a simple task.
471
00:14:05,737 -->
00:14:07,747
Our goal is, is transformation for
472
00:14:07,747 -->
00:14:09,907
you, your organization, your team, so
473
00:14:09,907 -->
00:14:11,317
that you're improving your performance,
474
00:14:11,317 -->
00:14:13,447
working with you, that's helping you be
475
00:14:13,447 -->
00:14:15,487
a better person that's augmenting you.
476
00:14:15,752 -->
00:14:18,152
That's giving you what you need to do to,
477
00:14:18,212 -->
00:14:20,852
to be able to improve, um, so that the
478
00:14:20,852 -->
00:14:22,472
objective function there is not just did
479
00:14:22,472 -->
00:14:24,362
we complete your task, but did we help
480
00:14:24,362 -->
00:14:26,852
you learn and grow through that process?
481
00:14:27,242 -->
00:14:28,652
And so, and that's
482
00:14:28,652 -->
00:14:29,432
fundamentally different.
483
00:14:29,432 -->
00:14:31,772
So we're optimizing for value over time.
484
00:14:32,072 -->
00:14:33,092
Like how is this getting
485
00:14:33,242 -->
00:14:33,902
for you over time?
486
00:14:33,902 -->
00:14:34,922
It is not even getting to know you.
487
00:14:34,952 -->
00:14:36,662
Is it personalized for you?
488
00:14:37,197 -->
00:14:38,877
And or are we also introducing
489
00:14:38,877 -->
00:14:39,837
productive friction?
490
00:14:40,017 -->
00:14:41,367
So are we stopping you from writing
491
00:14:41,367 -->
00:14:42,567
that email and saying like, oh,
492
00:14:42,567 -->
00:14:43,677
maybe that shouldn't be an email.
493
00:14:43,947 -->
00:14:45,177
Maybe you should go talk to that person.
494
00:14:45,537 -->
00:14:46,617
Or maybe this should, maybe you
495
00:14:46,617 -->
00:14:48,657
should, uh, do something else, right?
496
00:14:48,657 -->
00:14:50,637
So by the way, you've been, you've
497
00:14:50,637 -->
00:14:51,687
sent other emails in the past
498
00:14:51,687 -->
00:14:52,827
and they haven't been successful.
499
00:14:53,097 -->
00:14:54,447
And also we, you've been
500
00:14:54,447 -->
00:14:56,007
talking in new way this history.
501
00:14:56,247 -->
00:14:57,837
You, you have communication issues.
502
00:14:57,837 -->
00:14:59,307
Let's make sure that we work on those.
503
00:14:59,697 -->
00:15:00,772
So that's what we're, we're building.
504
00:15:02,012 -->
00:15:04,622
The future of a AI coach that can work
505
00:15:04,622 -->
00:15:06,872
with you, that helps you in your flow
506
00:15:06,872 -->
00:15:08,672
of work and be able, helps you kinda
507
00:15:08,672 -->
00:15:09,692
get things done, but also helps you
508
00:15:09,692 -->
00:15:11,612
grow and improve for you and your team.
509
00:15:11,792 -->
00:15:14,522
So kinda moving towards just, uh, an AI
510
00:15:14,522 -->
00:15:16,532
chat bot to actually, it knows your world.
511
00:15:16,652 -->
00:15:18,422
It knows it's connected to
512
00:15:18,422 -->
00:15:19,442
your organization, it has
513
00:15:19,442 -->
00:15:20,792
your organizational values.
514
00:15:21,197 -->
00:15:22,307
Let's learn and grow,
515
00:15:22,337 -->
00:15:23,747
grow with you over time.
516
00:15:24,017 -->
00:15:26,177
It knows your personality, your profile,
517
00:15:26,357 -->
00:15:27,917
how you work, and how you work with
518
00:15:27,917 -->
00:15:29,477
others so that it can really kind of
519
00:15:29,477 -->
00:15:31,787
coach you and, and do a better job than,
520
00:15:32,147 -->
00:15:34,427
um, because it's highly specialized
521
00:15:34,457 -->
00:15:36,017
for you and for your environment.
522
00:15:37,007 -->
00:15:37,847
Josh Rubin: Yeah, I hear a lot of
523
00:15:37,847 -->
00:15:40,397
things in there that I think are, uh,
524
00:15:40,397 -->
00:15:41,807
potentially technically challenging.
525
00:15:41,837 -->
00:15:45,287
Like I think the of personalization,
526
00:15:45,347 -->
00:15:46,517
you know, to you, to your
527
00:15:46,517 -->
00:15:49,697
org, I think the time horizon.
528
00:15:49,697 -->
00:15:51,827
How are you successful in the long
529
00:15:51,827 -->
00:15:54,137
term when you know the, the, uh, like
530
00:15:54,137 -->
00:15:55,577
you said, the objective function is.
531
00:15:56,507 -->
00:15:58,187
maybe harder to quantify when
532
00:15:58,187 -->
00:16:00,047
it's not directly observable
533
00:16:00,047 -->
00:16:01,007
in kind of little chunks.
534
00:16:01,007 -->
00:16:02,927
This I could see something like LLM
535
00:16:02,927 -->
00:16:05,717
as a judge struggling a little bit to
536
00:16:05,717 -->
00:16:08,867
provide a meaningful metric on sort
537
00:16:08,867 -->
00:16:11,777
of, um, you know, conversation turns
538
00:16:11,777 -->
00:16:14,567
or, uh, or even task, you know, uh,
539
00:16:14,597 -->
00:16:16,757
trace level kind of task completion.
540
00:16:16,907 -->
00:16:22,007
Um, do you, so, so tell, maybe you could
541
00:16:22,007 -->
00:16:23,927
say a little bit more about sort of.
542
00:16:25,637 -->
00:16:27,857
How you think about that problem and
543
00:16:27,857 -->
00:16:31,607
what differentiates it from, um, you
544
00:16:31,607 -->
00:16:35,207
know, what we can do with, you know,
545
00:16:35,477 -->
00:16:37,637
a clot or a chat GPT with a little bit
546
00:16:37,637 -->
00:16:39,677
of prompting, you know, a, a clever
547
00:16:39,677 -->
00:16:42,407
system prompt and some, some, you know,
548
00:16:42,407 -->
00:16:43,937
sort of simplistic memory mechanism.
549
00:16:44,992 -->
00:16:45,557
Where, where, where, where,
550
00:16:45,917 -->
00:16:46,847
where's the meat of this?
551
00:16:47,747 -->
00:16:50,147
Jeff Dalton: So, so there's a few
552
00:16:50,147 -->
00:16:51,197
different pillars we have here.
553
00:16:51,407 -->
00:16:52,847
So first, I think it's
554
00:16:52,847 -->
00:16:54,077
purpose-built to be a coach.
555
00:16:54,587 -->
00:16:56,477
So it's built, we have human coaches on
556
00:16:56,477 -->
00:16:58,997
staff, we have human coaching knowledge
557
00:16:59,027 -->
00:17:00,827
and best practices are encoded within it.
558
00:17:01,247 -->
00:17:02,717
Um, so you're, you're getting a kind
559
00:17:02,717 -->
00:17:04,517
of leading edge kind of coaching
560
00:17:04,517 -->
00:17:06,497
abilities kind of, that are grounded
561
00:17:06,497 -->
00:17:08,447
in, in fundamental principles that
562
00:17:08,447 -->
00:17:10,847
are, are there to facilitate growth and
563
00:17:10,847 -->
00:17:13,247
judgment and better actions over time.
564
00:17:13,667 -->
00:17:15,107
Um, then, uh, what we really
565
00:17:15,107 -->
00:17:16,847
have is like memory is a really
566
00:17:16,847 -->
00:17:19,097
first class object for us, right?
567
00:17:19,337 -->
00:17:20,417
So if we're going to learn and
568
00:17:20,417 -->
00:17:21,257
grow with you over time, we
569
00:17:21,257 -->
00:17:22,517
have to make sure that we are.
570
00:17:23,027 -->
00:17:24,587
Are we remembering the right things?
571
00:17:24,857 -->
00:17:26,567
We're looking at the past conversations.
572
00:17:26,627 -->
00:17:28,997
We're analyzing patterns in those.
573
00:17:29,387 -->
00:17:30,587
Uh, we're then making sure that
574
00:17:30,587 -->
00:17:32,567
we have the, the memories of that.
575
00:17:32,567 -->
00:17:33,857
You're, we're talking about the same,
576
00:17:33,857 -->
00:17:35,027
you're talking about the same people.
577
00:17:35,147 -->
00:17:36,167
So we know your people.
578
00:17:36,437 -->
00:17:37,607
We know that it's not just Josh,
579
00:17:37,637 -->
00:17:39,227
we know it's this Josh, right.
580
00:17:39,287 -->
00:17:40,067
That is there.
581
00:17:40,067 -->
00:17:41,207
So we've done kind of co reference
582
00:17:41,267 -->
00:17:42,557
resolution and understanding,
583
00:17:42,707 -->
00:17:43,757
and we give you control of that.
584
00:17:43,757 -->
00:17:45,437
You can see it, you can edit the people.
585
00:17:45,497 -->
00:17:46,487
Oh, that's the wrong Josh.
586
00:17:46,847 -->
00:17:48,317
So you have control over that.
587
00:17:48,317 -->
00:17:50,117
So those are all built into the product.
588
00:17:50,177 -->
00:17:50,807
That's 'cause part of
589
00:17:50,807 -->
00:17:51,647
the product experience.
590
00:17:52,382 -->
00:17:53,852
Uh, end to end, not just in the
591
00:17:53,852 -->
00:17:55,262
models in terms of what, what
592
00:17:55,262 -->
00:17:56,432
we, what we have for context.
593
00:17:56,552 -->
00:17:57,992
It's, it's about transparency and
594
00:17:57,992 -->
00:17:59,672
control for the user to see, this is
595
00:17:59,672 -->
00:18:01,292
what we know about you, this is what
596
00:18:01,292 -->
00:18:02,282
we're learning, these are the, the
597
00:18:02,282 -->
00:18:03,962
growth areas that we have for you.
598
00:18:04,292 -->
00:18:05,882
What, which ones do you wanna work on now?
599
00:18:05,882 -->
00:18:07,082
And giving you those choice
600
00:18:07,262 -->
00:18:08,522
choices in that process.
601
00:18:09,392 -->
00:18:11,042
Um, what we call, um.
602
00:18:12,242 -->
00:18:13,592
What makes kind of Nadia a little
603
00:18:13,592 -->
00:18:14,612
bit element there is what we
604
00:18:14,612 -->
00:18:16,142
call like the intelligence layer.
605
00:18:16,592 -->
00:18:17,942
So that's, you know, the intelligence
606
00:18:17,942 -->
00:18:20,402
to be able to take those past histories
607
00:18:20,402 -->
00:18:22,232
and be able to have a coaching lens
608
00:18:22,472 -->
00:18:24,032
or what we think are, the patterns are
609
00:18:24,032 -->
00:18:25,532
going to be for the hypotheses that we
610
00:18:25,532 -->
00:18:27,872
have for the future, um, but also that
611
00:18:27,872 -->
00:18:29,552
coaching intelligence at the, at the right
612
00:18:29,552 -->
00:18:31,472
time in the flow of the conversation.
613
00:18:32,297 -->
00:18:33,497
When do we pull in memory?
614
00:18:33,797 -->
00:18:34,817
Have you mentioned Josh?
615
00:18:34,907 -->
00:18:36,317
Oh, by the way, we're gonna pull in,
616
00:18:36,317 -->
00:18:37,937
go pull in the connections and pull
617
00:18:37,937 -->
00:18:39,497
in the relevant kind of retrieval
618
00:18:39,497 -->
00:18:41,417
parts, um, of the conversation
619
00:18:41,417 -->
00:18:42,602
that are relevant to you right now.
620
00:18:43,787 -->
00:18:45,497
Um, and be able to have that, and have
621
00:18:45,497 -->
00:18:47,057
that also be, we know that, you know,
622
00:18:47,117 -->
00:18:48,497
we know this information about Josh.
623
00:18:48,587 -->
00:18:49,487
This is the way he works,
624
00:18:49,487 -->
00:18:50,357
this is the way I work.
625
00:18:50,357 -->
00:18:51,827
This is, these are the failure modes
626
00:18:51,827 -->
00:18:53,267
that, that we could be experiencing.
627
00:18:53,807 -->
00:18:55,667
And so having that integrated into
628
00:18:55,697 -->
00:18:57,287
the kind of coaching knowledge as
629
00:18:57,287 -->
00:18:59,087
a kind of a fundamental pillar or
630
00:18:59,117 -->
00:19:00,677
how we work and how these people
631
00:19:00,677 -->
00:19:02,507
and the team should work together.
632
00:19:03,212 -->
00:19:04,982
As well as obviously for Nadia,
633
00:19:04,982 -->
00:19:06,932
it's about, um, it's it's foreign
634
00:19:06,932 -->
00:19:08,252
organization, so it's operating
635
00:19:08,252 -->
00:19:09,812
within the organizational constraints.
636
00:19:10,142 -->
00:19:11,522
It's operating with a safety
637
00:19:11,522 -->
00:19:12,632
that the organization has.
638
00:19:12,632 -->
00:19:14,162
We have a, we can talk about it, but one
639
00:19:14,162 -->
00:19:15,782
of the fundamental things that we have
640
00:19:15,782 -->
00:19:18,002
is our guardrails and a safety system.
641
00:19:18,542 -->
00:19:20,732
So we, we want the coaching that we
642
00:19:20,732 -->
00:19:22,917
have to be safe and tailored for,
643
00:19:23,042 -->
00:19:24,512
for you and for your organization.
644
00:19:24,602 -->
00:19:27,422
And I can feel free to pick wherever
645
00:19:27,422 -->
00:19:28,742
you see where you want to go next.
646
00:19:29,232 -->
00:19:29,712
Josh Rubin: Oh, yeah.
647
00:19:29,712 -->
00:19:30,822
Um, I, I, I think I
648
00:19:30,822 -->
00:19:31,632
wanna go to measurement.
649
00:19:31,782 -->
00:19:32,802
I, I, you know, I think when you,
650
00:19:32,802 -->
00:19:35,112
you, you, you, uh, raised my eyebrows
651
00:19:35,112 -->
00:19:36,072
when you were talking about sort of
652
00:19:36,072 -->
00:19:37,452
objective, objective functions, right?
653
00:19:38,552 -->
00:19:40,862
When you are building something that,
654
00:19:41,192 -->
00:19:43,622
you know, uh, it's easy to pull a,
655
00:19:44,042 -->
00:19:45,722
you know, a chatbot off the shelf
656
00:19:45,902 -->
00:19:48,152
now, uh, it's remarkably easy to build
657
00:19:48,152 -->
00:19:49,802
something that has some basic utility.
658
00:19:51,182 -->
00:19:54,152
you know, much like the conversa,
659
00:19:54,152 -->
00:19:55,862
the previous conversation about
660
00:19:55,862 -->
00:19:57,242
like not jumping to analyzing
661
00:19:57,242 -->
00:19:59,822
10,000 data points when you can
662
00:19:59,822 -->
00:20:01,472
get a whole lot out of 10 of them.
663
00:20:01,922 -->
00:20:02,522
Um.
664
00:20:04,787 -->
00:20:05,897
You also don't wanna build a whole lot
665
00:20:05,897 -->
00:20:07,667
of apparatus, a whole lot of scaffolding
666
00:20:07,877 -->
00:20:12,167
when the, know, the delta there in
667
00:20:12,467 -->
00:20:15,137
performance is, uh, you know, doesn't,
668
00:20:15,137 -->
00:20:16,787
doesn't support the investment, right?
669
00:20:16,792 -->
00:20:17,927
You, you wanna spend your energy
670
00:20:17,927 -->
00:20:19,607
where it's, it's most impactful.
671
00:20:19,607 -->
00:20:20,747
And I know, you know, from a Fiddler
672
00:20:20,747 -->
00:20:23,147
perspective, you know, we think a
673
00:20:23,147 -->
00:20:26,477
lot about how we support our, um, you
674
00:20:26,477 -->
00:20:28,547
know, our, our enterprise customers
675
00:20:28,547 -->
00:20:31,127
with, um, you know, our, our kind of.
676
00:20:32,012 -->
00:20:33,902
Three pillars are kind of production,
677
00:20:33,902 -->
00:20:36,152
Observability and guard, railing
678
00:20:36,152 -->
00:20:39,812
and evals experiments, experience.
679
00:20:39,842 -->
00:20:41,822
Uh, you know, we put a lot of sort
680
00:20:41,822 -->
00:20:46,622
of, uh, emphasis behind measurement
681
00:20:46,802 -->
00:20:49,802
as an important way of, know,
682
00:20:51,032 -->
00:20:53,882
identifying how well variant B is from
683
00:20:53,882 -->
00:20:56,702
variant A. Can you justify building
684
00:20:56,702 -->
00:20:58,172
out something more sophisticated?
685
00:20:58,172 -->
00:21:01,052
Also, um, uh.
686
00:21:02,627 -->
00:21:03,767
Where, where are the hotspots?
687
00:21:03,767 -->
00:21:04,697
Where is, what are, what
688
00:21:04,697 -->
00:21:05,987
are common failure modes?
689
00:21:06,257 -->
00:21:07,787
Maybe you wanna talk a little bit
690
00:21:07,787 -->
00:21:09,227
about, like, I, I guess I would really
691
00:21:09,227 -->
00:21:11,267
love to hear from you about, um, you
692
00:21:11,267 -->
00:21:12,767
know, what your perspective is on
693
00:21:12,767 -->
00:21:16,247
measurements and what kinds of, whether
694
00:21:16,247 -->
00:21:19,097
it's evals or human labeling or, uh.
695
00:21:19,967 -->
00:21:20,837
You know, any of those things.
696
00:21:20,837 -->
00:21:22,217
How do, how do you determine,
697
00:21:22,457 -->
00:21:23,387
you know, where the leverage
698
00:21:23,387 -->
00:21:24,497
is in improving your system?
699
00:21:24,527 -->
00:21:26,537
I guess it all goes back to like, what
700
00:21:26,537 -->
00:21:28,367
are the, you know, what's the objective
701
00:21:28,367 -->
00:21:29,627
function you're trying to optimize?
702
00:21:29,837 -->
00:21:32,207
So maybe, maybe you start there like, uh,
703
00:21:32,207 -->
00:21:33,617
what are some of your objective functions?
704
00:21:35,777 -->
00:21:36,467
Jeff Dalton: Well, I think I can,
705
00:21:36,827 -->
00:21:38,207
we'll talk about, I'll talk about
706
00:21:38,207 -->
00:21:39,977
objective functions, maybe a second may.
707
00:21:40,007 -->
00:21:41,537
Maybe first I can tell you a little bit
708
00:21:41,537 -->
00:21:44,537
about how we very consciously have built
709
00:21:44,537 -->
00:21:47,237
our system to be eval eval first, right?
710
00:21:47,297 -->
00:21:47,887
And be evaluatable.
711
00:21:49,127 -->
00:21:49,547
Um.
712
00:21:50,282 -->
00:21:52,202
So, for example, if people start out, if
713
00:21:52,202 -->
00:21:53,162
they're building, if they're building an
714
00:21:53,162 -->
00:21:54,362
agent system and they're building kind
715
00:21:54,362 -->
00:21:55,952
of, maybe they have some different prompts
716
00:21:55,952 -->
00:21:57,932
for different parts of the system, uh,
717
00:21:57,932 -->
00:21:59,252
you might start out with just some very
718
00:21:59,252 -->
00:22:01,052
simple prompts like, you know, do this
719
00:22:01,052 -->
00:22:03,002
or a natural language, and that's great.
720
00:22:03,122 -->
00:22:03,812
Um, and that, that,
721
00:22:03,812 -->
00:22:04,742
that's a first version.
722
00:22:04,742 -->
00:22:05,642
And that's where we started with the
723
00:22:05,642 -->
00:22:07,652
first version of Nadia, for example.
724
00:22:08,102 -->
00:22:09,902
But what we found was how do we trace it?
725
00:22:09,962 -->
00:22:10,797
How do we understand the behavior?
726
00:22:11,547 -->
00:22:12,597
What's, what's going on?
727
00:22:12,837 -->
00:22:14,247
And so we, we actually, actually,
728
00:22:14,307 -->
00:22:15,807
our approach is actually much closer
729
00:22:15,807 -->
00:22:17,697
to code, so it's closer to the
730
00:22:17,697 -->
00:22:19,377
prompts as code where they're really
731
00:22:19,377 -->
00:22:21,267
kind of semi-structured objects.
732
00:22:22,107 -->
00:22:23,547
Um, and so that allows us to say like,
733
00:22:23,577 -->
00:22:25,287
we are, we are taking an approach here.
734
00:22:25,287 -->
00:22:28,107
We are in grow coaching mode so that we
735
00:22:28,107 -->
00:22:29,637
know, and here's the set of instructions
736
00:22:29,667 -->
00:22:31,017
that we know that we we're in grow.
737
00:22:31,197 -->
00:22:33,117
And it has clearly defined steps and
738
00:22:33,117 -->
00:22:35,007
phases that were, that we can walk
739
00:22:35,007 -->
00:22:36,567
through so we can inspect those.
740
00:22:37,017 -->
00:22:38,577
So we take kinda a structured
741
00:22:38,577 -->
00:22:39,567
approach to the conversation
742
00:22:39,567 -->
00:22:40,707
so it's not just open-ended.
743
00:22:41,492 -->
00:22:43,682
We have kind of playbooks and kind of
744
00:22:43,682 -->
00:22:45,062
coaching approaches and playbooks that
745
00:22:45,062 -->
00:22:47,252
we're, that we're executing, and because
746
00:22:47,312 -->
00:22:48,842
we're executing kind of these kind of
747
00:22:48,842 -->
00:22:50,852
structured coaching programs that we
748
00:22:50,852 -->
00:22:53,462
can switch, switch on and, and transform
749
00:22:53,462 -->
00:22:55,412
over the kind of on the fly, what that
750
00:22:55,412 -->
00:22:57,362
allows us to do is to basically see like,
751
00:22:57,392 -->
00:22:58,742
oh, we're executing instructions one and
752
00:22:58,742 -->
00:23:01,142
three and grow, and so when something goes
753
00:23:01,142 -->
00:23:02,972
wrong, we can trace that back to like,
754
00:23:03,002 -->
00:23:05,522
oh, we need to go modify these lines of
755
00:23:05,522 -->
00:23:07,052
the prompt, right to, in order to make
756
00:23:07,052 -->
00:23:08,852
things, to be able to make things work.
757
00:23:09,722 -->
00:23:10,712
So there's, that's an
758
00:23:10,712 -->
00:23:11,972
element of like determinism.
759
00:23:12,062 -->
00:23:12,932
We can say like, should this
760
00:23:12,932 -->
00:23:14,642
be in G or should it be in R?
761
00:23:14,882 -->
00:23:16,922
So we can then evaluate offline
762
00:23:16,922 -->
00:23:18,602
with humans on label data sets like.
763
00:23:20,192 -->
00:23:22,442
Coaching is, is flowing as expected.
764
00:23:22,442 -->
00:23:23,732
So when we go to a model upgrade,
765
00:23:23,972 -->
00:23:25,712
we can say, ah, okay, the model is
766
00:23:25,712 -->
00:23:26,762
still transitioning at the right
767
00:23:26,762 -->
00:23:28,742
times, or, oh, something is stuck.
768
00:23:28,742 -->
00:23:29,792
You get stuck in G and
769
00:23:29,792 -->
00:23:30,512
we're not progressing.
770
00:23:30,512 -->
00:23:31,562
There's something broken here.
771
00:23:31,772 -->
00:23:33,662
So we have internal thinking and
772
00:23:33,662 -->
00:23:35,612
traceability from the model to be able
773
00:23:35,612 -->
00:23:37,232
to see what it's, what is actually,
774
00:23:37,292 -->
00:23:38,522
we have our own proprietary kind of
775
00:23:38,607 -->
00:23:40,172
thinking model that we have to be
776
00:23:40,172 -->
00:23:41,762
able to for those structured programs.
777
00:23:42,062 -->
00:23:43,532
And I think that's really, that's
778
00:23:43,532 -->
00:23:44,582
really kind of fundamental to make
779
00:23:44,582 -->
00:23:46,082
sure that you have those traces.
780
00:23:46,262 -->
00:23:47,432
So I'm really a fan of like, what, what
781
00:23:47,432 -->
00:23:48,552
you're doing with, with your work there.
782
00:23:49,847 -->
00:23:51,287
Um, I do wanna step back a
783
00:23:51,287 -->
00:23:52,307
little bit for talking about
784
00:23:52,337 -->
00:23:53,537
some of the objective function.
785
00:23:53,957 -->
00:23:55,667
Uh, what for us is, is a
786
00:23:55,667 -->
00:23:56,567
little bit more difficult.
787
00:23:56,627 -->
00:23:57,587
So some of the things, if you're
788
00:23:57,587 -->
00:23:58,787
drafting an email, that's fine, you
789
00:23:58,787 -->
00:24:00,107
can, you can kind of measure that.
790
00:24:00,557 -->
00:24:02,717
Um, but kind of best practice and kind of
791
00:24:02,717 -->
00:24:04,217
science-based approaches for evaluating
792
00:24:04,217 -->
00:24:05,867
the systems is based on rubrics.
793
00:24:06,317 -->
00:24:08,447
So grading things on with clearly
794
00:24:08,447 -->
00:24:10,097
defined a measurement benchmark.
795
00:24:10,097 -->
00:24:10,637
So we just say.
796
00:24:11,387 -->
00:24:12,947
Coach, you can grade a coach, you can
797
00:24:12,947 -->
00:24:14,897
grade a coaching conversation with rubric.
798
00:24:15,287 -->
00:24:16,757
This goes back to kind of teaching
799
00:24:16,757 -->
00:24:18,467
first principles as a professor, right?
800
00:24:18,467 -->
00:24:19,367
We have grading rubrics for
801
00:24:19,367 -->
00:24:20,957
when we graded grade papers.
802
00:24:20,957 -->
00:24:22,547
When we grade exams, we
803
00:24:22,547 -->
00:24:23,657
start with a rubric first.
804
00:24:24,317 -->
00:24:26,027
So for example, for our, we have
805
00:24:26,027 -->
00:24:27,677
our, our kind of rubric pillars.
806
00:24:27,677 -->
00:24:28,967
I can share a little bit about them.
807
00:24:29,357 -->
00:24:29,777
Um.
808
00:24:30,302 -->
00:24:31,592
We look at things like coaching,
809
00:24:31,592 -->
00:24:33,362
pro coaching as an explicit pillar.
810
00:24:33,362 -->
00:24:34,712
How good is, how good was the
811
00:24:34,712 -->
00:24:36,842
coaching in this conversation across
812
00:24:36,842 -->
00:24:38,282
different elements of coaching?
813
00:24:38,432 -->
00:24:41,312
Did we facilitate, um, facilitate change?
814
00:24:41,492 -->
00:24:43,292
Did we, was it person forced?
815
00:24:43,352 -->
00:24:45,992
Did we let, um, did we, uh.
816
00:24:46,662 -->
00:24:48,072
They were new things from
817
00:24:48,072 -->
00:24:49,302
coaching about them, about the
818
00:24:49,302 -->
00:24:50,442
world and about themselves.
819
00:24:51,132 -->
00:24:52,242
For example, as one of those
820
00:24:52,242 -->
00:24:53,952
elements we also have, how is
821
00:24:53,952 -->
00:24:55,122
the flow of the conversation?
822
00:24:55,392 -->
00:24:55,662
Right.
823
00:24:55,662 -->
00:24:57,702
Did was it, did it proceed as expected?
824
00:24:58,032 -->
00:24:59,022
And of course kind of user
825
00:24:59,022 -->
00:25:00,252
satisfaction, intent, right?
826
00:25:00,252 -->
00:25:01,302
It's kind of, kind of cross
827
00:25:01,302 -->
00:25:02,892
those di dimensions and we can,
828
00:25:02,897 -->
00:25:04,752
you can obviously dig into what
829
00:25:04,752 -->
00:25:05,802
those are, but those are kind.
830
00:25:06,932 -->
00:25:08,162
Some of those elements and kind of
831
00:25:08,162 -->
00:25:10,262
pillars of being able to evaluate
832
00:25:10,262 -->
00:25:11,822
a really subjective system.
833
00:25:12,032 -->
00:25:12,842
And that's where it takes kind
834
00:25:12,842 -->
00:25:14,402
of trained, calibrated humans.
835
00:25:14,792 -->
00:25:17,372
We spend a lot of time, um, calibrating
836
00:25:17,372 -->
00:25:19,922
our coaches so that when on those
837
00:25:19,922 -->
00:25:21,962
rubrics and aligning people so
838
00:25:21,962 -->
00:25:23,552
that we're getting clear signal.
839
00:25:23,672 -->
00:25:26,882
'cause the worst part is again, um,
840
00:25:27,902 -->
00:25:29,402
what's worse than having no evaluation,
841
00:25:29,402 -->
00:25:30,992
having bad evaluation, right.
842
00:25:31,412 -->
00:25:33,002
Because then you're, you're measuring
843
00:25:33,002 -->
00:25:34,022
the wrong thing and you're moving in
844
00:25:34,022 -->
00:25:35,582
the wrong direction, and you don't
845
00:25:35,582 -->
00:25:36,872
know if you're actually improving.
846
00:25:36,962 -->
00:25:38,762
That's a really common failure case.
847
00:25:38,762 -->
00:25:39,242
I see.
848
00:25:39,602 -->
00:25:41,312
Um, and experienced,
849
00:25:41,522 -->
00:25:42,602
um, personally as well.
850
00:25:43,762 -->
00:25:44,252
Josh Rubin: Yeah.
851
00:25:44,342 -->
00:25:45,182
Um, yeah.
852
00:25:45,182 -->
00:25:46,322
That's super, super interesting.
853
00:25:46,592 -->
00:25:49,532
Do, how do you think, so I, one,
854
00:25:49,532 -->
00:25:51,002
one thing that, that we think about
855
00:25:51,002 -->
00:25:52,677
a lot with our customers is, um.
856
00:25:55,202 -->
00:25:56,492
I dunno, I guess I call it sort
857
00:25:56,492 -->
00:25:58,502
of, um, aggregate and anecdote.
858
00:25:58,772 -->
00:26:02,222
Um, so, you know, things like aggregate
859
00:26:02,222 -->
00:26:04,622
metrics, you know, you can measure
860
00:26:04,622 -->
00:26:06,002
across you, you can sample, you can
861
00:26:06,002 -->
00:26:07,532
measure across, you can get a Gen A, a
862
00:26:07,532 -->
00:26:10,412
general tendency for performance along
863
00:26:10,412 -->
00:26:13,952
some parameter that you care about.
864
00:26:13,982 -->
00:26:15,152
You, you measure it in some way.
865
00:26:15,152 -->
00:26:18,402
You define some sort of, um, you know.
866
00:26:19,637 -->
00:26:20,807
Figure of merit, some sort of,
867
00:26:20,807 -->
00:26:22,427
some sort of metric that is, uh,
868
00:26:22,457 -->
00:26:24,227
or some set of metrics that you,
869
00:26:24,227 -->
00:26:27,647
you want to, uh, observe over time.
870
00:26:27,647 -->
00:26:28,937
And you certainly want to know if there
871
00:26:28,937 -->
00:26:30,527
are significant deviations over time,
872
00:26:30,917 -->
00:26:32,777
um, that that means something is awry.
873
00:26:32,777 -->
00:26:33,647
Usually something has
874
00:26:33,647 -->
00:26:34,517
changed in the world.
875
00:26:34,757 -->
00:26:35,537
People are asking some
876
00:26:35,537 -->
00:26:37,067
sort of new question that
877
00:26:37,247 -->
00:26:38,507
corresponds to some new event.
878
00:26:38,507 -->
00:26:39,437
Probably you guys.
879
00:26:40,352 -->
00:26:42,062
up having some experiences where, I
880
00:26:42,062 -->
00:26:43,682
mean, I, I could imagine, you know,
881
00:26:44,642 -->
00:26:46,772
world events or local events could
882
00:26:46,772 -->
00:26:49,052
push your models to have conversations
883
00:26:49,292 -->
00:26:50,367
in regimes that they were never.
884
00:26:51,392 -->
00:26:53,642
specifically, uh, tested
885
00:26:53,642 -->
00:26:55,772
in the, the other end.
886
00:26:55,772 -->
00:26:57,392
Let me, you one.
887
00:26:57,392 -->
00:26:59,372
Well, the anecdote is the other piece.
888
00:26:59,432 -->
00:27:00,512
I, I, I am sorry.
889
00:27:00,512 -->
00:27:01,772
I don't mean to overload your,
890
00:27:01,772 -->
00:27:03,692
uh, the, the, the stack with, with
891
00:27:03,692 -->
00:27:05,072
thoughts, but it's also like, I
892
00:27:05,072 -->
00:27:06,032
think the other piece that I think
893
00:27:06,032 -->
00:27:07,862
about is like, how do you catch the,
894
00:27:08,072 -->
00:27:09,842
the dangerous rare corner cases?
895
00:27:09,902 -->
00:27:11,522
And I think that piece is a little tougher
896
00:27:11,522 -->
00:27:14,732
from sort of sampling and, you know,
897
00:27:14,732 -->
00:27:16,442
I wouldn't quite know how to, you need
898
00:27:16,442 -->
00:27:18,602
some sort of probably a, a mechanism
899
00:27:18,602 -->
00:27:20,282
to identify those before you can.
900
00:27:20,687 -->
00:27:21,527
Give them to your high
901
00:27:21,527 -->
00:27:22,457
quality human labeler.
902
00:27:22,517 -->
00:27:24,917
I'm a huge fan of the, um, getting,
903
00:27:24,917 -->
00:27:26,477
getting humans to actually label stuff.
904
00:27:26,477 -->
00:27:28,577
I love the idea of calibrating your
905
00:27:28,577 -->
00:27:31,157
coaches on, on real human expertise.
906
00:27:31,577 -->
00:27:34,847
Um, yeah, so, so I'm not sure quite
907
00:27:34,847 -->
00:27:36,167
what the question was in there, except
908
00:27:36,167 -->
00:27:38,777
that I, I was gonna try to, I was
909
00:27:38,777 -->
00:27:39,797
getting, try to press you a little
910
00:27:39,797 -->
00:27:44,087
bit on, um, how you get at the sort of
911
00:27:44,357 -->
00:27:46,157
tricky corner cases that may not show
912
00:27:46,157 -->
00:27:47,807
up so much in a, in a random sample.
913
00:27:47,807 -->
00:27:48,077
Right.
914
00:27:48,542 -->
00:27:49,712
Those might be high stakes
915
00:27:49,712 -->
00:27:50,342
corner cases, right?
916
00:27:51,707 -->
00:27:53,597
Jeff Dalton: Yeah, well, I, I guess we'll,
917
00:27:53,597 -->
00:27:54,677
we'll start a little bit about some of
918
00:27:55,037 -->
00:27:56,807
what, maybe I'll share a few anecdotes,
919
00:27:57,227 -->
00:27:59,057
um, for example, and kind of where
920
00:27:59,057 -->
00:28:00,317
things, where things are challenging.
921
00:28:00,317 -->
00:28:01,757
So, you know, safety is fundamental.
922
00:28:01,757 -->
00:28:02,867
We have guardrail systems
923
00:28:02,867 -->
00:28:04,127
that, that we, that we build.
924
00:28:04,487 -->
00:28:05,777
Um, and then we're constantly
925
00:28:05,777 -->
00:28:07,397
onboarding new clients, um, and new
926
00:28:07,397 -->
00:28:08,897
clients are joining every, every day.
927
00:28:09,472 -->
00:28:11,452
Um, and um, one of those
928
00:28:11,452 -->
00:28:12,952
clients is a large legal firm.
929
00:28:13,492 -->
00:28:14,032
Um, and guess what?
930
00:28:14,032 -->
00:28:14,722
We have legal guardrail.
931
00:28:14,722 -->
00:28:15,772
Let's not give legal advice.
932
00:28:15,952 -->
00:28:17,362
But when people then start putting in,
933
00:28:17,392 -->
00:28:19,042
uploading their docs and like, help
934
00:28:19,042 -->
00:28:20,872
me coach, coach about this, about this
935
00:28:20,872 -->
00:28:23,302
document, um, are triggered our legal
936
00:28:23,302 -->
00:28:24,622
guardrails saying, I can't suddenly,
937
00:28:24,622 -->
00:28:26,332
I can't, um, give you legal advice.
938
00:28:26,722 -->
00:28:29,602
Um, and that, that was like, we quickly
939
00:28:29,602 -->
00:28:31,012
had, we saw that in our logs and
940
00:28:31,012 -->
00:28:32,212
we're like, okay, now we have to, we
941
00:28:32,212 -->
00:28:33,652
have to go back and quickly revisit.
942
00:28:33,947 -->
00:28:35,327
What, what the definition of
943
00:28:35,327 -->
00:28:37,337
legal advice means and what that,
944
00:28:37,337 -->
00:28:38,507
what that means for this client.
945
00:28:38,687 -->
00:28:40,187
Um, so making sure that things are
946
00:28:40,187 -->
00:28:41,747
both safe in the general, but also
947
00:28:41,747 -->
00:28:43,607
customized there to be able to address
948
00:28:43,607 -->
00:28:45,347
some of those, not just educators, but
949
00:28:45,827 -->
00:28:48,557
real domain shifts in terms of where
950
00:28:48,557 -->
00:28:49,997
things are being deployed and being
951
00:28:49,997 -->
00:28:51,617
able to see those, see those issues.
952
00:28:51,857 -->
00:28:53,177
So more for medical advice when
953
00:28:53,177 -->
00:28:54,647
we roll out to a health company,
954
00:28:55,097 -->
00:28:56,567
um, we have the, we have the same,
955
00:28:56,567 -->
00:28:57,767
we have the same issues, right?
956
00:28:57,767 -->
00:28:59,507
So we see those, we can anticipate some
957
00:28:59,507 -->
00:29:01,307
of them, but then it hit, we hit real
958
00:29:01,307 -->
00:29:03,377
users and the real users are, are messy.
959
00:29:03,962 -->
00:29:05,462
And real use is messy.
960
00:29:05,972 -->
00:29:07,082
And that's part of the, the fun
961
00:29:07,082 -->
00:29:08,402
kind of challenge that keeps us,
962
00:29:08,462 -->
00:29:09,632
um, that keeps us excited to be
963
00:29:09,632 -->
00:29:10,982
able to make these assistants kind
964
00:29:10,982 -->
00:29:12,572
of scalable for people over time.
965
00:29:12,932 -->
00:29:14,582
But another, you mentioned, for example,
966
00:29:15,302 -->
00:29:19,202
um, we're a coach and so, um, you can
967
00:29:19,202 -->
00:29:20,972
imagine when there was a, were some
968
00:29:20,972 -->
00:29:23,582
elections recently, and then we saw
969
00:29:23,642 -->
00:29:24,782
when, after those elections people
970
00:29:24,782 -->
00:29:25,742
came in, they're like, I'm depressed.
971
00:29:26,732 -->
00:29:28,832
And or on one side or the other,
972
00:29:29,072 -->
00:29:31,322
um, you have reactions to those.
973
00:29:31,622 -->
00:29:33,152
Um, and like how do we handle that in
974
00:29:33,152 -->
00:29:35,432
a way that's tactful, thoughtful, um,
975
00:29:35,432 -->
00:29:36,872
that doesn't just block on a political
976
00:29:36,872 -->
00:29:39,272
guardrail, but that still allows us to
977
00:29:39,272 -->
00:29:41,252
like, respond to your emotions while
978
00:29:41,252 -->
00:29:43,232
still maintaining our neutrality.
979
00:29:43,712 -->
00:29:45,047
Um, and kinda in a very.
980
00:29:45,827 -->
00:29:47,507
Um, in a very careful way.
981
00:29:48,287 -->
00:29:50,297
So that's some, some anecdotes from the
982
00:29:50,297 -->
00:29:52,877
front lines, um, from, from seeing sort,
983
00:29:53,027 -->
00:29:54,857
seeing some challenging situations there.
984
00:29:55,517 -->
00:29:57,617
Uh, the other kind of your kind
985
00:29:57,617 -->
00:29:58,427
of second question there is
986
00:29:58,427 -->
00:29:59,477
how do you find those cases?
987
00:29:59,477 -->
00:30:00,137
How do you see it?
988
00:30:00,977 -->
00:30:02,987
Um, and one of the things, one
989
00:30:02,987 -->
00:30:03,767
thing is there is that, that's
990
00:30:04,217 -->
00:30:04,732
where again, we all having LLMs.
991
00:30:05,562 -->
00:30:07,512
Kind of run, run as judges looking
992
00:30:07,512 -->
00:30:09,012
for those kind of edge cases,
993
00:30:09,012 -->
00:30:10,572
anomalies, where things were working.
994
00:30:11,442 -->
00:30:14,172
Um, we sample sample failures, so we
995
00:30:14,202 -->
00:30:16,422
over sample cases where we have low
996
00:30:16,662 -->
00:30:18,582
what, what, what that LLM thinks is a
997
00:30:18,582 -->
00:30:21,462
low quality conversation and then kind of
998
00:30:21,462 -->
00:30:24,942
prioritize those for anonymized review.
999
00:30:25,152 -->
00:30:26,682
Um, and, and kind of sampling
1000
00:30:27,042 -->
00:30:28,302
in privacy, preserving ways.
1001
00:30:32,132 -->
00:30:32,942
Josh Rubin: Yeah, that's, that's great.
1002
00:30:32,972 -->
00:30:33,722
That's really interesting.
1003
00:30:33,782 -->
00:30:36,992
Um, you know, I think, uh, you touched
1004
00:30:36,992 -->
00:30:38,612
on something that I think is, is I
1005
00:30:38,612 -->
00:30:40,142
think, relevant for both of us, which
1006
00:30:40,142 -->
00:30:42,872
is the, the challenges associated with,
1007
00:30:44,972 -->
00:30:45,662
you know, the general
1008
00:30:45,662 -->
00:30:46,922
versus the specific, right?
1009
00:30:46,922 -->
00:30:48,662
Like if you are able to roll
1010
00:30:48,662 -->
00:30:50,102
out a guardrail model that
1011
00:30:50,102 -->
00:30:52,472
is specific to constraint.
1012
00:30:53,432 -->
00:30:55,772
You know, uh, for legal, legal customers
1013
00:30:55,772 -->
00:31:00,962
or for healthcare, um, know, even, even
1014
00:31:00,962 -->
00:31:04,052
there, you're, the domain is reasonably
1015
00:31:04,052 -->
00:31:05,582
broad that it's not straightforward.
1016
00:31:05,582 -->
00:31:06,782
I think one of the, one of the things that
1017
00:31:06,782 -->
00:31:09,842
we a lot of time thinking about and uh,
1018
00:31:09,842 -->
00:31:11,462
is, is sort of a major challenge for, for,
1019
00:31:11,942 -->
00:31:14,192
you know, my team that's developing sort
1020
00:31:14,192 -->
00:31:16,052
of fairly general purpose guardrail models
1021
00:31:16,052 -->
00:31:19,922
is how do you deal with the, um, you know,
1022
00:31:20,042 -->
00:31:22,562
uh, dramatic differences between domains.
1023
00:31:23,642 -->
00:31:26,162
and you know, I think there's a
1024
00:31:26,162 -->
00:31:28,622
really interesting set of trade-offs
1025
00:31:28,652 -->
00:31:31,232
around, you know, I think for probably
1026
00:31:31,412 -->
00:31:32,762
many of our listeners here, right?
1027
00:31:32,762 -->
00:31:35,642
Like, where is the right place to draw
1028
00:31:35,642 -->
00:31:38,222
the balance for their particular use case?
1029
00:31:38,222 -->
00:31:40,352
I mean, not everyone is a startup who
1030
00:31:40,352 -->
00:31:46,502
gets to, um, uh, you know, focus on, know.
1031
00:31:46,667 -->
00:31:50,207
Uh, one kind of, uh, you know, uh,
1032
00:31:50,237 -->
00:31:52,127
coaching as a focus, you know, I
1033
00:31:52,127 -->
00:31:55,037
mean, I, I suspect we have, um, you
1034
00:31:55,037 -->
00:31:58,877
know, audience members who are, know,
1035
00:31:58,937 -->
00:32:01,847
overseeing different applications
1036
00:32:01,847 -->
00:32:03,767
across, you know, some financial services
1037
00:32:03,767 -->
00:32:06,662
organization and, um, you know, it.
1038
00:32:07,382 -->
00:32:08,942
Probably not a good use of resources
1039
00:32:08,942 -->
00:32:11,612
or practical to totally dial in
1040
00:32:11,612 -->
00:32:13,142
customization for every single
1041
00:32:13,142 -->
00:32:14,282
one of those use cases down.
1042
00:32:14,282 -->
00:32:16,352
You know, like how much can you repurpose
1043
00:32:16,352 -->
00:32:18,032
from off the shelf where there's some,
1044
00:32:18,032 -->
00:32:22,112
you know, expertise that's, uh, you
1045
00:32:22,112 -->
00:32:23,432
know, baked and doing, doing a pretty
1046
00:32:23,432 -->
00:32:26,192
good job at a thing versus, um, some
1047
00:32:26,192 -->
00:32:28,292
of the benefits you get out of the, the
1048
00:32:28,292 -->
00:32:31,382
very specific, uh, purpose-built thing.
1049
00:32:32,132 -->
00:32:35,042
Um, I think, uh, you know, one of
1050
00:32:35,042 -->
00:32:36,362
the things that I think sounds.
1051
00:32:36,737 -->
00:32:40,067
Appealing about, about your role is
1052
00:32:40,067 -->
00:32:41,477
you do really get to focus on this
1053
00:32:41,507 -->
00:32:44,837
particular use case and, um, generate
1054
00:32:44,837 -->
00:32:47,207
some sort of bespoke, uh, solutions to
1055
00:32:47,207 -->
00:32:48,437
the kinds of problems that you have.
1056
00:32:49,127 -->
00:32:50,507
How, maybe, maybe, I don't know, do,
1057
00:32:50,627 -->
00:32:52,487
do you have any sort of advice you
1058
00:32:52,487 -->
00:32:54,977
would give to the kind of spectrum
1059
00:32:54,977 -->
00:32:56,237
of people out there operating in
1060
00:32:56,237 -->
00:32:58,907
different orgs about, you know, when
1061
00:32:58,907 -->
00:33:00,917
it's a good idea to go kind of whole
1062
00:33:00,917 -->
00:33:05,802
hog into investing in, um, you know.
1063
00:33:06,677 -->
00:33:09,077
Bespoke parts of an agentic system versus
1064
00:33:09,077 -->
00:33:10,367
taking something that's much more off
1065
00:33:10,367 -->
00:33:11,417
the shelf in a common design pattern.
1066
00:33:13,367 -->
00:33:14,417
Jeff Dalton: Yeah, I mean, so
1067
00:33:14,417 -->
00:33:16,097
I'm, I'm, I'm gonna start, I
1068
00:33:16,097 -->
00:33:17,117
have a couple thoughts there.
1069
00:33:17,357 -->
00:33:18,797
One of, one of the, the first
1070
00:33:18,797 -->
00:33:20,162
challenges, do you have actually a clear.
1071
00:33:20,852 -->
00:33:21,572
Requirements.
1072
00:33:21,632 -->
00:33:22,592
Do you have, do you have a clear
1073
00:33:22,592 -->
00:33:23,942
spec for what things need to do?
1074
00:33:24,212 -->
00:33:25,742
Do you have some sample conversations,
1075
00:33:25,742 -->
00:33:28,142
some sample use cases people maybe might
1076
00:33:28,142 -->
00:33:29,612
rush to like, oh, I'm gonna make a prompt.
1077
00:33:29,612 -->
00:33:31,052
Like, whoa, whoa, whoa, stop.
1078
00:33:31,472 -->
00:33:32,432
Before you write the prompt,
1079
00:33:32,522 -->
00:33:34,292
what are, are you trying to do?
1080
00:33:34,352 -->
00:33:36,482
Do you have some data that those five, 10
1081
00:33:36,482 -->
00:33:38,492
examples at least kind of for that to be
1082
00:33:38,492 -->
00:33:40,052
able to know, at least have a sanity check
1083
00:33:40,052 -->
00:33:41,822
if things are, if things are working.
1084
00:33:42,122 -->
00:33:43,652
So if you, if you, again, if
1085
00:33:43,652 -->
00:33:44,822
you can't measure it, how do
1086
00:33:44,822 -->
00:33:46,082
you know if it's working or not?
1087
00:33:46,502 -->
00:33:47,822
I mean, so you need to have some data.
1088
00:33:48,127 -->
00:33:49,627
For that to be able to have some samples.
1089
00:33:49,627 -->
00:33:51,037
Now the big issue there is that the
1090
00:33:51,037 -->
00:33:52,567
data, data is different when you actually
1091
00:33:52,567 -->
00:33:54,337
deploy it, and then you actually have,
1092
00:33:54,547 -->
00:33:56,077
but these are actually kind of standard
1093
00:33:56,077 -->
00:33:57,277
known issues that we go, if you go
1094
00:33:57,277 -->
00:33:59,017
back to machine learning, we have model
1095
00:33:59,017 -->
00:34:00,937
drift and there's kind of just playbooks
1096
00:34:00,937 -->
00:34:02,917
on how you, on how you basically make
1097
00:34:02,917 -->
00:34:04,747
sure that your models are performing.
1098
00:34:04,807 -->
00:34:05,767
And that's, that's kind of
1099
00:34:05,767 -->
00:34:06,877
standard practice you can look at.
1100
00:34:07,187 -->
00:34:08,537
Some of the, the, some of the
1101
00:34:08,537 -->
00:34:10,727
literature from, from Google and others
1102
00:34:10,727 -->
00:34:12,287
who have been deploying ML models
1103
00:34:12,287 -->
00:34:14,567
at scale across, and many of your
1104
00:34:14,567 -->
00:34:15,527
listeners are doing the same thing.
1105
00:34:15,527 -->
00:34:16,877
So it's making sure that you're running
1106
00:34:16,877 -->
00:34:19,187
those same playbooks and running it with
1107
00:34:19,187 -->
00:34:20,957
a, with the agentic system so that that's
1108
00:34:20,957 -->
00:34:22,697
something that hasn't changed very much.
1109
00:34:22,727 -->
00:34:23,537
We just have to make sure that
1110
00:34:23,537 -->
00:34:25,457
we actually do it in these cases.
1111
00:34:25,937 -->
00:34:28,907
Um, I will say that for something
1112
00:34:28,907 -->
00:34:30,647
like safety or guardrails, you might
1113
00:34:30,647 -->
00:34:31,757
start with something off the shelf to
1114
00:34:31,757 -->
00:34:33,317
begin with and see how, how it works.
1115
00:34:33,797 -->
00:34:34,037
Um.
1116
00:34:35,102 -->
00:34:36,692
We actually, we started with
1117
00:34:36,722 -->
00:34:38,372
like, what, what was the simplest
1118
00:34:38,582 -->
00:34:39,572
possible version that we could
1119
00:34:39,572 -->
00:34:41,492
have as a first version as well?
1120
00:34:42,032 -->
00:34:43,052
And built it, built some
1121
00:34:43,052 -->
00:34:44,132
of those, our ourselves.
1122
00:34:44,582 -->
00:34:46,862
Um, and we continue to
1123
00:34:46,862 -->
00:34:47,492
build those ourselves.
1124
00:34:47,492 -->
00:34:48,632
'cause some of that's our, our business
1125
00:34:48,632 -->
00:34:49,772
value and it really matters to us.
1126
00:34:49,772 -->
00:34:51,182
So it that, that's as a,
1127
00:34:51,182 -->
00:34:52,532
as a safety feature, right?
1128
00:34:52,532 -->
00:34:53,582
For, for our users.
1129
00:34:53,762 -->
00:34:54,962
So if it really matters to you,
1130
00:34:55,082 -->
00:34:56,552
then you should own it and build it.
1131
00:34:56,882 -->
00:34:58,172
If it, if it's just something that
1132
00:34:58,172 -->
00:34:59,492
you need to have, that's just,
1133
00:34:59,492 -->
00:35:00,932
you know, that, that's just there.
1134
00:35:01,442 -->
00:35:03,272
Then go buy it and you know, take
1135
00:35:03,272 -->
00:35:04,922
something that's kind of mo off the shelf.
1136
00:35:05,312 -->
00:35:06,662
But I think from experience, what I
1137
00:35:06,662 -->
00:35:08,582
would've seen is quickly you hit issues
1138
00:35:08,582 -->
00:35:10,532
of, it doesn't quite work the way I
1139
00:35:10,532 -->
00:35:13,322
want it to, then I have to adapt it.
1140
00:35:13,472 -->
00:35:15,092
And if it's a big, complex framework
1141
00:35:15,152 -->
00:35:16,592
that's been built around it, then
1142
00:35:16,592 -->
00:35:18,062
it becomes, it's a lot harder to
1143
00:35:18,062 -->
00:35:19,832
change, um, and be able to get that
1144
00:35:19,832 -->
00:35:21,872
behavior, um, that you're looking for.
1145
00:35:22,887 -->
00:35:23,457
Josh Rubin: Gotcha.
1146
00:35:23,837 -->
00:35:25,247
Yeah, points.
1147
00:35:25,247 -->
00:35:26,927
I think that, um, you know, even within
1148
00:35:26,927 -->
00:35:29,177
our users, you know, we, while we try to
1149
00:35:29,177 -->
00:35:32,147
provide some set of stuff that mostly,
1150
00:35:32,147 -->
00:35:34,367
you know, works out of the box to provide
1151
00:35:34,367 -->
00:35:35,417
some sort of general coverage, you
1152
00:35:35,417 -->
00:35:37,307
know, safety, basic safety coverage and
1153
00:35:37,307 -->
00:35:39,107
Observability, you know, some general
1154
00:35:39,107 -->
00:35:41,957
purpose guardrails, um, you know.
1155
00:35:44,267 -->
00:35:46,217
That's fine for, for some applications
1156
00:35:46,217 -->
00:35:48,077
there are certain fairly standard things
1157
00:35:48,077 -->
00:35:50,987
that we see that, um, could be improved,
1158
00:35:50,987 -->
00:35:52,487
but are kind of at some reasonable level
1159
00:35:52,487 -->
00:35:54,257
of quality with the, with the basic stuff.
1160
00:35:54,257 -->
00:35:58,367
But, you know, one of the nice
1161
00:35:58,367 -->
00:36:00,257
things about, you know, our product
1162
00:36:00,257 -->
00:36:01,337
and the team I get to work with
1163
00:36:01,337 -->
00:36:03,437
here is that, you know, we have a
1164
00:36:03,437 -->
00:36:04,847
whole, you know, customer success
1165
00:36:04,847 -->
00:36:06,677
team that's really good at helping.
1166
00:36:07,007 -->
00:36:10,757
Our users, um, sort of adapt
1167
00:36:10,847 -->
00:36:12,437
our product to the particular
1168
00:36:12,437 -->
00:36:13,817
applications that the customers bring.
1169
00:36:13,817 -->
00:36:17,567
So some of that is, you know, uh, custom
1170
00:36:17,567 -->
00:36:20,147
metrics, you know, being able to throw,
1171
00:36:20,147 -->
00:36:23,987
uh, you know, uh, uh, calculations of
1172
00:36:23,987 -->
00:36:25,157
interest from a business perspective
1173
00:36:25,157 -->
00:36:27,587
up onto a dashboard or, um, you know,
1174
00:36:27,587 -->
00:36:29,657
customizable LLM as a judge that runs it.
1175
00:36:29,657 -->
00:36:31,697
Automation at scale so that you
1176
00:36:31,697 -->
00:36:32,837
know the specific thing that is
1177
00:36:32,837 -->
00:36:34,577
important to your, your use case.
1178
00:36:34,757 -->
00:36:35,797
At least you have a, you know.
1179
00:36:36,452 -->
00:36:38,462
The, the model's al not always the best,
1180
00:36:38,522 -->
00:36:41,492
but, um, you know, aggregate you have a
1181
00:36:41,492 -->
00:36:43,892
good proxy for how it performance might
1182
00:36:43,892 -->
00:36:45,512
be changing with time and stuff like that.
1183
00:36:45,902 -->
00:36:46,907
Um, so, uh.
1184
00:36:47,777 -->
00:36:48,827
Jeff Dalton: We certainly, and
1185
00:36:48,827 -->
00:36:49,787
I, I really appreciate that, and
1186
00:36:49,787 -->
00:36:50,687
I think that's fundamental to
1187
00:36:50,687 -->
00:36:52,187
making these systems successful.
1188
00:36:52,727 -->
00:36:54,527
Um, and like when, and for example,
1189
00:36:54,527 -->
00:36:55,637
when we roll out Nadia to a new
1190
00:36:55,637 -->
00:36:57,917
customer, um, there's a lot of
1191
00:36:57,917 -->
00:36:59,837
customization that goes into into that.
1192
00:36:59,837 -->
00:37:01,547
So making sure that we're curating the
1193
00:37:01,547 -->
00:37:02,987
right knowledge that Nadia knows about
1194
00:37:02,987 -->
00:37:04,697
you and your organization and how you work
1195
00:37:04,727 -->
00:37:06,497
and your terminology and your phrasing.
1196
00:37:07,037 -->
00:37:08,357
Um, having and having the
1197
00:37:08,357 -->
00:37:09,437
pilot so we can see that.
1198
00:37:09,437 -->
00:37:10,517
And then like what other, what
1199
00:37:10,517 -->
00:37:12,317
customization that needs to be done next.
1200
00:37:12,767 -->
00:37:13,577
Do you need a custom
1201
00:37:13,577 -->
00:37:15,347
workflow for, for feedback?
1202
00:37:15,347 -->
00:37:16,577
Do you need that custom?
1203
00:37:16,937 -->
00:37:16,997
Um.
1204
00:37:18,167 -->
00:37:20,837
Uh, the kind of custom role play,
1205
00:37:20,837 -->
00:37:23,117
for example, for, in terms of like
1206
00:37:23,117 -->
00:37:24,077
what you wanna role play with your
1207
00:37:24,077 -->
00:37:26,207
coach and how you approach role play.
1208
00:37:26,507 -->
00:37:28,217
Um, so those are kind of key things
1209
00:37:28,217 -->
00:37:29,657
that we wanna make sure that for, for
1210
00:37:29,657 -->
00:37:31,937
example, for a coach that we're, we're
1211
00:37:31,937 -->
00:37:33,887
clear in adding significant value.
1212
00:37:33,887 -->
00:37:35,867
And I think that's one thing why a
1213
00:37:35,867 -->
00:37:37,337
specialized coaching solution, like,
1214
00:37:37,367 -->
00:37:40,097
like, like Nadia and really kind
1215
00:37:40,097 -->
00:37:41,927
of beats co-pilots and, uh, other
1216
00:37:41,927 -->
00:37:43,217
kind of more generic solutions.
1217
00:37:44,462 -->
00:37:44,852
Josh Rubin: Yeah.
1218
00:37:45,062 -->
00:37:45,602
Yeah, totally.
1219
00:37:45,602 -->
00:37:46,502
I can totally imagine that.
1220
00:37:46,892 -->
00:37:47,522
Um,
1221
00:37:50,012 -->
00:37:50,822
how do you think about, how
1222
00:37:50,822 -->
00:37:52,142
do you think about privacy?
1223
00:37:52,142 -->
00:37:52,892
I mean, you're bringing up this
1224
00:37:52,892 -->
00:37:54,602
con concept of like, you know,
1225
00:37:54,602 -->
00:37:56,042
all these private conversations
1226
00:37:56,042 -->
00:37:57,932
with various kinds of role play,
1227
00:37:57,932 -->
00:38:01,442
and presumably there's, you know.
1228
00:38:02,987 -->
00:38:05,327
Some sort of, uh, org level
1229
00:38:05,327 -->
00:38:07,307
structure, org level customization
1230
00:38:07,307 -->
00:38:09,317
that tries to, has some interplay
1231
00:38:09,317 -->
00:38:11,417
between private conversations and,
1232
00:38:13,367 -->
00:38:17,327
you know, uh, org level oversight
1233
00:38:17,727 -->
00:38:18,147
Jeff Dalton: Yes.
1234
00:38:18,282 -->
00:38:18,417
Josh Rubin: this is.
1235
00:38:19,202 -->
00:38:20,642
Jeff Dalton: So, so I think our all, I
1236
00:38:20,642 -->
00:38:22,172
mean, organizations can provide their
1237
00:38:22,172 -->
00:38:23,672
customer guard guardrails and safety
1238
00:38:23,672 -->
00:38:25,112
in terms of what people can talk about.
1239
00:38:25,322 -->
00:38:27,152
They can seed it with, with organizational
1240
00:38:27,152 -->
00:38:29,072
knowledge that's important for the company
1241
00:38:29,072 -->
00:38:30,482
that's there for, for that's there.
1242
00:38:31,052 -->
00:38:32,462
Um, they can, they can have some
1243
00:38:32,462 -->
00:38:34,142
org tool calls for, to look up other
1244
00:38:34,142 -->
00:38:35,222
parts of knowledge to be able to
1245
00:38:35,222 -->
00:38:36,962
interact with for the key things.
1246
00:38:37,292 -->
00:38:38,672
But then I really kind of
1247
00:38:38,672 -->
00:38:39,932
fundamental first principle for us
1248
00:38:39,932 -->
00:38:41,252
is that your chats are your chats.
1249
00:38:41,322 -->
00:38:43,302
Privacy, they, no one
1250
00:38:43,302 -->
00:38:44,292
gets to see those, right?
1251
00:38:44,322 -->
00:38:45,492
That they're your, they're yours.
1252
00:38:45,492 -->
00:38:46,782
And that's a fundamental building
1253
00:38:46,782 -->
00:38:48,672
block of coaching is, is trust.
1254
00:38:49,062 -->
00:38:49,692
And that you have to be
1255
00:38:49,692 -->
00:38:50,412
able to trust the coach.
1256
00:38:50,442 -->
00:38:51,432
'cause you're, you have to be, you're
1257
00:38:51,432 -->
00:38:52,392
gonna be emotionally vulnerable.
1258
00:38:52,392 -->
00:38:53,292
You're gonna, you're gonna say, I'm
1259
00:38:53,292 -->
00:38:55,092
struggling with this and this is hard
1260
00:38:55,092 -->
00:38:56,472
and I'm having these people problems.
1261
00:38:56,742 -->
00:38:58,602
And that level of, of trust and
1262
00:38:58,602 -->
00:39:00,072
transparency is really fundamental
1263
00:39:00,072 -->
00:39:01,902
to growth, um, in terms of being
1264
00:39:01,902 -->
00:39:02,832
able to make things possible.
1265
00:39:02,832 -->
00:39:03,672
So that, that's really
1266
00:39:03,672 -->
00:39:04,992
just a fundamental pillar.
1267
00:39:05,242 -->
00:39:06,232
That we have kind of, that we've
1268
00:39:06,232 -->
00:39:07,252
built, kind of baked into the
1269
00:39:07,252 -->
00:39:08,752
product from, from the beginning.
1270
00:39:09,262 -->
00:39:10,702
Um, and also control and transparency.
1271
00:39:10,702 -->
00:39:12,202
So it's like you can go and you can
1272
00:39:12,202 -->
00:39:13,762
delete chats, you can delete insights
1273
00:39:13,762 -->
00:39:15,322
that the coaches learned about you,
1274
00:39:15,772 -->
00:39:17,422
and you can, you, you have full control
1275
00:39:17,422 -->
00:39:19,252
over that and not your organization.
1276
00:39:19,312 -->
00:39:19,582
Right.
1277
00:39:19,912 -->
00:39:22,582
Um, the also the ability, for
1278
00:39:22,582 -->
00:39:24,772
example, for you to share very clearly
1279
00:39:24,982 -->
00:39:26,152
with other people on your team.
1280
00:39:26,152 -->
00:39:27,832
So one of the, one the key things
1281
00:39:27,832 -->
00:39:29,452
is like if we want to move from
1282
00:39:29,452 -->
00:39:30,802
just coaching for me to coaching.
1283
00:39:31,157 -->
00:39:32,747
To a team, and we're moving.
1284
00:39:32,807 -->
00:39:33,827
In general, these agents are
1285
00:39:33,827 -->
00:39:35,327
gonna start working together.
1286
00:39:35,597 -->
00:39:36,197
They're gonna have to share
1287
00:39:36,197 -->
00:39:37,517
information somehow with each other.
1288
00:39:37,937 -->
00:39:39,677
So for example, um, in NAIA we
1289
00:39:39,677 -->
00:39:41,117
allow you to share information about
1290
00:39:41,117 -->
00:39:43,277
yourself publicly, and you control your
1291
00:39:43,307 -->
00:39:45,047
kind of public profile of yourself.
1292
00:39:45,407 -->
00:39:46,457
So your work profile.
1293
00:39:46,877 -->
00:39:48,317
So you might, you might be willing
1294
00:39:48,317 -->
00:39:49,787
to share your personality profile.
1295
00:39:49,907 -->
00:39:50,807
You might be willing to share
1296
00:39:50,807 -->
00:39:52,307
information about some of your calendar.
1297
00:39:52,937 -->
00:39:53,957
Share information about how you
1298
00:39:53,957 -->
00:39:55,667
worked, how to work best with you,
1299
00:39:55,907 -->
00:39:57,737
that then another Nadia can kind of
1300
00:39:57,737 -->
00:40:00,467
come and read and have access to, and
1301
00:40:00,467 -->
00:40:02,837
be able to see that information and,
1302
00:40:02,837 -->
00:40:04,187
and be able to then use that to be
1303
00:40:04,187 -->
00:40:05,507
able to customize the, the coaching.
1304
00:40:05,507 -->
00:40:07,577
So that kind of fundamental, um,
1305
00:40:07,577 -->
00:40:09,647
transparency around what's being
1306
00:40:09,647 -->
00:40:10,877
shared, how it's being shared
1307
00:40:10,877 -->
00:40:12,587
in a way that you control very
1308
00:40:12,587 -->
00:40:14,327
clearly is, is really important.
1309
00:40:14,747 -->
00:40:15,977
And I, I think will continue to be
1310
00:40:16,037 -->
00:40:18,557
important, um, for the en entire industry.
1311
00:40:19,112 -->
00:40:20,552
Um, when we, as, as these systems
1312
00:40:20,552 -->
00:40:22,232
become more complex and we, we are
1313
00:40:22,232 -->
00:40:23,462
increasingly sharing information.
1314
00:40:25,297 -->
00:40:26,807
Josh Rubin: Yeah, I, I think, um.
1315
00:40:27,707 -->
00:40:29,687
Um, on the side, I've tinkered around
1316
00:40:29,687 -->
00:40:30,887
with memory systems a little bit, and
1317
00:40:30,887 -->
00:40:32,507
I, you know, I think the, the idea at
1318
00:40:32,507 -->
00:40:34,367
least a few times has crossed my mind
1319
00:40:34,367 -->
00:40:37,907
that we're likely entering a world just
1320
00:40:38,567 -->
00:40:42,467
where, um, we have age agentic assistance
1321
00:40:42,467 -->
00:40:44,867
that are working for us, that have access
1322
00:40:44,867 -->
00:40:46,457
to a variety of privileged knowledge
1323
00:40:46,457 -->
00:40:48,977
about our own experiences and about our
1324
00:40:48,977 -->
00:40:51,587
family is they're useful to us when we
1325
00:40:51,947 -->
00:40:53,567
can share the specifics of our lives
1326
00:40:53,567 -->
00:40:55,817
in a personal way with them, you know.
1327
00:40:56,897 -->
00:40:59,567
but we're also already using them to make
1328
00:40:59,567 -->
00:41:02,027
reservations on the phone to restaurants
1329
00:41:02,027 -->
00:41:05,087
and you know, if we're not already
1330
00:41:05,087 -->
00:41:06,617
letting our agents talk to other people's
1331
00:41:06,617 -->
00:41:08,747
agents, you know, it's gonna be next week.
1332
00:41:08,747 -->
00:41:09,107
Right.
1333
00:41:09,467 -->
00:41:12,407
Um, I think it's a really interesting
1334
00:41:12,407 -->
00:41:17,357
question about how we federate knowledge.
1335
00:41:17,567 -->
00:41:18,377
Um.
1336
00:41:19,277 -->
00:41:22,307
In a way that's, uh, sort of, uh,
1337
00:41:22,337 -->
00:41:24,227
deterministically enforceable versus
1338
00:41:24,227 -->
00:41:28,907
just, um, sort of fall into the model.
1339
00:41:29,747 -->
00:41:30,557
If you, if you follow
1340
00:41:30,557 -->
00:41:31,547
what I'm trying to say,
1341
00:41:31,782 -->
00:41:32,072
Jeff Dalton: Yeah.
1342
00:41:32,147 -->
00:41:32,777
Josh Rubin: I don't think we be,
1343
00:41:32,807 -->
00:41:35,027
we've begun to solve that problem yet.
1344
00:41:35,087 -->
00:41:35,387
Um.
1345
00:41:36,032 -->
00:41:38,582
Jeff Dalton: And, um, it reminds
1346
00:41:38,582 -->
00:41:39,572
me, I think we're, we're going
1347
00:41:39,572 -->
00:41:40,832
to need to do some research here.
1348
00:41:40,892 -->
00:41:41,732
It's gonna take, it's gonna
1349
00:41:41,732 -->
00:41:42,782
take a little bit of evolution.
1350
00:41:43,322 -->
00:41:45,632
There's gonna be a, a future of, this is
1351
00:41:45,632 -->
00:41:47,222
my knowledge and how do I share this and
1352
00:41:47,222 -->
00:41:49,502
make this available with not just with
1353
00:41:49,502 -->
00:41:51,722
multiple agents in ways that have clear,
1354
00:41:52,652 -->
00:41:54,602
um, clear scope for where and how they
1355
00:41:54,602 -->
00:41:56,792
can be used and how they can be shared.
1356
00:41:57,047 -->
00:41:58,247
Um, and if they're remembered
1357
00:41:58,307 -->
00:41:59,537
or whether they're, whether,
1358
00:41:59,537 -->
00:42:00,887
whether they have to be forgotten.
1359
00:42:01,217 -->
00:42:02,447
So you, oh, you can use this to make
1360
00:42:02,447 -->
00:42:03,767
that reservation, but then don't
1361
00:42:03,767 -->
00:42:05,627
remember that I, you know, hate Thai
1362
00:42:05,627 -->
00:42:07,187
food other otherwise or something.
1363
00:42:07,307 -->
00:42:07,877
That fact, right?
1364
00:42:07,877 -->
00:42:08,807
I actually love Thai food.
1365
00:42:09,137 -->
00:42:09,777
Um, but the.
1366
00:42:10,502 -->
00:42:11,522
Whatever that preference, whatever
1367
00:42:11,522 -->
00:42:12,782
that user's preference is, right.
1368
00:42:12,872 -->
00:42:15,602
Um, the, if you, if you look at some of
1369
00:42:15,602 -->
00:42:17,192
these standards are starting to emerge,
1370
00:42:17,252 -->
00:42:18,962
it reminds me there was some work from
1371
00:42:18,992 -->
00:42:21,932
Tim Burners Lee, um, you know, founder
1372
00:42:21,932 -->
00:42:23,162
of the world or web, kind of looking
1373
00:42:23,162 -->
00:42:24,842
at, oh, privacy is gonna be fundamental.
1374
00:42:24,842 -->
00:42:26,672
We need it, we are going to need some new
1375
00:42:26,672 -->
00:42:29,942
foundational, um, layers and standards
1376
00:42:30,032 -->
00:42:31,712
that are being built for people's
1377
00:42:31,712 -->
00:42:33,452
knowledge and personal knowledge bases.
1378
00:42:33,902 -->
00:42:35,192
And that's gonna be increasingly
1379
00:42:35,192 -->
00:42:36,662
important when we're moving into.
1380
00:42:37,232 -->
00:42:39,302
Agents being, having those, those
1381
00:42:39,302 -->
00:42:41,132
knowledge stores, being able to take
1382
00:42:41,132 -->
00:42:42,422
them out to be able to control them.
1383
00:42:42,452 -->
00:42:43,322
I wanna share them with a
1384
00:42:43,322 -->
00:42:44,762
different agent as well.
1385
00:42:44,792 -->
00:42:46,262
How do, how do we make those portable?
1386
00:42:46,832 -->
00:42:48,782
There's no, there's very, there's not
1387
00:42:48,782 -->
00:42:49,892
clear standards for that today, and
1388
00:42:49,892 -->
00:42:51,362
we, and it's something that's gonna
1389
00:42:51,362 -->
00:42:53,012
be increasingly important, but I don't
1390
00:42:53,012 -->
00:42:54,302
have a good solution for you today.
1391
00:42:54,302 -->
00:42:55,742
But it's, it's really, it's
1392
00:42:55,742 -->
00:42:56,672
a really important problem.
1393
00:42:57,362 -->
00:42:57,722
But I also, a
1394
00:42:57,852 -->
00:42:58,072
Josh Rubin: one,
1395
00:42:58,142 -->
00:42:58,472
Jeff Dalton: lot of smart
1396
00:42:58,472 -->
00:42:59,192
people working on it.
1397
00:42:59,567 -->
00:43:00,227
Josh Rubin: one thing really
1398
00:43:00,227 -->
00:43:01,667
interesting that you raised that's worth
1399
00:43:01,667 -->
00:43:04,337
highlighting is the degree to which, um.
1400
00:43:05,837 -->
00:43:09,407
It sounds like Nadia is a sort of a
1401
00:43:09,407 -->
00:43:13,817
synthesis, some sort of hybrid of, um,
1402
00:43:13,967 -->
00:43:17,327
being large language model and being
1403
00:43:18,257 -->
00:43:20,297
some deterministic scaffolding around
1404
00:43:20,297 -->
00:43:23,837
tasks and, um, sort of, structured
1405
00:43:23,837 -->
00:43:25,727
explorations that are designed by experts.
1406
00:43:25,727 -->
00:43:25,877
Right.
1407
00:43:25,877 -->
00:43:26,777
I think, I think, um.
1408
00:43:29,102 -->
00:43:31,652
One pitfall is maybe sort of
1409
00:43:31,652 -->
00:43:34,202
giving too much freedom to the task
1410
00:43:34,202 -->
00:43:37,112
pursuing agent in every scenario.
1411
00:43:37,412 -->
00:43:39,392
Do you think that providing
1412
00:43:39,392 -->
00:43:42,092
sort of I'm gonna call it semi
1413
00:43:42,092 -->
00:43:43,352
deterministic structure, right?
1414
00:43:43,352 -->
00:43:47,582
Giving, uh, guiding the language
1415
00:43:47,582 -->
00:43:50,312
model with specific prompts that
1416
00:43:51,212 -->
00:43:53,132
follow a schedule that, that sort
1417
00:43:53,132 -->
00:43:53,792
of, I, I don't know if there's
1418
00:43:53,792 -->
00:43:56,282
a good name for, um, for that.
1419
00:43:56,492 -->
00:43:57,002
Uh.
1420
00:43:57,767 -->
00:44:01,187
Sort of paradigm of, know, deliberately
1421
00:44:01,187 -->
00:44:03,227
trying to guide the model, you know,
1422
00:44:03,227 -->
00:44:05,087
along something that's a little bit
1423
00:44:05,087 -->
00:44:07,517
more of a deterministic state machine.
1424
00:44:07,517 -->
00:44:09,287
But I, it seems to me that,
1425
00:44:09,747 -->
00:44:10,137
Jeff Dalton: I could,
1426
00:44:10,427 -->
00:44:10,847
Josh Rubin: yeah,
1427
00:44:11,217 -->
00:44:11,547
Jeff Dalton: I can dive
1428
00:44:11,547 -->
00:44:12,297
in a little bit there.
1429
00:44:12,717 -->
00:44:13,922
There's a. There's a long history
1430
00:44:13,922 -->
00:44:15,532
in dialogue systems like SIGDial
1431
00:44:15,662 -->
00:44:16,622
has been around a long time.
1432
00:44:16,622 -->
00:44:17,792
We've learned a lot about how
1433
00:44:17,792 -->
00:44:18,932
conversational systems work.
1434
00:44:19,442 -->
00:44:21,302
Um, and, and your past listeners have,
1435
00:44:21,332 -->
00:44:22,802
have heard people talking about, uh,
1436
00:44:22,862 -->
00:44:24,782
those some classic dialogue systems.
1437
00:44:25,202 -->
00:44:27,062
We had finite state machines.
1438
00:44:27,062 -->
00:44:28,472
We have dialogue trees.
1439
00:44:28,472 -->
00:44:29,702
If you've ever, you know, called a
1440
00:44:29,702 -->
00:44:31,047
phone, you've used a dialogue tree.
1441
00:44:31,927 -->
00:44:33,247
So you can go step by step.
1442
00:44:33,457 -->
00:44:34,717
We also then learned, we moved
1443
00:44:34,747 -->
00:44:36,007
the dialogue systems evolved to
1444
00:44:36,007 -->
00:44:37,687
frame based models, um, which
1445
00:44:37,687 -->
00:44:39,367
moved to like slots and values.
1446
00:44:39,367 -->
00:44:40,207
And you're like, I just need to make
1447
00:44:40,207 -->
00:44:41,767
sure that these things get filled.
1448
00:44:41,767 -->
00:44:43,267
I don't care the order you do them in.
1449
00:44:43,567 -->
00:44:44,737
So now that gave the system the
1450
00:44:44,737 -->
00:44:46,657
flexibility to ask multiple, get
1451
00:44:46,687 -->
00:44:48,127
multiple pieces of information in the
1452
00:44:48,127 -->
00:44:50,137
same turn, for example, from the user.
1453
00:44:50,527 -->
00:44:50,857
Um.
1454
00:44:51,447 -->
00:44:52,857
We're continuing to move.
1455
00:44:52,917 -->
00:44:54,867
Um, and now it's, it's largely
1456
00:44:54,867 -->
00:44:55,677
like we, now we're building
1457
00:44:55,677 -->
00:44:57,057
an artifact with the user.
1458
00:44:57,567 -->
00:44:59,097
And so we have to, basically, we are,
1459
00:44:59,097 -->
00:45:00,477
we're not just, it's not just, um,
1460
00:45:00,507 -->
00:45:01,917
there's a clear outcome that we're trying
1461
00:45:01,917 -->
00:45:03,537
to achieve, whether it's a coaching
1462
00:45:03,537 -->
00:45:05,757
plan or whether it's that, that a user
1463
00:45:05,757 -->
00:45:07,197
is committed to, or whether there's
1464
00:45:07,197 -->
00:45:08,517
a, an email that's getting written.
1465
00:45:09,017 -->
00:45:11,057
There's an actual, um, outcome that's
1466
00:45:11,057 -->
00:45:13,757
being drafted with the agent in the loop.
1467
00:45:14,327 -->
00:45:15,737
Um, and, and you can measure progress
1468
00:45:15,827 -->
00:45:17,177
and, and what's complete and what's
1469
00:45:17,177 -->
00:45:18,977
not complete, um, against that.
1470
00:45:19,487 -->
00:45:21,227
And so you can still have a, an LLM
1471
00:45:21,227 -->
00:45:23,117
kind of following structured process
1472
00:45:23,182 -->
00:45:23,602
Josh Rubin: Mm-hmm.
1473
00:45:23,657 -->
00:45:24,047
Jeff Dalton: to know that.
1474
00:45:24,932 -->
00:45:26,432
And if I think about how we,
1475
00:45:26,522 -->
00:45:28,802
how we think about Nadia, we
1476
00:45:28,802 -->
00:45:30,332
started really, really simple.
1477
00:45:30,752 -->
00:45:32,282
We started with a single, uh,
1478
00:45:32,282 -->
00:45:33,062
uh, we didn't start with a
1479
00:45:33,062 -->
00:45:34,622
complex multi-agent system.
1480
00:45:34,772 -->
00:45:36,032
We started with a kinda a single
1481
00:45:36,032 -->
00:45:37,712
threaded agent that's, that was kind of
1482
00:45:37,832 -->
00:45:39,842
working that would then had different
1483
00:45:39,842 -->
00:45:41,642
kind of workflows with code to be
1484
00:45:41,642 -->
00:45:43,592
able to pull in things like different
1485
00:45:43,592 -->
00:45:45,152
parts of memory at the right time.
1486
00:45:45,662 -->
00:45:46,772
Um, and that's kind of controlled
1487
00:45:46,772 -->
00:45:47,912
deterministically with code
1488
00:45:47,912 -->
00:45:48,962
and different kind of prompt
1489
00:45:48,962 -->
00:45:50,792
gates and models that we have.
1490
00:45:51,122 -->
00:45:51,452
So.
1491
00:45:51,842 -->
00:45:53,912
We can clearly see and, and inspect,
1492
00:45:54,092 -->
00:45:55,832
um, what that, what that workflow is
1493
00:45:56,072 -->
00:45:57,662
working, whether it's working, how it's
1494
00:45:57,662 -->
00:45:59,282
working, and be able to make changes.
1495
00:45:59,462 -->
00:46:00,992
And now it's pretty easy to change code.
1496
00:46:01,502 -->
00:46:03,752
Um, so if we want to evolve that system,
1497
00:46:03,782 -->
00:46:05,312
we can, we can change those rules and
1498
00:46:05,312 -->
00:46:06,632
make 'em more flexible and, and have
1499
00:46:06,632 -->
00:46:09,092
them be in code and where it makes
1500
00:46:09,092 -->
00:46:11,527
sense, um, pull things out into subagents
1501
00:46:11,702 -->
00:46:12,962
where, where they're appropriate.
1502
00:46:13,932 -->
00:46:14,987
Josh Rubin: I, I love this, Jeff.
1503
00:46:14,987 -->
00:46:17,237
I think that, um, so it sounds like
1504
00:46:17,237 -->
00:46:19,217
you come down somewhere with, with
1505
00:46:19,217 -->
00:46:21,377
respect to kind of federating memory,
1506
00:46:21,377 -->
00:46:23,117
it sounds like you see a lot of utility
1507
00:46:23,117 -->
00:46:24,497
in this kind of deterministic state
1508
00:46:24,497 -->
00:46:30,047
machine management of uh, state.
1509
00:46:31,407 -->
00:46:31,827
Jeff Dalton: Because you
1510
00:46:31,827 -->
00:46:32,697
don't need all memory at all,
1511
00:46:33,207 -->
00:46:35,067
everything at all times, right?
1512
00:46:35,427 -->
00:46:37,107
Um, we now have large language models that
1513
00:46:37,107 -->
00:46:39,417
have pretty huge context windows, massive
1514
00:46:39,417 -->
00:46:41,007
context windows, millions of tokens.
1515
00:46:41,007 -->
00:46:42,537
We can fit books, entire books
1516
00:46:42,867 -->
00:46:44,697
inside, inside of a context window.
1517
00:46:44,997 -->
00:46:47,127
So it's about how, how, what information
1518
00:46:47,127 -->
00:46:48,477
do we need to have all the time?
1519
00:46:48,747 -->
00:46:50,097
What information do we need to have some
1520
00:46:50,097 -->
00:46:52,917
of the time and be able to have relevance?
1521
00:46:53,067 -->
00:46:54,267
And so again, I have a search
1522
00:46:54,267 -->
00:46:55,947
background, so I always think in
1523
00:46:55,947 -->
00:46:56,852
the, in the concept of like what.
1524
00:46:57,172 -->
00:46:58,282
Piece of information was
1525
00:46:58,282 -->
00:47:00,412
relevant for right now.
1526
00:47:00,952 -->
00:47:02,302
And if I give you this information,
1527
00:47:02,302 -->
00:47:04,612
are you using it to materially change
1528
00:47:04,612 -->
00:47:06,712
the output in, in the, in the system?
1529
00:47:07,372 -->
00:47:09,652
Um, so there's obviously citations and,
1530
00:47:09,652 -->
00:47:11,542
and elements that we have then we, that we
1531
00:47:11,542 -->
00:47:12,952
know, we cited different information, that
1532
00:47:12,952 -->
00:47:14,572
we use different pieces of information.
1533
00:47:14,992 -->
00:47:17,182
And you can track that and see and, okay,
1534
00:47:17,302 -->
00:47:19,102
if we always had this information here, it
1535
00:47:19,102 -->
00:47:21,952
was never used by in any of the outputs.
1536
00:47:21,982 -->
00:47:22,912
So do we need it?
1537
00:47:23,152 -->
00:47:24,772
No, we can prune that from our context
1538
00:47:24,772 -->
00:47:26,392
windows and just, and put something else
1539
00:47:26,392 -->
00:47:28,612
in that's valuable instead that then
1540
00:47:28,612 -->
00:47:31,312
see an experiment with, um, different
1541
00:47:31,312 -->
00:47:33,592
types of context and different types of,
1542
00:47:33,712 -->
00:47:35,452
whether it's retrieval or whether it's
1543
00:47:35,452 -->
00:47:38,032
more deterministic, um, kind of elements
1544
00:47:38,032 -->
00:47:39,742
of rules that are sometimes always there.
1545
00:47:39,772 -->
00:47:41,242
Is it, sometimes we have to then
1546
00:47:41,242 -->
00:47:42,922
factor up, okay, well this is
1547
00:47:42,922 -->
00:47:43,672
never used, but we still have
1548
00:47:43,672 -->
00:47:45,052
to have it in these three cases.
1549
00:47:45,412 -->
00:47:46,132
Let's move it out.
1550
00:47:46,792 -->
00:47:47,182
Um, so.
1551
00:47:48,002 -->
00:47:49,562
You know, for example, do we need to
1552
00:47:49,562 -->
00:47:52,172
trigger memory on every single for for
1553
00:47:52,172 -->
00:47:54,062
every single system in every single turn?
1554
00:47:54,122 -->
00:47:54,452
No.
1555
00:47:54,782 -->
00:47:55,442
We have to be very
1556
00:47:55,442 -->
00:47:56,522
selective and be relevant.
1557
00:47:56,522 -->
00:47:58,982
It's like, this is what we think we need.
1558
00:47:59,162 -->
00:48:00,542
Oh, you mentioned a person, so we,
1559
00:48:00,602 -->
00:48:01,982
let's go look up that person's name.
1560
00:48:02,402 -->
00:48:03,332
Oh, you mentioned a project.
1561
00:48:03,332 -->
00:48:04,112
Let's go make sure that we've
1562
00:48:04,112 -->
00:48:04,982
like looked up when you've
1563
00:48:04,982 -->
00:48:06,362
mentioned that project recently.
1564
00:48:06,962 -->
00:48:08,252
And so it's about having the systems
1565
00:48:08,252 -->
00:48:10,322
and intelligence layer to be able to be
1566
00:48:10,322 -->
00:48:12,242
able to coordinate those, that memory,
1567
00:48:12,452 -->
00:48:13,982
those memory systems at the right time.
1568
00:48:14,492 -->
00:48:14,702
Josh Rubin: Nice.
1569
00:48:14,702 -->
00:48:15,482
That's a great answer.
1570
00:48:15,662 -->
00:48:17,912
Um, so I've finally just discovered
1571
00:48:17,912 -->
00:48:19,532
where the q and A resides with which
1572
00:48:19,532 -->
00:48:20,852
button I have to press to bring it up.
1573
00:48:20,852 -->
00:48:23,432
I, apparently I'm not very good at this.
1574
00:48:23,762 -->
00:48:25,232
Um, so let's, let's, now that we have.
1575
00:48:25,232 -->
00:48:26,342
Have I have some questions.
1576
00:48:26,402 -->
00:48:28,352
Uh, maybe we can quickly, you know,
1577
00:48:28,352 -->
00:48:29,882
spend five minutes or so kind of
1578
00:48:29,882 -->
00:48:33,122
addressing, uh, them and then,
1579
00:48:33,152 -->
00:48:34,622
uh, at least a couple of them.
1580
00:48:34,652 -->
00:48:37,172
So, uh, we have one question that
1581
00:48:37,172 -->
00:48:39,182
is, what kind of evaluations are
1582
00:48:39,182 -->
00:48:41,072
needed in production environments,
1583
00:48:41,552 -->
00:48:43,262
um, to assess if the agent is
1584
00:48:43,262 -->
00:48:45,482
going rogue or stuck in a loop.
1585
00:48:45,842 -->
00:48:47,072
What kind of monitoring is
1586
00:48:47,072 -->
00:48:48,842
necessary to address and trace
1587
00:48:48,842 -->
00:48:51,092
its memory, context and decisions?
1588
00:48:54,102 -->
00:48:54,792
Jeff Dalton: Well, I think, I think
1589
00:48:54,792 -->
00:48:56,142
you probably have some, some experience
1590
00:48:56,142 -->
00:48:57,312
there, so I'd love to have some of your
1591
00:48:57,312 -->
00:48:58,812
thoughts there In terms of, I'll share
1592
00:48:58,812 -->
00:49:00,402
a little bit of, kind of reiterate
1593
00:49:00,402 -->
00:49:01,662
a little bit of what I said earlier.
1594
00:49:01,662 -->
00:49:02,457
It's like we have our own.
1595
00:49:02,742 -->
00:49:04,722
Custom internal thinking structures that
1596
00:49:04,722 -->
00:49:06,192
we have, so we know what the agent thinks
1597
00:49:06,192 -->
00:49:08,292
it's trying to do, and we can trace, we
1598
00:49:08,622 -->
00:49:10,332
can trace those so when it, we can see
1599
00:49:10,332 -->
00:49:11,832
it getting stuck, oh, it's stuck in G
1600
00:49:11,832 -->
00:49:13,752
again, shoot, we have to go fix that.
1601
00:49:13,932 -->
00:49:14,982
There's some, there's some rule
1602
00:49:14,982 -->
00:49:16,392
that it, that there is a flag that's
1603
00:49:16,392 -->
00:49:17,742
not getting set to allow it to
1604
00:49:17,742 -->
00:49:19,092
progress the way we expect it to.
1605
00:49:19,362 -->
00:49:21,072
So if you have the, if you have those
1606
00:49:21,252 -->
00:49:23,202
reasoning traces, whether it's thought
1607
00:49:23,202 -->
00:49:24,852
traces of the model, you can actually
1608
00:49:24,852 -->
00:49:27,792
see and, and that's inspectable and
1609
00:49:27,792 -->
00:49:29,082
making sure that it's inspectable, but
1610
00:49:29,082 -->
00:49:30,732
also making sure that it's structured.
1611
00:49:31,102 -->
00:49:33,142
So that, so in, in structured ways
1612
00:49:33,142 -->
00:49:34,552
so that it's not just free text.
1613
00:49:35,032 -->
00:49:36,422
Um, that also is a kind of a, a
1614
00:49:36,422 -->
00:49:37,612
fundamental principle, I think to
1615
00:49:37,612 -->
00:49:39,082
be able to make that, uh, debug.
1616
00:49:39,562 -->
00:49:40,912
But I'm also curious, you, you probably
1617
00:49:40,912 -->
00:49:42,017
have some additional thoughts there, Josh.
1618
00:49:42,792 -->
00:49:43,482
Josh Rubin: A lot of thoughts.
1619
00:49:43,512 -->
00:49:45,072
Uh, it's, it's a little bit of
1620
00:49:45,072 -->
00:49:46,212
a tricky question, so I think,
1621
00:49:46,212 -->
00:49:47,352
you know, in production, so.
1622
00:49:47,832 -->
00:49:49,242
I'm, I'm, I, I assume that the
1623
00:49:49,242 -->
00:49:52,932
question at some level is about, um, so
1624
00:49:54,972 -->
00:49:57,342
in some ways guard railing is harder
1625
00:49:57,402 -->
00:49:59,982
than Observability because, or sort
1626
00:49:59,982 -->
00:50:02,262
of production Observability, know, uh,
1627
00:50:02,652 -->
00:50:04,182
guard railing is essentially trying
1628
00:50:04,182 -->
00:50:06,972
to stop an errant process in, in flow.
1629
00:50:06,972 -->
00:50:09,792
Uh, you know, the Observability, you
1630
00:50:09,792 -->
00:50:12,192
have time to use smart, smart models
1631
00:50:12,192 -->
00:50:13,992
and sort of chew on the dataset.
1632
00:50:14,492 -->
00:50:15,842
Um, stuff doesn't have to happen
1633
00:50:15,842 -->
00:50:17,882
quite in real time and still
1634
00:50:17,882 -->
00:50:19,802
gives you sort of debug ability
1635
00:50:19,802 -->
00:50:22,082
into what the failure modes are.
1636
00:50:22,322 -->
00:50:25,112
Um, so, you know, I think that there
1637
00:50:25,112 -->
00:50:28,742
are, you know, for egregious errors,
1638
00:50:28,922 -->
00:50:31,172
these are things that an LLM as a judge,
1639
00:50:31,892 -->
00:50:33,692
you know, capable of ingesting a lot
1640
00:50:33,692 -->
00:50:35,882
of data, some entire conversation,
1641
00:50:35,882 -->
00:50:37,352
trace some sort of trace level.
1642
00:50:37,802 -->
00:50:39,812
Um, uh.
1643
00:50:41,042 -->
00:50:43,412
You know, uh, conversation telemetry,
1644
00:50:43,412 -->
00:50:47,582
you can, you can capture, um, and
1645
00:50:47,582 -->
00:50:50,372
these LLMs are generally quite good
1646
00:50:50,372 -->
00:50:55,742
at identifying, um, you know, or
1647
00:50:55,742 -->
00:50:57,812
classifying common failure modes.
1648
00:50:57,812 -->
00:50:58,412
Hopefully this isn't
1649
00:50:58,412 -->
00:50:59,402
happening too commonly.
1650
00:50:59,762 -->
00:51:01,202
Um, but you, you also
1651
00:51:01,202 -->
00:51:02,372
have other proxies, right?
1652
00:51:02,372 -->
00:51:03,962
You can look at things that
1653
00:51:03,962 -->
00:51:05,342
raise guardrail errors.
1654
00:51:05,342 -->
00:51:06,032
You can look at.
1655
00:51:06,797 -->
00:51:08,987
know, ideally we get user feedback
1656
00:51:08,987 -->
00:51:11,657
sometimes when something's gone awry, and
1657
00:51:11,657 -->
00:51:13,457
that can sometimes be your sort of entry
1658
00:51:13,457 -->
00:51:15,797
point to investigating a certain mode.
1659
00:51:16,007 -->
00:51:17,117
You know, sometimes it's like the,
1660
00:51:17,357 -->
00:51:19,097
the human feedback is the hint at like
1661
00:51:19,097 -->
00:51:22,487
a, a vein of gold, defer some, um,
1662
00:51:22,727 -->
00:51:25,187
infrequent but important failure case.
1663
00:51:25,247 -->
00:51:28,637
Uh, you know, and so, um, you know,
1664
00:51:28,637 -->
00:51:29,747
there's a number of techniques.
1665
00:51:29,747 -->
00:51:30,797
I don't know that all of them
1666
00:51:30,797 -->
00:51:31,997
we are fielding today, but in
1667
00:51:31,997 -->
00:51:33,107
our near term roadmap, we should
1668
00:51:33,107 -->
00:51:34,517
have coverage for, you know.
1669
00:51:34,937 -->
00:51:37,067
Looking at, uh, classifying failure
1670
00:51:37,067 -->
00:51:38,867
mode types by doing some sort
1671
00:51:38,867 -->
00:51:40,487
of, um, you know, unstructured
1672
00:51:40,487 -->
00:51:44,327
clustering using embeddings of, um,
1673
00:51:44,447 -->
00:51:48,707
you know, SubT traces and exploring.
1674
00:51:49,382 -->
00:51:52,412
Small but common kind of error classes
1675
00:51:52,502 -->
00:51:55,172
or, um, you know, or, or surfacing
1676
00:51:55,172 -->
00:51:57,272
them with, with evaluator models that,
1677
00:51:57,692 -->
00:52:01,922
um, you know, are tasked with failure
1678
00:52:01,922 -->
00:52:04,172
modes in an, an offline capacity.
1679
00:52:04,412 -->
00:52:05,252
That was a little ranty.
1680
00:52:05,282 -->
00:52:06,452
I, I don't know if I had
1681
00:52:06,572 -->
00:52:07,502
a totally clear answer.
1682
00:52:07,502 -->
00:52:08,612
There's kind of a lot of stuff there.
1683
00:52:08,972 -->
00:52:09,932
Jeff Dalton: Couple quick thoughts there.
1684
00:52:10,147 -->
00:52:11,642
I, I think the fir some of the, the
1685
00:52:11,642 -->
00:52:13,142
first one there is again, having
1686
00:52:13,142 -->
00:52:14,372
some of that structure so you can
1687
00:52:14,372 -->
00:52:16,082
actually see those, see those elements
1688
00:52:16,082 -->
00:52:16,982
and see where, where, what the
1689
00:52:16,982 -->
00:52:18,632
traceability of that is, is fundamental.
1690
00:52:19,022 -->
00:52:20,042
Um, for something like safety
1691
00:52:20,042 -->
00:52:21,002
and guardrails, it's, it's
1692
00:52:21,002 -->
00:52:21,992
not, there's not one solution.
1693
00:52:21,992 -->
00:52:22,952
There's like, it's
1694
00:52:22,952 -->
00:52:24,002
actually defense in depth.
1695
00:52:24,002 -->
00:52:25,382
There's, it's a whole layer.
1696
00:52:25,502 -->
00:52:27,002
So there's foundation, model guard,
1697
00:52:27,362 -->
00:52:29,107
guardrails, then there's, there's.
1698
00:52:29,767 -->
00:52:31,357
Before we even get to input, there's,
1699
00:52:31,387 -->
00:52:33,157
there's like regexes and patterns and
1700
00:52:33,157 -->
00:52:34,267
things to make sure that the input
1701
00:52:34,267 -->
00:52:35,467
that's going doesn't have prompt
1702
00:52:35,467 -->
00:52:37,987
injection attacks and, and PII.
1703
00:52:37,987 -->
00:52:39,727
And so before that, even before
1704
00:52:39,727 -->
00:52:41,197
it even hits an LLM, we're met,
1705
00:52:41,197 -->
00:52:42,247
we're testing those things.
1706
00:52:42,667 -->
00:52:43,957
Then further on top of that we have
1707
00:52:43,957 -->
00:52:45,787
input guardrails that are maybe light
1708
00:52:45,817 -->
00:52:47,857
LLMs, and after that we have output,
1709
00:52:48,037 -->
00:52:49,417
um, guardrails as well that are
1710
00:52:49,417 -->
00:52:51,337
additional kind of LLM driven with.
1711
00:52:51,427 -->
00:52:52,117
And all of those have different
1712
00:52:52,117 -->
00:52:53,917
operating points of precision and recall.
1713
00:52:54,692 -->
00:52:56,612
But, and you're, you're really putting
1714
00:52:56,612 -->
00:52:58,412
that safety system together, overall
1715
00:52:58,412 -->
00:52:59,732
end to end and making sure that you're
1716
00:52:59,732 -->
00:53:01,622
measuring, um, each of the component
1717
00:53:01,622 -->
00:53:03,722
wise steps on, uh, as input and output.
1718
00:53:03,812 -->
00:53:06,842
But also is this, does that overall
1719
00:53:06,842 -->
00:53:08,942
aggregate behavior kind of lead to and
1720
00:53:08,942 -->
00:53:10,802
align with like what the target operating
1721
00:53:10,802 -->
00:53:12,002
point is that you're looking for?
1722
00:53:12,452 -->
00:53:13,862
And sometimes it very different.
1723
00:53:14,012 -->
00:53:15,722
A guardrail for something like,
1724
00:53:15,752 -->
00:53:17,102
um, for legal is gonna be very
1725
00:53:17,102 -->
00:53:18,632
different from one that says just.
1726
00:53:19,202 -->
00:53:20,672
Um, that has legal implications
1727
00:53:20,672 -->
00:53:21,632
gonna be very different from one that
1728
00:53:21,632 -->
00:53:24,422
has, um, kind of just, um, maybe a
1729
00:53:24,422 -->
00:53:26,462
date, a policy, a policy violation.
1730
00:53:26,717 -->
00:53:27,917
Josh Rubin: I think that's very well put.
1731
00:53:27,947 -->
00:53:29,447
I think that, you know, the fact that
1732
00:53:29,447 -->
00:53:31,397
we have magic now in language models
1733
00:53:31,637 -->
00:53:33,407
doesn't absolve us from system level
1734
00:53:33,407 -->
00:53:34,912
thinking about the components and.
1735
00:53:35,702 -->
00:53:37,262
You know, whatever the equivalent now
1736
00:53:37,262 -->
00:53:40,142
of, you know, unit tests, like, sort
1737
00:53:40,142 -->
00:53:42,152
of, uh, you know, instrumenting each
1738
00:53:42,152 -->
00:53:45,182
of the known steps with, um, ways of
1739
00:53:45,182 -->
00:53:46,562
measuring the things that can go wrong.
1740
00:53:46,562 -->
00:53:47,947
I mean, this goes back to our, uh.
1741
00:53:48,482 -->
00:53:50,552
You know, previous topic of, you know,
1742
00:53:50,672 -->
00:53:52,502
out of the box versus customizability,
1743
00:53:52,562 -->
00:53:54,602
I think, you know, something
1744
00:53:54,602 -->
00:53:57,152
that is, uh, has risk associated
1745
00:53:57,152 -->
00:53:59,432
with it is a high value use case.
1746
00:53:59,882 -->
00:54:01,562
I think you almost necessarily have
1747
00:54:01,562 -->
00:54:04,112
to think about the details of the
1748
00:54:04,112 -->
00:54:06,482
components, um, and build, build a
1749
00:54:06,482 -->
00:54:08,582
strategy for capturing and mitigating
1750
00:54:08,672 -->
00:54:10,322
the kinds of things that might be
1751
00:54:10,322 -->
00:54:11,817
expected to go wrong in, in that piece.
1752
00:54:11,887 -->
00:54:13,322
Jeff Dalton: Good agent systems
1753
00:54:13,472 -->
00:54:14,312
have, follow a lot of the same
1754
00:54:14,312 -->
00:54:15,992
principles as good software.
1755
00:54:16,502 -->
00:54:18,782
And so, um, so if you're building
1756
00:54:18,782 -->
00:54:20,072
a good, a good agent system, then
1757
00:54:20,072 -->
00:54:21,452
you, you also need to know, you need
1758
00:54:21,452 -->
00:54:22,352
to know some of the best principles
1759
00:54:22,352 -->
00:54:23,552
of how, what, what good software
1760
00:54:23,552 -->
00:54:25,292
engineering is and best practices.
1761
00:54:25,362 -->
00:54:26,862
I spent a lot of time at IBM
1762
00:54:26,862 -->
00:54:28,092
working on logging and tracing
1763
00:54:28,632 -->
00:54:29,862
in an enterprise application.
1764
00:54:29,922 -->
00:54:31,092
And if you want to debug some
1765
00:54:31,092 -->
00:54:32,712
of the really complex web sphere
1766
00:54:32,712 -->
00:54:34,032
application, you better have, you
1767
00:54:34,032 -->
00:54:35,652
better have really detailed traces,
1768
00:54:35,982 -->
00:54:37,692
um, so that, that you can, so that
1769
00:54:37,692 -->
00:54:39,042
you can understand that behavior.
1770
00:54:39,222 -->
00:54:39,852
And we need the same
1771
00:54:39,852 -->
00:54:41,112
thing in agent systems.
1772
00:54:42,872 -->
00:54:46,172
Josh Rubin: Um, so I, I just
1773
00:54:46,172 -->
00:54:47,162
glancing across the other
1774
00:54:47,162 -->
00:54:48,272
questions that we have here.
1775
00:54:48,272 -->
00:54:50,132
I think we've mostly addressed,
1776
00:54:50,222 -->
00:54:51,932
uh, the topics that were raised.
1777
00:54:52,382 -->
00:54:54,662
Um, how should we incorporate measurement
1778
00:54:54,662 -->
00:54:56,702
or evaluation tools and customized agents?
1779
00:54:56,702 -->
00:54:58,052
I think that's what we're talking about.
1780
00:54:58,652 -->
00:55:00,362
Um, would we need custom measurements?
1781
00:55:00,362 -->
00:55:03,822
I think sometimes depends on risk,
1782
00:55:03,852 -->
00:55:04,457
depends on what you're trying to do
1783
00:55:04,457 -->
00:55:04,727
Jeff Dalton: Yes.
1784
00:55:05,387 -->
00:55:06,167
Need custom measurements.
1785
00:55:06,197 -->
00:55:08,087
And for something like coaching, right?
1786
00:55:08,087 -->
00:55:09,857
We, we can, we can, and you know, we
1787
00:55:09,857 -->
00:55:11,477
can measure process quality, outcome,
1788
00:55:11,477 -->
00:55:12,677
quality, and whether it's, you know,
1789
00:55:12,797 -->
00:55:14,657
if that, if a session was successful.
1790
00:55:14,777 -->
00:55:15,857
But going back to like the research
1791
00:55:15,857 -->
00:55:18,077
question, how do we measure a journey?
1792
00:55:18,107 -->
00:55:19,007
How do we measures, measure
1793
00:55:19,007 -->
00:55:20,087
a user's journey over time?
1794
00:55:20,087 -->
00:55:21,977
Is there, is, is it, is
1795
00:55:21,977 -->
00:55:22,937
there a coherent plan?
1796
00:55:23,027 -->
00:55:24,317
Is user making progress and there
1797
00:55:24,377 -->
00:55:25,727
is trust being strengthened?
1798
00:55:26,147 -->
00:55:27,497
Um, all of those different elements of.
1799
00:55:28,517 -->
00:55:30,137
Part of what keeps me, uh, keeps me
1800
00:55:30,137 -->
00:55:31,607
here like working on how do you measure
1801
00:55:31,607 -->
00:55:34,037
a longitudinally, uh, kind of change.
1802
00:55:34,097 -->
00:55:36,107
Um, and, and that's so that, that is
1803
00:55:36,107 -->
00:55:37,877
a form of very, very custom, highly
1804
00:55:37,877 -->
00:55:40,097
specific, and not just conversation
1805
00:55:40,097 -->
00:55:41,897
level metric, but a system level
1806
00:55:41,897 -->
00:55:43,217
metric of what we're trying to
1807
00:55:43,217 -->
00:55:45,227
accomplish in the user's world.
1808
00:55:46,577 -->
00:55:47,087
Josh Rubin: That's great.
1809
00:55:47,777 -->
00:55:48,167
great.
1810
00:55:48,767 -->
00:55:49,937
I think we wrap here.
1811
00:55:50,027 -->
00:55:51,017
This has been a really, really
1812
00:55:51,017 -->
00:55:52,187
stimulating conversation.
1813
00:55:52,277 -->
00:55:53,927
Uh, yeah.
1814
00:55:53,987 -->
00:55:54,857
Uh, a lot of fun.
1815
00:55:54,857 -->
00:55:56,417
Thank you very much, Jeff, for sitting
1816
00:55:56,417 -->
00:55:58,457
and talking to me about, about this stuff.
1817
00:55:58,607 -->
00:55:59,177
Jeff Dalton: been a pleasure
1818
00:55:59,237 -->
00:56:00,342
and it's been great fun.
1819
00:56:00,747 -->
00:56:01,317
Josh Rubin: You've heard about
1820
00:56:01,317 -->
00:56:03,027
Valence and Nadia, check them out.
1821
00:56:03,267 -->
00:56:05,007
Um, and we're Fiddler AI.
1822
00:56:05,067 -->
00:56:06,567
Uh, you know, we'd love to work with
1823
00:56:06,567 -->
00:56:08,637
you on getting your instrumentation just
1824
00:56:08,637 -->
00:56:12,177
right on whatever collection of complex
1825
00:56:12,207 -->
00:56:14,517
agentic and predictive workflows you have.
1826
00:56:14,877 -->
00:56:16,827
Um, so thanks everybody for tuning in
1827
00:56:16,857 -->
00:56:19,527
and, uh, you know, have a great day.
1828
00:56:20,497 -->
00:56:20,657
Jeff Dalton: Have a great day.
1829
00:56:20,657 -->
00:56:21,027
Thank you very much.
