S1E5 | Are Software Factories the Future of AI Development? | Juri Strumpflohner

WEBVTT
1
00:00:01.439 --> 00:00:05.459
Welcome to the AI DevCraft podcast, formerly the Angular Plus show.
2
00:00:06.040 --> 00:00:09.769
We're talking AI, software development, the tools changing how we build,
3
00:00:10.060 --> 00:00:12.910
and the skills developers need next. And if you want
4
00:00:12.970 --> 00:00:15.869
even more, join us at AI DevCraft in Las Vegas,
5
00:00:16.010 --> 00:00:17.710
October 27th through the 28th of 2026. Now let's get
6
00:00:17.730 --> 00:00:18.070
into it.
7
00:00:25.179 --> 00:00:29.620
Hello, and welcome to another episode of the AI DevCraft podcast.
8
00:00:29.820 --> 00:00:32.289
I am one of your hosts. I'm Laura Newsome. With
9
00:00:32.329 --> 00:00:36.070
me today, I have Mike, Mike Hardington. How's it going today?
10
00:00:36.810 --> 00:00:37.929
Oh, it's going good.
11
00:00:38.149 --> 00:00:42.229
I am low on coffee, but I am well caffeinated.
12
00:00:42.810 --> 00:00:46.340
Low on coffee, high on anxiety. It's like almost like caffeine.
13
00:00:46.380 --> 00:00:46.799
Except it hurts.
14
00:00:46.820 --> 00:00:49.740
I was going to say that, but you know, sure,
15
00:00:49.799 --> 00:00:50.579
we'll go with that.
16
00:00:51.880 --> 00:00:53.719
Jay is with me as well. Jay, how's it going?
17
00:00:53.740 --> 00:00:54.439
Good.
18
00:00:55.020 --> 00:00:56.320
My coffee is just out.
19
00:00:56.759 --> 00:00:56.960
Yeah.
20
00:00:57.000 --> 00:00:59.539
You know, like I have this big old cup. Mine's
21
00:00:59.579 --> 00:01:02.280
not coffee. It's tea because my grandmother's British. So I'm
22
00:01:02.329 --> 00:01:04.359
legally required to drink.
23
00:01:04.790 --> 00:01:05.950
Yeah. Legally required.
24
00:01:06.010 --> 00:01:06.209
Yeah.
25
00:01:06.430 --> 00:01:07.450
They'll come for you if you don't.
26
00:01:07.849 --> 00:01:10.769
It has nothing to do with my digestive system or
27
00:01:10.810 --> 00:01:17.510
my tolerance. Yeah. So we have a great guest today.
28
00:01:17.530 --> 00:01:21.769
Our guest is Yuri from NX. Not only from NX,
29
00:01:21.829 --> 00:01:25.719
the Senior Director of Developer Experience at NX. Yuri, would
30
00:01:25.739 --> 00:01:27.819
you like to introduce yourself to our listeners?
31
00:01:28.239 --> 00:01:32.359
Yeah, I mean, you already said everything. So I'm Yuri.
32
00:01:32.659 --> 00:01:36.409
I currently work at NX, do martial arts in my
33
00:01:36.439 --> 00:01:38.670
free time. So those are the main passions. I have
34
00:01:39.049 --> 00:01:41.870
three kids at home, so lots of stuff going on there.
35
00:01:42.790 --> 00:01:47.129
Three small kids. But yeah, and recently I'm trying to
36
00:01:47.750 --> 00:01:53.500
navigate our new, weird, strange world of AI-driven development, or
37
00:01:53.519 --> 00:01:56.719
however you want to call that. and basically trying to
38
00:01:56.780 --> 00:01:59.120
write that thing and figuring out what's next.
39
00:01:59.650 --> 00:02:04.290
Yeah, absolutely. So you wrote in May that code generation
40
00:02:04.370 --> 00:02:10.650
isn't necessarily the bottleneck anymore, that verification is. So tell
41
00:02:10.689 --> 00:02:11.849
me a little more about that thought.
42
00:02:12.289 --> 00:02:16.849
Yeah, I mean, whoever already works, and pretty much like
43
00:02:16.909 --> 00:02:21.110
everyone now, right, works with AI assistants very closely, or
44
00:02:21.370 --> 00:02:26.139
let's say AI agents at this point. like data generate
45
00:02:26.319 --> 00:02:28.960
tons and tons and tons of code. So it's no
46
00:02:29.000 --> 00:02:31.439
more the problem of like, how long do I need
47
00:02:31.479 --> 00:02:35.180
to actually type out stuff in my editor after I've
48
00:02:35.199 --> 00:02:38.469
decided what I want to build? Because like that is
49
00:02:38.590 --> 00:02:41.289
now given, right? So AI agents are happily going and
50
00:02:41.389 --> 00:02:43.629
creating tons and tons of things. We just discussed how
51
00:02:43.930 --> 00:02:47.289
verbose they are and like how we need to tame them.
52
00:02:47.550 --> 00:02:52.129
So the whole taming is actually the bigger issue, right?
53
00:02:52.599 --> 00:02:55.139
So how do we make sure they just create what
54
00:02:55.180 --> 00:02:57.759
they actually need to create based on what we decided?
55
00:02:58.300 --> 00:03:01.939
How do we verify the whole discussion about reviewing? How
56
00:03:01.960 --> 00:03:04.460
do we even review all the tons and tons of
57
00:03:04.520 --> 00:03:07.759
PRs that are now being created? Where is even the
58
00:03:07.819 --> 00:03:10.189
place of the human of reviewing? What do we even review?
59
00:03:10.250 --> 00:03:13.550
Is it the same as before? It totally changed, right?
60
00:03:14.250 --> 00:03:15.810
That was a bit the thing that I mentioned initially
61
00:03:15.889 --> 00:03:18.889
in the intro, right? Like trying to navigate that new world.
62
00:03:18.909 --> 00:03:23.240
It's like, like what should I actually pay attention to nowadays, right?
63
00:03:23.280 --> 00:03:26.240
Because before I wrote the code, right? And it was
64
00:03:26.319 --> 00:03:28.979
smaller amount of pieces. You were very deep into it.
65
00:03:29.280 --> 00:03:32.159
You thought through, because like you thought through multiple times even, right?
66
00:03:32.180 --> 00:03:34.560
Like you start like thinking about the planning, et cetera,
67
00:03:34.900 --> 00:03:37.030
how you built out the architecture, then you go deep
68
00:03:37.060 --> 00:03:39.650
into the code phase and actually implement it. So you're
69
00:03:39.689 --> 00:03:42.689
deep into the weeds basically. And even as a reviewer
70
00:03:42.710 --> 00:03:45.050
on the other end, the amount of things you got
71
00:03:45.409 --> 00:03:48.030
in the PR are actually pretty smallish.
72
00:03:48.539 --> 00:03:48.699
Right.
73
00:03:48.759 --> 00:03:51.300
So they're kind of like more easy to, to grasp.
74
00:03:51.680 --> 00:03:54.740
And plus you were probably involved in multiple places throughout
75
00:03:54.780 --> 00:03:59.069
that whole creation phase of like coworker coming over or
76
00:03:59.330 --> 00:04:01.469
jumping on a big, quick call. If you're like a
77
00:04:01.650 --> 00:04:04.490
remote company of like having you pair on something or
78
00:04:04.530 --> 00:04:07.430
discuss an architectural piece, a lot of those things I
79
00:04:07.449 --> 00:04:08.710
feel are completely gone now.
80
00:04:09.409 --> 00:04:09.569
Right.
81
00:04:09.590 --> 00:04:12.129
So now it's basically the agent kind of shooting out
82
00:04:12.189 --> 00:04:15.250
all that code for you, right. Based on some script
83
00:04:15.310 --> 00:04:17.790
or some plan or whatever you created and I can
84
00:04:18.399 --> 00:04:21.860
gave it to get going. And so there's this distance there,
85
00:04:21.879 --> 00:04:24.139
but also the distance between the workers in this case, like, right?
86
00:04:24.279 --> 00:04:26.439
Like I probably less often, or I kind of can
87
00:04:26.480 --> 00:04:29.199
see it even, I less often reach out to someone, hey,
88
00:04:29.220 --> 00:04:30.819
can we pair on this? Like, this is a hard thing.
89
00:04:31.139 --> 00:04:33.529
You mostly, like a lot of us just iterate with
90
00:04:33.589 --> 00:04:37.209
our agent, right? Like going in loops of like, try better,
91
00:04:37.269 --> 00:04:40.029
try harder. Like, this doesn't look right, right? So a
92
00:04:40.050 --> 00:04:42.649
lot of that discussion happens there. And so that's why
93
00:04:42.730 --> 00:04:44.949
back then I've wrote that whole co-generation is no more
94
00:04:45.459 --> 00:04:48.459
that big of a deal i'm usually very cautious there
95
00:04:48.500 --> 00:04:50.579
as well it's like a lot of those frontier models
96
00:04:50.639 --> 00:04:54.230
obviously say oh code is soft right i don't know
97
00:04:54.550 --> 00:04:56.610
i'm personally not there yet but i kind of can
98
00:04:56.709 --> 00:04:59.689
see right if we continue with this speed that we've
99
00:04:59.709 --> 00:05:02.439
been going since just like last year right or not
100
00:05:02.480 --> 00:05:05.540
even last year just like january february it it heads
101
00:05:05.560 --> 00:05:07.959
in that direction right that at some point we might
102
00:05:08.040 --> 00:05:10.560
not be able to actually touching a lot of code
103
00:05:10.660 --> 00:05:11.680
by hand right.
104
00:05:12.079 --> 00:05:13.939
It's uh It's very funny.
105
00:05:14.040 --> 00:05:19.300
We went from being very cautious about how much context
106
00:05:19.339 --> 00:05:23.060
we gave an agent, and now our agents are just overloading.
107
00:05:22.740 --> 00:05:24.079
Us with additional context.
108
00:05:24.819 --> 00:05:27.060
And so we need to figure out, okay, how can
109
00:05:27.100 --> 00:05:32.370
we train these agents to be context-aware for the humans
110
00:05:32.430 --> 00:05:35.930
in the loop? And then, lo and behold, that's how
111
00:05:35.970 --> 00:05:38.470
we get AGI.
112
00:05:38.550 --> 00:05:41.910
We need our own vector stores so that we can do...
113
00:05:42.279 --> 00:05:45.579
cartesian lookups on our memories is what we need oh
114
00:05:45.600 --> 00:05:46.860
my god.
115
00:05:46.740 --> 00:05:50.360
I have not heard that yeah you're.
116
00:05:49.779 --> 00:05:53.180
Already gonna going out in the direction of automating yourself
117
00:05:53.639 --> 00:05:58.610
let me just vector index my thoughts so i don't
118
00:05:58.649 --> 00:06:01.910
have to yeah but yeah it very much goes in
119
00:06:01.949 --> 00:06:04.970
that direction of yeah how can we how can we
120
00:06:05.009 --> 00:06:06.889
control these beasts basically right.
121
00:06:07.259 --> 00:06:10.160
Yeah, so back in May, you gave a webinar, an
122
00:06:10.240 --> 00:06:14.220
NX webinar called Autonomous Software Factories, Are We There Yet?
123
00:06:15.699 --> 00:06:20.709
So let's start from the foundation. So what exactly do
124
00:06:20.730 --> 00:06:24.129
you mean by a software factory? Tell me a little
125
00:06:24.149 --> 00:06:24.589
more about that.
126
00:06:25.389 --> 00:06:25.649
Yeah.
127
00:06:26.649 --> 00:06:29.689
I mean, I think like I've been doing a lot
128
00:06:29.730 --> 00:06:33.490
of research, like digging around online, because from one end,
129
00:06:33.550 --> 00:06:36.060
this topic is highly interesting to me. like trying to
130
00:06:36.120 --> 00:06:38.110
figure out like, how can you automate away some of
131
00:06:38.149 --> 00:06:41.790
this stuff? And with the main thought of, if you
132
00:06:41.810 --> 00:06:44.670
think about what you work during the day, there's like
133
00:06:44.910 --> 00:06:47.769
those tasks that need your attention. There's a lot of
134
00:06:47.810 --> 00:06:50.870
tasks that are just chores, right? That is not even
135
00:06:50.889 --> 00:06:53.449
that interesting, to be honest, right? And so that was
136
00:06:53.490 --> 00:06:55.829
always kind of like, how can I automate that thing
137
00:06:55.870 --> 00:06:57.750
with like scripts? Like I had a lot of scripts
138
00:06:57.829 --> 00:06:59.779
on my computer trying to automate some of the stuff
139
00:06:59.800 --> 00:07:01.660
even before AI right so that's kind of like where
140
00:07:01.680 --> 00:07:04.920
I'm leaning in and so when you research software factories
141
00:07:05.120 --> 00:07:09.529
you see a ton of different descriptions right like because
142
00:07:09.569 --> 00:07:11.430
it's also the thing is also it can be the
143
00:07:11.529 --> 00:07:14.810
simplest thing ever like in terms of like an agentic
144
00:07:14.870 --> 00:07:16.930
environment it can be a skill right with a bunch
145
00:07:16.949 --> 00:07:19.689
of steps telling the agent what you should do and
146
00:07:19.709 --> 00:07:22.279
that's it right that's one file basically that is your
147
00:07:22.360 --> 00:07:27.620
software factory to like the full-blown cloud agent orchestration framework
148
00:07:27.660 --> 00:07:30.839
that kind of It's kind of even overloading just like
149
00:07:30.860 --> 00:07:33.399
from an architecture point of view. But at a very
150
00:07:33.439 --> 00:07:36.250
high level, what I see as a software factory is
151
00:07:36.410 --> 00:07:40.709
basically a workflow, right? There's some sort of trigger that
152
00:07:40.750 --> 00:07:43.329
can be automated, that can be on a schedule, that
153
00:07:43.350 --> 00:07:46.980
can be you. And then there's an agent working and
154
00:07:47.000 --> 00:07:49.300
then something gets back to you. That's it, right? So
155
00:07:49.720 --> 00:07:53.420
it can be the very simplest case, like every day
156
00:07:53.439 --> 00:07:58.490
in the morning, go scroll my GitHub issues, find the
157
00:07:58.550 --> 00:08:01.709
new ones that have been opened, run a first triage
158
00:08:01.769 --> 00:08:05.769
phase on it to understand, like, is this reproducible? Like,
159
00:08:06.910 --> 00:08:09.300
is it a duplicate of something? Are we already working
160
00:08:09.360 --> 00:08:11.139
on this? And then give me a report back so
161
00:08:11.180 --> 00:08:13.519
I can look at it, basically. Or if it is
162
00:08:13.639 --> 00:08:16.500
reproducible right away, it's a bug, I just go straight
163
00:08:16.560 --> 00:08:18.879
ahead and create a PR, and I review that PR, right?
164
00:08:19.459 --> 00:08:21.600
So that is what I see as a factory. So
165
00:08:21.899 --> 00:08:24.480
it kind of takes away some of the thoughts, like
166
00:08:24.500 --> 00:08:27.750
some of the work where you– probably at some point
167
00:08:27.769 --> 00:08:29.750
had like a calendar entry of saying like, okay, at
168
00:08:29.750 --> 00:08:31.560
9 a.m. the calendar entry goes up. Oh, let me
169
00:08:31.579 --> 00:08:34.159
do some GitHub triaging, right? And then you go through,
170
00:08:34.240 --> 00:08:36.919
read them and do all these things. And then you
171
00:08:36.940 --> 00:08:39.639
do the local reproduction and maybe you had some scripts
172
00:08:39.679 --> 00:08:42.759
for that, right? And so now the software factory does that.
173
00:08:42.789 --> 00:08:45.409
So that's the idea, right? So I just come back
174
00:08:45.490 --> 00:08:50.929
to a pre-triaged, sometimes better, sometimes worse kind of situation
175
00:08:50.990 --> 00:08:52.590
where I just look at those and then I take
176
00:08:52.629 --> 00:08:55.000
the decision of saying like, Oh, this looks worse or
177
00:08:55.019 --> 00:08:56.840
this looks bad or not. It's not even like what
178
00:08:56.860 --> 00:08:59.419
I wanted, et cetera. And that's where then the whole
179
00:08:59.840 --> 00:09:00.399
looping comes.
180
00:09:00.799 --> 00:09:03.659
Yeah. You're keeping a human in the loop, right? To
181
00:09:03.679 --> 00:09:05.440
actually make those kinds of decisions.
182
00:09:06.340 --> 00:09:06.559
Yeah.
183
00:09:06.759 --> 00:09:07.279
No, 100%.
184
00:09:07.279 --> 00:09:10.960
That's also something where like a software factory is not
185
00:09:11.080 --> 00:09:13.889
meant to how to say like i feed it the
186
00:09:14.009 --> 00:09:16.450
product i mean you could obviously define it like i
187
00:09:16.470 --> 00:09:19.179
feed the product idea and then i don't know hundreds
188
00:09:19.240 --> 00:09:21.799
agents go off and they build and then like a
189
00:09:21.840 --> 00:09:23.799
month later or not a week later or whatever i
190
00:09:24.220 --> 00:09:25.960
get the product and i'm like oh this is cool
191
00:09:25.980 --> 00:09:29.970
right so it's it's not that even though like there's
192
00:09:29.990 --> 00:09:32.610
some misconception even going on because like i think anthropic
193
00:09:32.730 --> 00:09:35.350
also occurs so there was at one point those that
194
00:09:35.389 --> 00:09:38.590
that phase where they're like oh we let bunch of
195
00:09:38.649 --> 00:09:41.399
agents go for a week and see what they do, right?
196
00:09:41.460 --> 00:09:43.960
Kind of thing, which that would probably go in that
197
00:09:44.000 --> 00:09:46.120
direction of like a factory working on its own and
198
00:09:46.159 --> 00:09:48.379
like starting from just a single product prompt.
199
00:09:49.000 --> 00:09:50.059
Or hacking HackingFace.
200
00:09:50.679 --> 00:09:51.519
Yeah, yeah, exactly.
201
00:09:51.740 --> 00:09:52.039
Exactly.
202
00:09:52.080 --> 00:09:53.830
There was, yeah.
203
00:09:53.889 --> 00:09:56.690
Or hacking certain other companies, third-party companies, yeah.
204
00:09:57.389 --> 00:09:59.970
There are a lot of companies out there that promise products.
205
00:10:00.120 --> 00:10:02.600
like that too, where they're promising that like, all you
206
00:10:02.620 --> 00:10:04.059
got to do is give us a thing and we'll,
207
00:10:04.519 --> 00:10:06.639
it's going to be working, a full stack working software
208
00:10:06.700 --> 00:10:07.190
at the end.
209
00:10:07.259 --> 00:10:11.080
So, yeah. But I think like that phase, I've seen
210
00:10:11.100 --> 00:10:15.330
a bunch of times in my career, like that, like
211
00:10:15.419 --> 00:10:18.250
you model driven, like model in the sense of like,
212
00:10:18.649 --> 00:10:23.220
You draw things, driven development, you don't need developers anymore, blah, blah, right?
213
00:10:23.480 --> 00:10:23.620
Yeah.
214
00:10:23.820 --> 00:10:27.220
Now, I think we're gotten a bit closer to that
215
00:10:27.279 --> 00:10:29.539
in a sense of like, I mean, we just said before,
216
00:10:29.620 --> 00:10:32.440
if you identify with just writing code and that's your
217
00:10:32.519 --> 00:10:36.720
developer identity, well, yeah, some of that seems to start
218
00:10:36.779 --> 00:10:39.299
going away, right? But I think there's much more to it, right?
219
00:10:39.340 --> 00:10:42.460
And so that's also why I see a software factory
220
00:10:42.720 --> 00:10:45.860
absolutely not as like removes the human from the loop,
221
00:10:46.220 --> 00:10:49.379
but just changes what you need to look at right
222
00:10:50.110 --> 00:10:52.629
i don't want to look at as i mentioned kind
223
00:10:52.649 --> 00:10:56.690
of before that things that are just chores or things
224
00:10:56.710 --> 00:10:58.309
that just need to be done right which is part
225
00:10:58.330 --> 00:11:00.850
of the product building those i might be able to
226
00:11:00.889 --> 00:11:03.509
just automate right which i'm also might be also happy
227
00:11:03.549 --> 00:11:05.960
about it um and so i can kind of point
228
00:11:06.000 --> 00:11:08.639
my leverage a bit more to a more decision making
229
00:11:08.700 --> 00:11:11.840
thing where like or bringing in um i really like
230
00:11:11.899 --> 00:11:14.200
what what adios money had a couple good articles on
231
00:11:14.240 --> 00:11:17.809
this he mentions, he brought up with like taste, right?
232
00:11:17.909 --> 00:11:20.049
I think that kind of nails it a bit because
233
00:11:20.389 --> 00:11:24.139
at least right now, there's not really tastes in AI models.
234
00:11:24.159 --> 00:11:26.320
They try to follow some just guidelines that they got
235
00:11:26.340 --> 00:11:29.080
from the training or pre-training, right? But it's still a
236
00:11:29.120 --> 00:11:30.620
human that kind of like needs to go in and
237
00:11:30.639 --> 00:11:33.320
say like, okay, we built this or we will go
238
00:11:33.340 --> 00:11:35.879
in that direction. Like that makes these decisions of like
239
00:11:35.960 --> 00:11:39.330
what to build, the how actually we can more and
240
00:11:39.370 --> 00:11:42.750
more remove ourselves from, right? The details and the degree
241
00:11:42.789 --> 00:11:45.759
of like how things are wired together. Those things actually
242
00:11:45.799 --> 00:11:48.799
start looking more and more that can be taken by
243
00:11:48.820 --> 00:11:49.200
an agent.
244
00:11:49.600 --> 00:11:49.860
Yeah.
245
00:11:49.879 --> 00:11:55.049
So, okay, you talked about four primitives that should exist
246
00:11:55.429 --> 00:12:03.029
in a software factory. So context, isolation, feedback loops, recovery.
247
00:12:04.409 --> 00:12:06.409
When you look at what teams are building that they're
248
00:12:06.429 --> 00:12:09.830
calling factories, what do teams tend to miss in that process?
249
00:12:09.850 --> 00:12:10.230
Yeah.
250
00:12:13.340 --> 00:12:15.549
It really depends where they are in the whole kind
251
00:12:16.090 --> 00:12:18.669
of like, I was about to say almost like. the
252
00:12:18.710 --> 00:12:21.610
factory maturity model or whatever. I don't even know if
253
00:12:21.669 --> 00:12:24.730
that exists. I don't know why that came in like software.
254
00:12:25.110 --> 00:12:27.720
Wasn't there a term like software capability maturity models or
255
00:12:27.740 --> 00:12:28.279
something like that?
256
00:12:28.460 --> 00:12:31.460
It was like the STLC. Yeah, I don't know. Something
257
00:12:31.500 --> 00:12:31.860
like that.
258
00:12:31.879 --> 00:12:33.919
Yeah, something like that. I don't know where this came from.
259
00:12:33.940 --> 00:12:36.840
Somebody throws out a new term like every 10 minutes.
260
00:12:36.860 --> 00:12:38.080
Yeah.
261
00:12:39.070 --> 00:12:40.529
Wait, is this just a new one or do we
262
00:12:40.570 --> 00:12:41.970
all know this? Like I'm confused.
263
00:12:42.210 --> 00:12:45.190
Exactly. It really kind of depends where you are in
264
00:12:45.250 --> 00:12:47.090
that flow, right? Because like it can be as simple
265
00:12:47.129 --> 00:12:49.889
as I said before, like, You just have that skill
266
00:12:50.289 --> 00:12:52.269
and then you kick off an agent and it does
267
00:12:52.309 --> 00:12:54.769
some triaging, it connects to your GitHub issues, right? GitHub
268
00:12:54.789 --> 00:12:57.990
CLI or something, triages them and that's it, right? It
269
00:12:58.289 --> 00:13:00.590
stores them on some file system that you then review,
270
00:13:01.129 --> 00:13:03.809
et cetera. So that's a pretty basic thing. I think
271
00:13:03.850 --> 00:13:05.860
the main parts or the biggest one that is being
272
00:13:05.909 --> 00:13:10.259
missed is the part when you, or basically let's say
273
00:13:10.779 --> 00:13:15.500
where you see a different quality outcome is the starting point,
274
00:13:15.519 --> 00:13:17.299
which is the context engineering part.
275
00:13:18.000 --> 00:13:18.139
So.
276
00:13:18.730 --> 00:13:20.230
Because if you just give an agent a prompt and
277
00:13:20.250 --> 00:13:22.950
it doesn't have any other context from it, then, well,
278
00:13:22.970 --> 00:13:25.210
that's it, right? It will just try to do its
279
00:13:25.250 --> 00:13:27.240
best and it will try all sorts of weird things.
280
00:13:27.620 --> 00:13:29.779
And so then it can go completely sideways. So it's
281
00:13:29.820 --> 00:13:33.740
important to, like, when you start a thing, figure out
282
00:13:33.860 --> 00:13:36.320
what does the agent have context at, right? Does it
283
00:13:36.480 --> 00:13:39.639
know what in that specific area of the software product
284
00:13:39.679 --> 00:13:42.070
has been done before? Is the only access it has
285
00:13:42.110 --> 00:13:44.269
the code level access, right? Does it have access to,
286
00:13:44.769 --> 00:13:49.779
I know, previous agent, histories agent logs maybe there's something
287
00:13:49.850 --> 00:13:52.070
about like a lot of people talk about memory or
288
00:13:52.110 --> 00:13:55.490
some central facts of like here's what we do here
289
00:13:55.529 --> 00:13:57.889
is how our software usually looks like here is like
290
00:13:57.909 --> 00:14:01.629
the guidelines here's like so high level constraints the next
291
00:14:01.669 --> 00:14:04.500
thing is usually then the actual implementation phase so what
292
00:14:04.539 --> 00:14:06.740
does your code base look like i talked to some
293
00:14:06.779 --> 00:14:09.960
people recently where they're like oh yeah we we struggle
294
00:14:10.019 --> 00:14:12.659
really hard in like our existing code base because we
295
00:14:12.710 --> 00:14:16.220
have patterns from angular one of the first versions to
296
00:14:16.320 --> 00:14:19.919
signals whatnot if i let my agent lose it it
297
00:14:20.019 --> 00:14:23.049
just doesn't understand what's going on right and i often
298
00:14:23.090 --> 00:14:26.309
make that analogy of uh like you can have the
299
00:14:26.370 --> 00:14:29.009
best senior engineer and expert and throw it into a
300
00:14:29.049 --> 00:14:31.570
code base if your code base is super messy messy
301
00:14:31.909 --> 00:14:34.110
it will be able to start cleaning it up but
302
00:14:34.470 --> 00:14:37.830
the person won't be productive from the get-go right Right.
303
00:14:37.929 --> 00:14:40.850
Versus if you have a kind of like well-organized code base,
304
00:14:40.870 --> 00:14:42.269
like we all know what it is, right? We all
305
00:14:42.289 --> 00:14:44.230
worked in these types of like code bases and good
306
00:14:44.269 --> 00:14:47.129
and bad ones. And like the productivity level is just
307
00:14:47.169 --> 00:14:51.289
like completely different, right? And agents are not that different
308
00:14:51.350 --> 00:14:54.330
in that case, in that sense, right? As I said,
309
00:14:54.370 --> 00:14:56.590
like if it identifies, if it needs to create a
310
00:14:56.629 --> 00:14:58.889
new form or some new IPs or lists, and it
311
00:14:58.929 --> 00:15:00.970
has like five different patterns of how it connects to
312
00:15:00.990 --> 00:15:04.750
the data stream from the backend, it's going to guess, right?
313
00:15:04.789 --> 00:15:07.370
Unless you have some kind of like constraints in place.
314
00:15:07.409 --> 00:15:10.769
And that's where that implementation phase part comes from, where
315
00:15:11.330 --> 00:15:13.529
you kind of need to figure out how is my
316
00:15:13.740 --> 00:15:17.120
code base structure, like, is it modularized properly? So the
317
00:15:17.159 --> 00:15:19.419
agent has a high level understanding of where the regions are.
318
00:15:20.080 --> 00:15:21.919
Do I have constraints in place that if it runs
319
00:15:21.960 --> 00:15:24.539
the wrong direction, does it get errors, right? And these
320
00:15:24.559 --> 00:15:27.360
can be simple things like a lint rule that keeps failing, right?
321
00:15:27.860 --> 00:15:31.519
If it touches parts it shouldn't touch, right? Good test coverage,
322
00:15:31.879 --> 00:15:37.779
which is also an interesting topic now with agents. test coverage, right?
323
00:15:37.799 --> 00:15:40.620
That false green signal that it's not as green as
324
00:15:40.659 --> 00:15:46.110
it looks like. So like that, those things help drastically
325
00:15:46.269 --> 00:15:48.110
improve the quality that they get out of it, right?
326
00:15:48.129 --> 00:15:50.289
And that's often why you hear some people saying like, oh,
327
00:15:50.330 --> 00:15:54.669
my code is completely like, forget it. Like we're not there. Absolutely.
328
00:15:54.690 --> 00:15:56.549
And some are like, oh, this is fantastic. Like I
329
00:15:56.590 --> 00:16:02.269
had like a hundred PRs just today. I question very
330
00:16:02.320 --> 00:16:05.730
much those kind of statements, but like still that can
331
00:16:05.769 --> 00:16:08.429
make a huge difference. And the last part to close
332
00:16:08.450 --> 00:16:11.610
the loop is usually the phase after, which I see
333
00:16:11.690 --> 00:16:14.789
actually not a lot of people do at all, which
334
00:16:14.830 --> 00:16:16.629
is even if you just interact with your agent, right?
335
00:16:16.649 --> 00:16:19.570
Like do something like a retro or something with your agent.
336
00:16:19.610 --> 00:16:22.289
You go in a coding session, even if it's interactive, right?
337
00:16:22.330 --> 00:16:23.990
Not automated. You just look at it and you go
338
00:16:24.029 --> 00:16:26.509
back and forth. At the end, let it go through
339
00:16:26.529 --> 00:16:29.110
its logs, right? Like where does it go sideways, right?
340
00:16:29.230 --> 00:16:32.629
What went wrong? And then can I just backfill some
341
00:16:32.669 --> 00:16:36.740
of that into my existing skills, some constraints, et cetera.
342
00:16:37.159 --> 00:16:39.419
You would be surprised how much information you get out
343
00:16:39.440 --> 00:16:42.200
of that. Like I tried to do it like not
344
00:16:42.240 --> 00:16:44.519
on like after every run because you could just keep forgetting,
345
00:16:44.960 --> 00:16:47.370
but just like twice a week or so, like you
346
00:16:47.389 --> 00:16:49.909
just let it pass through the logs. Like where are
347
00:16:49.970 --> 00:16:51.509
things that you keep failing and like give you just
348
00:16:51.549 --> 00:16:53.779
like indicators of patterns that you should look for, right?
349
00:16:53.799 --> 00:16:57.059
Where it just backtracked or you actually stopped it and
350
00:16:57.139 --> 00:16:59.759
told it to do something completely different. It's actually pretty
351
00:16:59.779 --> 00:17:01.840
good of like extracting some of that and then kind
352
00:17:01.860 --> 00:17:05.829
of feeding that back into your system. And that again,
353
00:17:05.869 --> 00:17:08.730
like obviously over time can help then kind of like
354
00:17:08.769 --> 00:17:11.619
steer it more into where you want to go. And
355
00:17:11.680 --> 00:17:14.200
so it becomes more autonomous. And that's even a kind
356
00:17:14.220 --> 00:17:18.339
of a common prerequisite of having like a mini software factory,
357
00:17:18.380 --> 00:17:19.339
if you want to call it like that.
358
00:17:19.859 --> 00:17:24.619
Yeah, absolutely. I find myself saying a lot of like,
359
00:17:24.900 --> 00:17:27.180
when I get code, I'll get utility functions. And I'm like,
360
00:17:27.220 --> 00:17:29.250
this has to be somewhere in this code base already.
361
00:17:29.369 --> 00:17:31.970
Just having to go back through and be like, okay, yeah,
362
00:17:32.009 --> 00:17:34.089
I found this exact implementation like five times.
363
00:17:34.369 --> 00:17:35.329
Should I extract it out.
364
00:17:35.250 --> 00:17:35.589
Of the utility?
365
00:17:35.789 --> 00:17:37.529
Which is pretty gone pattern. Yeah.
366
00:17:38.069 --> 00:17:40.390
Well, it's like looking at any code base and you
367
00:17:40.410 --> 00:17:45.200
see the same tailwind class merge utility five different times.
368
00:17:45.319 --> 00:17:48.369
And it just becomes like a, How is this not
369
00:17:48.670 --> 00:17:52.150
standardized or how is this not maintained in like isolated
370
00:17:52.170 --> 00:17:56.029
into its own thing? That's just past experience and trauma.
371
00:17:57.690 --> 00:17:59.210
No, I understand. And I think like a lot of
372
00:17:59.250 --> 00:18:03.650
the actual engineering now is very much directed towards that, right?
373
00:18:04.069 --> 00:18:05.589
Like a lot of the, that's why I was saying
374
00:18:05.630 --> 00:18:08.750
also initially, like if you identify with just actually writing
375
00:18:08.789 --> 00:18:11.609
out classes and like connecting them and stuff like that,
376
00:18:11.769 --> 00:18:14.470
and that is very much like your main focus. And
377
00:18:14.490 --> 00:18:17.410
I know a lot of extremely good, like senior engineers
378
00:18:17.710 --> 00:18:20.359
that I've had a super hard time to adopt that
379
00:18:20.420 --> 00:18:22.720
whole agent pattern. Because if that is your.
380
00:18:22.599 --> 00:18:23.200
Home, right?
381
00:18:23.240 --> 00:18:26.019
Like you're not going to get anyone inside there, right?
382
00:18:26.039 --> 00:18:29.160
Like that's how I build stuff, how you construct stuff.
383
00:18:29.539 --> 00:18:31.880
So I'm not going to have some automated thing scrape
384
00:18:31.980 --> 00:18:33.799
over it, right? So I feel like a lot of
385
00:18:33.839 --> 00:18:36.420
that engineering practices are being kind of like a bit
386
00:18:36.720 --> 00:18:39.859
lifted upwards where you look more at integration points, architecture,
387
00:18:39.920 --> 00:18:43.329
and these kind of feedback mechanism where you're like, oh,
388
00:18:43.440 --> 00:18:46.259
it did go sideways. This is very much a tool
389
00:18:46.299 --> 00:18:49.299
I use now to create products. So if it goes sideways,
390
00:18:49.319 --> 00:18:51.500
I need to figure out how to fix that and
391
00:18:51.619 --> 00:18:54.029
ideally fix it in a way where my teammate can
392
00:18:54.049 --> 00:18:57.230
benefit from it as well. Because otherwise my agent starts
393
00:18:57.250 --> 00:18:59.470
behaving as I would want it to behave in that
394
00:18:59.490 --> 00:19:02.190
code base. But the agent of my other teammates just
395
00:19:02.269 --> 00:19:04.829
goes sideways and keeps going sideways. And those are a
396
00:19:04.829 --> 00:19:06.869
lot of the issues that I also hear when talking
397
00:19:06.890 --> 00:19:08.309
to people recently.
398
00:19:09.470 --> 00:19:10.250
So bringing this back.
399
00:19:10.710 --> 00:19:12.170
Or not bring it back to you, but like, let's
400
00:19:12.549 --> 00:19:14.970
talk about, uh, this with like software factories in NX.
401
00:19:15.230 --> 00:19:17.880
So I'm looking at the blog post. It looks like
402
00:19:17.910 --> 00:19:21.099
Victor wrote it, a software factories, not a product. Um,
403
00:19:21.140 --> 00:19:23.819
that one you talk a lot about, or he, I guess,
404
00:19:23.940 --> 00:19:28.380
talks a lot about polygraph in there. Um, so if
405
00:19:28.680 --> 00:19:32.079
a software factory is a workflow, where does NX fit
406
00:19:32.140 --> 00:19:35.960
in here? Where's polygraph fit in here? So that's outside
407
00:19:35.980 --> 00:19:39.339
of the NX ecosystem, like what are they using or
408
00:19:39.380 --> 00:19:41.819
what are their methods to, you know, if they're not
409
00:19:41.859 --> 00:19:46.349
using Polygraph, what would they be doing instead? Talk me
410
00:19:46.410 --> 00:19:49.809
through what the workflow actually looks like here in terms
411
00:19:49.849 --> 00:19:51.359
of the tooling side of things.
412
00:19:51.880 --> 00:19:56.200
Yeah, this touches actually on multiple points, I would say.
413
00:19:56.220 --> 00:19:59.720
So at a very high level, an X is simply
414
00:19:59.819 --> 00:20:02.619
underlying tooling mechanism that your agent is going to use, right?
415
00:20:03.420 --> 00:20:05.539
And that very much feeds into what we're just talking
416
00:20:05.579 --> 00:20:07.460
about in the sense of how can I control the
417
00:20:07.500 --> 00:20:08.420
implementation part?
418
00:20:08.880 --> 00:20:09.019
Right.
419
00:20:09.039 --> 00:20:12.079
How can I control how the agent runs tests and
420
00:20:12.160 --> 00:20:13.640
like builds, et cetera.
421
00:20:13.700 --> 00:20:13.859
Right.
422
00:20:14.359 --> 00:20:15.769
If I can make that faster.
423
00:20:16.029 --> 00:20:16.200
Right.
424
00:20:16.240 --> 00:20:18.329
And then X has been a big goal of that
425
00:20:18.390 --> 00:20:21.769
has always been making things run faster as a developer.
426
00:20:22.289 --> 00:20:24.509
But it also benefits like if an agent is getting
427
00:20:24.589 --> 00:20:27.869
faster feedback and always goes the same route of invoking things,
428
00:20:28.410 --> 00:20:31.990
it just doesn't run into like. like roadblocks, because if
429
00:20:32.019 --> 00:20:34.359
you just like observe sometimes in a code base, an
430
00:20:34.420 --> 00:20:37.400
agent where it tries to do the TypeScript type checking, right?
431
00:20:37.420 --> 00:20:39.059
So just like goes and tries to figure out how
432
00:20:39.079 --> 00:20:41.269
can you do it? It goes through the package JSON script. Oh,
433
00:20:41.279 --> 00:20:43.380
there's a TSC installed. Let me just like use that
434
00:20:43.400 --> 00:20:46.140
and point to the product, to the TSC config JSON
435
00:20:46.740 --> 00:20:48.900
and just run a type check, which might miss a
436
00:20:48.940 --> 00:20:51.359
whole bunch of other things, right? Like just imagine you
437
00:20:51.400 --> 00:20:53.009
are on a monorepo, you have a bunch of other
438
00:20:53.049 --> 00:20:55.569
projects that are dependent on it. Those would not be touched, right?
439
00:20:55.589 --> 00:20:58.589
They just wouldn't know, right? And so ICNX kind of
440
00:20:58.609 --> 00:21:01.920
like as that like, tooling layer where the agent directly
441
00:21:01.940 --> 00:21:06.900
just like uses to run down underlying things. Obviously there's
442
00:21:06.980 --> 00:21:11.170
other parts to it, which now actually, interestingly and like happily,
443
00:21:11.190 --> 00:21:13.589
I'm happily kind of that those things kind of come
444
00:21:13.670 --> 00:21:15.769
up again, which is the whole thing about, for instance,
445
00:21:16.089 --> 00:21:18.710
module boundaries that we have been proven. It's actually a
446
00:21:18.710 --> 00:21:20.049
lint rule in the end, right? It's not even an
447
00:21:20.250 --> 00:21:22.380
X specific or super X specific. We just shipped it
448
00:21:22.430 --> 00:21:24.839
as a plugin, but it now starts, it was always
449
00:21:24.859 --> 00:21:27.500
kind of important, but like some people just like didn't
450
00:21:27.519 --> 00:21:30.910
care and like, yeah, whatever. And, we kind of handle
451
00:21:30.990 --> 00:21:33.509
it like throughout code reuse, et cetera, which might actually
452
00:21:33.529 --> 00:21:36.069
be fine. The problem is now the velocity with agents
453
00:21:36.130 --> 00:21:38.650
is so much bigger and so much faster, right? Before,
454
00:21:39.099 --> 00:21:41.099
like even if like a team member on your team
455
00:21:41.140 --> 00:21:43.220
kind of like added something in that's kind of like
456
00:21:43.640 --> 00:21:46.160
violating those rules, while you would either catch it in
457
00:21:46.220 --> 00:21:48.079
a review or in the next PR you're going to
458
00:21:48.119 --> 00:21:50.619
fix it. Now agents, and I always say kind of
459
00:21:50.640 --> 00:21:51.980
bring up the analogy, it's kind of like a small
460
00:21:52.019 --> 00:21:55.369
hole in your system, right? Now you have like huge
461
00:21:55.430 --> 00:21:58.240
holes that these agents just punch in Because there's so
462
00:21:58.279 --> 00:22:02.069
much faster chipping, right? So now automation becomes even more important, right?
463
00:22:02.470 --> 00:22:03.670
And so an X is kind of like, I see
464
00:22:03.710 --> 00:22:07.369
that as that layer where it just provides utilities to
465
00:22:07.450 --> 00:22:10.549
do that. Verification loops, if you want to some degree
466
00:22:10.589 --> 00:22:13.759
and constraints, those types of things. Same obviously for the
467
00:22:13.799 --> 00:22:17.950
CI part, right? We just saw an article from Anthropic
468
00:22:17.970 --> 00:22:21.470
a couple of days ago of them being completely overloaded
469
00:22:22.150 --> 00:22:24.970
with the amount of PRs and tests that now run
470
00:22:25.009 --> 00:22:27.789
on CI. And so it doesn't matter if you ship
471
00:22:27.829 --> 00:22:30.569
hundreds and hundreds of PRs per day, per person, right?
472
00:22:30.990 --> 00:22:34.210
But CI just can't keep up, right? Processing all those, right?
473
00:22:34.269 --> 00:22:37.119
And running all those tests, which you still want to do, right?
474
00:22:37.730 --> 00:22:40.619
And as you said before, agents are really happy at
475
00:22:40.640 --> 00:22:42.740
producing much more code and tests are a nice way
476
00:22:42.779 --> 00:22:46.049
to express themselves, right? And so like we saw like
477
00:22:46.069 --> 00:22:49.119
a bunch more on them. Like I just even personally
478
00:22:49.130 --> 00:22:51.839
on my personal GitHub account, I never ran out of
479
00:22:51.880 --> 00:22:55.119
like my monthly allowance and now mid-month I'm through. It's
480
00:22:55.140 --> 00:22:57.200
just like, oh, you need to kind of went to
481
00:22:57.220 --> 00:22:59.079
the next plan or something. Otherwise you're not going to
482
00:22:59.099 --> 00:23:01.400
be able to run GitHub actions, which is surprising. I
483
00:23:01.420 --> 00:23:02.440
never thought even about that.
484
00:23:02.500 --> 00:23:02.660
Right.
485
00:23:02.700 --> 00:23:05.339
It's just my side projects, but now it's so easy
486
00:23:05.380 --> 00:23:07.119
to spin up a new PR and just goes and
487
00:23:07.339 --> 00:23:09.960
runs through. And so that's where kind of NX fits in.
488
00:23:10.579 --> 00:23:13.579
Polygraph is a new product that we've been starting developing
489
00:23:14.170 --> 00:23:16.980
It was like May, April, May. I think we launched
490
00:23:17.019 --> 00:23:20.339
it in June. It's still now, it's still in open beta.
491
00:23:20.359 --> 00:23:22.079
So everyone can try it out for free and play
492
00:23:22.119 --> 00:23:25.420
around with it. And that is more our approach of like,
493
00:23:25.619 --> 00:23:29.779
how can we kind of like make a genetic development
494
00:23:30.319 --> 00:23:34.259
nice in terms of economics from a various parts, basically.
495
00:23:34.480 --> 00:23:37.940
Like one initial starting point was basically, what if you
496
00:23:37.980 --> 00:23:40.730
don't have a monorepo, right? Like we, I obviously, an
497
00:23:40.789 --> 00:23:44.029
ex comes from the monorepo scene. And every time, we're
498
00:23:44.049 --> 00:23:46.150
still huge promoters of that, obviously, but a lot of
499
00:23:46.210 --> 00:23:48.819
companies just have that one monorepo and then a bunch
500
00:23:48.839 --> 00:23:52.359
of other separate repos, or they want to migrate to one,
501
00:23:52.400 --> 00:23:54.420
and then they have still to orchestrate all these different
502
00:23:54.460 --> 00:23:57.180
other repositories. And we were like, what if we could
503
00:23:57.240 --> 00:24:00.119
provide some solution for that, specifically for agents, because that
504
00:24:00.160 --> 00:24:02.599
seems where everyone is heading. And so the initial thought
505
00:24:02.619 --> 00:24:06.589
of Polygraph was, okay, let's just create a layer on
506
00:24:06.619 --> 00:24:09.789
top of the harnesses, so Cloud Code and Codex, which
507
00:24:09.809 --> 00:24:13.740
we call a meta-harness, there's a page we create, metaharness.tools,
508
00:24:14.460 --> 00:24:15.960
which kind of like summarizes a bit the idea. There's
509
00:24:15.980 --> 00:24:18.380
a bunch of these metaharness around and like more come
510
00:24:18.440 --> 00:24:21.269
out like every week, it seems, which is kind of
511
00:24:21.289 --> 00:24:24.130
like that layer around to provide some niceties to your
512
00:24:24.309 --> 00:24:27.190
agent harness. Because like if you just launch Cloud Code
513
00:24:27.230 --> 00:24:30.150
in a repo, it just works there and that's it, right?
514
00:24:30.529 --> 00:24:32.670
But if you say, oh, I want to change that
515
00:24:32.730 --> 00:24:34.890
thing in that design system library and I want to
516
00:24:34.990 --> 00:24:38.859
update the corresponding projects or test them, directing those projects,
517
00:24:38.900 --> 00:24:41.069
it has no idea where that comes from. But if
518
00:24:41.170 --> 00:24:44.349
it has that overlay arching graph across these repos, which
519
00:24:44.369 --> 00:24:46.569
you can figure out and index, right, which is where
520
00:24:46.630 --> 00:24:48.769
Polygraph comes in, then you can feed that to an
521
00:24:48.809 --> 00:24:52.490
agent and just brings it in from this, from the graph.
522
00:24:52.910 --> 00:24:55.789
And so an agent can now implement something across these repos.
523
00:24:56.250 --> 00:24:59.230
And so that is, so where this fits in into
524
00:24:59.269 --> 00:25:03.710
these workflows, et cetera, that Jay mentioned is it provides
525
00:25:03.730 --> 00:25:06.869
this orchestration layer where you can just tell an agent, okay,
526
00:25:06.930 --> 00:25:09.579
implement this change in the design system and update the
527
00:25:09.619 --> 00:25:12.210
dependent project. And you don't have to think about or
528
00:25:12.259 --> 00:25:14.839
worry about, like, where's my local checkout of all these repos,
529
00:25:15.539 --> 00:25:18.700
et cetera. It will just clone them down, connect them,
530
00:25:18.740 --> 00:25:21.059
spawn an agent on all of them, and just run
531
00:25:21.079 --> 00:25:24.079
this implementation. Which, again, just as with Cloud Code, you
532
00:25:24.099 --> 00:25:27.299
can just guide it interactively. Or you could have it
533
00:25:27.359 --> 00:25:28.880
as part of a software factor so it becomes like
534
00:25:28.900 --> 00:25:31.900
the underlying running layer almost for making sure things don't
535
00:25:32.140 --> 00:25:35.119
run into conflicts. And the second part of Polygraph, which
536
00:25:35.140 --> 00:25:38.099
kind of emerged as we just heavily started using it internally,
537
00:25:38.579 --> 00:25:42.180
is the whole... kind of memory aspect. Memory is such
538
00:25:42.200 --> 00:25:44.079
a loaded term, so I'm always a bit cautious with that.
539
00:25:44.099 --> 00:25:46.619
Because what even is memory, right? But it's a combination
540
00:25:46.640 --> 00:25:50.619
of like facts as in memory, but also mostly traces
541
00:25:50.759 --> 00:25:54.220
or if you want conversation histories with agents, but across teams, right?
542
00:25:54.259 --> 00:25:57.809
Not just yours. For those that might not know, you
543
00:25:57.829 --> 00:25:59.829
can look up all the session logs of an agent
544
00:25:59.869 --> 00:26:02.390
locally in your file system. It just stores them. You
545
00:26:02.430 --> 00:26:04.230
can just go through them, kind of have an agent
546
00:26:04.289 --> 00:26:07.710
parse through them. But it's just your session logs, right?
547
00:26:07.789 --> 00:26:10.130
And As I mentioned initially, now a lot of the
548
00:26:10.170 --> 00:26:12.970
development happens between you and your agent and like Mike
549
00:26:13.009 --> 00:26:14.849
and his agents and Jay is his agent, et cetera, right?
550
00:26:14.869 --> 00:26:18.220
We work on a team. A lot of the implementation
551
00:26:18.319 --> 00:26:21.740
reasoning is the agent sometimes deciding, right? Because if you
552
00:26:21.759 --> 00:26:24.099
just decide roughly the architecture and how things should relate
553
00:26:24.140 --> 00:26:28.119
together and you're fine with that, right? Which you shouldn't always, right?
554
00:26:28.579 --> 00:26:30.700
But like, let's assume that we're going in that direction
555
00:26:30.740 --> 00:26:34.430
where you think more about the architectural parts and system architecture,
556
00:26:34.849 --> 00:26:37.789
but not the lower level things. The agent decides sometimes,
557
00:26:38.269 --> 00:26:41.750
that how things should be wired together. Or you stop
558
00:26:41.829 --> 00:26:43.589
it and say like, oh, no, no, no, you shouldn't
559
00:26:43.609 --> 00:26:45.950
do that because there was like that regression thing or
560
00:26:45.970 --> 00:26:48.910
a customer or something. Do this other thing instead, right?
561
00:26:49.170 --> 00:26:51.450
So it's like you kind of making it do it
562
00:26:51.529 --> 00:26:54.369
on like a 180 degree turn or something in our direction.
563
00:26:54.869 --> 00:26:56.650
If then another team member comes in and wants to
564
00:26:56.690 --> 00:26:58.890
go on top of that implementation and add some other stuff,
565
00:26:59.029 --> 00:27:01.069
it doesn't have that context at all, right? You won't
566
00:27:01.109 --> 00:27:03.890
see it from the source code unless your agent, as
567
00:27:03.930 --> 00:27:05.859
they do, like has like tons and tons of comments
568
00:27:05.910 --> 00:27:09.109
on it. which might be very quickly outdated. It's not
569
00:27:09.119 --> 00:27:13.130
the original part, right? And so that's where that whole, like, traces,
570
00:27:13.529 --> 00:27:16.470
session log sharing came in. Because now with Polygraph, you
571
00:27:16.490 --> 00:27:18.349
can just tell it, okay, I'm going to implement that
572
00:27:18.390 --> 00:27:21.829
piece here, get the context from previous parts that implemented
573
00:27:21.869 --> 00:27:24.390
that function, for instance, and is able to understand from
574
00:27:24.410 --> 00:27:27.019
the session logs from the team, pull them in, analyze them,
575
00:27:27.039 --> 00:27:28.680
and say, okay, like, we can add it on top
576
00:27:28.720 --> 00:27:32.059
of this. This decision was taken because the user that
577
00:27:32.380 --> 00:27:35.269
guided that agent did that because of that article. customer
578
00:27:35.289 --> 00:27:37.170
ticket that came in and broke something and so we
579
00:27:37.210 --> 00:27:39.349
have to do the workaround for something right so basically
580
00:27:39.569 --> 00:27:42.930
encode these things that you wouldn't otherwise know where to
581
00:27:43.019 --> 00:27:46.900
find it's almost auto-documenting some of these facts which before
582
00:27:46.940 --> 00:27:50.059
lyft and the heads of the developers right so that
583
00:27:50.099 --> 00:27:52.180
is kind of like where those two things.
584
00:27:52.220 --> 00:27:55.319
Kind of kick in right now what i like about
585
00:27:55.359 --> 00:27:58.500
this is that from it's kind of solving one of
586
00:27:58.519 --> 00:28:02.849
those very very tough problems that everyone has focused on,
587
00:28:02.890 --> 00:28:07.269
and that is how do you have that consistency, but
588
00:28:07.329 --> 00:28:11.269
also that reliability when working with agents? It's like bringing
589
00:28:11.299 --> 00:28:16.119
in the deterministic aspect that AI just so desperately.
590
00:28:15.660 --> 00:28:17.039
Needs and does not have.
591
00:28:17.059 --> 00:28:20.000
But then there becomes a real split of this is
592
00:28:20.079 --> 00:28:23.869
in NX, this is in the agent. what, like, how
593
00:28:23.880 --> 00:28:26.529
do you, like, where do you see that split happening
594
00:28:26.569 --> 00:28:29.549
and like encourage teams? Like this should live inside of your,
595
00:28:29.569 --> 00:28:32.430
you know, your agents MD, this should live as something
596
00:28:32.470 --> 00:28:36.450
else that NX can benefit and maybe non agent, like
597
00:28:36.630 --> 00:28:39.349
maybe humans, non agents can.
598
00:28:39.509 --> 00:28:42.410
Uh, can, uh, utilize. It's kind of weird to think
599
00:28:42.430 --> 00:28:45.150
about non agents, human.
600
00:28:44.869 --> 00:28:47.369
Developers, the original programmers.
601
00:28:47.910 --> 00:28:48.109
Yeah.
602
00:28:49.349 --> 00:28:51.230
Artisanally hand-coded.
603
00:28:51.329 --> 00:28:55.309
Artisanally crafted code.
604
00:28:55.349 --> 00:29:03.960
We are a species in front of extinction. It's really
605
00:29:04.079 --> 00:29:07.619
hard to say, like, so my rule usually goes where
606
00:29:08.220 --> 00:29:12.259
everything that I can encode in an automation where it's
607
00:29:12.279 --> 00:29:14.289
basically like, let's frame it like that. Like there's different
608
00:29:14.329 --> 00:29:17.369
levels of how I see it. So if I don't
609
00:29:17.410 --> 00:29:20.970
give an agent any instruction. like not over agents MD
610
00:29:21.009 --> 00:29:24.799
and not additional prompt context or something like that, and
611
00:29:24.839 --> 00:29:27.960
it runs something and it fails and gives it proper feedback,
612
00:29:28.259 --> 00:29:29.720
I want to do that, right? So for instance, a
613
00:29:29.720 --> 00:29:32.200
lint rule with a proper feedback, an agent will run
614
00:29:32.240 --> 00:29:36.410
into that regardless whether it read that instruction in agent's
615
00:29:36.470 --> 00:29:39.339
MDE in that external linked file or something, which we
616
00:29:39.380 --> 00:29:42.880
know agents sometimes are just ignoring for whatever reason. So
617
00:29:42.900 --> 00:29:45.240
I'm trying to bake that into that part, right? So
618
00:29:45.299 --> 00:29:47.059
if it goes through the NXT line, in this case,
619
00:29:47.079 --> 00:29:49.299
for instance, or whatever you're using, and you have a
620
00:29:49.339 --> 00:29:53.069
mechanism that the lint rule pushes back some proper setup
621
00:29:53.230 --> 00:29:55.569
and trend test or something pushes back, well, that's what
622
00:29:55.609 --> 00:29:58.750
an agent will use. Either locally or at least your
623
00:29:58.789 --> 00:30:01.170
CI will fail, right? If an agent misses its run,
624
00:30:01.329 --> 00:30:04.670
your CI will fail, an agent or yourself will read
625
00:30:04.730 --> 00:30:06.309
it and you push it back to an agent and
626
00:30:06.690 --> 00:30:09.670
looks at the log and then fixes it. So I'm
627
00:30:09.769 --> 00:30:13.910
still highly believing that those tools have a place and
628
00:30:14.170 --> 00:30:17.130
you should even automate more. Like I've recently also starting
629
00:30:17.150 --> 00:30:20.509
to advocate more people to just use agents to automate
630
00:30:20.529 --> 00:30:24.140
a lot of that stuff, right? For instance, An X,
631
00:30:24.299 --> 00:30:26.880
I'm just mentioning an X because obviously that's where I'm
632
00:30:26.940 --> 00:30:28.660
living day to day, but like you could just create
633
00:30:28.680 --> 00:30:30.660
your custom lint rules, right? An X had had a
634
00:30:30.680 --> 00:30:34.140
mechanism for that for a long time, but it's always like, yeah,
635
00:30:34.180 --> 00:30:36.849
but like just, yeah, whatever. It takes away time. I
636
00:30:36.869 --> 00:30:38.329
need to go and like figure out how to do
637
00:30:38.369 --> 00:30:40.829
that even, right. Even though there was a generator that
638
00:30:40.849 --> 00:30:42.349
you could just run and then fit in the details,
639
00:30:42.490 --> 00:30:44.970
but still, right. It takes away work from like a
640
00:30:45.390 --> 00:30:48.940
feature development. I keep telling people. Agents are really good
641
00:30:48.980 --> 00:30:49.279
at that.
642
00:30:49.299 --> 00:30:49.960
Really good.
643
00:30:49.980 --> 00:30:53.680
Because it's such a close and deterministic task. Like this
644
00:30:53.720 --> 00:30:56.579
is like, this is the error I have. You see
645
00:30:56.599 --> 00:30:58.309
it because you run the lint rule or you run
646
00:30:58.349 --> 00:30:59.990
something and it doesn't fail, for instance, but you want
647
00:31:00.029 --> 00:31:02.710
it to fail. You can exactly tell it, whenever you
648
00:31:02.750 --> 00:31:05.470
see this pattern, you should fail, create a lint rule
649
00:31:05.529 --> 00:31:08.309
for me. And it can loop until it reaches that
650
00:31:08.369 --> 00:31:10.329
point and you can do other stuff in the meantime
651
00:31:10.349 --> 00:31:12.009
and we'll do it in background. And I can kind
652
00:31:12.049 --> 00:31:15.200
of like confidently say, well, probably nail it. I've done
653
00:31:15.220 --> 00:31:17.230
it a bunch of times. And this can just be
654
00:31:17.250 --> 00:31:19.690
a rule that lifts in your monorail or whatever, and
655
00:31:19.710 --> 00:31:21.450
it keeps running from now on. It will fail on CI,
656
00:31:21.529 --> 00:31:24.490
et cetera. So you baked it in, right? Same with
657
00:31:24.549 --> 00:31:26.450
like workspace level generators.
658
00:31:27.049 --> 00:31:27.329
Yeah.
659
00:31:27.910 --> 00:31:29.859
Some people created it. I remember when we did a
660
00:31:29.880 --> 00:31:31.700
lot of consulting, that was one of the things that
661
00:31:31.720 --> 00:31:35.180
we did a lot for companies, just like standardized things, right?
662
00:31:35.240 --> 00:31:37.779
Here's how our React library looks like. We don't want this.
663
00:31:37.819 --> 00:31:40.460
We want this test runner. So that like every engineer
664
00:31:40.500 --> 00:31:43.410
creating new things, they didn't have to go through docs.
665
00:31:43.430 --> 00:31:45.789
Like it's automated basically. But again, it took work, right?
666
00:31:45.809 --> 00:31:48.299
It takes effort to do. Like usually you had platform
667
00:31:48.319 --> 00:31:49.940
teams doing that for you or we came in as
668
00:31:49.980 --> 00:31:52.220
consultants that helped you do that. Now an agent can
669
00:31:52.259 --> 00:31:54.720
do a lot of that automation, right? And that also
670
00:31:54.740 --> 00:31:57.059
means in turn, an other agent running, not even just
671
00:31:57.079 --> 00:31:59.799
your coworker or like junior developer that comes in, an
672
00:31:59.839 --> 00:32:03.089
other agent running will also use that generator and not
673
00:32:03.170 --> 00:32:06.349
just try to vibe out that new React library, new workspace,
674
00:32:06.390 --> 00:32:09.069
but it uses a consistent pattern. So that again, like our,
675
00:32:09.589 --> 00:32:12.670
Like very deterministic rules I can set in place. The
676
00:32:12.690 --> 00:32:15.309
next level is obviously then agent MD files and skills, right?
677
00:32:15.490 --> 00:32:17.809
Whatever you cannot fill into the workspace, the next level
678
00:32:17.829 --> 00:32:20.670
will be, okay, have an agent MD file, have some skills,
679
00:32:20.740 --> 00:32:24.700
link them in, and basically have an agent read those
680
00:32:24.759 --> 00:32:28.240
as a second layer of safety net. I've also seen
681
00:32:28.259 --> 00:32:32.079
a lot of people adopt these architectural decision records patterns,
682
00:32:32.980 --> 00:32:36.799
which I think are really beneficial because Mostly for not
683
00:32:36.819 --> 00:32:40.119
having to re-explain previous taken decisions. That's that thing. You
684
00:32:40.140 --> 00:32:42.359
shouldn't tell agent document what you did.
685
00:32:42.380 --> 00:32:43.039
No, no.
686
00:32:43.220 --> 00:32:46.480
Like document where I pushed back. I think, I think where, um,
687
00:32:46.680 --> 00:32:49.559
I think on. Can see dots repos. Like he does
688
00:32:49.579 --> 00:32:51.599
that heavily. Maybe he has had an article or something
689
00:32:51.640 --> 00:32:53.089
or video about it. I saw it in a bunch
690
00:32:53.119 --> 00:32:55.200
of places. I think like his pattern was really good
691
00:32:55.220 --> 00:32:57.910
because he was very specific of saying, don't try just
692
00:32:57.930 --> 00:33:01.089
to document what that session just did in terms of architecture, right?
693
00:33:01.109 --> 00:33:03.349
Because that can be encoded in the code base, right?
694
00:33:03.569 --> 00:33:06.069
Can read it from the code base. Just document where
695
00:33:06.089 --> 00:33:08.069
I push back, right? If it wants to do some
696
00:33:08.430 --> 00:33:10.410
or use some type of pattern, you don't want to
697
00:33:10.509 --> 00:33:12.619
use it, then document that and should go in there.
698
00:33:12.660 --> 00:33:14.680
And so next time it should read those before it
699
00:33:14.720 --> 00:33:17.680
even tries to suggest you some new pattern. That'll be
700
00:33:17.700 --> 00:33:18.099
the second.
701
00:33:18.500 --> 00:33:19.880
Yeah. Oh, I should say one thing I wanted to
702
00:33:19.920 --> 00:33:22.319
point out too, like with the lint rules and the generators,
703
00:33:22.359 --> 00:33:26.279
like the deterministic output is that it also helps prevent
704
00:33:26.559 --> 00:33:28.420
one problem we ran into is that we were putting
705
00:33:28.680 --> 00:33:32.150
so much instruction into our context. Like we do this
706
00:33:32.250 --> 00:33:34.210
and we also do this and we always do this,
707
00:33:34.490 --> 00:33:36.569
but all of that was captured in our lint rules.
708
00:33:36.650 --> 00:33:38.950
So if it did that, it would fail. And so
709
00:33:38.970 --> 00:33:41.970
we were able to cut out a lot of context
710
00:33:42.029 --> 00:33:43.839
that we were trying to feed the agent every time
711
00:33:43.859 --> 00:33:46.519
and we're getting better results when we took those out. So.
712
00:33:46.799 --> 00:33:49.160
Yeah, that's a really good point because context is obviously valuable.
713
00:33:49.240 --> 00:33:52.380
There's this term about context rot, which is absolutely still
714
00:33:52.420 --> 00:33:54.720
a case, right? So if it gets too much, the
715
00:33:54.799 --> 00:33:55.680
agent just gets lost.
716
00:33:56.259 --> 00:33:57.960
I want to point on the.
717
00:33:58.200 --> 00:34:03.289
I want to point on the pushing back thing because that,
718
00:34:03.549 --> 00:34:07.049
to me, just really was like, oh, wow, we really
719
00:34:07.069 --> 00:34:10.269
should be pushing back because that's how you determine taste.
720
00:34:10.750 --> 00:34:12.750
And that's how you get the agent to understand.
721
00:34:12.829 --> 00:34:16.449
It's like, here's the human-inquired taste that we've got and
722
00:34:16.929 --> 00:34:19.250
How do you determine that? Well, you don't just enforce
723
00:34:19.349 --> 00:34:21.929
the good, you enforce the things that you don't want.
724
00:34:22.730 --> 00:34:23.809
Yeah, yeah, yeah.
725
00:34:24.170 --> 00:34:26.409
Yeah, and also if it, and that goes a bit
726
00:34:26.429 --> 00:34:28.489
in the direction of these architectural decision records, which is
727
00:34:28.590 --> 00:34:31.190
one pattern, right? It's not, you should go that direction.
728
00:34:31.210 --> 00:34:33.130
It's like one of those patterns that you could look at,
729
00:34:33.550 --> 00:34:35.590
but it's one of those things where it keeps, if
730
00:34:35.610 --> 00:34:37.769
these things keep coming up, like the immediate thing on
731
00:34:37.809 --> 00:34:40.590
your head as an engineer should be, why does it
732
00:34:40.670 --> 00:34:43.269
try to do this thing, right? Which I don't want
733
00:34:43.329 --> 00:34:45.690
it to do. Can I prevent it such that my
734
00:34:45.730 --> 00:34:47.789
coworker doesn't run into the same issue? I think that's
735
00:34:48.469 --> 00:34:51.210
one of the main forces that you have to make
736
00:34:51.250 --> 00:34:55.010
sure your co-based keeps healthy or stays healthy nowadays is
737
00:34:55.429 --> 00:34:58.360
create these automations and boundaries and like feedback things into
738
00:34:58.389 --> 00:35:00.800
place such that other agents run into it and then
739
00:35:00.840 --> 00:35:02.000
we're course correct, hopefully.
740
00:35:03.440 --> 00:35:03.599
Yeah.
741
00:35:04.500 --> 00:35:05.019
Yeah, exactly.
742
00:35:05.039 --> 00:35:07.300
And then the third layer will be something like polygraph, right?
743
00:35:07.320 --> 00:35:10.280
Where you have some external context that you can pull in.
744
00:35:10.300 --> 00:35:13.260
I have seen a lot of people try, like a
745
00:35:13.280 --> 00:35:16.219
lot of, This is pretty new still. So there's a
746
00:35:16.239 --> 00:35:18.530
lot of like research being done. Like how do you
747
00:35:18.570 --> 00:35:22.510
best organize such a central memory storage? As I mentioned initially,
748
00:35:22.550 --> 00:35:25.429
like what is it even, right? On the Matterharness tools,
749
00:35:25.550 --> 00:35:29.059
we have, we try to categorize it into different types
750
00:35:29.099 --> 00:35:32.699
of memory. So like structural memory, semantic memory, like where
751
00:35:32.719 --> 00:35:34.980
does what hold? Like what is code even? Because code
752
00:35:35.000 --> 00:35:36.739
is also obviously a part of the memory, right? You
753
00:35:36.780 --> 00:35:40.090
can just read code structure and deduce things from there.
754
00:35:40.110 --> 00:35:42.510
So you shouldn't repeat that. because then you go in
755
00:35:42.610 --> 00:35:45.769
even conflicting statements where an agent then gets lost, doesn't
756
00:35:46.449 --> 00:35:48.889
know what to do. And so that will be the
757
00:35:48.929 --> 00:35:51.190
third layer where you can pull in information from those
758
00:35:51.230 --> 00:35:55.550
sources as an additional provider to context, which I've seen already,
759
00:35:56.130 --> 00:36:00.230
even in simple cases, work out surprisingly well. Like stupid
760
00:36:00.250 --> 00:36:03.670
things like, for instance, I had some page where I
761
00:36:03.730 --> 00:36:06.130
think it might even have been a MetaHarris tools page
762
00:36:06.170 --> 00:36:10.610
where I did some SEO optimization and then change something
763
00:36:11.150 --> 00:36:14.789
on a title and implement that, push WPR, went online.
764
00:36:14.809 --> 00:36:16.489
And the idea back then, like when I interact with
765
00:36:16.510 --> 00:36:18.670
the agent was like, okay, let's try this out. Let's
766
00:36:19.289 --> 00:36:21.489
stay there for like two, three weeks, not change it
767
00:36:21.510 --> 00:36:24.539
at all to actually see how Google impacts and search results,
768
00:36:24.559 --> 00:36:26.659
et cetera. And I was doing some other work and
769
00:36:26.679 --> 00:36:28.320
the agent was going in, oh, that title looks weird.
770
00:36:28.340 --> 00:36:31.800
Let me just fix that, right? And that's because why?
771
00:36:31.820 --> 00:36:34.400
Because like the title wasn't super optimal, but it would
772
00:36:34.440 --> 00:36:37.909
have completely messed up the previous kind of like, data
773
00:36:37.949 --> 00:36:40.110
retrieval that we were currently doing, right? And it was
774
00:36:40.150 --> 00:36:42.690
just giving the agent an access to these previous sessions.
775
00:36:42.929 --> 00:36:44.670
It just looked it up and it's like, oh, wait,
776
00:36:44.750 --> 00:36:47.090
there was something else. It found that we had discussed
777
00:36:47.130 --> 00:36:50.230
about this like two weeks of data collection and analysis.
778
00:36:50.289 --> 00:36:54.300
And so it just completely reverted the decision that it took, right? Obviously, right?
779
00:36:54.340 --> 00:36:57.179
It's like a human not having context, right? It looks
780
00:36:57.219 --> 00:37:00.059
at the problem. That's the best solution. Without knowing that
781
00:37:00.400 --> 00:37:03.760
there was this workaround before, it wouldn't know basically what
782
00:37:03.780 --> 00:37:05.369
to do, right? It would just let go with the
783
00:37:05.409 --> 00:37:08.659
best possible decision it could make. Um, so yeah, those
784
00:37:08.699 --> 00:37:10.659
are roughly the, the levels I would say.
785
00:37:11.059 --> 00:37:11.940
So, okay.
786
00:37:12.380 --> 00:37:14.360
If you have your boss comes to you, they've been
787
00:37:14.400 --> 00:37:16.800
reading about software factories and they're like, we need a
788
00:37:16.820 --> 00:37:20.389
software factory because you know, they're doing, somebody's going to
789
00:37:20.429 --> 00:37:25.550
do that. Um, how does the team either know that? Yes,
790
00:37:25.670 --> 00:37:28.250
we're ready to do something like that or no, we
791
00:37:28.289 --> 00:37:29.989
need to push back and we need to buy some
792
00:37:30.030 --> 00:37:32.760
time to be ready to do that. I think.
793
00:37:33.760 --> 00:37:35.800
It goes a bit with some of the parts we
794
00:37:35.820 --> 00:37:39.239
talked about before in a sense of like, do you
795
00:37:39.260 --> 00:37:43.019
have some of the other things like these feedback loops,
796
00:37:43.980 --> 00:37:46.690
like cloak construct, do you have those in place such
797
00:37:46.710 --> 00:37:47.550
that it can even work?
798
00:37:48.130 --> 00:37:48.329
Right.
799
00:37:48.570 --> 00:37:50.789
Because it doesn't, like a software factor usually is not
800
00:37:50.829 --> 00:37:52.789
just like, okay, let's create a software factor. When that
801
00:37:53.030 --> 00:37:55.429
ask comes, it's more like, oh, we need to have
802
00:37:55.489 --> 00:37:58.380
agents work in the background. So like you don't have
803
00:37:58.389 --> 00:38:00.820
to care. And so we increase our productivity and so
804
00:38:01.420 --> 00:38:04.239
we make more money. Right. We don't need more people, right? Like,
805
00:38:04.280 --> 00:38:05.920
it usually goes in that direction, which is, like, a
806
00:38:06.340 --> 00:38:07.340
bit worrying, but, like, we know.
807
00:38:07.239 --> 00:38:07.860
How it goes, right?
808
00:38:07.880 --> 00:38:11.550
So people see more agents, more productivity. It's all going
809
00:38:11.570 --> 00:38:16.289
to be good. So that obviously doesn't work. What I
810
00:38:16.329 --> 00:38:20.730
would start doing is start it on the simplest possible problem. Like,
811
00:38:20.750 --> 00:38:23.860
the GitHub triaging, for instance, is one very classic case
812
00:38:24.280 --> 00:38:27.000
where you cannot really do a lot of harm, but
813
00:38:27.039 --> 00:38:29.940
you can already start getting that feel of, like, that
814
00:38:30.119 --> 00:38:32.900
mini workflow, right? I would just come back to that
815
00:38:32.920 --> 00:38:36.099
before even talking about like an actual factory. Because when
816
00:38:36.119 --> 00:38:38.760
you try it on these things, you immediately run into
817
00:38:38.860 --> 00:38:42.039
these roadblocks or something where, oh, it cannot access GitHub.
818
00:38:42.840 --> 00:38:45.159
Then the next question is like, oh, should I actually
819
00:38:45.199 --> 00:38:47.829
give it full access to GitHub? Maybe that's also a
820
00:38:47.849 --> 00:38:50.250
bad idea, right? So you start thinking about security.
821
00:38:51.329 --> 00:38:51.570
Exactly.
822
00:38:51.610 --> 00:38:53.769
But you start thinking about security in the sense of like,
823
00:38:54.309 --> 00:38:55.949
maybe the token I give to my agent should be
824
00:38:55.989 --> 00:38:58.489
able to delete repos or push new repos. Maybe it
825
00:38:58.510 --> 00:39:01.469
should just be able to kind of read specific, like
826
00:39:01.610 --> 00:39:05.170
give you that specific grant for parsing issues, parsing PRs,
827
00:39:05.210 --> 00:39:07.829
et cetera, right? So all those questions come up, but
828
00:39:07.849 --> 00:39:09.909
they are very much on that simple use case of
829
00:39:09.989 --> 00:39:12.989
triaging issues, right? And so I would start doing that
830
00:39:13.090 --> 00:39:15.829
and then you can extend that, right? Okay, let the
831
00:39:15.909 --> 00:39:19.869
agent do some classification. Issues that need human review because
832
00:39:19.889 --> 00:39:23.309
there's not enough context or issues, oh, we perfectly, that's
833
00:39:23.329 --> 00:39:26.820
the buck, perfectly reproduced it. I know the fix. I
834
00:39:26.840 --> 00:39:28.840
can submit a PR, which then a human will review.
835
00:39:29.360 --> 00:39:31.599
And just to prove, or push back right because if
836
00:39:31.619 --> 00:39:33.900
you then succeed with that flow you will already see
837
00:39:34.119 --> 00:39:37.199
oh that's amazing right because those simple bucks that no
838
00:39:37.260 --> 00:39:39.530
one attended to but they were there for like two
839
00:39:39.570 --> 00:39:41.989
months right and like if just someone had paid attention
840
00:39:42.010 --> 00:39:44.210
it would be like a half an hour job yeah
841
00:39:44.539 --> 00:39:47.019
are now just going through for free almost, right? And
842
00:39:47.059 --> 00:39:49.039
so I would start doing that because then you get
843
00:39:49.079 --> 00:39:51.840
a perfect feel of like the whole flow end to end.
844
00:39:52.119 --> 00:39:54.530
And then just like start improving it because like a
845
00:39:54.590 --> 00:39:57.570
big part of these software factories is like how much
846
00:39:57.690 --> 00:39:59.230
trust do you have in that process?
847
00:40:00.030 --> 00:40:00.829
Yeah, absolutely.
848
00:40:00.849 --> 00:40:03.369
Yeah, we finally got to the point where like I
849
00:40:03.429 --> 00:40:05.989
can spin up work. I can say, hey, open a work,
850
00:40:06.090 --> 00:40:08.880
cut a branch, open in a work tree, fix this bug.
851
00:40:09.300 --> 00:40:12.210
When you're done, push it. open a pull request and
852
00:40:12.250 --> 00:40:14.429
I'll review it. And I can get like three bugs
853
00:40:14.510 --> 00:40:16.579
going at once while I'm working on something else.
854
00:40:17.039 --> 00:40:17.980
Exactly. Exactly.
855
00:40:18.079 --> 00:40:20.300
Like that's a great workflow to be able to get into.
856
00:40:20.599 --> 00:40:21.900
I do want to point out too, like I work
857
00:40:21.920 --> 00:40:25.039
for Cisco. So we have an entire team of people
858
00:40:25.099 --> 00:40:27.929
that are working on securing our agents. Like I don't
859
00:40:27.969 --> 00:40:30.530
have to think about it. So when you're in smaller companies,
860
00:40:30.670 --> 00:40:33.929
like that is a major concern and you really do
861
00:40:33.989 --> 00:40:36.389
have to think about what kind of access you give
862
00:40:36.409 --> 00:40:36.969
your agents.
863
00:40:37.210 --> 00:40:37.409
So.
864
00:40:37.409 --> 00:40:37.949
100%. 100%.
865
00:40:37.949 --> 00:40:41.849
Like that, that, That should be an immediate and constant
866
00:40:41.969 --> 00:40:45.349
thought that you have because obviously the more you also
867
00:40:45.449 --> 00:40:47.409
integrate it with tools, which you want to do, right?
868
00:40:47.510 --> 00:40:49.909
Because ideally you give it some access to linear or
869
00:40:49.929 --> 00:40:52.239
whatever you're managing your tasks, right? So it has context
870
00:40:52.260 --> 00:40:54.800
from there. Give it some access to Slack because there
871
00:40:54.840 --> 00:40:57.719
might have been a conversation that talked about this or
872
00:40:58.000 --> 00:41:01.019
might even have originated from Slack directly, right? So you
873
00:41:01.039 --> 00:41:05.239
will need these tooling integration at some point, but on
874
00:41:05.400 --> 00:41:07.989
every step of these, you should think about what is
875
00:41:08.010 --> 00:41:10.659
the security implication of that. Because an agent to some
876
00:41:10.699 --> 00:41:13.880
degree is like, I don't want to say a thinking thing,
877
00:41:13.920 --> 00:41:16.639
but like it can, it's indeterministic. That's a better word, right?
878
00:41:17.179 --> 00:41:19.239
Before you had a script, well, the script might have
879
00:41:19.260 --> 00:41:22.300
been buggy, but I couldn't do that much harm potentially.
880
00:41:22.360 --> 00:41:26.099
I mean, it depends on the script, right? I was
881
00:41:26.139 --> 00:41:28.000
just thinking, oh, I might have put down production at
882
00:41:28.059 --> 00:41:30.420
one point. But like in that sense, like it is
883
00:41:30.480 --> 00:41:32.739
a deterministic thing. You can audit, you can go through,
884
00:41:32.780 --> 00:41:35.239
you can have multiple, you look at it. An agent
885
00:41:35.400 --> 00:41:38.219
gets some instruction from your Slack conversations you had with
886
00:41:38.340 --> 00:41:40.530
a a colleague, and then you just tell it, okay,
887
00:41:40.550 --> 00:41:43.130
go fix it. It might have misinterpreted something and just
888
00:41:43.170 --> 00:41:45.610
goes nuts. Right. So that's, I think that's the thing that,
889
00:41:45.789 --> 00:41:47.730
that you'd obviously need to keep in mind because there
890
00:41:47.769 --> 00:41:51.070
is this indeterminism. Um, and yeah, and from there I
891
00:41:51.090 --> 00:41:53.489
would just iterate, right. How can we make this better?
892
00:41:53.510 --> 00:41:56.659
How can we have the entire team benefit from this? Um,
893
00:41:56.780 --> 00:41:58.489
and then the next step could already be, okay, how
894
00:41:58.550 --> 00:42:01.019
can we share some of the stuff? Like do I
895
00:42:01.039 --> 00:42:03.880
see runs from my team? Uh, because otherwise like we're
896
00:42:03.900 --> 00:42:06.460
triaging the same issues, right. Is there some shared infrastructure?
897
00:42:07.119 --> 00:42:10.760
it can go like super wild, right? Like some factories
898
00:42:10.780 --> 00:42:12.800
then spawning cloud agents because they don't want to have
899
00:42:12.860 --> 00:42:15.940
it run locally in my machine, right? I can, some
900
00:42:15.960 --> 00:42:18.119
of these triaging can just happen in the cloud, right?
901
00:42:18.159 --> 00:42:20.090
It does it and I see the PR, that's it, right?
902
00:42:20.170 --> 00:42:21.530
I don't even run it on my machine so it's
903
00:42:21.570 --> 00:42:24.309
more secure, it's sandboxed. There is a lot of these
904
00:42:24.329 --> 00:42:27.409
different levels that you can go through. Maybe we need
905
00:42:27.469 --> 00:42:33.409
that factory capability maturity model or something like that.
906
00:42:33.570 --> 00:42:33.860
Right.
907
00:42:34.219 --> 00:42:37.690
Where, at what point they're like, what does an org's
908
00:42:37.750 --> 00:42:40.809
existing architecture in terms of like their, maybe their code
909
00:42:40.829 --> 00:42:42.929
base or their workflows, like what does that need to
910
00:42:42.989 --> 00:42:45.409
look like before introducing an agent loop? Is there a
911
00:42:45.510 --> 00:42:47.809
point where like you need to do a bunch of
912
00:42:47.869 --> 00:42:50.550
cleanup before you get to the point of introducing a
913
00:42:50.590 --> 00:42:53.719
factory of some maker model, whether that be the trivial,
914
00:42:54.340 --> 00:42:57.400
I'm spinning things off to agents or the less trivial,
915
00:42:57.539 --> 00:43:00.179
I have a whole factory workflow kind of in place. Like,
916
00:43:00.219 --> 00:43:03.099
is there a, can, can anybody just get started with
917
00:43:03.139 --> 00:43:06.090
this or do they have to like get their, their context,
918
00:43:06.190 --> 00:43:07.269
I guess, into a.
919
00:43:07.929 --> 00:43:12.190
Good quote-unquote state. Yeah. No, I think it will always
920
00:43:12.250 --> 00:43:15.239
be an incremental process, right? As with all the things, right?
921
00:43:15.260 --> 00:43:17.900
In the sense of like, it's hard to say, okay,
922
00:43:17.920 --> 00:43:20.219
I sit down, I make my code base clean, and
923
00:43:20.300 --> 00:43:22.679
now I let the agent loose and they will produce
924
00:43:22.719 --> 00:43:25.599
great results, right? So the looping part actually is a
925
00:43:25.639 --> 00:43:28.460
good one that you mentioned because you can already, regardless
926
00:43:28.500 --> 00:43:30.840
of whether you have an agent or not, just try
927
00:43:30.860 --> 00:43:34.329
those things out. There's good articles around also for loop engineering, which...
928
00:43:34.539 --> 00:43:38.099
Like we have now every two weeks a new hot term, right?
929
00:43:38.139 --> 00:43:40.599
So loop engineering was hot at some point in June,
930
00:43:40.659 --> 00:43:43.860
I think, or May, I don't remember even. But it
931
00:43:43.949 --> 00:43:47.070
is a good practice in a sense of like, how
932
00:43:47.170 --> 00:43:50.190
confident are you in your code base to implement a
933
00:43:50.190 --> 00:43:52.730
new feature and have the agent do it on its own, right?
934
00:43:53.070 --> 00:43:55.250
And it can be like, it can be you looking
935
00:43:55.329 --> 00:43:57.250
at the log in front, right? And just let it
936
00:43:57.289 --> 00:43:59.690
go through. But how often do you have to intervene,
937
00:43:59.989 --> 00:44:02.250
course correct, et cetera? I think that's a good metric
938
00:44:02.309 --> 00:44:05.070
for understanding what it, something like a more autonomous agent
939
00:44:05.090 --> 00:44:07.860
is even possible, right? Because like, can it, does it
940
00:44:07.880 --> 00:44:09.800
go through the whole entire thing? Does it open up
941
00:44:09.840 --> 00:44:12.099
the browser? Does it check the actual, can it navigate
942
00:44:12.119 --> 00:44:15.039
to the feature it implemented and verify that? Because even that,
943
00:44:15.079 --> 00:44:18.820
for instance, made huge changes like impact for me. Like
944
00:44:18.840 --> 00:44:20.820
I tested out on one of my side projects where
945
00:44:20.840 --> 00:44:23.760
I'm actually building that factory more of like also fun thing.
946
00:44:24.280 --> 00:44:25.610
And it was like, oh, it would be cooler if
947
00:44:25.670 --> 00:44:28.170
it was an Electron app because it's an installable thing, right?
948
00:44:28.550 --> 00:44:31.050
For whatever reason, I didn't rebuild a ton of Electron
949
00:44:31.070 --> 00:44:34.869
apps in the past. And, The initial bump that I
950
00:44:34.889 --> 00:44:38.670
ran into, it's much harder to verify the actual work, right?
951
00:44:38.690 --> 00:44:41.789
The UI. But actually it's not, because like apparently there
952
00:44:41.829 --> 00:44:44.170
is a skill, like agent browser is one that I
953
00:44:44.210 --> 00:44:47.250
found out at some point, like early in summer, that
954
00:44:47.269 --> 00:44:50.570
just can also launch the Electron app and navigate it
955
00:44:50.929 --> 00:44:53.119
as a web browser. Because in the end, it's a
956
00:44:53.119 --> 00:44:55.619
web browser underneath, right? And so once I had plugged
957
00:44:55.679 --> 00:44:59.059
that into the process, and it was simply an instruction
958
00:44:59.079 --> 00:45:01.699
in my MD files, right? We were saying like, whenever
959
00:45:01.780 --> 00:45:04.380
I tell you to implement a feature, create a work tree,
960
00:45:04.840 --> 00:45:07.179
you go through, we just plan first the whole thing,
961
00:45:07.199 --> 00:45:09.610
like think through like some of the decision points, and
962
00:45:09.630 --> 00:45:11.670
then just go off and do it until the PR
963
00:45:11.750 --> 00:45:14.750
comes up. And seeing the agent open up the actual UI,
964
00:45:15.050 --> 00:45:18.610
go in, click the button, see it break, go back,
965
00:45:18.710 --> 00:45:21.159
fix it again, do it again. Like, otherwise it would,
966
00:45:21.380 --> 00:45:24.119
on the first thing, it would say, okay, done, looks good.
967
00:45:24.659 --> 00:45:26.480
I look at the screenshot, it looks good, click the button,
968
00:45:26.519 --> 00:45:29.579
everything breaks, right? Stack trace done, right? So having these
969
00:45:29.659 --> 00:45:31.639
loops and see how good they work, I think is
970
00:45:31.659 --> 00:45:34.760
the incremental step, as Jay said, like to get confidence.
971
00:45:34.900 --> 00:45:36.880
Like for me, it was all about like seeing that
972
00:45:37.090 --> 00:45:39.289
kind of work through. I was like, oh, maybe now
973
00:45:39.389 --> 00:45:42.250
I could actually just let it go alone without me
974
00:45:42.309 --> 00:45:45.010
actually looking at it, right? You just spawn a work tree,
975
00:45:45.250 --> 00:45:48.769
runs in some terminal in the background. I occasionally chime
976
00:45:48.789 --> 00:45:51.679
in to see what's going on and it does it, right?
977
00:45:52.119 --> 00:45:56.150
And I tried, on that factory project, even without using
978
00:45:56.170 --> 00:45:58.210
the factory. For instance, I use, recently I started using
979
00:45:58.250 --> 00:46:02.599
a workflow where I use Herder, which is the terminal multiplex,
980
00:46:02.619 --> 00:46:04.940
which you can use, like the Supersight is a good one,
981
00:46:04.960 --> 00:46:06.960
which you use a long time as well. But the
982
00:46:06.980 --> 00:46:09.940
main thing that I liked about Herder is the fact
983
00:46:10.019 --> 00:46:14.429
it can communicate between terminal paints, right? So TMAX can
984
00:46:14.449 --> 00:46:16.760
do that as well, right? And when I discovered that,
985
00:46:16.780 --> 00:46:18.679
I was like, oh, that is actually interesting, right? Because
986
00:46:18.699 --> 00:46:21.460
the agents have this mechanism of like communicating between sub-agents,
987
00:46:21.480 --> 00:46:23.920
et cetera, but it was never as transparent, right? Because
988
00:46:23.940 --> 00:46:26.480
the sub-agent gets spawned and you can look into its logs, obviously,
989
00:46:26.760 --> 00:46:29.699
but it's not as transparent. I was like, okay, what
990
00:46:29.760 --> 00:46:33.780
if I have this central coordinator, which I'm talking to, right?
991
00:46:33.820 --> 00:46:35.739
And I use a higher model for that, like Fable
992
00:46:35.760 --> 00:46:38.559
or something, like to talk to and plan through and
993
00:46:38.579 --> 00:46:40.539
think about all the edge cases. And once you have
994
00:46:40.559 --> 00:46:42.449
a good plan, I just tell it, okay, this is
995
00:46:42.630 --> 00:46:45.510
roughly the plan we should go through. Spawn Codex to
996
00:46:45.570 --> 00:46:48.159
implement it, right? And so it just creates a work
997
00:46:48.199 --> 00:46:50.960
tree and prompts Codex in a much better prompt than
998
00:46:50.980 --> 00:46:53.380
I would ever be able to do. And just tell Codex,
999
00:46:53.400 --> 00:46:56.840
when you're done, use that herder communication path to prompt
1000
00:46:56.880 --> 00:46:59.340
back into the main coordinator, right? And so you have
1001
00:46:59.360 --> 00:47:02.289
a communication now established and it works. And I started
1002
00:47:02.309 --> 00:47:04.449
doing that at the beginning of summer. Nowadays, I think even,
1003
00:47:04.469 --> 00:47:07.610
I think Claude, if I'm not wrong, has now some
1004
00:47:07.670 --> 00:47:09.570
sort of messaging mechanism built in as well.
1005
00:47:09.590 --> 00:47:09.670
Yeah.
1006
00:47:10.530 --> 00:47:13.230
Grok bot now talks about in a similar fashion, right?
1007
00:47:13.250 --> 00:47:16.949
You have this coordinator mechanism that talks to different sub-agents
1008
00:47:17.010 --> 00:47:21.619
and coordinates work almost like a team. But there were
1009
00:47:21.639 --> 00:47:23.900
some articles around beginning of the year from Anthropic about
1010
00:47:23.920 --> 00:47:27.639
that team, squad, whatever that builds the thing, right? But
1011
00:47:27.679 --> 00:47:30.260
it always sounded like weird to me. And I kind
1012
00:47:30.280 --> 00:47:33.050
of like this approach because you see the full thing
1013
00:47:33.090 --> 00:47:35.349
going on because you can just like open that sub-terminal, right?
1014
00:47:35.389 --> 00:47:37.429
Look at it and see the logs and stop it there, right?
1015
00:47:37.829 --> 00:47:39.880
So you have kind of that control feeling. But you
1016
00:47:39.900 --> 00:47:42.699
still have the coordinating mechanism where the main coordinator just
1017
00:47:42.719 --> 00:47:45.119
talks to these and gets the results, tests and verifies,
1018
00:47:45.139 --> 00:47:47.820
then pushes back. And so some of that thing back
1019
00:47:47.860 --> 00:47:50.840
and forth happens now without me having to kind of
1020
00:47:50.880 --> 00:47:53.469
like interrupt, right? So that's what I mean. That's the
1021
00:47:53.489 --> 00:47:56.309
incremental steps where you go up a bit and you
1022
00:47:56.329 --> 00:47:57.869
gain more trust and see, oh, it goes in the
1023
00:47:58.110 --> 00:48:01.250
correct direction. And until you say like, okay, now I
1024
00:48:01.349 --> 00:48:04.030
could see how I could potentially, some of the easier
1025
00:48:04.090 --> 00:48:08.530
things just delegate complete to my factory, whatever that looks like, right? Yeah.
1026
00:48:08.989 --> 00:48:12.840
How to go through so yuri we're getting towards the
1027
00:48:12.920 --> 00:48:17.099
end of uh of our episode uh we close out
1028
00:48:17.139 --> 00:48:21.079
our episodes with a series of hot takes um very
1029
00:48:21.139 --> 00:48:24.639
short answers two or three words gut don't think about
1030
00:48:24.679 --> 00:48:27.380
it too long just kind of whatever whatever comes to
1031
00:48:27.420 --> 00:48:31.900
your mind um you get one it depends all right.
1032
00:48:31.760 --> 00:48:35.929
So this is You cannot keep answering, it depends.
1033
00:48:37.610 --> 00:48:39.409
We know you're a senior, so we know you're used
1034
00:48:39.449 --> 00:48:40.929
to saying, well, it depends.
1035
00:48:40.969 --> 00:48:44.070
Yeah, I was saying that this changes the rules quite
1036
00:48:44.090 --> 00:48:44.429
a bit.
1037
00:48:44.690 --> 00:48:45.789
Yeah, you only get one.
1038
00:48:46.650 --> 00:48:47.809
From the gut. Okay.
1039
00:48:48.369 --> 00:48:51.769
With the move from actual code to just like, you know,
1040
00:48:51.789 --> 00:48:53.909
a lot of agent and session logs, are we getting
1041
00:48:54.030 --> 00:48:55.130
more or less creative?
1042
00:48:55.989 --> 00:48:56.210
More.
1043
00:48:56.670 --> 00:48:56.909
More.
1044
00:48:56.929 --> 00:48:57.730
Okay.
1045
00:48:58.130 --> 00:48:58.730
I'd agree with that.
1046
00:48:59.150 --> 00:48:59.340
Yeah.
1047
00:48:59.550 --> 00:48:59.989
Interesting.
1048
00:49:00.429 --> 00:49:00.760
All right.
1049
00:49:01.300 --> 00:49:04.199
So you said verification is the new bottleneck. By the
1050
00:49:04.380 --> 00:49:07.510
end of 2027, do we fix it? Is it solved?
1051
00:49:07.860 --> 00:49:10.949
Or is the bottleneck just now someplace else?
1052
00:49:11.409 --> 00:49:13.230
Do I know what other questions are coming so I
1053
00:49:13.250 --> 00:49:15.980
know when to spend my defense? No.
1054
00:49:16.019 --> 00:49:18.079
You don't know what my question is going to be yet.
1055
00:49:18.099 --> 00:49:22.039
You could just say, like, other factors may apply. I
1056
00:49:22.039 --> 00:49:23.599
know you can.
1057
00:49:23.639 --> 00:49:24.800
No, I don't think.
1058
00:49:24.880 --> 00:49:27.179
Short answers, everybody. Short answers.
1059
00:49:27.239 --> 00:49:30.090
Verification. The thing is verification has different levels. I don't
1060
00:49:30.110 --> 00:49:31.070
think we solve all of them.
1061
00:49:31.550 --> 00:49:31.690
Yeah.
1062
00:49:32.150 --> 00:49:32.369
Okay.
1063
00:49:32.730 --> 00:49:32.969
Fair enough.
1064
00:49:32.989 --> 00:49:34.750
That was my it depends workaround, right?
1065
00:49:35.050 --> 00:49:35.570
There you go.
1066
00:49:35.889 --> 00:49:37.250
We'll give that a good one.
1067
00:49:38.710 --> 00:49:42.130
Mine is very topical. Mine's very topical.
1068
00:49:42.170 --> 00:49:45.449
If you were paying attention to Discourse, I think maybe
1069
00:49:45.469 --> 00:49:48.389
it was last week or the week before. Do you
1070
00:49:48.449 --> 00:49:52.480
review every AI-generated or agent-generated PR, or do you have
1071
00:49:52.539 --> 00:49:53.860
rules when you don't?
1072
00:49:54.360 --> 00:49:57.099
I have rules when I don't. Like, if it's super
1073
00:49:57.139 --> 00:50:00.420
simple things, you look, it's one file change, something like,
1074
00:50:00.659 --> 00:50:03.280
just go through it, that's done. Or it really depends. Like,
1075
00:50:03.719 --> 00:50:07.590
it doesn't really depend. Like, it's about, it's about, it's about, like, the...
1076
00:50:08.019 --> 00:50:13.909
Impact of the issue. So what you're saying is context
1077
00:50:13.949 --> 00:50:14.710
is important.
1078
00:50:15.289 --> 00:50:15.670
Exactly.
1079
00:50:15.690 --> 00:50:17.369
It all comes back to context.
1080
00:50:18.070 --> 00:50:19.010
Ah, context.
1081
00:50:19.570 --> 00:50:21.090
Classic. All right.
1082
00:50:21.230 --> 00:50:23.449
So is there anything that we should have asked you
1083
00:50:23.469 --> 00:50:24.250
that we did not?
1084
00:50:25.190 --> 00:50:25.949
No, I don't think so.
1085
00:50:25.969 --> 00:50:26.849
All right.
1086
00:50:26.869 --> 00:50:27.760
It wouldn't come to my mind.
1087
00:50:28.340 --> 00:50:28.820
It depends.
1088
00:50:29.539 --> 00:50:30.280
It depends.
1089
00:50:31.019 --> 00:50:31.880
Where do you want to go? So.
1090
00:50:35.000 --> 00:50:39.710
For the listener, if you enjoyed this episode, you should
1091
00:50:39.750 --> 00:50:44.190
subscribe to keep getting more of these episodes. If you
1092
00:50:44.230 --> 00:50:49.050
want to continue conversations around AI, specifically in person with
1093
00:50:49.159 --> 00:50:53.119
some of your favorite developers, you should go get your
1094
00:50:53.159 --> 00:50:56.179
tickets right now to the AI DevCraft Conference, which is
1095
00:50:56.260 --> 00:50:59.760
October 27th and 28th at the Red Rock Resort in
1096
00:50:59.800 --> 00:51:03.739
Las Vegas. Um, so the prices go up the closer
1097
00:51:03.760 --> 00:51:06.429
we get to that date. So sweet talk your boss
1098
00:51:06.489 --> 00:51:09.570
into getting you tickets now. Yuri, if folks would like
1099
00:51:09.590 --> 00:51:11.690
to reach out to you, how is the best way
1100
00:51:11.750 --> 00:51:12.190
to do that?
1101
00:51:12.730 --> 00:51:16.769
Uh, the best way is on X probably, or sometimes LinkedIn,
1102
00:51:16.829 --> 00:51:19.929
but not as often, uh, or just like go to yuri.dev.
1103
00:51:20.550 --> 00:51:22.690
I believe I have some contact information there, so you
1104
00:51:22.710 --> 00:51:24.110
should be able to reach me. So yeah.
1105
00:51:24.590 --> 00:51:25.800
Nice. So it depends.
1106
00:51:26.159 --> 00:51:29.059
You do post a lot of good content on LinkedIn. Um,
1107
00:51:29.079 --> 00:51:33.949
so I'd, I tend to bookmark a lot of what
1108
00:51:33.989 --> 00:51:37.809
you post. And you also just put out a lot
1109
00:51:37.849 --> 00:51:41.139
of great content in general. Mike, if the listeners want
1110
00:51:41.159 --> 00:51:41.900
to reach out to you.
1111
00:51:42.760 --> 00:51:46.039
They can find me basically everywhere as mhardington. That could
1112
00:51:46.079 --> 00:51:50.940
be LinkedIn, websites, GitHub. I'm just a very, very simple.
1113
00:51:50.960 --> 00:51:53.280
Paul Bearded Man at a conference might not be Mike,
1114
00:51:53.400 --> 00:51:54.340
but might be Mike.
1115
00:51:54.400 --> 00:51:56.179
Might not be Mike. Could be Mike.
1116
00:51:56.840 --> 00:51:58.679
Could be Mike. Could be the guy that kind of
1117
00:51:58.699 --> 00:52:01.130
looks like Mike. Jay?
1118
00:52:01.889 --> 00:52:05.489
You can find me everywhere as Jay Cooper Bell for
1119
00:52:05.510 --> 00:52:07.590
the most part on most platforms.
1120
00:52:07.949 --> 00:52:10.929
You can mostly just find me on LinkedIn. There you go.
1121
00:52:11.050 --> 00:52:17.000
I guess I quit. All right. Thank you, Yuri, so
1122
00:52:17.039 --> 00:52:19.199
much for joining us. Thank you so much for everything
1123
00:52:19.239 --> 00:52:22.960
you give to the community and all of your great contributions.
1124
00:52:24.019 --> 00:52:26.920
Software engineering would not be what it is today without
1125
00:52:27.280 --> 00:52:31.250
you and people like you. we just appreciate that.
1126
00:52:32.230 --> 00:52:33.710
So, um, yeah, thanks.
1127
00:52:35.170 --> 00:52:38.289
Absolutely. Uh, so to the listener, thank you for joining
1128
00:52:38.369 --> 00:52:40.170
us and we hope to catch you next time.
1129
00:52:42.630 --> 00:52:45.230
Thanks for listening to the AI dev craft podcast.
1130
00:52:45.829 --> 00:52:46.369
If you liked the.
1131
00:52:46.349 --> 00:52:49.110
Episode, follow the show and share it with a fellow developer.
1132
00:52:50.449 --> 00:52:53.489
And don't forget AI dev craft is coming to Las Vegas,
1133
00:52:53.570 --> 00:52:57.429
October 27th through the 28th, 2026. Come join us in
1134
00:52:57.469 --> 00:53:00.869
person and keep the conversation going. See you next time.