Safer Agent Automation with GitHub Agentic Workflows

1
00:00:00,000 --> 00:00:02,800
Bret: Say something's gone wrong in
your repository, you don't wanna have

2
00:00:02,800 --> 00:00:05,260
to get up in the morning, and say,
oh my God, something's gone wrong.

3
00:00:05,260 --> 00:00:07,630
Should I have coffee or
should I investigate it?

4
00:00:07,630 --> 00:00:09,710
You shouldn't have to make that
trade-off because the agent should

5
00:00:09,710 --> 00:00:10,940
have already investigated it for you.

6
00:00:11,140 --> 00:00:14,403
So you should be able to just go to
your breakfast, have your coffee, read

7
00:00:14,403 --> 00:00:18,666
the report, read the analysis, and
it says, here's a possible fix for it.

8
00:00:18,666 --> 00:00:20,226
Here's actually meta pool request for it.

9
00:00:20,636 --> 00:00:22,886
And you go, oh yeah man,
uh, we, we gotta fix that.

10
00:00:22,886 --> 00:00:23,586
Let's get that in.

11
00:00:23,586 --> 00:00:25,496
it should be there, ready for you.

12
00:00:25,496 --> 00:00:29,026
the agent should be proactive
and should be immersed in a

13
00:00:29,026 --> 00:00:30,526
world of cooperative agents.

14
00:00:30,526 --> 00:00:34,067
Bret (2): Welcome to the Agentic DevOps
podcast, and I am your host, Bret Fisher,

15
00:00:34,167 --> 00:00:38,887
back with another episode about one of my
most exciting things that I'm working on

16
00:00:38,887 --> 00:00:45,477
this year, the project that I am adopting
on GitHub, and I am trying to dig into

17
00:00:45,477 --> 00:00:49,907
the weeds as much as possible because
I think that GitHub is onto something.

18
00:00:49,939 --> 00:00:54,097
Specifically GitHub Next and Microsoft
Research, which are both the research arms

19
00:00:54,097 --> 00:00:58,427
of both those organizations, and they're
working together to evolve GitHub Actions

20
00:00:58,867 --> 00:01:02,367
to what it would be if it was AI native.

21
00:01:02,367 --> 00:01:08,027
What would it be if AI was there in a
safe and reproducible way that we could

22
00:01:08,077 --> 00:01:15,136
sandbox and protect in a, in a very
detailed and heavily scrutinized way?

23
00:01:15,397 --> 00:01:16,607
And it's not what you think.

24
00:01:16,617 --> 00:01:21,207
It is not simply just adding
LLM prompts into GitHub Actions,

25
00:01:21,207 --> 00:01:23,157
which you could do today, and
you've been able to do for years.

26
00:01:23,177 --> 00:01:26,587
It's not simply just adding Claude
Code as a step in your GitHub Action.

27
00:01:26,967 --> 00:01:28,387
Those are things that already existed.

28
00:01:28,397 --> 00:01:33,377
But out of GitHub Next last year,
we heard about the early alpha beta

29
00:01:33,397 --> 00:01:37,477
release of something called Agentic
Workflows, which is technically what I

30
00:01:37,477 --> 00:01:42,347
would call a feature of GitHub Actions,
but it's a whole website now with tons

31
00:01:42,347 --> 00:01:44,777
of examples, a team working on it.

32
00:01:45,067 --> 00:01:49,467
And when you really dig into the details
of what this is, I think this is the

33
00:01:49,507 --> 00:01:54,967
only way we should be doing anything
with an AI inside of GitHub Actions.

34
00:01:54,977 --> 00:01:58,947
If we are prompting in GitHub
Actions, they should be using

35
00:01:58,987 --> 00:02:00,227
this tool, in my opinion.

36
00:02:00,267 --> 00:02:04,167
As I dig more into this, and then
the recent Claude Code security

37
00:02:04,407 --> 00:02:07,457
concern that we had, and then all
of the security concerns we've had

38
00:02:07,457 --> 00:02:11,257
around GitHub Actions lately related
specifically not to the supply chain

39
00:02:11,257 --> 00:02:16,607
per se, but specifically to workflows
that are getting basically prompt

40
00:02:16,607 --> 00:02:19,587
injected through untrusted prompt input.

41
00:02:19,627 --> 00:02:23,267
That is one of the biggest risks anywhere
we put a model, whether it's in a

42
00:02:23,267 --> 00:02:28,077
chatbot or it's in our automation or
it's in front of some of our systems.

43
00:02:28,537 --> 00:02:35,757
If someone can put untrusted text in a
place that somehow doesn't get verified

44
00:02:35,767 --> 00:02:39,837
by a trusted member of our team before
it goes to a model, that's a risky

45
00:02:39,837 --> 00:02:41,897
place to be, and that's part of our job.

46
00:02:41,907 --> 00:02:44,727
Platform engineers, DevOps,
security engineers, like we're all

47
00:02:44,737 --> 00:02:45,877
very concerned about that, right?

48
00:02:46,131 --> 00:02:49,831
We're maybe a little bit trepidatious
on what we should be doing with these

49
00:02:49,841 --> 00:02:54,011
things today, which is automating more
of our systems because we don't wanna be

50
00:02:54,011 --> 00:02:58,881
that person on the team that gets caught
putting risky AI stuff into automation.

51
00:02:58,901 --> 00:03:03,611
'Cause as we know, automation
can make good things really easy.

52
00:03:03,611 --> 00:03:06,721
It can reduce toil, but it can also
automate the bad things if we're not

53
00:03:06,721 --> 00:03:10,891
careful, and you can sometimes automate
yourself all the way into an outage.

54
00:03:11,039 --> 00:03:14,689
So we have on the show this
time Don from GitHub Next.

55
00:03:14,689 --> 00:03:19,229
That's their research arm full of PhDs
and experts that are trying to figure

56
00:03:19,229 --> 00:03:20,889
out tooling for the future of GitHub.

57
00:03:21,149 --> 00:03:23,139
Personally, I just love GitHub Next.

58
00:03:23,139 --> 00:03:26,489
I'm always on their website,
githubnext.com, and looking at what

59
00:03:26,499 --> 00:03:29,839
they're working on because to me it's
like reading the tea leaves of where the

60
00:03:29,849 --> 00:03:34,289
big money is researching for the future
of software development life cycle.

61
00:03:34,643 --> 00:03:39,063
And we also have Pele from Microsoft
Research, who's also on the team building

62
00:03:39,063 --> 00:03:42,833
this product And using it daily, which
we get into exactly how they use it.

63
00:03:43,073 --> 00:03:47,673
And I'm really excited because I feel
like this system has a lot of rigor

64
00:03:47,683 --> 00:03:49,953
to it, which is what I'm looking for.

65
00:03:49,973 --> 00:03:53,453
And when I'm thinking about
implementing AI anywhere in my

66
00:03:53,493 --> 00:03:58,973
automation, especially when it comes
to CI/CD automation, possibly anything

67
00:03:58,973 --> 00:04:03,433
around my code repos, I'm wanting
that to be as secure as possible.

68
00:04:03,443 --> 00:04:07,223
So I look to make my steps deterministic
with traditional programming, and then

69
00:04:07,243 --> 00:04:12,823
only as a last resort when I need a
judgment or a, maybe a consolidation

70
00:04:12,853 --> 00:04:16,753
of text or a summary of text, that's
when I consider putting in models.

71
00:04:16,903 --> 00:04:21,193
And I used to just attach Claude Code,
maybe build it a workflow around that,

72
00:04:21,193 --> 00:04:25,823
or attach Codex or Copilot as a step,
but this is something totally different.

73
00:04:25,833 --> 00:04:30,183
You use command line tools to generate
something that is a very long workflow.

74
00:04:30,193 --> 00:04:35,023
You use CLI tools to create a lock file
to make sure that that doesn't change.

75
00:04:35,273 --> 00:04:39,273
You establish rigor around making
sure that the prompts and the

76
00:04:39,283 --> 00:04:43,073
things that you're trying to create
with AI are trustworthy and can be

77
00:04:43,073 --> 00:04:45,543
sandboxed and protected properly.

78
00:04:45,753 --> 00:04:50,073
And I feel like the depths that I've
gone into after this show that we're

79
00:04:50,073 --> 00:04:54,943
gonna get into in a second, I feel
like I now can say this is how I'm only

80
00:04:54,943 --> 00:04:56,303
gonna be doing the AI in GitHub Actions.

81
00:04:56,313 --> 00:04:59,553
So welcome to the show Don
and Pele, and let's get into

82
00:04:59,553 --> 00:04:59,663
it.

83
00:04:59,682 --> 00:05:01,712
Bret: Don tell us who you
are and how you got here.

84
00:05:01,812 --> 00:05:02,982
Don: Bret, thank you so much.

85
00:05:02,982 --> 00:05:06,052
the introduction is through our
mutual friend, Ben, who works as a

86
00:05:06,102 --> 00:05:09,732
product director at GitHub Actions,
and yeah, we are having a blast.

87
00:05:09,732 --> 00:05:13,552
I work at GitHub Next, and I am now
working on, I guess, what we'd call

88
00:05:13,592 --> 00:05:18,422
ai, agentic DevOps so continuous
ai, agentic software, automated

89
00:05:18,422 --> 00:05:21,412
agentic, software engineering,
and lots of associated topics.

90
00:05:21,722 --> 00:05:25,302
And yeah, I got a long background in
kind of programming language design

91
00:05:25,302 --> 00:05:29,312
and runtime design and done a lot
of product delivery over the years

92
00:05:29,312 --> 00:05:30,992
i- in Microsoft and other places.

93
00:05:31,092 --> 00:05:35,002
A lot of DevOps along the way, and
yeah, I've been at GitHub Next a few

94
00:05:35,002 --> 00:05:37,102
years now, and I absolutely love it.

95
00:05:37,152 --> 00:05:39,892
Yeah, we got a charter to investigate
the frontier of software development,

96
00:05:39,892 --> 00:05:42,932
and there's never been a more exciting
time to Mm-hmm … be doing that.

97
00:05:43,012 --> 00:05:47,132
A time where things are so nascent
and things are in formation and in

98
00:05:47,132 --> 00:05:51,032
change and, turning upside down,
And you're really able to kind of

99
00:05:51,142 --> 00:05:53,652
radically rethink a lot of things.

100
00:05:53,652 --> 00:05:56,352
I mean, a lot of it's, turns
your head upside down, just

101
00:05:56,352 --> 00:05:57,742
how much things are changing.

102
00:05:57,742 --> 00:05:59,902
But, you know, I, I'm
enjoying that very much.

103
00:05:59,932 --> 00:06:00,052
It's

104
00:06:00,352 --> 00:06:01,662
Bret: a good moment to be here, I think.

105
00:06:01,742 --> 00:06:05,552
I keep reminding myself, I don't know, a
couple times a week, it feels like that

106
00:06:05,652 --> 00:06:09,142
just reflect on the fact that you're in
the middle of this and that, like, when

107
00:06:09,142 --> 00:06:13,232
you look back on it, these are the stories
we will tell and like, be present because.

108
00:06:13,332 --> 00:06:17,112
When I look back at the cloud and the
pc, the mainframe, to PC migration, and

109
00:06:17,112 --> 00:06:20,402
like all the sort of large infrastructure
evolutions that I've been a part of,

110
00:06:20,502 --> 00:06:23,292
you know, when you're in the middle
of it, it's, I was too young to

111
00:06:23,292 --> 00:06:25,372
realize that it was of significance.

112
00:06:25,442 --> 00:06:28,112
and as I've grown older, I've started
to see the patterns and so I can

113
00:06:28,112 --> 00:06:29,802
sort of feel like I, this is it.

114
00:06:29,802 --> 00:06:30,542
This is a thing.

115
00:06:30,642 --> 00:06:31,422
It's, this is exciting.

116
00:06:31,422 --> 00:06:33,962
this will not happen, this
will not be like this forever.

117
00:06:34,002 --> 00:06:36,502
and we will look back on
this as, wow, that was crazy.

118
00:06:36,602 --> 00:06:37,132
so Yeah.

119
00:06:37,162 --> 00:06:37,282
yeah.

120
00:06:37,282 --> 00:06:37,402
Yeah.

121
00:06:37,432 --> 00:06:37,712
Welcome.

122
00:06:37,712 --> 00:06:38,572
I'm glad to have you here.

123
00:06:38,772 --> 00:06:39,732
Peli: my name is Philippe Duhaleu.

124
00:06:39,732 --> 00:06:42,842
I'm a engineer in Microsoft
Research in a group called

125
00:06:42,942 --> 00:06:44,622
Research in Software Engineering.

126
00:06:44,622 --> 00:06:49,022
We're also very interested in things
like verification testing, uh, all

127
00:06:49,022 --> 00:06:50,922
this kind of stuff over the years.

128
00:06:51,022 --> 00:06:57,102
And I had been looking at, LLM Automation
back in the days before agents, and

129
00:06:57,102 --> 00:06:59,442
got in contact with Don in GitHub Next.

130
00:06:59,442 --> 00:07:02,862
And, we decided to kind of work
together on this idea of continuous ai.

131
00:07:02,962 --> 00:07:06,552
This kind of started this project,
and that was a year ago, roughly.

132
00:07:06,652 --> 00:07:11,822
Uh, but we had been working with
Don from a distance from Microsoft

133
00:07:11,922 --> 00:07:14,432
on various projects It's--
we've been around for a while.

134
00:07:14,532 --> 00:07:14,922
Yeah.

135
00:07:15,052 --> 00:07:16,292
yeah, so that's the intro.

136
00:07:16,632 --> 00:07:19,722
I've been working on developer
tools for professionals, but

137
00:07:19,722 --> 00:07:21,252
also for kids for a while.

138
00:07:21,732 --> 00:07:23,942
I've built, K12 coding platforms.

139
00:07:24,242 --> 00:07:24,702
Bret: Nice.

140
00:07:25,052 --> 00:07:25,122
Yeah.

141
00:07:25,122 --> 00:07:29,452
Is there an analogy between a K12,
learning platform and agent harnesses?

142
00:07:29,452 --> 00:07:29,952
It is, it

143
00:07:29,952 --> 00:07:30,162
is

144
00:07:30,212 --> 00:07:33,352
Peli: There's a lot of things that people
don't realize in the way it's designed.

145
00:07:33,352 --> 00:07:36,352
It is designed as a sandbox,
just like I build the coding

146
00:07:36,352 --> 00:07:37,972
infrastructure for Minecraft.

147
00:07:38,072 --> 00:07:41,312
Mm. So when kids learn to code in
Minecraft, we, so there are some design

148
00:07:41,312 --> 00:07:43,072
patterns that, that are applicable.

149
00:07:43,172 --> 00:07:46,242
Uh, you know, these agents
are finicky little monsters.

150
00:07:46,292 --> 00:07:50,162
so we gotta talk a lot about kind
of sandbox design and API design and

151
00:07:50,252 --> 00:07:53,022
things we do under the hood to make
it more reliable when people say,

152
00:07:53,022 --> 00:07:54,622
"I know, agents aren't reliable.

153
00:07:54,722 --> 00:07:55,042
or- Right.

154
00:07:55,042 --> 00:07:57,912
So we do a lot of work under
the hood to remove that.

155
00:07:58,082 --> 00:07:59,792
But also we talk a lot about safety.

156
00:07:59,892 --> 00:08:03,002
' cause when we have a real sandbox, you
can't just experiment You can build

157
00:08:03,002 --> 00:08:05,022
castles, without destroying the world.

158
00:08:05,022 --> 00:08:06,782
So there's a lot of that is baked in.

159
00:08:06,782 --> 00:08:10,132
but it's made for professional, it's
made for DevOps, it's made for people

160
00:08:10,132 --> 00:08:12,012
who are actions users and stuff.

161
00:08:12,052 --> 00:08:12,112
Yeah.

162
00:08:12,412 --> 00:08:16,302
But yeah, it's uh, actually, it happened,
I didn't realize I, I was designing a

163
00:08:16,302 --> 00:08:20,522
system like that and then after the fact
it's like, whoa, this kind of feels like

164
00:08:20,622 --> 00:08:24,122
I've rebuilt a system that I've built
for so many times for that environment.

165
00:08:24,172 --> 00:08:24,362
yeah.

166
00:08:24,462 --> 00:08:24,722
Bret: Okay.

167
00:08:24,752 --> 00:08:27,272
So it like it became apparent
after it was happening, yes.

168
00:08:27,632 --> 00:08:31,762
and I, I should back up because I think,
uh, we got connected because I heard

169
00:08:31,762 --> 00:08:36,262
about Agentic workflows last year,
when it was before, GitHub Universe.

170
00:08:36,612 --> 00:08:39,662
And it was like early beta
or maybe even before that.

171
00:08:40,102 --> 00:08:44,912
And I think I was like requesting early
access to get in because it was right up

172
00:08:44,912 --> 00:08:49,512
my alley and I had no idea what it was
and just knew that, uh, at the time I was

173
00:08:49,512 --> 00:08:53,932
building a GitHub Actions course, still
am recording videos, uh, week by week.

174
00:08:53,962 --> 00:08:58,472
And I've been a big GitHub Actions fan
for, gosh, at least five or six years,

175
00:08:58,582 --> 00:09:01,952
since before we had reusable workflows
and a lot of the niceties we have

176
00:09:01,952 --> 00:09:05,942
nowadays ' cause I really kind of saw
that as the evolution of the everyone's

177
00:09:05,942 --> 00:09:10,912
CI almost kind of what Jenkins represented
in the industry for so long, where it

178
00:09:10,912 --> 00:09:15,802
was sort of the default and I was so
anxious for the, another winner to,

179
00:09:15,802 --> 00:09:18,822
to replace Jenkins 'cause I was tired
of maintaining the infrastructure.

180
00:09:18,922 --> 00:09:23,192
And then, you know, got onto GitHub
Actions, saw the advantages of having it

181
00:09:23,192 --> 00:09:27,122
built right into where our code storage
was and where everything else was.

182
00:09:27,122 --> 00:09:29,952
And that just really fit, I
think with a lot of my clients

183
00:09:29,952 --> 00:09:31,432
and a lot of my community.

184
00:09:31,532 --> 00:09:34,242
And so over the years, uh,
GitHub Next, which we should

185
00:09:34,242 --> 00:09:35,462
probably talk about what that is.

186
00:09:35,882 --> 00:09:38,752
I've always looked at GitHub Next, you
know, a couple times a year to try to see,

187
00:09:38,752 --> 00:09:43,282
try to re- see the tea leaves that you all
are reading and figure out where, what's

188
00:09:43,282 --> 00:09:46,482
next for GitHub, what's the exciting
thing that might come to fruition?

189
00:09:46,482 --> 00:09:49,862
Whether it's a font, uh, font
package, which is my favorite fonts

190
00:09:49,862 --> 00:09:54,052
that I use everywhere today or, some
feature in the UI of GitHub that

191
00:09:54,052 --> 00:09:58,362
was considered a new experiment in
how we represent the information.

192
00:09:58,362 --> 00:10:00,232
And it's been fun to watch that.

193
00:10:00,232 --> 00:10:02,532
'cause y- you feel like you're
kind of seeing the skunk works

194
00:10:02,532 --> 00:10:06,052
or the, like the, the deep think
of an organization, in public.

195
00:10:06,052 --> 00:10:08,312
And that's been a really cool thing that
I don't think a lot of people know about.

196
00:10:08,312 --> 00:10:10,682
Obviously Microsoft research is the
same thing that's been around for

197
00:10:10,682 --> 00:10:14,552
decades, but, um, uh, it was cool
to see that so close to the coding

198
00:10:14,552 --> 00:10:16,062
platforms that we're all using every day.

199
00:10:16,402 --> 00:10:19,362
So last year, if I'm setting
this up correctly, there was an

200
00:10:19,362 --> 00:10:22,672
announcement around GitHub Agentic
workflows as an idea, and then

201
00:10:22,672 --> 00:10:24,352
you announced it at Universe,

202
00:10:24,432 --> 00:10:27,252
And then since then, it, I feel like it's
been like off to the races, incredible

203
00:10:27,252 --> 00:10:31,812
documentation, slide decks that are
available that you all are putting out.

204
00:10:31,812 --> 00:10:34,412
and there's a lot of information that
I think has to come along with this

205
00:10:34,412 --> 00:10:39,742
because it's not an obvious evolution
necessarily of how it's not just like

206
00:10:39,742 --> 00:10:42,392
one little feature added to a workflow.

207
00:10:42,422 --> 00:10:46,372
So who wants to take up the mantle
of describing, elevator pitch,

208
00:10:46,372 --> 00:10:48,722
the agentic workflow concept?

209
00:10:49,022 --> 00:10:51,702
Don: I mean, I'll just run through
the kind of three principles, right?

210
00:10:51,702 --> 00:10:54,162
One is the idea of agentic
repository automation.

211
00:10:54,252 --> 00:10:58,292
You know, uh, we, the… when people
talk about agents, I think there's been

212
00:10:58,292 --> 00:10:59,542
something missing in the conversation.

213
00:10:59,812 --> 00:11:00,062
Okay.

214
00:11:00,062 --> 00:11:02,422
Which is about, you want
agents that are proactive.

215
00:11:02,522 --> 00:11:04,022
That's what I think of
when I think of an agent.

216
00:11:04,052 --> 00:11:05,842
I don't know where the idea
came in around the world.

217
00:11:05,842 --> 00:11:08,872
That agent was something you invoked
from your chat session, right?

218
00:11:09,372 --> 00:11:10,262
I don't know who invented that.

219
00:11:10,262 --> 00:11:12,252
because when I think of the
word agent, I think of something

220
00:11:12,332 --> 00:11:13,942
that kind of is there around me.

221
00:11:13,942 --> 00:11:14,502
It does stuff.

222
00:11:14,502 --> 00:11:16,972
It knows the, it knows of What's
happening in the world around me.

223
00:11:16,972 --> 00:11:19,452
And in the context of a repository,
that means it kind of knows what's

224
00:11:19,452 --> 00:11:20,522
happening in the repository.

225
00:11:20,522 --> 00:11:26,192
And, when I think about things like, I
want to, refactor my code so that all

226
00:11:26,192 --> 00:11:29,392
the files are under a certain size, I
wanna check, uh, my error messages to

227
00:11:29,392 --> 00:11:32,302
check the language is age-appropriate
for a particular target audience,

228
00:11:32,302 --> 00:11:35,572
or hundreds and hundreds of other
things I wanna do in my repository.

229
00:11:35,872 --> 00:11:37,902
I, those are things I
wanna do continuously.

230
00:11:38,322 --> 00:11:40,952
They're not something I
wanna just do once, okay.

231
00:11:41,012 --> 00:11:44,402
I don't wanna have to stand up
every day and kind of say, oh

232
00:11:44,402 --> 00:11:45,852
my, we had some error messages.

233
00:11:45,852 --> 00:11:48,012
Gotta re reestablish
that kind of principle.

234
00:11:48,012 --> 00:11:50,372
that the error messages are all
in, in good shape or whatever.

235
00:11:50,792 --> 00:11:54,632
You know, A GitHub repository
fundamentally is a continuous growing,

236
00:11:54,722 --> 00:11:57,522
evolving, collaborative kind of space.

237
00:11:57,582 --> 00:12:01,792
And it, it's got a history that's very
interesting and long from wherever

238
00:12:01,792 --> 00:12:03,772
it came from, and it's got future.

239
00:12:03,822 --> 00:12:07,572
and the agents, uh, the, anything
that happens in the repository

240
00:12:07,602 --> 00:12:11,782
has to be established to
happen on a continuous basis.

241
00:12:11,822 --> 00:12:15,462
It's gotta be able to be adaptive to
what- to the change that's happening in

242
00:12:15,462 --> 00:12:17,212
the, in, in the, re- in the repository.

243
00:12:17,312 --> 00:12:19,812
If you're gonna have AI working
in a pull request is also a

244
00:12:19,812 --> 00:12:20,992
continuous kind of object.

245
00:12:20,992 --> 00:12:22,942
You know, it's gotta kind of go with you.

246
00:12:23,052 --> 00:12:25,162
the journey all the way through
the repository's a journey.

247
00:12:25,162 --> 00:12:29,542
And we've been digging, to me
that's the heart of what GitHub is.

248
00:12:29,582 --> 00:12:32,592
it's like that's just central.

249
00:12:32,592 --> 00:12:36,142
That's why it's the place where
everybody comes to work together.

250
00:12:36,142 --> 00:12:38,912
It's why continuous integration
and continuous, uh, deployment,

251
00:12:38,912 --> 00:12:40,562
make sense there and.

252
00:12:40,685 --> 00:12:45,695
So if you're gonna do agentic working, you
wanna establish it on a continuous basis.

253
00:12:45,695 --> 00:12:46,805
we all know the examples.

254
00:12:46,805 --> 00:12:49,635
here, continuous documentation,
continuous code improvement,

255
00:12:49,735 --> 00:12:53,375
continuous fault analysis,
proactive, kind of fault analysis.

256
00:12:53,845 --> 00:12:54,815
You don't wanna have to,

257
00:12:54,915 --> 00:12:57,965
Bret: say something's gone wrong in
your repository, you don't wanna have

258
00:12:58,035 --> 00:13:00,915
to wait, you don't wanna have to kind
of get up in the morning, and say,

259
00:13:00,975 --> 00:13:02,425
oh my God, something's gone wrong.

260
00:13:02,525 --> 00:13:05,145
Should I have coffee or
should I investigate it?

261
00:13:05,275 --> 00:13:05,535
Don: Right.

262
00:13:05,535 --> 00:13:07,615
Bret: You shouldn't have to make that
trade-off because the agent should

263
00:13:07,615 --> 00:13:08,845
have already investigated it for you.

264
00:13:09,045 --> 00:13:12,005
'cause you, the flow is to
investigate a fault in CI or a

265
00:13:12,005 --> 00:13:13,765
fault in your website or whatever.

266
00:13:13,765 --> 00:13:15,685
there's where the signal,
the fault signals are coming

267
00:13:15,685 --> 00:13:17,785
from the flows are encodable.

268
00:13:17,785 --> 00:13:19,345
you know, all the steps you wanna do.

269
00:13:19,345 --> 00:13:22,595
You've probably written them out in your
Mm …kind of documentation and so on.

270
00:13:22,935 --> 00:13:23,365
So

271
00:13:23,695 --> 00:13:26,185
you should be able to just go to
your breakfast, have your coffee,

272
00:13:26,225 --> 00:13:28,075
read the report, read the analysis,

273
00:13:28,155 --> 00:13:28,465
Don: uh,

274
00:13:28,465 --> 00:13:31,435
Bret: and you know, it says,
here's a possible fix for it.

275
00:13:31,435 --> 00:13:32,995
Here's actually meta pool request for it.

276
00:13:33,405 --> 00:13:35,655
And you go, oh yeah man,
uh, we, we gotta fix that.

277
00:13:35,655 --> 00:13:36,355
Let's get that in.

278
00:13:36,355 --> 00:13:36,625
Right?

279
00:13:36,625 --> 00:13:38,635
So it should be there, ready for you.

280
00:13:38,715 --> 00:13:42,635
the agent should be proactive
and should be immersed in a

281
00:13:42,635 --> 00:13:44,135
world of cooperative agents.

282
00:13:44,135 --> 00:13:45,815
Don: And when you think about
that kind of vision, there's,

283
00:13:45,955 --> 00:13:48,415
that's quite close to the idea of.

284
00:13:48,515 --> 00:13:52,975
uh, Assistance in your repository,
sort of a virtual team in a way.

285
00:13:52,975 --> 00:13:53,685
They're not humans.

286
00:13:53,685 --> 00:13:53,835
Okay.

287
00:13:53,835 --> 00:13:55,945
We, I don't like the
anthropomorphization, but they,

288
00:13:56,155 --> 00:13:56,705
Peli: yeah, it's hard

289
00:13:56,705 --> 00:13:56,875
not

290
00:13:56,875 --> 00:13:57,155
Don: to though.

291
00:13:57,155 --> 00:13:57,515
they're gonna

292
00:13:57,515 --> 00:13:57,585
Bret: be

293
00:13:57,765 --> 00:13:58,055
Don: It's hard.

294
00:13:58,055 --> 00:13:58,485
Bret: to, yeah.

295
00:13:58,585 --> 00:13:58,815
Don: Yeah.

296
00:13:58,815 --> 00:14:05,395
And, and there's so many times
in my working life I've wanted

297
00:14:05,695 --> 00:14:07,955
assistance in various repositories.

298
00:14:07,985 --> 00:14:11,445
I think, there's so much, you know,
I, I, there's, I don't know how to

299
00:14:11,445 --> 00:14:14,905
do really good engineering uh, in,
in various dimensions, in every

300
00:14:14,905 --> 00:14:16,695
repository I have to kind of touch.

301
00:14:16,725 --> 00:14:20,525
Uh, so bringing in this
proactive assistance.

302
00:14:20,955 --> 00:14:21,195
I know.

303
00:14:21,295 --> 00:14:24,035
And so, when you think
about it, where are we?

304
00:14:24,035 --> 00:14:25,135
Let's make that real.

305
00:14:25,135 --> 00:14:28,835
We're in, sitting in 2025,
and let's make that real.

306
00:14:28,835 --> 00:14:30,655
What's the ideal experience?

307
00:14:30,655 --> 00:14:34,325
What's the most, let's platform
fit that onto GitHub as a concept.

308
00:14:34,355 --> 00:14:37,625
And you go Actions actually, yeah.

309
00:14:37,625 --> 00:14:39,105
Actions has got a lot of trouble.

310
00:14:39,265 --> 00:14:39,655
Yeah.

311
00:14:39,655 --> 00:14:42,405
We, we, you know, you start thinking,
well, we automatically generate

312
00:14:42,405 --> 00:14:45,735
some big complex YAML, which kind
of is like compilation of some

313
00:14:45,735 --> 00:14:46,785
intent or something like that.

314
00:14:46,785 --> 00:14:49,885
And then you go, then you realize
actually, no, I just wanna run

315
00:14:50,105 --> 00:14:53,495
those amazing coding agents, which
will also appear around that time.

316
00:14:53,865 --> 00:14:58,105
I just wanna run them in actions
on a kind of continuous basis.

317
00:14:58,175 --> 00:14:58,425
Okay.

318
00:14:58,525 --> 00:15:01,735
And what would be the ideal
simple, let's make that simple.

319
00:15:01,735 --> 00:15:04,025
Let's make that as simple as possible.

320
00:15:04,285 --> 00:15:08,015
That would be Check in a markdown
file, which describes your prompts.

321
00:15:08,115 --> 00:15:12,955
and It runs and you have, check and it
and give some triggers like in actions.

322
00:15:13,055 --> 00:15:14,115
Everything just works.

323
00:15:14,145 --> 00:15:18,195
That's the kind of core idea of
GitHub agentic workflows is to make

324
00:15:18,335 --> 00:15:22,195
it, to capture the simplicity, the
beautiful simplicity of GitHub Actions.

325
00:15:22,195 --> 00:15:24,655
YAML, you know, people have got all
sorts of complaints about GitHub

326
00:15:24,655 --> 00:15:28,345
Actions, and YAML and so on, but
it's got a massive advantage, which

327
00:15:28,345 --> 00:15:30,655
is, it is damn really simple to use.

328
00:15:31,035 --> 00:15:35,355
You just check in a file
and you have magic, right?

329
00:15:35,515 --> 00:15:35,685
Peli: Yeah.

330
00:15:35,715 --> 00:15:39,965
Don: And that's uh, existing's ya
YAML A- and one way that people put

331
00:15:39,965 --> 00:15:44,805
it, I think Ben puts it this way, is
that actions is a way for a- average

332
00:15:44,805 --> 00:15:50,355
developers repository maintainers to
get access to some cloud resources.

333
00:15:50,385 --> 00:15:53,535
They get access to cloud compute
networking and some storage

334
00:15:53,605 --> 00:15:54,995
in the context of a repo.

335
00:15:55,095 --> 00:15:58,505
And they don't have to go ask the
security team a permission grant

336
00:15:58,505 --> 00:16:02,505
has been given by the company that
decides to let you use Git, GitHub,

337
00:16:02,505 --> 00:16:04,225
or your open source repositories.

338
00:16:04,605 --> 00:16:08,055
You get this amazing playground
and it's not just a… It's a, it's

339
00:16:08,055 --> 00:16:10,225
actually like a factory space, right?

340
00:16:10,305 --> 00:16:15,035
It's actually like, it's got storage, it's
got network, it's got so much compute, and

341
00:16:15,035 --> 00:16:20,005
these days it's now we're bringing kind of
coding agents into that kind of factory.

342
00:16:20,105 --> 00:16:22,505
And, uh, so that's the simplicity of it.

343
00:16:22,505 --> 00:16:25,235
You're running through the kind of,
uh, add wizard kind of thing here,

344
00:16:25,235 --> 00:16:28,735
which is kind of, uh, saying, you know,
yeah, helping you, uh, set up your

345
00:16:28,735 --> 00:16:32,245
tokens as well, uh, in, in the video.

346
00:16:32,245 --> 00:16:35,765
And it's, it, it says, would you
also like to trigger the workflow?

347
00:16:35,865 --> 00:16:40,955
And then bang, you've got your first
repository, automation running, and, uh,

348
00:16:41,045 --> 00:16:43,405
that thing I think is set up to run daily.

349
00:16:43,895 --> 00:16:47,475
Uh, you can kind of change it to run,
weekly on different kind of schedules,

350
00:16:47,505 --> 00:16:48,965
uh, different triggers and so on.

351
00:16:49,065 --> 00:16:53,445
And bang, you've just got, you
just made your, brought the most

352
00:16:53,445 --> 00:16:58,495
powerful coding tools into your
factory, onto your factory floor.

353
00:16:58,885 --> 00:17:01,335
And the, the potential is limitless.

354
00:17:01,415 --> 00:17:01,735
Now.

355
00:17:01,965 --> 00:17:07,495
You have got all the power to automate
everything, and creative things,

356
00:17:07,495 --> 00:17:12,515
analytical things, uh, problem-solving
things, code improvement things.

357
00:17:12,615 --> 00:17:16,235
everything is set up, in, in,
in the GitHub repository now.

358
00:17:16,635 --> 00:17:16,775
Okay.

359
00:17:16,775 --> 00:17:18,325
So that's the vision repository.

360
00:17:18,325 --> 00:17:21,005
Automation with the coding
agents you know and love.

361
00:17:21,125 --> 00:17:22,175
You can use Claude.

362
00:17:22,335 --> 00:17:25,295
You can use copilot, CLI,
you can use Gemini, CLI.

363
00:17:25,385 --> 00:17:26,515
And I think we're adding some more.

364
00:17:26,615 --> 00:17:27,575
codecs as well.

365
00:17:27,665 --> 00:17:31,315
And, uh, we, we will take, uh,
proposals for new ones as well.

366
00:17:31,415 --> 00:17:31,855
So that's it.

367
00:17:31,855 --> 00:17:34,515
Repository automations with the
coding agents, you know, and love

368
00:17:34,875 --> 00:17:36,805
safely with strong guardrails.

369
00:17:36,905 --> 00:17:39,225
We'll get onto that in GitHub Actions.

370
00:17:39,685 --> 00:17:40,295
That's the formula.

371
00:17:40,395 --> 00:17:40,945
Nice and simple.

372
00:17:41,245 --> 00:17:41,775
Bret: Nice.

373
00:17:41,875 --> 00:17:42,885
That's a good elevator pitch.

374
00:17:43,145 --> 00:17:47,215
one of the challenges I think in all of,
in all, like, like just if we think of

375
00:17:47,215 --> 00:17:50,755
the entire software life cycle on GitHub,
I've always experienced this with every

376
00:17:50,755 --> 00:17:55,865
piece of software I've implemented as sort
of a DevOps ops person, is anything that

377
00:17:55,865 --> 00:18:01,015
I put in, even if its goal as a product
or a tool to, to automate something.

378
00:18:01,045 --> 00:18:04,585
Like even putting in GitHub Actions
as a thing to help me automate

379
00:18:04,585 --> 00:18:06,735
traditional, bef- before we had agents.

380
00:18:07,035 --> 00:18:10,465
There always was an addition, the,
a, a cost of toil that was added on

381
00:18:10,465 --> 00:18:12,645
top of that, that I could not escape.

382
00:18:12,745 --> 00:18:16,515
And when I teach GitHub Actions, a lot
of, you know, it's obviously there's,

383
00:18:16,515 --> 00:18:22,395
there's the day zero creation of these
YAMLs, but the day two stuff is arguably

384
00:18:22,395 --> 00:18:26,025
even more important because it rarely
gets discussed, like in, in getting

385
00:18:26,025 --> 00:18:27,585
started guides and stuff like that.

386
00:18:28,065 --> 00:18:32,435
And so over the years, GitHub Actions
has added functionality to, you know,

387
00:18:32,435 --> 00:18:37,655
reusable workflows and templates and,
the GitHub repo op opportunities there.

388
00:18:37,655 --> 00:18:41,475
And there's, there's been a certain level
of trying to manage this giant beast that

389
00:18:41,475 --> 00:18:45,535
we've all created when you have hundreds
of repos, and now that means possibly

390
00:18:45,535 --> 00:18:47,645
thousands of YAML files to manage.

391
00:18:47,645 --> 00:18:50,575
And, uh, you know, I hate to, I don't
like using the word at scale, I think

392
00:18:50,575 --> 00:18:53,115
like it's way overused and a lot of
us aren't even dealing with scale.

393
00:18:53,115 --> 00:18:54,395
We're just small teams.

394
00:18:54,895 --> 00:18:58,695
but I always felt like there was this
challenge of, oh yeah, we're gonna

395
00:18:58,695 --> 00:19:02,355
implement this CI tool for you, but you
probably need someone's, you know, half

396
00:19:02,355 --> 00:19:06,725
a half a day, a week, maybe, or more
to, you know, fix the broken workflows,

397
00:19:06,725 --> 00:19:12,175
to update the GitHub, or approve the
PRs for the action upgrades to make

398
00:19:12,175 --> 00:19:15,135
sure that the Dependabot is configured
correctly with all the latest stuff.

399
00:19:15,135 --> 00:19:17,825
And so there's a lot of
this sort of hidden toil.

400
00:19:17,855 --> 00:19:21,625
And I was gonna throw the question to you
of, do either one of you see this future

401
00:19:21,625 --> 00:19:25,975
where it, it feels like there might be
a possibility here where we can add.

402
00:19:26,075 --> 00:19:29,595
We can add features or functionality
or, or, or solve problems without

403
00:19:29,625 --> 00:19:34,365
adding additional toil, but also
maybe that toil might be managing

404
00:19:34,365 --> 00:19:38,305
the agents, like up updating the
agent skills or the configuration,

405
00:19:38,305 --> 00:19:43,205
or like, that becomes the new human
toil layer that we have to maintain.

406
00:19:43,205 --> 00:19:46,815
Or do we think that this is
the, the gain is far exceeding

407
00:19:46,815 --> 00:19:49,165
possibly the, the toil involved?

408
00:19:49,215 --> 00:19:51,385
Um, with that, what do you,
what do you think about that?

409
00:19:51,715 --> 00:19:51,865
You-

410
00:19:51,965 --> 00:19:55,145
Peli: So essentially no,
we compile down to action.

411
00:19:55,245 --> 00:19:55,435
Yeah.

412
00:19:55,495 --> 00:19:56,865
So our markdown becomes an action.

413
00:19:56,865 --> 00:20:00,795
So we inherit all the toil today, Today
we have all the toil that you have to do.

414
00:20:00,825 --> 00:20:03,715
However, we're working closely
with the GitHub action team,

415
00:20:03,795 --> 00:20:05,205
and we're looking at that toil.

416
00:20:05,235 --> 00:20:08,345
we're looking at the sources of that,
and we're looking for solution to get

417
00:20:08,345 --> 00:20:12,705
rid of these, you know, the requirement
to, oh, there's no way to push an

418
00:20:12,705 --> 00:20:14,625
action over an entire org, you know?

419
00:20:14,625 --> 00:20:14,655
Mm-hmm.

420
00:20:14,655 --> 00:20:16,235
You always have to push
files, things like that.

421
00:20:16,235 --> 00:20:17,975
It's like, can we do make this better?

422
00:20:18,075 --> 00:20:21,635
Uh, so we're looking at these
problems where you start to have

423
00:20:21,635 --> 00:20:24,705
hundreds and thousands of repos and
everything becomes a scale problem.

424
00:20:24,805 --> 00:20:29,735
so there is definitely work being there
for the action product itself to be able

425
00:20:29,735 --> 00:20:31,505
to fix those oh, you mean But in general

426
00:20:31,505 --> 00:20:31,635
sort of a

427
00:20:31,635 --> 00:20:35,145
Bret: deterministic level, like something
that's just a feature of GitHub is a

428
00:20:35,145 --> 00:20:36,665
deterministic feature of action to be just

429
00:20:37,035 --> 00:20:38,125
Peli: yeah …better at this.

430
00:20:38,195 --> 00:20:40,505
But aside from that, we can go and.

431
00:20:40,605 --> 00:20:42,965
Attack any toil and automate it, anything.

432
00:20:43,065 --> 00:20:46,695
Because now we have, we have a, we
have the ultimate reasoning hammer

433
00:20:46,785 --> 00:20:51,005
that we can just point at a problem,
just go systematically on every

434
00:20:51,005 --> 00:20:52,545
repo, do some reasoning, right?

435
00:20:52,925 --> 00:20:56,445
You can take, in fact, any deterministic
tool and wrap it into, with an

436
00:20:56,445 --> 00:20:59,685
agent and point it at any repo
and, you know, hope for the best.

437
00:21:00,015 --> 00:21:01,195
It might give you a result.

438
00:21:01,235 --> 00:21:05,355
But, let's say your Dependabot thing, it's
always slightly different, For every repo.

439
00:21:05,525 --> 00:21:08,255
But now you have an agent that actually
pretty good at dealing with that.

440
00:21:08,605 --> 00:21:13,465
So you can go and, and run campaigns
over your repos and, and fix complying

441
00:21:13,465 --> 00:21:16,655
issues and things like that, that
a deterministic tool would fail.

442
00:21:16,755 --> 00:21:20,885
These agents are able to go and, and I
think that's what you mentioned by toil.

443
00:21:20,985 --> 00:21:21,215
Yeah.

444
00:21:21,275 --> 00:21:25,475
Or I, yeah, but we also inherit Yeah
…the, the limitation of actions today.

445
00:21:25,475 --> 00:21:27,695
because, we're working on that,
but, you know, that takes time.

446
00:21:27,795 --> 00:21:28,075
but

447
00:21:28,215 --> 00:21:28,525
Don: Yeah …it's on

448
00:21:28,525 --> 00:21:28,985
Peli: our radar.

449
00:21:29,335 --> 00:21:31,265
Don: I got a different thing for the toil.

450
00:21:31,265 --> 00:21:32,415
Mm. It's kind of like, yeah.

451
00:21:32,415 --> 00:21:36,335
it does take time, take costs something to
kind of create automations that take kind

452
00:21:36,335 --> 00:21:39,955
of cost something, costs you personal time
and there's ongoing maintenance time for,

453
00:21:39,955 --> 00:21:42,125
to create existing, uh, GitHub Actions.

454
00:21:42,125 --> 00:21:42,505
YAML.

455
00:21:42,605 --> 00:21:44,245
Couple of things kind of change a bit.

456
00:21:44,275 --> 00:21:46,615
One is that because we're
dealing with coding agents, it's

457
00:21:46,615 --> 00:21:48,975
possible to create extremely um.

458
00:21:49,075 --> 00:21:51,285
You might think of it as ambiguous,
but it's actually kind of

459
00:21:51,285 --> 00:21:54,055
general, kind of, um, workflows.

460
00:21:54,515 --> 00:21:57,825
So you can kind of create, one
of my favorite workflows is to

461
00:21:57,825 --> 00:22:00,345
do with, repository maintenance
and it's called Repo Assist.

462
00:22:00,445 --> 00:22:03,465
and, um, and it's a multitask
workflow and it kind of, each day

463
00:22:03,465 --> 00:22:07,085
it's got like 11 tasks I think, and
it kind of rolls the dice each day.

464
00:22:07,165 --> 00:22:08,505
to say, Hey, what am I going to do today?

465
00:22:08,915 --> 00:22:09,145
Okay.

466
00:22:09,195 --> 00:22:10,695
So, because you don't
want it to do all of them.

467
00:22:10,695 --> 00:22:13,815
because it, you know, Yeah … only do the
first three, so it just rolls the dice.

468
00:22:13,815 --> 00:22:16,015
Just to kind of get
things nice and balanced.

469
00:22:16,115 --> 00:22:19,295
One of them is, for instance, to
label issues, to just to make sure

470
00:22:19,295 --> 00:22:23,155
all the issues are, can check most
recent issues It can look at a backlog

471
00:22:23,155 --> 00:22:26,085
and just make sure all the labels
are kind, you don't have to write

472
00:22:26,085 --> 00:22:27,875
that out algorithmically, right?

473
00:22:27,875 --> 00:22:32,105
You just kind of tell it the end state
that you want and maybe some hints about

474
00:22:32,105 --> 00:22:35,945
how to get there, about what the journey
is, it has to make and, uh, you know,

475
00:22:36,135 --> 00:22:37,925
pretty much what I just said really.

476
00:22:38,025 --> 00:22:40,695
And it'll work out all
the rest of the details.

477
00:22:40,825 --> 00:22:44,225
Uh, it, you know, in the sense that
that's the prompting that will be

478
00:22:44,225 --> 00:22:47,975
running each day or every third,
fourth, fifth day or whatever it runs.

479
00:22:48,075 --> 00:22:56,625
Uh, and, um, it sorts out the rest in any
repository for any set of labels for any.

480
00:22:57,035 --> 00:22:59,175
Language, you don't have to
work in English, you could

481
00:22:59,175 --> 00:23:00,485
be working something else.

482
00:23:00,555 --> 00:23:04,045
So whereas in a traditional setting, you
might have had to configure the exact

483
00:23:04,045 --> 00:23:08,595
labels to use, configure the kind of,
uh, the, the, the, the kind of heuristics

484
00:23:08,595 --> 00:23:10,395
to use to label issues and so on.

485
00:23:10,495 --> 00:23:14,485
In this setting, it can be
made to work everything.

486
00:23:14,915 --> 00:23:16,925
And that's really golden.

487
00:23:16,925 --> 00:23:19,725
'cause that means you have these
very generic workflows, which can

488
00:23:19,725 --> 00:23:21,605
be used in many different settings.

489
00:23:21,605 --> 00:23:25,135
I mean, other Yeah …tasks in that repo
are things like take, uh, work through the

490
00:23:25,135 --> 00:23:28,145
issues and just analyze them one by one.

491
00:23:28,245 --> 00:23:29,165
not all in one run.

492
00:23:29,165 --> 00:23:34,355
just do a bit of that frontier work, uh,
uh, and, and do a depth investigation,

493
00:23:34,355 --> 00:23:37,455
do a reproduction, and give some advice
about what to do about this issue.

494
00:23:37,555 --> 00:23:40,305
And it's sort of what you'd
do with a coding agent today.

495
00:23:40,305 --> 00:23:42,775
You might check it out locally and
investigate the kind of thing, but

496
00:23:42,775 --> 00:23:44,365
it's all done for you, proactively.

497
00:23:44,465 --> 00:23:46,715
Um, and again, it's very general, right?

498
00:23:46,715 --> 00:23:49,095
You don't have, it's the sort of thing
we could never have programmed up

499
00:23:49,095 --> 00:23:51,905
two or three years ago because it's
amazing, you know, these things Yeah

500
00:23:51,905 --> 00:23:52,675
these agents do.

501
00:23:52,775 --> 00:23:57,035
But it's also very general and that
means the burden I use repo assist

502
00:23:57,035 --> 00:23:59,715
in, I think, um, 12 different repos.

503
00:23:59,815 --> 00:24:03,435
They're all different and I haven't had
to change it really between any of them.

504
00:24:03,435 --> 00:24:07,365
So you can have these very generic,
powerful, tools which help you

505
00:24:07,365 --> 00:24:11,005
make progress, uh, expressed at
the right level, low enough that it

506
00:24:11,005 --> 00:24:12,535
actually kinda knows what it's doing.

507
00:24:12,535 --> 00:24:13,985
It doesn't just do something canned.

508
00:24:14,285 --> 00:24:16,525
but general enough, it's really
applicable to everything.

509
00:24:16,825 --> 00:24:20,975
Bret: I'm starting to see enough
teams that are tiptoeing into agents

510
00:24:20,975 --> 00:24:22,515
as, in their CI, essentially, right.

511
00:24:22,515 --> 00:24:26,855
P- Somehow, somewhere sticking a model
and writing a prompt in some fashion to

512
00:24:26,855 --> 00:24:28,075
a model, whatever we wanna call that.

513
00:24:28,545 --> 00:24:31,655
And they've, I think they, a lot
of the teams that I'm working with,

514
00:24:31,655 --> 00:24:35,825
see, the first experience is sort of,
it's a checkbox feature in GitHub.

515
00:24:35,855 --> 00:24:39,885
Like they might turn on the PR review
agent for copilot and it's sort of an o

516
00:24:40,165 --> 00:24:44,425
on/off thing, or you, you can opt in to
each PR and it just becomes a feature.

517
00:24:44,425 --> 00:24:48,285
So it doesn't really feel, I mean, even
it's providing automation and obviously

518
00:24:48,285 --> 00:24:50,765
it does, you know, it puts comments
in the PR it does all these things,

519
00:24:50,765 --> 00:24:53,765
but it, it's not something that they
have to hobby kit implement, right?

520
00:24:53,765 --> 00:24:56,575
They're not writing a YAML file
necessarily on day one of that.

521
00:24:56,965 --> 00:24:58,535
So I feel like that's the first phase.

522
00:24:58,535 --> 00:24:59,985
And the second phase is where.

523
00:25:00,375 --> 00:25:05,235
They wanna actually ha- you know, see
everybody else putting Claude, code into a

524
00:25:05,235 --> 00:25:08,595
CI run, or they, you know, they, I notice
nowadays it's actually getting pretty

525
00:25:08,595 --> 00:25:14,315
rare to look at the, uh, maintainers
or, or the contributors to a repo.

526
00:25:14,595 --> 00:25:17,365
And that Claude logo is always
there, it seems like nowadays.

527
00:25:17,685 --> 00:25:21,865
people are sort of figuring out how they
can either write code or, or review code.

528
00:25:21,965 --> 00:25:26,565
That feels almost like, for me, that
wasn't the most interesting part to me.

529
00:25:26,845 --> 00:25:30,160
So, someone who had to maintain the
CI, I was always looking for things

530
00:25:30,160 --> 00:25:33,130
that would, you know, automatically
troubleshoot a failed check, right?

531
00:25:33,130 --> 00:25:37,510
And try to provide an automated, uh,
'cause I, I'm that person who's usually

532
00:25:37,510 --> 00:25:41,630
responsible when the, the checks fail
because the dev team's gonna push back and

533
00:25:41,630 --> 00:25:43,300
say, yeah, there's a configuration issue.

534
00:25:43,300 --> 00:25:44,480
It's not our fault, blah, blah, blah.

535
00:25:44,840 --> 00:25:46,450
And so we're gonna have
that back and forth there.

536
00:25:46,920 --> 00:25:51,640
And I, I've, I've started to describe
this, because people aren't even really

537
00:25:51,640 --> 00:25:54,960
sure where to start, and I love that
the Agentic Workflows website, the

538
00:25:54,960 --> 00:25:58,450
documentation really is starting to
categorize these things into certain

539
00:25:58,450 --> 00:26:01,990
areas where you're finding success
and you're seeing the good metrics

540
00:26:01,990 --> 00:26:03,330
coming out of tho- those results.

541
00:26:03,430 --> 00:26:07,410
But I've often tried to describe it
to them as just find a place where

542
00:26:07,410 --> 00:26:09,560
there's huge human judgment involved.

543
00:26:09,660 --> 00:26:14,250
that You've previously had to involve a
human, but it wasn't a deliberate gate.

544
00:26:14,630 --> 00:26:18,500
because a lot of people get nervous
about, I think, the idea of AI in their

545
00:26:18,500 --> 00:26:21,380
CI because the first thing they might
even, l- or one of their first thoughts

546
00:26:21,380 --> 00:26:24,510
might be, well, I don't want it to
automatically deploy to production.

547
00:26:24,510 --> 00:26:25,940
Mm. And to me that's like, of course.

548
00:26:25,940 --> 00:26:26,060
Yeah.

549
00:26:26,060 --> 00:26:29,030
That's probably the last thing I'm going
to automate Mm. with any sort of, Mm.

550
00:26:29,030 --> 00:26:32,860
uh, model it, you know, that is to me,
sometimes a lot of teams, it's a manual

551
00:26:32,860 --> 00:26:37,710
gate we intentionally put there, and
so we're enforcing a human stop point.

552
00:26:37,950 --> 00:26:40,850
But there's so much other, that's like
you mentioned, labels, uh, automatically

553
00:26:40,850 --> 00:26:45,700
labeling automatically, you know, maybe
approving like low risk, patch releases

554
00:26:45,700 --> 00:26:47,280
of Dependabot updates or something.

555
00:26:47,280 --> 00:26:49,370
Like, there's probably some
low hanging fruit there.

556
00:26:49,780 --> 00:26:52,770
I certainly have been a part of teams
where we've had to implement that

557
00:26:52,770 --> 00:26:57,380
cross repo l- auto labeling, even just
synchronizing back when we didn't have it.

558
00:26:57,380 --> 00:27:01,090
Uh, synchronizing label names across
repos, you know, just a lot of sort

559
00:27:01,090 --> 00:27:04,740
of silly stuff that maybe just wasn't
a feature in the product yet and we

560
00:27:04,740 --> 00:27:08,080
were backfilling it with some sort
of manual automation that feels

561
00:27:08,080 --> 00:27:12,420
like the ripe, uh, opportunities
for, uh, people getting started.

562
00:27:12,420 --> 00:27:16,180
But do you, f- do you frame it that way of
look for opportunities where there's, y-

563
00:27:16,260 --> 00:27:20,460
human judgment that we have today, but it
we, it wasn't because we enforced a human

564
00:27:20,460 --> 00:27:24,560
to get involved, we just didn't have a way
to automate that with a, for each loop.

565
00:27:24,660 --> 00:27:26,930
Is that something that you,
you frame it that way for.

566
00:27:27,030 --> 00:27:30,610
Don: Yeah, I personally like to begin
with uh, a chat about what are the

567
00:27:30,610 --> 00:27:32,560
problems people are having in the repo?

568
00:27:32,650 --> 00:27:35,150
Like what are the actual
struggles that are happening?

569
00:27:35,210 --> 00:27:35,520
Okay.

570
00:27:35,520 --> 00:27:39,130
So, um, in the case of, repo assist,
you know, having that chat with

571
00:27:39,170 --> 00:27:43,540
myself, the problem is we've got
an issue backlog of 200 issues that

572
00:27:43,540 --> 00:27:45,170
go back years and years and years.

573
00:27:45,260 --> 00:27:47,400
And every time I come to this
repo as a maintainer, I don't

574
00:27:47,400 --> 00:27:48,060
know what to do with them.

575
00:27:48,060 --> 00:27:49,040
I don't wanna close them out.

576
00:27:49,140 --> 00:27:50,220
I, 'cause there's value there.

577
00:27:50,220 --> 00:27:52,530
I know there are bugs there, and I
don't want to leave bugs that people

578
00:27:52,530 --> 00:27:54,020
have found lying around this software.

579
00:27:54,120 --> 00:27:57,820
so, my problem is one of kind of
the burden of being a maintainer,

580
00:27:57,820 --> 00:28:00,950
the guilt in a way, you kind of, you
know, it's not, I, haven't got on top.

581
00:28:00,950 --> 00:28:02,580
I love to get on top of the repo.

582
00:28:02,580 --> 00:28:05,680
I'd love to get it meaning get
that issue count down in the below

583
00:28:05,680 --> 00:28:08,140
100, below 50, below 10, uh, and.

584
00:28:08,610 --> 00:28:14,360
So the flow is designed to help me solve
that problem and reach where I wanna be.

585
00:28:14,460 --> 00:28:17,350
So I kind of like to have the
discussions about like, what

586
00:28:17,350 --> 00:28:18,150
are you trying to achieve here?

587
00:28:18,150 --> 00:28:19,050
What are your goals?

588
00:28:19,100 --> 00:28:20,720
What are your quality goals?

589
00:28:20,720 --> 00:28:24,150
You're after, uh, you know,
is performance the top thing?

590
00:28:24,420 --> 00:28:25,830
That's your, your problem?

591
00:28:25,830 --> 00:28:31,320
is sort of, is quality and crunching
out the bugs, the problem is integration

592
00:28:31,320 --> 00:28:33,420
and kind of cross-repo kind of working.

593
00:28:33,420 --> 00:28:36,580
That's absolutely in a lot of
settings, a ki- kind of problem.

594
00:28:36,580 --> 00:28:40,390
Making things regular across
multiple repos is a good example.

595
00:28:40,390 --> 00:28:43,540
Uh, so once you start the
conversation about, tell me

596
00:28:43,660 --> 00:28:45,580
what, what's making life painful?

597
00:28:45,710 --> 00:28:49,800
What's Yeah …causing you to lose sleep,
then, what causes you to disengage from

598
00:28:49,800 --> 00:28:53,340
the repo or, whatever I don't know, Peli,
How, how do you begin conversations?

599
00:28:53,640 --> 00:28:57,090
Peli: I'm doing extreme agentic
development in the way that I'm

600
00:28:57,090 --> 00:28:59,440
at, what, 400 plus PRs a week?

601
00:28:59,740 --> 00:29:02,640
exclusively through, I mean, a-
agentic workflows is written with

602
00:29:02,640 --> 00:29:06,850
agents and the challenge is how do you
create quality software doing that?

603
00:29:06,950 --> 00:29:07,190
Yeah.

604
00:29:07,490 --> 00:29:12,750
And this involves many, many agents,
uh, looking at the generated code,

605
00:29:12,750 --> 00:29:17,560
cleaning it, adding tests, extracting
specs, generating tests from specs.

606
00:29:17,660 --> 00:29:19,190
So there's an entire.

607
00:29:19,290 --> 00:29:23,870
Intricate set of agents that are running
and that are powered by agentic workflows

608
00:29:24,370 --> 00:29:27,460
that are running in the repo and that are,
look, you know, it's really exploring,

609
00:29:27,460 --> 00:29:33,450
okay, what is, so we can, we can generate
code at a incredible rate now, but we've

610
00:29:33,450 --> 00:29:37,000
always known that creating a feature
was a tiny piece of the equation.

611
00:29:37,030 --> 00:29:41,360
You had to do test plans, you had to
design features, research, you had

612
00:29:41,360 --> 00:29:43,090
to write documentation, maintain it.

613
00:29:43,190 --> 00:29:45,780
All that stuff is ripe for automation.

614
00:29:46,140 --> 00:29:47,830
and where does the human fit in?

615
00:29:47,830 --> 00:29:50,620
Where does the engineer come
in and, you know, say no.

616
00:29:50,720 --> 00:29:53,750
In my, stat, I think 20%
of the PRs, I refuse them.

617
00:29:53,850 --> 00:29:58,710
there's still quite a bit of
engineering involvement I do maybe

618
00:29:58,710 --> 00:30:00,880
three up, three intervention per PR.

619
00:30:00,980 --> 00:30:05,230
so there's still quite a bit of steering
from the, from me, but I have a lot

620
00:30:05,230 --> 00:30:09,300
of tools that are agentic workflows
that are optimizing very specific

621
00:30:09,300 --> 00:30:14,040
angle of a code base, like reducing
code duplicates, fixing linter issues.

622
00:30:14,340 --> 00:30:17,990
And you know, I mean these are
kind of sound practices that

623
00:30:18,290 --> 00:30:20,170
software engineers have been doing.

624
00:30:20,270 --> 00:30:20,750
Um, Yeah.

625
00:30:20,850 --> 00:30:23,370
and then we go down the rabbit
hole and we're like, oh, well.

626
00:30:23,670 --> 00:30:24,910
We always fix something.

627
00:30:24,910 --> 00:30:26,290
So now we extract linters.

628
00:30:26,290 --> 00:30:29,360
So we infer linters from
our, our own practices.

629
00:30:29,480 --> 00:30:31,080
your devs are always fixing the same bugs.

630
00:30:31,280 --> 00:30:32,470
Maybe it should be a linter.

631
00:30:32,570 --> 00:30:35,490
then once you have a linter, you
have something that is very scalable

632
00:30:35,490 --> 00:30:37,080
in terms of compute and cost.

633
00:30:37,250 --> 00:30:40,150
And then, but there's, there's
something we discover with the

634
00:30:40,150 --> 00:30:44,130
agent they're very meta agents can
help, agents can generate agents.

635
00:30:44,130 --> 00:30:47,510
And agentic workflows
are an instance of that.

636
00:30:47,510 --> 00:30:49,990
They, the workflows are really
specialized tools in my mind.

637
00:30:50,090 --> 00:30:53,000
You know, they use agents and
everything's an agent, so we

638
00:30:53,000 --> 00:30:54,070
have to put different names.

639
00:30:54,070 --> 00:30:57,910
But I build tools, uh, and one of
the unblocker of this experiment

640
00:30:57,910 --> 00:31:02,040
is that I can go from an idea to a
tool in five minutes that is running

641
00:31:02,040 --> 00:31:03,190
in the CI and giving me a result.

642
00:31:03,290 --> 00:31:07,590
The first version will be trash, you
know, it will not work, it will crash.

643
00:31:07,590 --> 00:31:11,200
And then there's a self-reporting loop and
two or three iteration, you have something

644
00:31:11,200 --> 00:31:12,470
that actually creating the value.

645
00:31:12,570 --> 00:31:14,200
and then, you know, add more loops.

646
00:31:14,200 --> 00:31:16,770
You get, you start optimizing
and, and saving tokens.

647
00:31:17,070 --> 00:31:21,320
But …that's essentially the
key thing is zero friction from

648
00:31:21,320 --> 00:31:24,470
your idea to an automation that's
bringing value to your project.

649
00:31:24,770 --> 00:31:25,270
That's paramount.

650
00:31:25,370 --> 00:31:25,650
Yeah.

651
00:31:25,950 --> 00:31:30,300
Don: Yeah, the repository as the
kinda agent foundry, the agent host,

652
00:31:30,300 --> 00:31:34,660
the place where you can just create
and deploy your automation and, um,

653
00:31:34,960 --> 00:31:38,670
get it to do everything over, you
know, the fabric of the repository.

654
00:31:38,880 --> 00:31:42,590
Create issues, It can add issue
comments, It can read existing issues.

655
00:31:42,960 --> 00:31:46,160
It can create pull requests, it
can add to existing pull requests.

656
00:31:46,160 --> 00:31:48,440
Those are the kind of, uh,
discussions of the other elements

657
00:31:48,440 --> 00:31:49,320
of the information fabric.

658
00:31:49,350 --> 00:31:52,180
But that's, and it can read the
security reports, it can look at the

659
00:31:52,180 --> 00:31:54,060
actions, it can look at the CI runs.

660
00:31:54,160 --> 00:31:56,610
And so that's the fabric information
fabric you're working over.

661
00:31:56,710 --> 00:31:59,800
And, um, it, it makes you look at a lot
of the stuff that people are doing with

662
00:31:59,800 --> 00:32:03,130
agent harnesses differently because
you don't really need to think about

663
00:32:03,130 --> 00:32:04,770
where you put your to-to-do list.

664
00:32:04,840 --> 00:32:06,820
Let's just put it in an issue, right?

665
00:32:06,940 --> 00:32:11,160
You don't need to think, where do you
put the output, the analysis of, say

666
00:32:11,160 --> 00:32:14,740
you've got an agent, which is kind of,
uh, checking performance every, night,

667
00:32:14,800 --> 00:32:18,410
different dimensions of performance,
and running through them, uh, uh, or

668
00:32:18,410 --> 00:32:21,540
checking your, get your, your, your
getting started guide, reading your

669
00:32:21,540 --> 00:32:24,370
docs, and kind of running through the
kind of getting started material and

670
00:32:24,370 --> 00:32:26,870
making sure it's simple and making
sure everything kind of works right.

671
00:32:26,970 --> 00:32:28,950
Those basic kind of
walkthrough kind of things.

672
00:32:29,050 --> 00:32:30,690
where does it put its output.

673
00:32:31,030 --> 00:32:31,300
I don't.

674
00:32:31,350 --> 00:32:36,060
wanna have to think about deploying
this to some Agent Foundry and

675
00:32:36,060 --> 00:32:37,810
some other platform, right?

676
00:32:37,890 --> 00:32:41,250
I I, it's just gonna run in GitHub and
it's gonna write its output to GitHub then

677
00:32:41,320 --> 00:32:42,880
it's gonna create an issue or add, right?

678
00:32:42,880 --> 00:32:44,750
A comment to an existing issue.

679
00:32:44,780 --> 00:32:47,540
Uh, or it's just gonna create a pull
request and fix the the thing directly.

680
00:32:47,540 --> 00:32:49,320
Depends on the design that you want.

681
00:32:49,420 --> 00:32:54,270
And, th- so it's that it's, you mentioned
it earlier with GitHub Actions about

682
00:32:54,270 --> 00:32:57,300
how having it right there next to
your code is just a, a great thing.

683
00:32:57,380 --> 00:32:59,950
And, uh, it's the same with this.

684
00:32:59,980 --> 00:33:03,550
It's like right there on the information
fabric we all know and use it knows how

685
00:33:03,550 --> 00:33:05,420
to use the GitHub information fabric.

686
00:33:05,500 --> 00:33:06,110
really well.

687
00:33:06,450 --> 00:33:08,900
It's all about issues and how
to query, them, how to search,

688
00:33:09,010 --> 00:33:10,810
about pull requests and so on.

689
00:33:10,810 --> 00:33:13,220
Uh, so, um, I love that.

690
00:33:13,410 --> 00:33:18,560
I lo- I love that it's operating in
my home, in my, my fact, my place

691
00:33:18,590 --> 00:33:20,140
where we get work done together.

692
00:33:20,240 --> 00:33:23,400
And Yeah …I love, love that I
can dig back to the actions log

693
00:33:23,400 --> 00:33:24,570
and see exactly what happened.

694
00:33:24,690 --> 00:33:26,750
I love Right … when, when the
agents of the actions log to

695
00:33:26,750 --> 00:33:28,100
work out what went wrong, right?

696
00:33:28,310 --> 00:33:28,490
Peli: Yeah.

697
00:33:28,490 --> 00:33:30,770
The inherent every time you come
back, yeah, every time you come

698
00:33:30,770 --> 00:33:34,170
back to the graph, to GitHub,
the engineer can intervene.

699
00:33:34,270 --> 00:33:35,290
The human is back in the loop.

700
00:33:35,620 --> 00:33:35,760
Right.

701
00:33:36,040 --> 00:33:38,110
The agent does some
computation, creates an issue.

702
00:33:38,110 --> 00:33:38,140
Uh,

703
00:33:38,280 --> 00:33:38,890
Don: th- That's right.

704
00:33:38,890 --> 00:33:40,990
It gives that natural place for
the human to be in the loop.

705
00:33:40,990 --> 00:33:41,050
Yeah.

706
00:33:41,050 --> 00:33:44,760
And, and that's, that can be
so confusing when you kind of

707
00:33:44,760 --> 00:33:46,380
disassociate from the factory.

708
00:33:46,430 --> 00:33:48,220
you take it to the outside, from the repo.

709
00:33:48,320 --> 00:33:51,450
Uh, and you wonder where's the, h-
where's the human gonna be l- in the

710
00:33:51,450 --> 00:33:52,620
loop in this kind of stuff, right.

711
00:33:52,730 --> 00:33:53,470
I'm drawing out.

712
00:33:53,890 --> 00:33:55,690
And the answer's really simple right?

713
00:33:55,740 --> 00:33:56,720
it's in the pull request.

714
00:33:56,720 --> 00:33:56,920
Right?

715
00:33:56,960 --> 00:33:58,740
It's okay if the, uh, the, i- yeah.

716
00:33:58,840 --> 00:33:59,090
Okay.

717
00:33:59,130 --> 00:34:00,260
First of all, it's actions.

718
00:34:00,310 --> 00:34:03,740
What it does to the issue set of issues
has to be really tightly constrained.

719
00:34:03,740 --> 00:34:05,340
And that's where we can start
to talk about guardrails.

720
00:34:05,370 --> 00:34:06,870
You know, it's not allowed
to delete every issue.

721
00:34:06,870 --> 00:34:08,010
It's not allowed to close every issue.

722
00:34:08,010 --> 00:34:10,470
It's not allowed to comment on
every issue or randomly kind of

723
00:34:10,510 --> 00:34:12,440
write ha ha ha all over the place.

724
00:34:12,510 --> 00:34:15,240
It's got really strong limits
over what it's, what it can do.

725
00:34:15,620 --> 00:34:17,930
It can add a comment to a
single issue, for example.

726
00:34:17,960 --> 00:34:19,130
That's a super strong limit.

727
00:34:19,230 --> 00:34:19,880
Uh, okay.

728
00:34:19,880 --> 00:34:21,150
But it can do that automatically.

729
00:34:21,150 --> 00:34:23,500
it's not gonna boil the ocean
or whatever its expression to

730
00:34:23,500 --> 00:34:25,760
let it act on the issue section.

731
00:34:25,830 --> 00:34:31,000
so it's kind of n- it's a big enormous
scratch pad for these agents to work on,

732
00:34:31,180 --> 00:34:31,310
right?

733
00:34:31,310 --> 00:34:32,990
Uh, and, th- and that again,
it makes it clarifying.

734
00:34:32,990 --> 00:34:34,610
'cause you know, it's
not writing to the repo.

735
00:34:34,610 --> 00:34:39,240
It's not like writing a to do or goals.md
or tasks.md into the repo, like a lot

736
00:34:39,240 --> 00:34:42,560
of people are doing, it's very tempting
to use the repo as the scratch pad.

737
00:34:42,860 --> 00:34:44,510
Uh, But it's not gonna do that.

738
00:34:44,510 --> 00:34:47,900
Uh, uh, So it's, it's, we've got our
scratch pad that is issues and it's

739
00:34:47,900 --> 00:34:51,740
got its way of proposing forward action
in the world, which is pull requests.

740
00:34:51,840 --> 00:34:56,350
And, uh, or you can also propose
an issue or propose a pull

741
00:34:56,350 --> 00:34:57,890
request, an actual concrete change.

742
00:34:57,890 --> 00:34:58,590
It's very common.

743
00:34:58,590 --> 00:35:01,680
We get the, uh, agentic workflows
to create issues instead of

744
00:35:02,070 --> 00:35:03,270
going straight to a pull request.

745
00:35:03,270 --> 00:35:06,880
Because again, it, g- it's, it kind
of divided into a world where there's,

746
00:35:06,950 --> 00:35:11,130
it, is it, it's roughly a, like,
is it gonna take one kind of check?

747
00:35:11,160 --> 00:35:11,630
Yes.

748
00:35:11,970 --> 00:35:15,380
Do it thing in the pull request, or
does it kind of …offer choices?

749
00:35:15,380 --> 00:35:17,910
If it's offers offering choices,
then you'd better create an

750
00:35:17,970 --> 00:35:21,320
issue first, because the human
really needs some guidance.

751
00:35:21,730 --> 00:35:23,930
The agent really needs
some guidance about Right.

752
00:35:23,930 --> 00:35:25,440
What's the next step?

753
00:35:25,890 --> 00:35:30,140
But in that, the human in
the loop is really simple.

754
00:35:30,550 --> 00:35:35,450
It is just at digesting what the agent
does in the issue space, the comments

755
00:35:35,450 --> 00:35:41,090
it adds the options it gives and acting
in the pull request space to make a

756
00:35:41,130 --> 00:35:46,620
actual, big, uh, actual change progress
forward in, in the code towards the goals

757
00:35:46,650 --> 00:35:48,590
that, that everyone's pursuing together.

758
00:35:48,690 --> 00:35:49,180
I love that.

759
00:35:49,260 --> 00:35:52,120
I lo- I love that I know what
human in the loop m- means.

760
00:35:52,380 --> 00:35:52,990
I love that.

761
00:35:53,290 --> 00:35:57,460
it just very clarifying in what is,
and it's a- also something I can trust.

762
00:35:57,560 --> 00:36:00,960
if all it's gonna do is create a
pull request and just one of them

763
00:36:00,960 --> 00:36:04,520
is that, it's almost certainly
going to be useful what it creates.

764
00:36:04,800 --> 00:36:07,620
So at first like, that's great, but
it's certainly not, you know, I can

765
00:36:07,620 --> 00:36:12,120
trust, I can sleep well at night
that the thing is doing things in a

766
00:36:12,380 --> 00:36:15,670
positive direction or certainly not
in a significantly bad direction.

767
00:36:15,770 --> 00:36:18,450
So guardrails and security,
super, super important.

768
00:36:18,450 --> 00:36:21,680
Do very worried about something you said
earlier, Bret, just to be controversial.

769
00:36:21,680 --> 00:36:26,320
you were saying, run Claude
code or these, uh, coding agent

770
00:36:26,320 --> 00:36:28,610
CLIs directly in GitHub Actions.

771
00:36:28,710 --> 00:36:32,390
I understand the temptation of that
and, uh, in fact, the origins of GitHub

772
00:36:32,390 --> 00:36:36,180
agentic workflows are in that kind of
space, but it's really dangerous, right?

773
00:36:36,280 --> 00:36:38,570
The- We call that kind of running
naked, We're kind of Mm-hmm …running

774
00:36:38,570 --> 00:36:41,790
without a security architecture or
rolling your own security architecture

775
00:36:41,790 --> 00:36:44,880
or attempting to do the re-analysis
of the security architecture.

776
00:36:44,880 --> 00:36:48,980
At every time, every workflow you do,
That's really easy to make mistakes.

777
00:36:49,080 --> 00:36:50,900
And you're being exposed to inputs.

778
00:36:50,900 --> 00:36:54,350
You you've got the world's
most powerful coding tools.

779
00:36:54,410 --> 00:36:58,180
And frankly, they can be used for good
and they can be used for bad, right?

780
00:36:58,260 --> 00:37:03,790
And they are running in GitHub Actions
with potentially access to secrets under

781
00:37:03,790 --> 00:37:08,470
the direction of your, of arbitrary
people working up, walking up to the

782
00:37:08,470 --> 00:37:10,780
repo of people feeding in information.

783
00:37:10,890 --> 00:37:11,070
Okay?

784
00:37:11,070 --> 00:37:13,970
So it's a little bit like you
hire an amazing team of people.

785
00:37:14,400 --> 00:37:17,630
They're sitting in an office and you're
allowing people to walk in through the

786
00:37:17,630 --> 00:37:21,310
front security gate and just feed them
notes about what to do under the door.

787
00:37:21,800 --> 00:37:22,420
And they read them.

788
00:37:22,420 --> 00:37:23,640
They go, oh yeah, I'll do that.

789
00:37:23,760 --> 00:37:24,560
You know- And they think those are

790
00:37:24,560 --> 00:37:24,810
"I'll just do

791
00:37:24,810 --> 00:37:26,610
Bret: it" … just as
important as the boss's notes.

792
00:37:26,610 --> 00:37:26,670
Yeah.

793
00:37:26,670 --> 00:37:26,850
And they'll do

794
00:37:26,850 --> 00:37:29,850
Don: any They'll just… You know, I
mean, yeah, you can try and box it and

795
00:37:29,850 --> 00:37:31,030
try and tell it's not so important.

796
00:37:31,030 --> 00:37:35,130
And of course you do all those things,
but it's still, you kind of want the

797
00:37:35,130 --> 00:37:38,740
security guard at the desk, and you
want the security guard you particularly

798
00:37:38,740 --> 00:37:42,530
want, I mean, you want any outward
action, any write action from that

799
00:37:42,530 --> 00:37:46,630
team, any act external action in the
world, any information sent outside.

800
00:37:46,690 --> 00:37:48,960
if you're, if you're worried
about private information leaking.

801
00:37:49,070 --> 00:37:52,930
you want, it's gotta have a security
architecture, you've gotta, you, you,

802
00:37:52,960 --> 00:37:57,820
must, if you're gonna run automated
coding agents in the context of anything

803
00:37:57,820 --> 00:38:01,570
sensitive whatsoever, you must have
a security architecture full stop.

804
00:38:01,850 --> 00:38:03,850
That's Do not leave home without that.

805
00:38:03,900 --> 00:38:04,730
in any serious way.

806
00:38:04,760 --> 00:38:06,750
And- And that's what GitHub-
the security landscape

807
00:38:06,750 --> 00:38:09,910
Peli: is different than
what people think is secure.

808
00:38:10,010 --> 00:38:10,040
Mm-hmm.

809
00:38:10,210 --> 00:38:11,950
It is not just running in a container.

810
00:38:12,050 --> 00:38:12,170
Right.

811
00:38:12,170 --> 00:38:13,070
There's a lot more threat.

812
00:38:13,070 --> 00:38:14,140
These things are intelligent.

813
00:38:14,530 --> 00:38:14,830
Don: Yeah.

814
00:38:14,950 --> 00:38:17,920
So for instance, running in a
container, but in our model, we

815
00:38:17,920 --> 00:38:19,280
have some very strong guarantees.

816
00:38:19,330 --> 00:38:21,650
Uh, And, we really care about security.

817
00:38:21,650 --> 00:38:25,350
not just because we're trying to stop
people getting work done or anything, but

818
00:38:25,350 --> 00:38:29,170
because we think the better the guardrails
you have, the faster you can run.

819
00:38:29,270 --> 00:38:29,590
Okay?

820
00:38:29,590 --> 00:38:32,350
The faster the automation can go,
the better the train tracks are.

821
00:38:32,530 --> 00:38:35,940
The faster a train can go, the
more you can ramp it up, ramp

822
00:38:35,940 --> 00:38:37,560
up the TGV up to high speed.

823
00:38:37,560 --> 00:38:41,020
because you trust the rails you're on and
nothing's gonna go wrong along the way.

824
00:38:41,120 --> 00:38:45,290
And that's what, that's how it actually
works with GitHub agentic workflows.

825
00:38:45,340 --> 00:38:48,140
Now, let's just run through some of
those guardrailing kind of things.

826
00:38:48,140 --> 00:38:52,900
Perhaps one of the really big ones is that
the agentic step, the actual coding agent,

827
00:38:52,900 --> 00:38:55,300
the reasoning the… runs read-only, okay.

828
00:38:55,300 --> 00:38:59,290
With just one Mm …narrow safe
output that it's allowed to make.

829
00:38:59,760 --> 00:38:59,880
Okay.

830
00:38:59,980 --> 00:39:02,890
And that is a really
strong thing, All right?

831
00:39:02,890 --> 00:39:06,150
That it, we are not giving these
things, write Access to MCPs.

832
00:39:06,190 --> 00:39:10,440
uh, In our MCP docs, we say, if you're
gonna add in extra mcps, you should not.

833
00:39:10,890 --> 00:39:13,090
Give them write Access
to the external world.

834
00:39:13,150 --> 00:39:16,610
You should design these safe outputs
instead, which are very tightly

835
00:39:16,610 --> 00:39:18,430
controll- controlled handover points.

836
00:39:18,740 --> 00:39:21,270
And when you really digest
that, it's running read-only

837
00:39:21,310 --> 00:39:22,830
without access to any secrets.

838
00:39:22,830 --> 00:39:25,820
'cause we go through our gateway
to access, uh, inside the

839
00:39:25,820 --> 00:39:27,300
container and got the ga- gateway.

840
00:39:27,600 --> 00:39:29,990
Those two together, huge relief.

841
00:39:30,350 --> 00:39:34,670
There's no chance it's gonna
leak all your repository secrets.

842
00:39:34,770 --> 00:39:35,300
Right, Uh- right.

843
00:39:35,300 --> 00:39:36,030
'cause it never had access

844
00:39:36,030 --> 00:39:36,150
to

845
00:39:36,150 --> 00:39:36,650
Bret: begin with.

846
00:39:36,650 --> 00:39:36,760
Yeah.

847
00:39:36,760 --> 00:39:36,890
It

848
00:39:36,890 --> 00:39:38,190
Don: never had access.

849
00:39:38,290 --> 00:39:40,790
And it's got a firewall around
it for network access as well.

850
00:39:41,250 --> 00:39:41,480
Peli: Yeah.

851
00:39:41,480 --> 00:39:41,560
There's

852
00:39:41,560 --> 00:39:43,890
Don: still concerns about there,
but they're known concerns.

853
00:39:43,890 --> 00:39:46,620
you can calibrate what's
going on in that landscape.

854
00:39:46,900 --> 00:39:53,380
And, uh, and I, it's that allows me to
sleep well at night when these things run.

855
00:39:53,380 --> 00:39:56,220
When we first started running coding
agents and actions, I was worried that we

856
00:39:56,220 --> 00:40:00,170
were creating actions that were gonna be
hackable, com, subvertible, compromisable,

857
00:40:00,170 --> 00:40:02,070
and, and you do get those people.

858
00:40:02,390 --> 00:40:09,110
Uh, and, but it's this very tight
containerization, readonly, no access

859
00:40:09,110 --> 00:40:14,780
to secrets and, um, and, and very narrow
scope of action in the safe output.

860
00:40:14,820 --> 00:40:19,660
And I, t- together these allow me, I, I'm
a guy who worries, Worried about things.

861
00:40:19,960 --> 00:40:20,480
as we should,

862
00:40:20,480 --> 00:40:20,660
And I think that

863
00:40:20,660 --> 00:40:21,260
Bret: will- as we should.

864
00:40:21,260 --> 00:40:22,420
I think that's been a hot topic.

865
00:40:22,420 --> 00:40:25,930
In fact, I'm doing, uh, some workshops and
I'm actually speaking at a conference this

866
00:40:25,930 --> 00:40:28,530
summer around GitHub Actions security.

867
00:40:28,530 --> 00:40:32,200
And I've also got a, a little
plug for a open source tool I'm

868
00:40:32,280 --> 00:40:35,830
about to release called, uh, GASA,
GitHub Action security assessment.

869
00:40:35,830 --> 00:40:39,820
That really just kind of takes the
top 10 things that I see, or the top

870
00:40:39,820 --> 00:40:43,020
dozen that teams I work with or, uh,
you know, and I've been studying a

871
00:40:43,020 --> 00:40:46,770
lot of these GitHub action, uh, supply
chain attacks and trying to understand

872
00:40:46,770 --> 00:40:48,860
where the core misconfigurations are.

873
00:40:48,860 --> 00:40:49,970
'cause that's really what
we're talking about Mm.

874
00:40:49,970 --> 00:40:51,430
-a lot in almost all these cases Yeah.

875
00:40:51,430 --> 00:40:51,660
It's really

876
00:40:51,660 --> 00:40:53,710
Don: just where's the problem
in this seven-step attack.

877
00:40:54,200 --> 00:40:54,690
Bret: Yeah.

878
00:40:54,770 --> 00:40:55,910
Don: where did that go wrong?

879
00:40:56,120 --> 00:40:56,190
Yeah.

880
00:40:56,250 --> 00:40:56,460
Right.

881
00:40:56,460 --> 00:40:59,900
Bret: What, what step in there
can we secure immediately and, and

882
00:40:59,900 --> 00:41:02,890
without Mm. consequence or, you know,
usually without breaking anything.

883
00:41:02,890 --> 00:41:05,260
And so that's one of the, I, I, I
mean- I'm trying to build a little

884
00:41:05,260 --> 00:41:07,080
tool that helps people discover that.

885
00:41:07,080 --> 00:41:09,310
'cause a lot of these things are just
like Mm. features of the platform that

886
00:41:09,630 --> 00:41:12,750
people don't thoroughly Mm. understand
that event in a GitHub action workflow

887
00:41:12,750 --> 00:41:16,470
or that particular Mm. checkbox in
the security settings of Mm. GitHub

888
00:41:16,470 --> 00:41:19,800
Actions in their repo settings, that
they just, they left it by default

889
00:41:19,800 --> 00:41:20,790
'cause they didn't understand it.

890
00:41:20,790 --> 00:41:21,910
They don't know the caveats.

891
00:41:21,910 --> 00:41:23,160
And, and so I've, go ahead.

892
00:41:23,190 --> 00:41:24,760
I was gonna set you up real quick Yeah.

893
00:41:24,760 --> 00:41:25,870
for this and just say, Okay.

894
00:41:25,870 --> 00:41:26,300
Go, go ahead.

895
00:41:26,300 --> 00:41:26,520
Yeah.

896
00:41:26,520 --> 00:41:30,780
Like, what if people aren't fully aware
of what's what we're talking about here?

897
00:41:31,170 --> 00:41:35,410
If you just add like a step in a
workflow and maybe your workflow

898
00:41:35,410 --> 00:41:39,260
today, your workflow before didn't
exist, and you add a new workflow to.

899
00:41:39,360 --> 00:41:41,900
Take, to assess an issue, right?

900
00:41:41,900 --> 00:41:43,510
Like you they- Issue triage.

901
00:41:43,510 --> 00:41:47,900
you need, you need some automation around
the title, and maybe for it to, you're

902
00:41:47,900 --> 00:41:53,500
thinking, I wanna have a model intuitively
s- select the label based on what it

903
00:41:53,500 --> 00:41:55,220
sees in the description, and the title.

904
00:41:55,220 --> 00:42:00,890
And so we've seen some m- disastrous
often cases where people just, they add

905
00:42:00,890 --> 00:42:04,910
a step to a workflow, they put a prompt
in there that basically says, please read

906
00:42:04,910 --> 00:42:07,020
through this, and then decide the label.

907
00:42:07,290 --> 00:42:11,490
And here's my GitHub, uh, you know,
workflow, my action token out of the

908
00:42:11,490 --> 00:42:15,770
gate by default, which may or may not
have full privileges to the entire repo.

909
00:42:16,140 --> 00:42:20,420
and you think that "Well, I'm giving
it a prompt to just change the issue

910
00:42:20,450 --> 00:42:24,470
or just uh, to just change the, label
of an issue or, maybe add a comment or

911
00:42:24,470 --> 00:42:29,290
something at most, but they don't really
understand the effect or the causality of

912
00:42:29,290 --> 00:42:31,060
what they just did and prompt injection.

913
00:42:31,060 --> 00:42:35,700
So can you maybe set that up, as like,
the problem with that and how this helps

914
00:42:35,900 --> 00:42:38,370
prevent that, obviously you said like
removing secret access and all that.

915
00:42:38,370 --> 00:42:39,160
but- I, I can take on

916
00:42:39,160 --> 00:42:39,720
Peli: on that.

917
00:42:40,170 --> 00:42:40,400
Yeah.

918
00:42:40,480 --> 00:42:43,090
looking at the token you have
with the agent, let's say

919
00:42:43,090 --> 00:42:44,090
you have to write an issue.

920
00:42:44,120 --> 00:42:47,690
So you're doing issue write Can
create, delete, update any issues in

921
00:42:47,690 --> 00:42:49,920
your repo at an insane rate, right?

922
00:42:50,020 --> 00:42:54,820
And the title might be a prompt injection
that changes the goal of the agent.

923
00:42:55,220 --> 00:42:58,960
Suddenly the goal, the agent has been
reprogrammed to close all your issues.

924
00:42:59,060 --> 00:43:03,740
Or to implant malware to all your
issues or, you know, but basically

925
00:43:03,740 --> 00:43:07,590
there's nothing preventing the agent
from saying, oh sure, let me go and,

926
00:43:07,590 --> 00:43:11,990
you know, I'll curl 500 times and
pass the token and do it, or so.

927
00:43:12,090 --> 00:43:15,270
So that's the danger that these
agents are extremely powerful.

928
00:43:15,590 --> 00:43:17,540
Uh, and the security posture.

929
00:43:17,540 --> 00:43:19,790
And the security construct we
have, you know, when we say

930
00:43:19,890 --> 00:43:21,480
read-only, it's at the token level.

931
00:43:21,580 --> 00:43:24,940
It is a deterministic guarantee
by the GitHub action platform.

932
00:43:25,150 --> 00:43:27,210
It is not based on any kind of prompting.

933
00:43:27,310 --> 00:43:31,080
This is a strong guarantee that you
basically adhere to because you're

934
00:43:31,080 --> 00:43:32,730
using the permission object in action.

935
00:43:32,830 --> 00:43:35,910
The same thing where we say there's no
co- there's no sequence in a container.

936
00:43:36,370 --> 00:43:38,830
Then you kind of, some level of
trust containers and so forth.

937
00:43:39,020 --> 00:43:44,830
So our security story is based on
DevOps primitives that are well-known

938
00:43:44,830 --> 00:43:49,250
in industry containers, and permission
scopes and things like that.

939
00:43:49,690 --> 00:43:53,720
And that is the deterministic
secure box we built.

940
00:43:53,820 --> 00:43:53,920
Yeah.

941
00:43:53,920 --> 00:43:56,360
So we don't rely on agentic.

942
00:43:56,680 --> 00:44:00,670
we, we have some agentic protection,
but as much as possible, we wanna

943
00:44:00,670 --> 00:44:06,620
build a box that is deterministically,
provably safe to some extent, right?

944
00:44:07,040 --> 00:44:07,960
Some guarantees.

945
00:44:08,060 --> 00:44:09,120
Don: Vibe security, Yeah.

946
00:44:09,190 --> 00:44:10,660
Yeah …that's what we call it.

947
00:44:10,910 --> 00:44:15,000
So when you, um, when you use GitHub
agentic workflows, you write this

948
00:44:15,000 --> 00:44:16,390
markdown and you write some front matter.

949
00:44:16,490 --> 00:44:16,810
Okay.

950
00:44:16,810 --> 00:44:18,030
It's, it's, it's lovely, right?

951
00:44:18,030 --> 00:44:19,480
The front matter looks a lot like actions.

952
00:44:19,480 --> 00:44:19,800
YAML.

953
00:44:19,800 --> 00:44:21,080
It will be fa- familiar to people.

954
00:44:21,080 --> 00:44:25,180
There's some differences, but, uh,
it's pretty, pretty familiar territory.

955
00:44:25,280 --> 00:44:30,300
Um, and you, you run this step
called, uh, GHAW Compile, okay?

956
00:44:30,340 --> 00:44:34,140
And it can produce the lock.yaml, which
is a YAML that actually runs and that,

957
00:44:34,140 --> 00:44:36,930
that compile actually, I, I, I wish
we'd chosen a different word for that.

958
00:44:36,930 --> 00:44:39,770
And we might make it a synonym,
which would be something like Harden.

959
00:44:40,140 --> 00:44:40,320
Okay?

960
00:44:40,320 --> 00:44:40,400
Mm-hmm.

961
00:44:40,400 --> 00:44:42,570
'cause what you're actually
doing is taking that prompting

962
00:44:42,670 --> 00:44:44,480
and you're kind of hardening it.

963
00:44:44,540 --> 00:44:46,660
You're m- you're putting a
security architecture around it.

964
00:44:46,660 --> 00:44:48,450
You're saying, I'm gonna run
that prompting, I'm gonna

965
00:44:48,450 --> 00:44:49,860
run it in in a coding agent.

966
00:44:49,890 --> 00:44:53,440
But that's gonna be, you know, we're
gonna create the YAML, which puts it in

967
00:44:53,440 --> 00:44:57,010
a nice box and gives it a nice, secure
thing and puts in a threat detection

968
00:44:57,010 --> 00:44:59,020
step to as a, as just a extra step.

969
00:44:59,120 --> 00:45:03,010
And I think that's the right kind
of model to use is what I want an

970
00:45:03,010 --> 00:45:06,910
automated tool, which will just make
me feel good, safe about running

971
00:45:06,910 --> 00:45:10,670
that, uh, about r- uh, running that
coding agent in GitHub Actions.

972
00:45:10,840 --> 00:45:15,520
And where that, we run that YAML
through several checkers as well.

973
00:45:15,570 --> 00:45:19,000
Actionlint, and uh, zizamor and,
uh, there's another one as well,

974
00:45:19,000 --> 00:45:19,570
Probably- Poutine … Poutine.

975
00:45:19,670 --> 00:45:21,020
and, um, Poutine, Runner

976
00:45:21,020 --> 00:45:21,670
Peli: guard.

977
00:45:21,910 --> 00:45:22,160
Don: Okay,

978
00:45:22,160 --> 00:45:22,240
And

979
00:45:22,240 --> 00:45:23,240
Bret: so we run- Ooh,
uh, Spell that for me.

980
00:45:23,240 --> 00:45:24,050
'cause I don't know about that one.

981
00:45:24,150 --> 00:45:27,200
Peli: P-O, P-O-U, you know,
like the food, like the Canadian

982
00:45:27,200 --> 00:45:29,300
Bret: fries, like the
gravy Cheese and the fries.

983
00:45:29,300 --> 00:45:30,220
I love, yeah, the gravy.

984
00:45:30,220 --> 00:45:30,350
Yep.

985
00:45:30,350 --> 00:45:30,490
Yeah.

986
00:45:30,800 --> 00:45:31,190
Okay.

987
00:45:31,390 --> 00:45:31,860
I'm already in.

988
00:45:31,860 --> 00:45:32,330
Sign me up.

989
00:45:32,430 --> 00:45:32,500
Yeah.

990
00:45:32,800 --> 00:45:33,010
Don: Yeah.

991
00:45:33,010 --> 00:45:35,440
so we, run those tools, uh,
they caught some things in the

992
00:45:35,440 --> 00:45:37,590
YAML, uh, in our hardened, YAML.

993
00:45:38,060 --> 00:45:40,210
And, um, so there's a
lot of goodness there.

994
00:45:40,210 --> 00:45:42,800
And there's something you can hand off
to the, it means your security team,

995
00:45:42,800 --> 00:45:45,930
if you're doing this in an enterprise,
can actually check that YAML as well.

996
00:45:45,930 --> 00:45:46,930
They get to see what's going on.

997
00:45:46,930 --> 00:45:48,560
They get to see the full
security architecture and check

998
00:45:48,560 --> 00:45:49,740
it matches up with what we say.

999
00:45:49,740 --> 00:45:52,510
They get to see the exact
container settings and its firewall

1000
00:45:52,610 --> 00:45:56,250
settings and what's mounted and
what, what's not, and so on.

1001
00:45:56,420 --> 00:46:01,000
Uh, and yeah, so yeah,
the guardrails are good.

1002
00:46:01,080 --> 00:46:05,640
And I love the security architecture
and the confidence is it gives me to

1003
00:46:05,670 --> 00:46:07,770
run fast, uh, with agentic automation.

1004
00:46:07,870 --> 00:46:07,940
Peli: Yeah.

1005
00:46:07,940 --> 00:46:13,100
So back to, building educational runtimes
and, you know, you think about velocity,

1006
00:46:13,200 --> 00:46:17,140
what this sandbox gives you is in
the strict mode, a guarantee, because

1007
00:46:17,240 --> 00:46:18,660
that you're not gonna leak secrets.

1008
00:46:18,660 --> 00:46:21,010
So you're gonna have a read-only
token when you're a, you know, the

1009
00:46:21,010 --> 00:46:25,290
agent won't see your agents, and that
you have a very specific, you know,

1010
00:46:25,290 --> 00:46:28,720
precisely where the agent's gonna
be able to mutate the world, right?

1011
00:46:28,720 --> 00:46:31,930
Because all the writes are transactional
and then we validated them.

1012
00:46:32,030 --> 00:46:35,700
there's a layered, a number of layer
of guarantees, and you're in full

1013
00:46:35,700 --> 00:46:38,610
control saying, I will allow you
to do one issue as Don said, or "I

1014
00:46:38,610 --> 00:46:40,040
will allow you to open an, a PR.

1015
00:46:40,140 --> 00:46:40,270
Right?

1016
00:46:40,270 --> 00:46:41,530
So from a practitioner's point of view.

1017
00:46:41,530 --> 00:46:43,910
These are guar- deterministic guarantees.

1018
00:46:44,210 --> 00:46:48,840
Then this allows you to go wild on
the prompt side and do back star and

1019
00:46:48,840 --> 00:46:52,560
do YOLO because you're gonna YOLO
inside of the container, not on your

1020
00:46:52,560 --> 00:46:54,540
dev box where all the secrets are.

1021
00:46:54,640 --> 00:46:57,650
Um, you're gonna YOLO in a container
that has no secrets that, you

1022
00:46:57,650 --> 00:46:59,820
know, you can't escape unless
you break out of the container.

1023
00:46:59,920 --> 00:47:03,000
But, that is insanely empowering.

1024
00:47:03,300 --> 00:47:06,260
because now you can try things
without second-guessing everything.

1025
00:47:06,660 --> 00:47:10,910
You can try things faster, more tools,
without taking down the whole house.

1026
00:47:11,010 --> 00:47:15,160
That will take you from trying
things very carefully with AI to

1027
00:47:15,160 --> 00:47:19,030
actually go, go, go, go, go much
faster because you have safety.

1028
00:47:19,500 --> 00:47:21,510
So people underestimate the fact that,

1029
00:47:21,680 --> 00:47:21,800
To,

1030
00:47:22,270 --> 00:47:25,930
Don: to give an example, uh, on,
on, on it means you're gonna write

1031
00:47:25,930 --> 00:47:29,940
prompting of things like, okay,
agent, work out the test coverage

1032
00:47:29,940 --> 00:47:31,970
in this repository and improve it.

1033
00:47:32,070 --> 00:47:32,420
Okay?

1034
00:47:32,520 --> 00:47:36,830
Find the big holes and assess the value,
uh, and choose the highest value bits

1035
00:47:36,830 --> 00:47:38,500
and fill in improve the test coverage.

1036
00:47:38,600 --> 00:47:42,330
That means it's actually going
to in- possibly install tools

1037
00:47:42,380 --> 00:47:44,110
to be taking test coverage.

1038
00:47:44,110 --> 00:47:47,830
It's gonna be working out the command
line invocations to kind of do that.

1039
00:47:48,270 --> 00:47:51,620
you know, stuff, if this is a c
thing, you know, the impossible

1040
00:47:51,620 --> 00:47:55,070
stuff of ever taking coverage of
a c repo or something like that.

1041
00:47:55,170 --> 00:47:57,730
And, uh, and it's gonna
be reading the files.

1042
00:47:57,800 --> 00:48:00,710
Uh, and it's gonna be, uh, it's just
gonna be doing everything right.

1043
00:48:00,710 --> 00:48:00,840
Yeah.

1044
00:48:00,840 --> 00:48:03,590
And you can keep your, it means
you can keep your prompt in general

1045
00:48:03,660 --> 00:48:07,000
and the agents will use the full
power of the software engineering

1046
00:48:07,000 --> 00:48:09,710
toolkits that it's got, uh, available.

1047
00:48:09,770 --> 00:48:13,210
And one of the magical things as well,
you're running in GitHub Actions and,

1048
00:48:13,350 --> 00:48:16,810
people take those VMs for granted,
but, and the system side of what's

1049
00:48:16,890 --> 00:48:17,850
built there, 'cause that's a Sure.

1050
00:48:17,850 --> 00:48:19,330
I don't know how big are the images now?

1051
00:48:19,460 --> 00:48:21,260
Like hundreds of gigabytes or something?

1052
00:48:21,260 --> 00:48:21,640
Terabytes.

1053
00:48:21,680 --> 00:48:22,120
Oh, really?

1054
00:48:22,120 --> 00:48:24,040
there's a lot of secret
ingredients There's- …that

1055
00:48:24,040 --> 00:48:25,710
Peli: make action an amazing platform.

1056
00:48:25,710 --> 00:48:25,910
Yeah.

1057
00:48:26,090 --> 00:48:26,270
Don: Yeah.

1058
00:48:26,270 --> 00:48:30,100
Mm. So that means every time the agent,
every time your agentic workflows, or

1059
00:48:30,100 --> 00:48:34,020
in fact your YAML workflows are kind
of waking up, uh, or, or running.

1060
00:48:34,020 --> 00:48:37,340
They're running with all the world
software engineering tools, kind of very,

1061
00:48:37,340 --> 00:48:40,210
very efficiently available and well-known
install locations and all sorts of things.

1062
00:48:40,210 --> 00:48:40,500
Yes.

1063
00:48:40,600 --> 00:48:45,170
And, uh, that gives them a super
powerful, um, I mean, they're just,

1064
00:48:45,170 --> 00:48:46,620
they're just incredible what they can do.

1065
00:48:46,720 --> 00:48:49,720
Peli: So, for example, if you think
about the actions and the features you

1066
00:48:49,720 --> 00:48:51,330
have, you know, every run is recorded.

1067
00:48:51,430 --> 00:48:52,990
Every run you can store artifacts.

1068
00:48:53,410 --> 00:48:54,920
You can, You have APIs to read them.

1069
00:48:54,950 --> 00:48:57,890
So we store the agent session, the
sessions that are sitting on your dev

1070
00:48:57,890 --> 00:48:59,760
box individually, they're kind of lost.

1071
00:48:59,820 --> 00:49:00,320
We store them.

1072
00:49:00,700 --> 00:49:03,430
So we analyze them, we optimize
them, then we debug that.

1073
00:49:03,530 --> 00:49:09,180
Uh, so the automated agentic workflows
are primed to be optimizable,

1074
00:49:09,280 --> 00:49:11,260
debuggable because we have full history.

1075
00:49:11,560 --> 00:49:15,710
You run things five times, you realize
you always do the same MCP calls.

1076
00:49:15,710 --> 00:49:16,170
What happens?

1077
00:49:16,170 --> 00:49:19,610
You tell the agent, move
these MCP calls to steps.

1078
00:49:19,860 --> 00:49:20,380
And guess what?

1079
00:49:20,380 --> 00:49:24,300
Because we're action, we can
do a mix of deterministic, just

1080
00:49:24,490 --> 00:49:26,650
good old steps and agentic.

1081
00:49:26,650 --> 00:49:30,770
So let's say your agent does, you
know, give me the the 10 first pull

1082
00:49:30,770 --> 00:49:32,550
requests, and it does the GitHub MCP.

1083
00:49:32,810 --> 00:49:33,870
That is an agentic step.

1084
00:49:33,870 --> 00:49:35,410
It eats a bunch of token.

1085
00:49:35,910 --> 00:49:41,180
But now you move that into a GH PR call
as a step, drop it into a JSON file

1086
00:49:41,280 --> 00:49:43,390
and let the agent do JQ bash on that.

1087
00:49:43,480 --> 00:49:47,350
Suddenly your agent is eating
10, 10 less turns, 20 less turns.

1088
00:49:47,350 --> 00:49:48,730
Mm. you've moved the dial.

1089
00:49:49,100 --> 00:49:52,230
That's one of the interesting, these are
all interesting things we've discovered

1090
00:49:52,550 --> 00:49:57,650
in actions that being able to pull the
dial between deterministic and agentic.

1091
00:49:57,650 --> 00:50:01,620
Because in your CI we're 100%
deterministic historically, And

1092
00:50:01,620 --> 00:50:03,380
people have gone 100% agentic.

1093
00:50:03,380 --> 00:50:06,650
But the truth is Yeah …it's
gonna be in the middle.

1094
00:50:06,760 --> 00:50:07,980
And the more you are.

1095
00:50:08,080 --> 00:50:08,370
Yeah.

1096
00:50:08,410 --> 00:50:10,360
The more you're deterministic,
the cheaper it is.

1097
00:50:10,810 --> 00:50:11,660
The most, powerful- And the less…

1098
00:50:12,050 --> 00:50:12,340
Don: Yes.

1099
00:50:12,340 --> 00:50:13,820
they are, uh, that's right.

1100
00:50:13,820 --> 00:50:18,070
So, and that's why we love being in the
actions ecosystem, because those steps

1101
00:50:18,070 --> 00:50:23,880
can also use the full existing, GitHub
Actions, everything in that ecosystem.

1102
00:50:23,880 --> 00:50:24,110
Yeah.

1103
00:50:24,180 --> 00:50:26,590
And, uh, and that's,
that's really powerful.

1104
00:50:26,620 --> 00:50:31,270
Peli: You can see this gradient coming
right in your CI, you know, 100% CI, 100%

1105
00:50:31,270 --> 00:50:35,900
deterministic, but now you start squeezing
in a bit of agentic as much as you want.

1106
00:50:35,900 --> 00:50:39,160
Maybe just a little reasoning at
the end of your test run, you know?

1107
00:50:39,370 --> 00:50:39,450
Yeah,

1108
00:50:39,750 --> 00:50:39,980
Don: yeah.

1109
00:50:39,980 --> 00:50:43,170
I do like to point out to those,
uh, people who are really big

1110
00:50:43,170 --> 00:50:47,290
on, like, CI, and CD need to be
deterministic, and we absolutely

1111
00:50:47,290 --> 00:50:50,540
have to kind of nail that in order to
'cause it is That's absolutely true.

1112
00:50:50,570 --> 00:50:51,910
we don't challenge that.

1113
00:50:52,030 --> 00:50:52,420
Okay.

1114
00:50:52,420 --> 00:50:55,880
We say there's a, a third area,
a new area, like a third leg of a

1115
00:50:55,880 --> 00:50:57,070
stool that we didn't know was there.

1116
00:50:57,070 --> 00:50:58,810
Right, Which is like continuous ai.

1117
00:50:58,920 --> 00:50:59,220
Right.

1118
00:50:59,540 --> 00:51:02,680
Which includes some of these subjective
steps and it's got different properties,

1119
00:51:02,960 --> 00:51:04,790
but the continuity is a big part.

1120
00:51:04,790 --> 00:51:08,770
The automation is what is and the kind
of always aligning with the state of the

1121
00:51:08,770 --> 00:51:12,100
repo as it is today, which is what we
kind of mean by continuity, continuous.

1122
00:51:12,200 --> 00:51:12,790
Um- the claim

1123
00:51:12,790 --> 00:51:13,330
Peli: is bolder.

1124
00:51:13,430 --> 00:51:16,850
The claim, you need CI and
deterministic and fast.

1125
00:51:16,950 --> 00:51:18,470
This is how you're gonna tame the agents.

1126
00:51:18,570 --> 00:51:19,910
The agents are, you know, little monsters.

1127
00:51:19,910 --> 00:51:20,350
Don: Absolutely.

1128
00:51:20,350 --> 00:51:21,190
It's the ultimate guide.

1129
00:51:21,340 --> 00:51:21,710
This is the

1130
00:51:21,710 --> 00:51:22,910
Peli: golden era of CI.

1131
00:51:23,330 --> 00:51:23,610
Don: Yeah.

1132
00:51:23,670 --> 00:51:23,990
Peli: Yeah.

1133
00:51:24,320 --> 00:51:24,780
The Teams

1134
00:51:24,780 --> 00:51:27,480
that don't have CI will
not benefit from this.

1135
00:51:27,550 --> 00:51:28,090
The Boost.

1136
00:51:28,190 --> 00:51:28,510
Bret: Yeah.

1137
00:51:28,540 --> 00:51:32,680
I've often thought or described it to
people that, uh, are asking me like,

1138
00:51:32,680 --> 00:51:36,580
you know, where I get started and I,
I talked to them about that, you know,

1139
00:51:36,580 --> 00:51:40,570
you're probably not gonna rewrite your
Docker build workflow to be agentic.

1140
00:51:40,860 --> 00:51:42,370
that's probably not the area of focus.

1141
00:51:42,400 --> 00:51:45,610
I would imagine that a lot of these,
the majority of these are n- net new

1142
00:51:46,020 --> 00:51:47,740
workflows or expanding Absolutely.

1143
00:51:47,740 --> 00:51:50,360
An existing workflow to do
things it couldn't previously do.

1144
00:51:50,360 --> 00:51:52,960
Not rewriting my reliable test runs.

1145
00:51:52,960 --> 00:51:56,210
I mean, maybe there's an AI that's gonna
help me automate parallelism and all that,

1146
00:51:56,210 --> 00:51:57,870
but it's separate from maybe the run.

1147
00:51:57,870 --> 00:51:58,010
itself.

1148
00:51:58,010 --> 00:51:58,130
Yeah.

1149
00:51:58,130 --> 00:51:58,290
And It's

1150
00:51:58,290 --> 00:52:00,390
Don: really, really important.

1151
00:52:00,390 --> 00:52:05,140
We get that deeply because the way one
of my friends put it is like the CI/CD.

1152
00:52:05,460 --> 00:52:09,310
And in fact, GitHub in many ways is
where the grownups are in software.

1153
00:52:09,670 --> 00:52:10,960
Development process, right?

1154
00:52:10,960 --> 00:52:14,910
That that's, Yeah …you know, we're all
going crazy about doing our coding agents

1155
00:52:14,910 --> 00:52:19,570
and YOLOing on our local machines and
you know, whatever kind of cra But in the

1156
00:52:19,570 --> 00:52:24,230
end, the place where you build confidence
in an organization where you feel you

1157
00:52:24,230 --> 00:52:26,480
can deploy, where they're in your CI/CD.

1158
00:52:26,480 --> 00:52:30,750
and we've got to keep that
grown-up mentality about quality.

1159
00:52:30,800 --> 00:52:35,830
And we want, you know, when we talk
about code improvement, we we're talking

1160
00:52:35,830 --> 00:52:39,720
about proposed code improvements, which
have to get through the gates, right?

1161
00:52:39,720 --> 00:52:39,780
Yeah.

1162
00:52:39,780 --> 00:52:43,390
We're not talking about things that they
should come with test improvements, right?

1163
00:52:43,390 --> 00:52:46,020
That kind of match that just like
you'd expect from any pull request.

1164
00:52:46,020 --> 00:52:50,570
so we absolutely have to keep CID being
the place where the grownups are and keep

1165
00:52:50,570 --> 00:52:52,780
the mentality that goes along with that.

1166
00:52:52,870 --> 00:52:56,000
Uh, as well as having a more
flexible idea about automation.

1167
00:52:56,000 --> 00:52:58,750
And we wanna really
empower the DevOps people.

1168
00:52:58,780 --> 00:53:03,560
This is like, what it's all about is like
we feel there's just this missing piece

1169
00:53:03,560 --> 00:53:10,030
of the puzzle in the AI story, which is
where we empower the people who run the

1170
00:53:10,030 --> 00:53:12,950
repositories to use AI to their benefit.

1171
00:53:12,950 --> 00:53:15,510
And we know some people are suffering
in the open source world from

1172
00:53:15,510 --> 00:53:17,900
AI coming in from third parties.

1173
00:53:18,370 --> 00:53:18,500
Right.

1174
00:53:18,500 --> 00:53:20,050
We wanna empower people.

1175
00:53:20,190 --> 00:53:24,600
So the maintainers and the people
who create the repos decide What

1176
00:53:24,600 --> 00:53:29,180
automation runs in their repositories
for what goals, under what c- cost,

1177
00:53:29,180 --> 00:53:32,920
trade-offs, what, you know, what
quality trade-offs and everything.

1178
00:53:32,920 --> 00:53:35,510
They're the ones who can balance
those things in the context of the

1179
00:53:35,510 --> 00:53:37,950
business goals or the open source
goals that they kind of have.

1180
00:53:38,050 --> 00:53:43,120
And, uh, yeah, empower them and, and
not, not, don't just make them suffer

1181
00:53:43,120 --> 00:53:44,530
and kind of the recipients of Make them.

1182
00:53:44,960 --> 00:53:45,140
Yeah.

1183
00:53:45,140 --> 00:53:47,490
the- And they'll see so
many new uses for it.

1184
00:53:47,490 --> 00:53:51,030
That's one of the things, this is job
creation all over the place in the sense

1185
00:53:51,030 --> 00:53:55,750
there's so often, so many opportunities to
do work we could never have done before.

1186
00:53:56,150 --> 00:53:57,180
Performance optimization.

1187
00:53:57,180 --> 00:54:00,730
is a good one There's, there's-
this is literal job or at least

1188
00:54:00,730 --> 00:54:05,260
work creation because the people up
close to the repos are the ones who

1189
00:54:05,260 --> 00:54:07,890
know what, where the suffering is.

1190
00:54:08,000 --> 00:54:11,510
where the, where the un, where the
unenforced invariance, unenforced

1191
00:54:11,540 --> 00:54:15,740
quality, uh, the, the opportunities for
improvement, which were never explored.

1192
00:54:16,200 --> 00:54:21,610
Uh, and the, the legacy code, which
can actually be brought back alive and

1193
00:54:21,610 --> 00:54:26,180
actually serve a, a, a role going forward
or transition to a new system or whatever.

1194
00:54:26,360 --> 00:54:26,500
Peli: Yeah.

1195
00:54:26,500 --> 00:54:29,080
Don: So many opportunities for
work, once you get into the

1196
00:54:29,080 --> 00:54:31,500
right mindset and, uh, yeah.

1197
00:54:31,670 --> 00:54:35,960
It's, it's a golden age for DevOps people,
uh, who, um, to create There's lots

1198
00:54:35,960 --> 00:54:37,810
to learn, but it's, it's a golden age.

1199
00:54:37,910 --> 00:54:38,150
Bret: Yeah.

1200
00:54:38,150 --> 00:54:42,800
D- I, I have often, The more I've
understood the mindset behind your

1201
00:54:42,800 --> 00:54:45,870
creation of agentic workflows and how,
you know, like my first realization

1202
00:54:45,870 --> 00:54:49,070
was if I start looking at this lock
file, most of this is deterministic.

1203
00:54:49,400 --> 00:54:49,480
Yeah.

1204
00:54:49,870 --> 00:54:50,860
it's relatively long.

1205
00:54:51,000 --> 00:54:53,660
It's not, it doesn't look anything like
something I would write in a GitHub

1206
00:54:53,660 --> 00:54:58,200
action workflow, but it is really
just a, mostly a framework around, at

1207
00:54:58,200 --> 00:55:01,880
least when I first got started around
controlling and protecting and guiding

1208
00:55:01,880 --> 00:55:05,060
it, it is to this very small part
that's actually a model prompt, Yeah,

1209
00:55:05,210 --> 00:55:05,350
Don: yeah.

1210
00:55:05,380 --> 00:55:07,040
It's, it's, yes, it's, that's right.

1211
00:55:07,040 --> 00:55:10,320
There's somewhere in the middle there's
a invocation of a coding agent, but

1212
00:55:10,320 --> 00:55:13,180
you put all this apparatus around it
to say, we were gonna make that safe.

1213
00:55:13,550 --> 00:55:14,270
We're gonna make that guardrail.

1214
00:55:14,370 --> 00:55:14,980
And we, and yeah.

1215
00:55:15,270 --> 00:55:15,570
Go on, Pavel.

1216
00:55:15,870 --> 00:55:16,050
Peli: Yeah.

1217
00:55:16,050 --> 00:55:16,930
There is something new.

1218
00:55:16,930 --> 00:55:20,580
Actually, you know, if you look at
agentic workflows, it's, it's a big YAML.

1219
00:55:20,610 --> 00:55:24,470
I mean, by this time, we support
a lot of features, but you

1220
00:55:24,470 --> 00:55:25,740
don't even edit this yourself.

1221
00:55:26,130 --> 00:55:28,930
I mean, at least for on
the, in the ideation phase.

1222
00:55:28,970 --> 00:55:32,990
And, you know, until you reach
your 90% done, this is gone.

1223
00:55:33,040 --> 00:55:34,100
This is done through an agent.

1224
00:55:34,490 --> 00:55:37,060
You don't actually have, you
have to come in with your intent,

1225
00:55:37,160 --> 00:55:38,090
what you're trying to achieve.

1226
00:55:38,470 --> 00:55:41,900
And we, we've done a lot of research
and we've done this, we have this prompt

1227
00:55:41,900 --> 00:55:45,680
that is gonna try to generate the best
agentic workflow for you as a starter,

1228
00:55:45,780 --> 00:55:50,870
but there's also a completely new
experience where as an automator, you use

1229
00:55:50,870 --> 00:55:55,250
an agent to design that automation, and
then you can fine-tune, you know, and,

1230
00:55:55,250 --> 00:55:57,080
and run the compiler deterministically.

1231
00:55:57,570 --> 00:56:00,160
But this will get you from 0 to 80%.

1232
00:56:00,460 --> 00:56:02,300
Without actually having
to read the documentation.

1233
00:56:02,510 --> 00:56:06,430
'cause you come in and say, um, you
know, you, you name your scenarios and

1234
00:56:06,430 --> 00:56:12,800
you, you know the keywords, you know,
issues PRs, build workflow, run test.

1235
00:56:13,210 --> 00:56:17,600
The agent has access to your agent md
the agent has access to all your actions.

1236
00:56:17,810 --> 00:56:21,050
So if you already have CI/CD,
the agent can read your CI build,

1237
00:56:21,440 --> 00:56:23,910
figure out how you build your
software, how you run your test.

1238
00:56:24,010 --> 00:56:27,570
And one thing that is great about action
is that it is baked into the LLMs.

1239
00:56:27,670 --> 00:56:30,800
LLMs today know very well action.

1240
00:56:30,950 --> 00:56:34,190
They know how to write the YAML, they
know the entire schema, they know how

1241
00:56:34,190 --> 00:56:37,630
to refactor it because people have been
blogging about yeah actions forever.

1242
00:56:37,730 --> 00:56:41,120
Uh, so there's like this free, you
know, we have this format is actually

1243
00:56:41,120 --> 00:56:45,040
designed to be close to what the
agent would expect because then you

1244
00:56:45,040 --> 00:56:48,220
get this magic where it just knows
it, there's no fine-tuning needed.

1245
00:56:48,320 --> 00:56:50,470
You can tell it to refactor
the prompt into steps.

1246
00:56:50,800 --> 00:56:51,880
And it's like, sure.

1247
00:56:51,880 --> 00:56:54,580
I mean, yeah, I, and
it knows the ecosystem.

1248
00:56:54,710 --> 00:56:58,320
It's gonna go and pull in the right
custom actions to, you know, to

1249
00:56:58,320 --> 00:57:01,720
do actions GitHub scripts, or to
do checkout and all this stuff.

1250
00:57:01,720 --> 00:57:06,090
So that is also part of the magic here, is
that not only we're leveraging a platform,

1251
00:57:06,550 --> 00:57:10,940
but we're leveraging the fact that the
platform is already trained in the model.

1252
00:57:11,340 --> 00:57:11,380
Yeah.

1253
00:57:11,410 --> 00:57:14,190
You don't need to load a
skill to learn actions.

1254
00:57:14,350 --> 00:57:15,880
It is already in- This
is some new product.

1255
00:57:15,910 --> 00:57:16,480
Bret: Yes.

1256
00:57:16,580 --> 00:57:19,810
Don: Bret- I- uh, ca- can I just
share, uh, my screen briefly?

1257
00:57:20,185 --> 00:57:21,155
Oh yeah, sure.

1258
00:57:21,425 --> 00:57:25,215
Yeah, I, just wanna, just wanna show one,
one thing to kind of get across why Yeah.

1259
00:57:25,335 --> 00:57:27,435
why I'm so excited by this.

1260
00:57:27,505 --> 00:57:27,995
uh- yeah,

1261
00:57:27,995 --> 00:57:28,095
let

1262
00:57:28,095 --> 00:57:28,665
Bret: me, um,

1263
00:57:28,965 --> 00:57:29,425
Don: All right hold

1264
00:57:29,525 --> 00:57:30,055
Bret: on a second.

1265
00:57:30,105 --> 00:57:31,605
let me pull that in.

1266
00:57:31,705 --> 00:57:33,455
I don't have that button
on my stream deck.

1267
00:57:33,455 --> 00:57:33,945
One second.

1268
00:57:34,245 --> 00:57:34,695
You can see it.

1269
00:57:34,995 --> 00:57:38,755
I can see it, but I need to put it on
the screen for, everyone else to see it.

1270
00:57:38,755 --> 00:57:40,655
I need to give it a guest place.

1271
00:57:41,135 --> 00:57:43,035
And then, let's

1272
00:57:43,035 --> 00:57:44,235
Don: see.

1273
00:57:44,235 --> 00:57:44,725
There we go.

1274
00:57:45,025 --> 00:57:46,035
All right, brilliant.

1275
00:57:46,035 --> 00:57:49,465
I, so I just wanna briefly mention
this, this particular workflow.

1276
00:57:49,645 --> 00:57:51,395
This is a, uh, this is a workflow.

1277
00:57:51,395 --> 00:57:53,525
You just install one of these
in your repo and it kind of

1278
00:57:53,525 --> 00:57:54,985
helps you maintain the repo.

1279
00:57:55,165 --> 00:57:55,425
Okay.

1280
00:57:55,425 --> 00:57:56,965
This is the thing I
mentioned before, repo.

1281
00:57:57,035 --> 00:58:01,155
This is… and, uh, it, it's
really simple getting started and.

1282
00:58:01,255 --> 00:58:04,315
it's super, and this is kind of how,
how it works There's a diagram here

1283
00:58:04,415 --> 00:58:05,905
kind of selects a couple of tasks.

1284
00:58:05,905 --> 00:58:07,845
It reads the memory, and these
are the different tasks It

1285
00:58:07,845 --> 00:58:09,425
might do issue labeling for you.

1286
00:58:09,425 --> 00:58:12,305
It might do an issue investigation
and the other things.

1287
00:58:12,435 --> 00:58:15,075
And you can configure this and
you can edit, you can say, add

1288
00:58:15,075 --> 00:58:16,725
a new task to do this or this.

1289
00:58:16,725 --> 00:58:19,955
And, you know, and it kind of just
works on a daily rhythm or hourly

1290
00:58:19,955 --> 00:58:21,605
rhythm or whatever rhythm you want.

1291
00:58:21,705 --> 00:58:25,095
And I kind of wr- I've written up, uh,
written that up in a kind of blog post

1292
00:58:25,095 --> 00:58:26,225
and you can kind of see how it works.

1293
00:58:26,275 --> 00:58:28,345
but I just wanted to sh- share
this really, which is this

1294
00:58:28,345 --> 00:58:32,425
report we've written, uh, on
the impact of using repo assist.

1295
00:58:32,425 --> 00:58:33,635
So if you kind of look
at what's on the screen.

1296
00:58:33,635 --> 00:58:37,655
You can probably guess where we started
to use repo assist in this particular

1297
00:58:37,655 --> 00:58:40,365
repository There's a number of issues
that were open in the repository.

1298
00:58:40,365 --> 00:58:43,565
So this was a pretty much dormant
repository, but with a backlog, right?

1299
00:58:43,565 --> 00:58:47,285
It's got a… you know, I, I as a
maintainer sort of stopped engaging with

1300
00:58:47,285 --> 00:58:50,675
this because I didn't actually, each of
these issues would've taken me sort of

1301
00:58:50,675 --> 00:58:54,475
a, a night, probably in the traditional
way to kind of reengage with the issue.

1302
00:58:54,475 --> 00:58:57,135
And even if I was doing it manually
with a coding agent, it would've

1303
00:58:57,185 --> 00:59:01,535
taken me significant, 20 minutes,
30 minutes, an hour for each issue.

1304
00:59:01,795 --> 00:59:02,085
Okay.

1305
00:59:02,185 --> 00:59:06,505
And instead you've got the automated
AI effectively r- burning through

1306
00:59:06,505 --> 00:59:09,745
the backlog, commenting on it,
making pull requests for it, and

1307
00:59:09,745 --> 00:59:11,775
like actively, proactively kind of.

1308
00:59:11,875 --> 00:59:14,435
make solving all the, all of that backlog.

1309
00:59:14,525 --> 00:59:19,465
And I mean, boy, it allowed me either
to close out the backlog or actually fix

1310
00:59:19,465 --> 00:59:23,165
the backlog and make I think three major,
some of this was feature requests as well.

1311
00:59:23,165 --> 00:59:27,255
So it actually kind of took the
repository forward as well as implementing

1312
00:59:27,325 --> 00:59:32,395
features and got three major new
versions of this, uh, of this component

1313
00:59:32,465 --> 00:59:34,715
out in, uh, as open source releases.

1314
00:59:34,765 --> 00:59:37,255
And, that repository
is now in a good state.

1315
00:59:37,255 --> 00:59:39,055
Reposys continues to run.

1316
00:59:39,445 --> 00:59:42,375
So if there's any more, uh, it's
now running sort of on a weekly

1317
00:59:42,375 --> 00:59:44,695
kind of basis to as a cost control.

1318
00:59:44,795 --> 00:59:49,445
And if any new input comes into
the repo, uh, new issues, uh,

1319
00:59:49,445 --> 00:59:50,845
it will, it will start to do.

1320
00:59:50,845 --> 00:59:51,905
Its, its kind of magic.

1321
00:59:51,905 --> 00:59:53,415
It will look after that for me.

1322
00:59:53,515 --> 00:59:55,065
Uh, and of course I'm still in control.

1323
00:59:55,065 --> 00:59:58,825
The human's still in the loop, but
you can just get the very dramatic

1324
00:59:58,905 --> 01:00:03,065
difference it makes between software
with bugs to software that is actually

1325
01:00:03,545 --> 01:00:05,015
maintainable and fully usable.

1326
01:00:05,015 --> 01:00:06,085
And it's not just one repo.

1327
01:00:06,085 --> 01:00:07,155
Here's another repo.

1328
01:00:07,155 --> 01:00:10,095
The same workflow, uh, with,
uh, here's another one.

1329
01:00:10,095 --> 01:00:11,235
This is a different maintainer.

1330
01:00:11,235 --> 01:00:13,765
So it's not just me, other,
other maintainers picking it up,

1331
01:00:13,765 --> 01:00:14,745
a slightly different percent.

1332
01:00:14,845 --> 01:00:18,585
Uh, but you know, after, after a month
of sort of this thing ticking away,

1333
01:00:18,765 --> 01:00:21,095
uh, it, the repo's in excellent shape.

1334
01:00:21,195 --> 01:00:22,105
Uh, Here's another one.

1335
01:00:22,105 --> 01:00:25,305
This is something I co-maintain
with somebody else, uh, it's

1336
01:00:25,305 --> 01:00:26,905
a, slight different trajectory.

1337
01:00:27,005 --> 01:00:30,985
And, uh, and here's another one where
there were good reasons to leave a lot

1338
01:00:30,985 --> 01:00:35,585
of, uh, feature suggestions, uh, lying
around the repo at the end of the kind

1339
01:00:35,685 --> 01:00:37,275
of, uh, as it acquiesces at the end.

1340
01:00:37,365 --> 01:00:37,435
Yeah.

1341
01:00:37,625 --> 01:00:42,105
So, um, That's cool …super,
super happy with how this is, uh,

1342
01:00:42,175 --> 01:00:46,365
this is going, uh, this one, the
maintainer actually only comes back.

1343
01:00:46,425 --> 01:00:48,515
He said, So didn't wanna
work on this full, time.

1344
01:00:48,515 --> 01:00:49,905
didn't wanna crunch a whole lot away.

1345
01:00:49,905 --> 01:00:52,145
was just kind of happy just to
kind of come back to it every

1346
01:00:52,145 --> 01:00:55,895
few months and kind of this graph
will keep going down step by step.

1347
01:00:55,965 --> 01:00:59,045
So the, the report we're looking
at is, uh, just to bring it up to

1348
01:00:59,045 --> 01:01:02,685
the top, is the impact of automated
repository maintenance assistance.

1349
01:01:02,745 --> 01:01:05,065
and you wrote a blog about this, right?

1350
01:01:05,485 --> 01:01:06,045
on our GitHub Next site.

1351
01:01:06,275 --> 01:01:12,465
And yes, there is a, uh, blog, the blog
about Reposys in general, and on my,

1352
01:01:12,465 --> 01:01:18,375
uh, just grabbing this is a, is a link
to our new report, from GitHub Next,

1353
01:01:18,415 --> 01:01:20,345
it's on our GitHub Next site as well.

1354
01:01:20,565 --> 01:01:21,505
Yeah, so check that out.

1355
01:01:21,575 --> 01:01:22,665
Uh, where was that report?

1356
01:01:22,915 --> 01:01:22,975
Um…

1357
01:01:23,275 --> 01:01:23,595
Bret: Yeah.

1358
01:01:23,705 --> 01:01:27,475
Uh, I think to me one of the most
exciting things about all of this,

1359
01:01:27,475 --> 01:01:31,105
and, and we're, we've been hinting
at this the whole time, is that the.

1360
01:01:31,405 --> 01:01:36,385
As a CI maintainer, uh, uh, a-as someone
who's m- I call it the middle gray area

1361
01:01:36,385 --> 01:01:39,685
of the software development life cycle,
where it's post-commit of the developer,

1362
01:01:40,095 --> 01:01:43,345
but it's, it's pre-production running.

1363
01:01:43,385 --> 01:01:46,915
And everything in that middle has been
like, I, I, r- I can remember, we, if we

1364
01:01:46,915 --> 01:01:52,475
go back to even 2018, I can distinctly
remember at DockerCon and at KubeCon,

1365
01:01:52,825 --> 01:01:56,685
we were talking about what was the next
wave of innovation, because we felt like

1366
01:01:56,965 --> 01:02:01,575
the container ecosystem had matured, and
we kind of knew what that looked like

1367
01:02:01,575 --> 01:02:03,245
and how to move things around as images.

1368
01:02:03,245 --> 01:02:04,975
And that was all well-defined.

1369
01:02:04,975 --> 01:02:08,025
And w- at the time, we were all
talking about the CI platform

1370
01:02:08,025 --> 01:02:09,295
as the next piece of innovation.

1371
01:02:09,295 --> 01:02:11,825
And there was all this discussion around
different startups that were getting

1372
01:02:12,115 --> 01:02:15,495
funding because that was gonna be the
n-next opportunity for innovation.

1373
01:02:15,875 --> 01:02:18,855
And uh, we, there was experiments
that, you know, GitHub Actions, uh,

1374
01:02:18,855 --> 01:02:20,765
workflows were probably part of that.

1375
01:02:20,765 --> 01:02:22,175
Like that was a part of that wave.

1376
01:02:22,505 --> 01:02:25,925
It didn't, it was awesome, but also
didn't seem to always fulfill the

1377
01:02:25,925 --> 01:02:29,745
promise of what we were trying to
innovate on and reinvent in the CI space.

1378
01:02:30,135 --> 01:02:33,315
But I feel like we're
finally at this moment where.

1379
01:02:33,415 --> 01:02:36,395
I might just be able to do
all the things I always wanted

1380
01:02:36,395 --> 01:02:38,545
to do to fixing the platform.

1381
01:02:38,545 --> 01:02:41,065
You know, the maintenance, the
toil, the backlog, the endless

1382
01:02:41,065 --> 01:02:45,335
backlog of things that needed to be
optimized or locked down or scanned

1383
01:02:45,335 --> 01:02:46,965
or improved, like documentation.

1384
01:02:47,365 --> 01:02:52,405
And that w-we… management always tended
to, you know, focus on the feature set.

1385
01:02:52,405 --> 01:02:54,985
You know, th- those of us in DevOps
are always trying to help, help

1386
01:02:54,985 --> 01:02:57,795
them understand and that there's
more than just adding features.

1387
01:02:57,795 --> 01:02:58,925
We need to maintain the system.

1388
01:02:58,955 --> 01:03:01,945
SREs are a thing now, so we all
get this, you know, we at least get

1389
01:03:01,945 --> 01:03:04,715
someone in production that's helping
to optimize the production information.

1390
01:03:04,715 --> 01:03:09,195
But I feel like the CI platform is still
this sort of, uh, if redheaded stepchild

1391
01:03:09,195 --> 01:03:13,155
is a thing we still say like, it feels
like the thing that still doesn't get the

1392
01:03:13,155 --> 01:03:14,845
love and nurturing that it always needed.

1393
01:03:14,845 --> 01:03:17,325
and- Mm. I mean- …I've lost
count of the number of places going

1394
01:03:17,425 --> 01:03:20,825
Don: Yeah …my view, it's
the center of the factory.

1395
01:03:20,825 --> 01:03:24,285
It's the, it is the software factory
where all the grown-up stuff happens.

1396
01:03:24,385 --> 01:03:27,825
So much of the forward progress
happens beyond the- yeah.

1397
01:03:27,955 --> 01:03:30,725
Maybe feature development, which
might be done by agentic, local,

1398
01:03:30,725 --> 01:03:34,085
or Pele actually does a whole lot
in the CI It's just Everything.

1399
01:03:34,585 --> 01:03:38,735
He wants a feature implemented, he just
writes an issue and, uh, or, or yeah,

1400
01:03:38,855 --> 01:03:40,705
everything comes through it's CI system.

1401
01:03:40,865 --> 01:03:41,375
Amazing.

1402
01:03:41,375 --> 01:03:43,555
It, You know, the, the software
factory- So you're- …is real.

1403
01:03:43,555 --> 01:03:43,885
And

1404
01:03:43,935 --> 01:03:48,265
Bret: so you're not prompting you're
issuing You're issue prompting?

1405
01:03:48,265 --> 01:03:48,365
Uh, no.

1406
01:03:48,415 --> 01:03:49,525
Peli: I skip the issue.

1407
01:03:49,525 --> 01:03:52,215
Uh, I set a prompt
directly, but a lot of it.

1408
01:03:52,645 --> 01:03:54,075
Are agents that create issues.

1409
01:03:54,105 --> 01:03:55,445
So the issue is Oh, okay …a work queue.

1410
01:03:55,445 --> 01:03:55,665
Right.

1411
01:03:56,075 --> 01:03:56,455
Mostly.

1412
01:03:56,695 --> 01:04:00,075
So a part of the work, which is
maintenance and code improvement,

1413
01:04:00,195 --> 01:04:04,435
uh, or documentation updates, would
be produced daily by, by workers.

1414
01:04:04,535 --> 01:04:05,885
but you know, what you're saying is right.

1415
01:04:05,925 --> 01:04:10,220
It's, you know, up until now you could
have a sloppy CI and for example, you

1416
01:04:10,220 --> 01:04:13,580
could rely on a good dev team to kind
of trust your, you know, they would do

1417
01:04:13,580 --> 01:04:15,290
the right thing, you know, the quality.

1418
01:04:15,680 --> 01:04:16,480
This is not true anymore.

1419
01:04:16,570 --> 01:04:20,700
The only thing, the only way you're
gonna leverage these agent is to have

1420
01:04:20,700 --> 01:04:25,600
a very, very tight CI with a very good
test suite and not just one test suite.

1421
01:04:25,630 --> 01:04:27,910
You need test of the test
integration test first test.

1422
01:04:27,910 --> 01:04:28,860
I mean, you bring it.

1423
01:04:28,960 --> 01:04:29,130
Yeah.

1424
01:04:29,130 --> 01:04:32,840
Because you need to triangulate them
so that they cannot escape that box.

1425
01:04:32,890 --> 01:04:36,140
And then you, and then once you
have that, you can get the boost.

1426
01:04:36,180 --> 01:04:38,410
The boost is the cloud, your dev box.

1427
01:04:38,410 --> 01:04:41,280
There's only so many eyes and so
many terminals you guys can handle.

1428
01:04:41,330 --> 01:04:44,540
So I don't know if you're, if you're
a spider, you get eight eyes and

1429
01:04:44,540 --> 01:04:48,860
you can maybe do 64 terminals in the
cloud, I can easily run hundreds.

1430
01:04:49,160 --> 01:04:49,420
Okay.

1431
01:04:49,420 --> 01:04:50,690
So let, me- There's,
there's like no compare.

1432
01:04:50,790 --> 01:04:51,220
Bret: Yeah.

1433
01:04:51,250 --> 01:04:54,250
Uh, let me ask real quick, 'cause
I think one of the things I love

1434
01:04:54,250 --> 01:04:58,060
about this, or I, I try to optimize
on this show is to change behavior.

1435
01:04:58,450 --> 01:05:01,280
Give people an insight that will actually
cause them to do something different

1436
01:05:01,670 --> 01:05:03,770
rather than just executive over-overviews.

1437
01:05:04,140 --> 01:05:05,850
not that we've been doing that, we've
been digging in the weeds, but I,

1438
01:05:05,850 --> 01:05:07,710
I'm, I'm actually very curious like.

1439
01:05:08,070 --> 01:05:12,970
if When you're on the forefront, can
you talk through what it, what your

1440
01:05:13,280 --> 01:05:16,900
activity looks like when you want
to create something new with the CI?

1441
01:05:16,900 --> 01:05:20,620
So you're, you're prompting the
LLM on your local harness, right?

1442
01:05:20,620 --> 01:05:23,050
It sounds like it's creating
the issue on your behalf.

1443
01:05:23,150 --> 01:05:25,440
You've got a bunch of automation
running in the background.

1444
01:05:25,840 --> 01:05:31,480
Are you asking it to like develop the
PR and then you're gonna wait for it

1445
01:05:31,480 --> 01:05:33,980
to tell you when the checks are ready?

1446
01:05:33,980 --> 01:05:37,970
Like are you even going to GitHub
or are you like harness first?

1447
01:05:37,970 --> 01:05:40,510
Like t-talk to a little bit through
that so that we can get an idea.

1448
01:05:40,510 --> 01:05:40,570
Okay.

1449
01:05:40,570 --> 01:05:40,730
I'm,

1450
01:05:40,730 --> 01:05:41,520
Peli: a bit extreme.

1451
01:05:41,800 --> 01:05:42,150
Sure.

1452
01:05:42,200 --> 01:05:43,970
Um- That's what I want, I want,

1453
01:05:43,970 --> 01:05:45,240
Bret: I want the red pill all the way.

1454
01:05:45,370 --> 01:05:45,460
Peli: down.

1455
01:05:45,530 --> 01:05:46,840
So first of all, it's fully async.

1456
01:05:46,840 --> 01:05:46,940
Okay.

1457
01:05:47,240 --> 01:05:50,580
there is no discussion on
my part with the agent.

1458
01:05:50,740 --> 01:05:54,800
I fire and forget through, git, the
GitHub cloud agent most of the time.

1459
01:05:54,860 --> 01:05:58,950
So, you know, you go either to github.com
and you do new agentic session.

1460
01:05:59,020 --> 01:06:00,250
I mostly use my phone.

1461
01:06:00,670 --> 01:06:02,190
So I do that from the iOS app.

1462
01:06:02,290 --> 01:06:03,160
So I'm not waiting.

1463
01:06:03,260 --> 01:06:06,110
That means I can have five to
10 agents running at all times.

1464
01:06:06,420 --> 01:06:08,540
And it's just like playing
chess on multiple boards.

1465
01:06:08,640 --> 01:06:09,480
Now they take time.

1466
01:06:09,780 --> 01:06:12,820
So if I forget what do you do between
the agents, you think about your future.

1467
01:06:12,920 --> 01:06:15,130
you have more time to think or you
talk to people In fact you have more

1468
01:06:15,130 --> 01:06:17,990
time to talk to people because the
agents are doing the work when they,

1469
01:06:17,990 --> 01:06:19,720
s- how is this gonna change your work?

1470
01:06:20,020 --> 01:06:20,340
So.

1471
01:06:20,640 --> 01:06:21,580
That's a very big one.

1472
01:06:21,670 --> 01:06:25,100
people are into the token, you
know, they look at the token

1473
01:06:25,100 --> 01:06:28,950
flowing down and it becomes a slot
machine, and they get addicted.

1474
01:06:29,070 --> 01:06:30,120
They get headaches and stuff.

1475
01:06:30,180 --> 01:06:30,710
They're tired.

1476
01:06:30,810 --> 01:06:30,920
Yeah.

1477
01:06:30,920 --> 01:06:32,870
Don't have, you know,
just schedule the work.

1478
01:06:33,100 --> 01:06:34,330
let it churn for a while, come back.

1479
01:06:34,710 --> 01:06:35,650
so that's One big thing.

1480
01:06:35,700 --> 01:06:39,380
Now when I determine that there's
a pattern that I'm doing the same

1481
01:06:39,380 --> 01:06:42,730
stuff all the time, or there's like
something I'm thinking in my head of an

1482
01:06:42,730 --> 01:06:44,680
automator, whoa, I could do that again.

1483
01:06:44,680 --> 01:06:46,920
You know, there's like,
there's some value to that.

1484
01:06:46,920 --> 01:06:50,300
So the most obvious one, one of the
first one we wrote, was like, this

1485
01:06:50,300 --> 01:06:52,010
agent creates a lot of duplicate code.

1486
01:06:52,110 --> 01:06:55,650
And it did it in kind
of a, a very subtle way.

1487
01:06:55,700 --> 01:06:59,560
it would rewrite string functions,
like string start, string, you know,

1488
01:06:59,560 --> 01:07:01,200
string split, but in different ways.

1489
01:07:01,300 --> 01:07:04,960
so I had this idea I was like, okay,
maybe we look at the, we look at

1490
01:07:04,960 --> 01:07:10,190
the function titles and we ask the
agent to bucketize them by intent, So

1491
01:07:10,290 --> 01:07:14,130
because the body of the function may
be s- completely different, but the

1492
01:07:14,130 --> 01:07:15,250
intent of the function, is the same.

1493
01:07:15,550 --> 01:07:20,370
so you open your phone and you say, create
me a daily agentic workflow that uses some

1494
01:07:20,370 --> 01:07:25,390
LSP to list all the functions or regex
to list all the function in code base.

1495
01:07:25,390 --> 01:07:28,950
Bucketize them by intent, pick the
biggest bucket and now generate

1496
01:07:28,950 --> 01:07:31,270
a prompt that says, remove all
these duplicates." That's it.

1497
01:07:31,370 --> 01:07:32,310
That's all you have to do.

1498
01:07:32,340 --> 01:07:34,340
Wait five minutes, you
get an agentic workflow.

1499
01:07:34,440 --> 01:07:37,640
As a PR, you review the PR,
you look at the safe outputs,

1500
01:07:37,740 --> 01:07:38,950
which are saved by default.

1501
01:07:39,350 --> 01:07:40,320
This is your first version.

1502
01:07:40,450 --> 01:07:43,230
probably not efficient, but it's gonna
run, it's gonna burn a lot of tokens.

1503
01:07:43,620 --> 01:07:46,100
It's not, it doesn't have the
right MCP, you know, it's not

1504
01:07:46,100 --> 01:07:48,420
optimized, but it kind of works.

1505
01:07:48,520 --> 01:07:48,810
Yeah.

1506
01:07:49,110 --> 01:07:51,020
And you're like, concept
this is real value.

1507
01:07:51,320 --> 01:07:52,660
Then, you know, we iterate and so forth.

1508
01:07:53,130 --> 01:07:57,580
And this concept of, I've got something
that, that annoys me in my code base

1509
01:07:57,650 --> 01:08:02,500
and I can get goodness I can get slight
improvement in a human consumable

1510
01:08:02,500 --> 01:08:06,800
way, which is basically the daily
newspaper concept is insanely good.

1511
01:08:07,020 --> 01:08:08,340
First of all, it's super fun.

1512
01:08:08,670 --> 01:08:11,280
You feel good because like, "Whoo,
you know, my code is better now.

1513
01:08:11,380 --> 01:08:15,040
it's a new way to also handle the
agent, because upstream that means

1514
01:08:15,040 --> 01:08:16,220
you don't need the perfect PR.

1515
01:08:16,320 --> 01:08:17,350
You can work in a feature branch.

1516
01:08:17,350 --> 01:08:20,350
You can go faster by
not doing the 17 nits.

1517
01:08:20,450 --> 01:08:22,050
Mm. because you have cleaners now.

1518
01:08:22,150 --> 01:08:24,490
You have tools that are
looking for patterns that are

1519
01:08:24,490 --> 01:08:25,790
known to happen with agents.

1520
01:08:26,040 --> 01:08:26,780
You know, they're gonna happen.

1521
01:08:26,780 --> 01:08:28,600
by the way, humans were
terrible at coding too.

1522
01:08:28,700 --> 01:08:29,420
We forgot that.

1523
01:08:29,810 --> 01:08:32,820
But, uh, so now you clean
everything, human or non-human,

1524
01:08:32,920 --> 01:08:34,240
you clean all the patterns.

1525
01:08:34,430 --> 01:08:36,160
and we've been, I mean
by now we have what?

1526
01:08:36,160 --> 01:08:38,380
27 or 50 running.

1527
01:08:38,380 --> 01:08:40,470
So we have also a summarizer
that looks at them.

1528
01:08:40,570 --> 01:08:41,630
I don't have time to look at them.

1529
01:08:41,630 --> 01:08:43,320
I have something that
mines them, We track them.

1530
01:08:43,630 --> 01:08:47,790
But it's always started from there is
something I'm doing all the time and I

1531
01:08:47,840 --> 01:08:49,650
wonder if the agent could actually help.

1532
01:08:50,030 --> 01:08:54,150
And this is important, like, I don't know
at the moment I'm writing the agentic

1533
01:08:54,150 --> 01:08:59,430
workflow whether it's gonna work or not,
Mm. but my time, my experiment costs

1534
01:08:59,430 --> 01:09:03,055
me five minutes, 10 minutes, So I'm not
investing three months to build a static

1535
01:09:03,055 --> 01:09:06,525
analysis tool like we used to, like,
Hey, maybe this works and some of them

1536
01:09:06,525 --> 01:09:08,255
are just terrible or just too expensive.

1537
01:09:08,355 --> 01:09:08,855
You, know, or,

1538
01:09:08,905 --> 01:09:10,905
they're, but a lot of them
are like, surprisingly good.

1539
01:09:11,005 --> 01:09:12,705
Then you put the right MCPS and so forth.

1540
01:09:12,805 --> 01:09:13,795
Uh, then we optimize.

1541
01:09:13,795 --> 01:09:14,435
By optimize.

1542
01:09:14,435 --> 01:09:17,235
I mean, think of an agentic
workflow as a concretized plan.

1543
01:09:17,235 --> 01:09:21,105
You've done sash plan, you paid
for opus, you burn a lot of tokens.

1544
01:09:21,105 --> 01:09:22,085
You've got a really good plan.

1545
01:09:22,185 --> 01:09:25,455
That one is now set in stone
in your agentic workflow.

1546
01:09:25,555 --> 01:09:27,545
That means you can go for
a lower model to implement.

1547
01:09:27,645 --> 01:09:27,685
Mm-hmm.

1548
01:09:27,685 --> 01:09:30,955
And then you can start splitting
into submodels and everything.

1549
01:09:31,055 --> 01:09:32,025
Lower your tokens.

1550
01:09:32,125 --> 01:09:36,445
And all of these are hyper-specialized
tools because they, the clearer the

1551
01:09:36,445 --> 01:09:38,105
goal, the better the agent gonna be.

1552
01:09:38,205 --> 01:09:40,475
So these are all kind of intuitions
that we build, that we've kind of

1553
01:09:40,475 --> 01:09:45,465
measured, that we use, um, so it's like
a plethora, like a, an insane amount of.

1554
01:09:45,765 --> 01:09:48,295
Very, very specific tools.

1555
01:09:48,395 --> 01:09:48,635
Yeah.

1556
01:09:48,735 --> 01:09:52,165
Linters are a good example you know, this
kind of tools we've built over the years.

1557
01:09:52,465 --> 01:09:53,965
Yeah, I love- Linters have rules.

1558
01:09:54,465 --> 01:09:57,425
Bret: My my favorite thing, and I think
I might even have it in my global agents

1559
01:09:57,425 --> 01:10:00,195
file on my machine is always, always lint.

1560
01:10:00,295 --> 01:10:04,365
Uh, at the end of every edit you make,
uh, or at the end of every run of an edit

1561
01:10:04,365 --> 01:10:08,055
you make, because I, I don't even wanna
s- I don't even wanna look at what your

1562
01:10:08,055 --> 01:10:11,745
output is unless it's passed linters and,
uh, at, I think the minimum, like you

1563
01:10:11,745 --> 01:10:14,515
ment- you mentioned the actionlint and the
zismore and, yeah, these are Yeah …like

1564
01:10:14,515 --> 01:10:16,665
table stakes for me, for GitHub workflows.

1565
01:10:16,895 --> 01:10:17,885
Uh, I was just curious real quick.

1566
01:10:17,885 --> 01:10:17,945
Yeah.

1567
01:10:17,945 --> 01:10:19,845
What is a, what does
optimization look like?

1568
01:10:19,845 --> 01:10:22,945
Is that just improving prompt,
it, uh, when you're So many

1569
01:10:22,945 --> 01:10:23,255
Peli: things?

1570
01:10:23,555 --> 01:10:23,785
Yeah.

1571
01:10:23,815 --> 01:10:26,935
So you wanna have the same
performance, the same reasoning,

1572
01:10:27,035 --> 01:10:28,275
at a lower cost, right?

1573
01:10:28,775 --> 01:10:29,655
that's really the end game.

1574
01:10:29,755 --> 01:10:33,855
you don't wanna degrade your performance,
but also you don't wanna pay, you

1575
01:10:33,855 --> 01:10:37,445
cannot just pay opus all the time, Uh,
so there's a lot of tricks in the bag.

1576
01:10:37,445 --> 01:10:40,605
And we talked about moving,
turns into the step side of

1577
01:10:40,605 --> 01:10:42,145
the action Pre-computing data.

1578
01:10:42,315 --> 01:10:43,825
Also, this grounds the agent.

1579
01:10:43,925 --> 01:10:47,555
You do the computation, you cook
some python, you're given the pre-fed

1580
01:10:47,555 --> 01:10:51,165
computation with all the aggregates and
say, now reason on this and don't do any

1581
01:10:51,165 --> 01:10:53,035
computation, don't make up numbers, right?

1582
01:10:53,065 --> 01:10:54,975
So that's also a trick too.

1583
01:10:55,075 --> 01:10:57,485
But then other things
are using small models.

1584
01:10:57,585 --> 01:11:02,285
So splitting a monolithic prompt into
a prompt plus small models, typically.

1585
01:11:02,285 --> 01:11:07,375
For example, if your task says, go and
summarize files, the file summary can be

1586
01:11:07,375 --> 01:11:09,025
done by a small agent, a subagent, Yeah.

1587
01:11:09,125 --> 01:11:10,195
Then comes back with the summary.

1588
01:11:10,405 --> 01:11:16,965
Right now, you've actually went from a 6X
model sonnet style, you know, 6X, to 0.3.

1589
01:11:17,315 --> 01:11:19,615
you've dropped by 20X your cost per token.

1590
01:11:19,715 --> 01:11:23,125
Um, and of course, you know,
there's a prompting and we have

1591
01:11:23,125 --> 01:11:24,485
AB testing in the platform.

1592
01:11:24,485 --> 01:11:28,725
So when you run at scale over thousands of
repos, you can start doing campaigns and

1593
01:11:28,815 --> 01:11:33,225
field test, prompt improvement and measure
in a scientific way, in a reasoned way.

1594
01:11:33,225 --> 01:11:36,355
Just like, you know, think these
are really websites, right?

1595
01:11:36,495 --> 01:11:38,075
at the scale you're going
to run these agents.

1596
01:11:38,375 --> 01:11:41,535
My belief is that it's not really an eval
thing, it's more like a website where you

1597
01:11:41,535 --> 01:11:45,345
do ab testing on features, you're going to
do ab testing on prompts or model and so

1598
01:11:45,345 --> 01:11:49,215
forth, and measure as you're spending the
money measure whether they work or not.

1599
01:11:49,535 --> 01:11:52,845
It's really hard to build evals
when you have hundreds of agents and

1600
01:11:52,845 --> 01:11:54,025
everything's moving all the time.

1601
01:11:54,125 --> 01:11:54,315
Right.

1602
01:11:54,315 --> 01:11:54,885
Uh, but- That's

1603
01:11:54,885 --> 01:11:55,005
that's

1604
01:11:55,005 --> 01:11:57,875
Bret: a lot of eval runs in addition
to the agent runs themselves.

1605
01:11:57,875 --> 01:11:58,135
Yeah.

1606
01:11:58,605 --> 01:11:59,025
But AB

1607
01:11:59,025 --> 01:12:03,025
Peli: testing is a proven way of
fielding, uh, improvements and measuring.

1608
01:12:03,325 --> 01:12:03,485
Don: Yeah.

1609
01:12:03,485 --> 01:12:06,875
And in terms of, you know, the
people on the call watching this, uh.

1610
01:12:06,975 --> 01:12:10,515
you know, One of the big behavior
changes and mindset changes is

1611
01:12:10,515 --> 01:12:13,615
like, what's your future work
gonna be in an enterprise, right?

1612
01:12:13,715 --> 01:12:18,175
It's not just running over one repo, but
you're gonna be the agentic maestro or a

1613
01:12:18,175 --> 01:12:24,255
team of agentic maestros who are able to
do things at scale across tens, hundreds

1614
01:12:24,295 --> 01:12:29,535
of repos in say, a major, uh, a major
organization that might be about like,

1615
01:12:29,575 --> 01:12:34,575
uh, applying security patches, uh, or
making judgment calls about how the impact

1616
01:12:34,625 --> 01:12:36,295
of what it means to roll things out.

1617
01:12:36,395 --> 01:12:39,655
It might be, uh, making things
more regular across those repos.

1618
01:12:39,655 --> 01:12:42,425
It might be, um, improving
the test, uh, assessing the

1619
01:12:42,425 --> 01:12:44,605
test coverage across the repo.

1620
01:12:44,665 --> 01:12:49,515
You know, if I was the CTO of a major,
um, all companies, software companies

1621
01:12:49,545 --> 01:12:53,145
these days, I'd kind of want a report
a summary report, up of like, what is

1622
01:12:53,145 --> 01:12:54,895
the, what are all the repos we have?

1623
01:12:55,055 --> 01:12:57,065
What are, what's the status of them all?

1624
01:12:57,065 --> 01:12:58,915
What, how do they Right
…cluster together?

1625
01:12:59,335 --> 01:13:02,235
How do they, how, what
technologies do we actually use?

1626
01:13:02,235 --> 01:13:05,975
And not just based on what we think
we use, but actually assessing,

1627
01:13:06,155 --> 01:13:07,565
what we actually depend on.

1628
01:13:07,565 --> 01:13:11,305
And so there's so much that's even on
information reporting, kind of working,

1629
01:13:11,305 --> 01:13:14,405
kind of going up even before you get
to kind of taking, improving all of

1630
01:13:14,405 --> 01:13:16,165
those different, uh, repositories.

1631
01:13:16,265 --> 01:13:20,015
Yeah, I, I, I, I think there's a, you
know, I've been worried for a long

1632
01:13:20,015 --> 01:13:21,965
time that some of these, you know,
I'm in London and the there are a

1633
01:13:21,965 --> 01:13:24,385
whole lot of investment banks down
the road, I've worked with some of

1634
01:13:24,385 --> 01:13:26,905
them over the years, and I actually
am really worried about some of them.

1635
01:13:26,905 --> 01:13:27,975
You know, I'm worried
about their software.

1636
01:13:27,975 --> 01:13:30,395
They, they tell me they've got
20,000 production systems or

1637
01:13:30,395 --> 01:13:31,555
something like that, right?

1638
01:13:31,555 --> 01:13:33,515
And It's just like,
Yeah … uh, it's, it's insane.

1639
01:13:33,515 --> 01:13:36,465
It's, the, the, the, The software
legacy debt that they have across

1640
01:13:36,465 --> 01:13:37,825
those systems is just huge.

1641
01:13:37,925 --> 01:13:40,945
Luckily for them, the tools have now
come along, which can deal with that.

1642
01:13:40,985 --> 01:13:44,655
But they need agentic Maestros to come
up with the workflows to get a grip

1643
01:13:44,745 --> 01:13:50,030
on that software complexity in uh,
through summarization and action and

1644
01:13:50,030 --> 01:13:52,870
all sorts of other, yeah, along various
dimensions are kind of kind of working.

1645
01:13:52,970 --> 01:13:55,610
So we're not just talking one repo,
we're talking a whole, the agentic

1646
01:13:55,720 --> 01:13:59,370
organization and how that actually
maps down to actually working

1647
01:13:59,370 --> 01:14:00,690
with real software artifacts.

1648
01:14:00,690 --> 01:14:03,460
It's not just something in theory
or something kind of, Some of these

1649
01:14:03,460 --> 01:14:06,720
automation platforms like, uh, you
know, Asana and NAD and the other

1650
01:14:06,720 --> 01:14:09,100
ones, they're much more on the
kind of information working side,

1651
01:14:09,100 --> 01:14:12,780
like working with the HR systems or
your, uh, ERP systems or whatever.

1652
01:14:12,880 --> 01:14:17,050
this makes it very concrete to me,
like I now know what the agentic

1653
01:14:17,050 --> 01:14:20,270
enterprise means for the whole
software side of the enterprise.

1654
01:14:20,310 --> 01:14:21,670
Yeah, it's very real.

1655
01:14:21,670 --> 01:14:24,300
It's like, it, it, there's a
lot of work to be done to make.

1656
01:14:24,400 --> 01:14:28,240
To actually crank the handle on
that and auth, a lot of this becomes

1657
01:14:28,280 --> 01:14:31,790
auth constrained, for example, who's
allowed to do this stuff, right?

1658
01:14:31,830 --> 01:14:31,920
Yeah.

1659
01:14:31,920 --> 01:14:36,940
the ideal job in this world is where you
have maximal auth and you are trusted.

1660
01:14:37,040 --> 01:14:41,900
And that is what the maestro really, the,
uh, the grand wizard of the enterprise.

1661
01:14:41,900 --> 01:14:44,110
That's the ideal job to have
in this kind of situation.

1662
01:14:44,110 --> 01:14:47,740
You've got lots of power, lots of
tokens to spend you, or can find

1663
01:14:47,740 --> 01:14:52,350
out what the actual business value
work, uh, to be done actually is.

1664
01:14:52,450 --> 01:14:55,500
And you can actually make it happen
not just shout from the sidelines a-

1665
01:14:55,630 --> 01:14:57,490
across some organizational divide.

1666
01:14:57,570 --> 01:15:01,090
So yeah, you want to be part of the grow-
If you're looking for a new job to, or

1667
01:15:01,090 --> 01:15:05,440
a new career direction you want to be,
you want to be leading the conversation,

1668
01:15:05,690 --> 01:15:10,960
the agentic software, automated software
conversation in your whole company.

1669
01:15:11,240 --> 01:15:12,040
However, that is

1670
01:15:12,140 --> 01:15:13,850
Bret: the continuous AI czar.

1671
01:15:14,220 --> 01:15:14,820
maybe we'll workshop the name.

1672
01:15:14,930 --> 01:15:15,720
Or the group of czars.

1673
01:15:15,720 --> 01:15:17,440
Don: title, I mean Yeah, The group.

1674
01:15:17,560 --> 01:15:19,710
I mean, it doesn't have to
be a single czar, but yeah.

1675
01:15:19,710 --> 01:15:24,920
The, the maestros, the, uh, the thought
leaders in the company, the people who

1676
01:15:24,920 --> 01:15:29,320
see this in multidimensional ways, uh, who
aren't just, they're not just nutty and

1677
01:15:29,320 --> 01:15:34,100
evangelistic, they've grown up about it,
but they can use re- repository automation

1678
01:15:34,140 --> 01:15:35,990
at scale for positive action across the

1679
01:15:36,020 --> 01:15:36,330
Peli: whole.

1680
01:15:36,360 --> 01:15:36,500
org.

1681
01:15:36,500 --> 01:15:40,350
it will be the catalyst for a
reorganization of software production.

1682
01:15:40,450 --> 01:15:44,440
The way we build software will change
because we will design new processes

1683
01:15:44,540 --> 01:15:47,520
between agents and humans where,
you know, we've been doing the pull

1684
01:15:47,520 --> 01:15:51,060
request for a while, the agents are
kind of, you know, kicked in the

1685
01:15:51,060 --> 01:15:52,850
door and starting to shake things up.

1686
01:15:52,910 --> 01:15:56,250
But there will be new ways to
build software and they will.

1687
01:15:56,350 --> 01:15:59,980
And you know, and we are seeing, in
a way we're experimenting with all

1688
01:15:59,980 --> 01:16:03,510
these new kind of flow, information
flow and production flow that involve

1689
01:16:03,510 --> 01:16:06,860
agents, human agents But, we do it fast.

1690
01:16:06,960 --> 01:16:08,450
We have the means to do it safely.

1691
01:16:08,550 --> 01:16:08,810
Yeah.

1692
01:16:08,810 --> 01:16:12,680
Don: My, yeah, the one I'm
currently, um, which is a kind

1693
01:16:12,680 --> 01:16:14,630
of a software factory, uh, image.

1694
01:16:14,630 --> 01:16:16,830
I've used that kind of
terminology quite a lot.

1695
01:16:17,010 --> 01:16:19,350
And, uh, when you said
like, what's the action?

1696
01:16:19,350 --> 01:16:21,390
How do I, how do I
start my design process?

1697
01:16:21,390 --> 01:16:25,670
At the moment it's about saying, let's
build a factory, uh, where there's

1698
01:16:25,710 --> 01:16:27,190
actually lots of inputs flowing in.

1699
01:16:27,230 --> 01:16:29,830
They might be issues of some
kind, but they might not be just

1700
01:16:29,830 --> 01:16:30,990
issues in a maintenance sense.

1701
01:16:30,990 --> 01:16:34,370
It might be like in, in the case of
GitHub, we have automated tools which

1702
01:16:34,370 --> 01:16:38,780
find problems in, in, in GitHub problems,
in the logic of how we use our databases

1703
01:16:38,780 --> 01:16:40,520
N+1 problems and things similar.

1704
01:16:40,960 --> 01:16:43,970
So you're kind of going to get these
to flow in and then you've got a

1705
01:16:43,970 --> 01:16:49,390
whole automation human, a mixture of
human and the factory is a, is a place

1706
01:16:49,390 --> 01:16:51,340
where both agents and humans work.

1707
01:16:51,540 --> 01:16:51,950
Crucially.

1708
01:16:52,050 --> 01:16:52,200
Okay.

1709
01:16:52,200 --> 01:16:54,760
And When you think of it as a
factory, then things can get

1710
01:16:54,760 --> 01:16:56,790
blocked at the human point.

1711
01:16:56,890 --> 01:17:00,050
Like the, uh, even when you get all
the automation set up, which is really,

1712
01:17:00,050 --> 01:17:04,800
really great, you, it can still get
blocked by overwhelming the human

1713
01:17:04,800 --> 01:17:06,850
with too much kind of generation.

1714
01:17:07,260 --> 01:17:09,690
And you can either scale that back
or you can increase the quality.

1715
01:17:09,690 --> 01:17:11,540
There may be good reasons why
they're doing that, or you can

1716
01:17:11,540 --> 01:17:13,620
actually turn off the whole
factory because it's not actually

1717
01:17:13,620 --> 01:17:15,560
serving the humans' needs properly.

1718
01:17:15,660 --> 01:17:15,910
Okay.

1719
01:17:15,910 --> 01:17:21,950
So the, the aim of the agentic
maestro is to design that human, i-e,

1720
01:17:21,980 --> 01:17:26,820
that, that, that agent-human factory
and make it flow, make it work.

1721
01:17:26,820 --> 01:17:30,630
'cause when it does flow, you get those
really dramatic results on quality.

1722
01:17:30,930 --> 01:17:31,400
Bret: Awesome.

1723
01:17:31,600 --> 01:17:36,430
I feel like it's been, this has been
a good discussion around, uh, agentic

1724
01:17:36,430 --> 01:17:40,780
workflows because I think that's not,
it's not a feature that we see in the UI

1725
01:17:40,780 --> 01:17:45,950
n- yet, so I feel like it's still really
early days in terms of getting everyone

1726
01:17:45,950 --> 01:17:49,040
to be aware that this thing exists
and how to go about implementing it.

1727
01:17:49,410 --> 01:17:52,840
So I'm excited about talking more about
it, and especially now that I, I feel

1728
01:17:52,840 --> 01:17:55,480
like you're giving me more reasons
to pay more attention to it because

1729
01:17:55,480 --> 01:17:59,240
I'm realizing that I've, even though
I've, I've dove into some of it and

1730
01:17:59,240 --> 01:18:02,670
implemented some of it that I'm, I'm
still, I feel s- I'm a babe in the woods.

1731
01:18:02,720 --> 01:18:04,150
I'm a babe in the woods
right now, but, um, so, so

1732
01:18:04,150 --> 01:18:04,550
are

1733
01:18:04,550 --> 01:18:04,750
Don: we.

1734
01:18:05,000 --> 01:18:05,160
Yeah.

1735
01:18:05,260 --> 01:18:06,470
So are we, yeah.

1736
01:18:06,780 --> 01:18:07,290
I wanted to ask you.

1737
01:18:07,290 --> 01:18:07,780
It's an exciting

1738
01:18:07,780 --> 01:18:11,710
Bret: time the last question or the
last topic before we wrap this up.

1739
01:18:11,810 --> 01:18:14,670
you're both, you're bo- o-
one of you is GitHub Next one

1740
01:18:14,670 --> 01:18:15,900
of you is Microsoft Research.

1741
01:18:15,900 --> 01:18:18,560
You're basically both
already thinking years out.

1742
01:18:18,610 --> 01:18:21,070
my assumption is that there's,
there's things that are coming.

1743
01:18:21,070 --> 01:18:23,560
So like what, this is all brand new to us.

1744
01:18:23,610 --> 01:18:25,450
It's, I'm sure it's still very new to you.

1745
01:18:25,450 --> 01:18:28,780
Like what, what is the thing that's
coming, not that this is ever replacing

1746
01:18:28,780 --> 01:18:31,870
this, but like what, what are you
excited about for the rest of the year?

1747
01:18:32,010 --> 01:18:38,150
what else do you, is it more pl- finding
more places that this can operate or like

1748
01:18:38,150 --> 01:18:43,200
sussing out the real value of where these
workflows are running in sort of m- like

1749
01:18:43,200 --> 01:18:46,990
maybe, maybe maybe making t- the top five
list or the top 10 implementation list?

1750
01:18:46,990 --> 01:18:50,060
Like where is it that you, you're
looking to in the short term?

1751
01:18:50,190 --> 01:18:50,740
take this.

1752
01:18:51,040 --> 01:18:51,570
Anyone wanna I'll let

1753
01:18:51,570 --> 01:18:51,980
Don: you first?

1754
01:18:51,980 --> 01:18:52,120
I'll

1755
01:18:52,220 --> 01:18:54,060
Peli: let you we're
still in techno preview.

1756
01:18:54,160 --> 01:18:54,440
Yeah.

1757
01:18:54,490 --> 01:18:57,340
So In a sense it's, we haven't,
we're still in first gear.

1758
01:18:57,640 --> 01:19:02,240
so very excited to see where, where
the product is gonna go and we're

1759
01:19:02,240 --> 01:19:06,630
gonna really be able to, to go out and
try to, and we have intuitions about

1760
01:19:06,630 --> 01:19:08,090
what's gonna happen when life scales.

1761
01:19:08,090 --> 01:19:09,030
We haven't really done it.

1762
01:19:09,130 --> 01:19:09,380
Yeah.

1763
01:19:09,640 --> 01:19:11,940
And very excited to actually learn.

1764
01:19:12,240 --> 01:19:14,570
Everything's fine when you
have one repo, 10 repo.

1765
01:19:14,570 --> 01:19:19,480
But we are very much looking forward
to 1000, 10,000, 100,000 scenarios.

1766
01:19:19,940 --> 01:19:23,750
And now looking at large scale,
agentics and all these scales, economies

1767
01:19:23,750 --> 01:19:25,630
are gonna happen, uh, through that.

1768
01:19:25,930 --> 01:19:28,330
personally I think it's
a golden era of CI.

1769
01:19:28,330 --> 01:19:32,190
I mean, if there's one thing out of
this discussion is yeah, stay in CI,

1770
01:19:32,640 --> 01:19:36,210
it's gonna get good because everybody's
gonna turn to you and say, how do I

1771
01:19:36,210 --> 01:19:38,220
run my agents in your, in your CI?

1772
01:19:38,270 --> 01:19:42,540
how do I do more agent stuff and
you know, and do all these, I heard

1773
01:19:42,540 --> 01:19:45,000
this and I wanna do it in your
CI and they're gonna turn to you.

1774
01:19:45,120 --> 01:19:45,410
Don: Yeah.

1775
01:19:45,410 --> 01:19:46,280
Peli: And they, they're
they're also, they're,

1776
01:19:46,280 --> 01:19:48,580
Don: al- they're also gonna
turn up and say, Hey, can I use

1777
01:19:48,580 --> 01:19:50,140
OpenClaw inside the enterprise?

1778
01:19:50,140 --> 01:19:50,280
You know?

1779
01:19:50,280 --> 01:19:52,810
'cause I wanna automate my,
like, uh, I wanna automate doing

1780
01:19:52,810 --> 01:19:54,440
my PRs inside the repository.

1781
01:19:54,440 --> 01:19:57,950
And it's just like, you know, maybe
you could go learn ag- get agentic

1782
01:19:57,950 --> 01:20:00,840
workflows because that's, that's
actually like a, you know, pre- it's

1783
01:20:00,840 --> 01:20:02,900
pretty safe way of doing automation.

1784
01:20:02,900 --> 01:20:03,050
Right.

1785
01:20:03,050 --> 01:20:04,030
And they're gonna turn up.

1786
01:20:04,030 --> 01:20:06,010
with… there's, lo- we're seeing
lots of other, We already kind of

1787
01:20:06,010 --> 01:20:08,770
touched on the, like run the naked
coding agents kind of approach.

1788
01:20:08,770 --> 01:20:11,830
And it's like, the answer to that is
go use GitHub agentic workflows, right?

1789
01:20:11,830 --> 01:20:13,930
'cause that's got, that's
got a security architecture.

1790
01:20:13,930 --> 01:20:16,010
You can also make, there'll
be other options as well.

1791
01:20:16,010 --> 01:20:17,210
There'll be other security, architectures.

1792
01:20:17,260 --> 01:20:17,600
Yeah.

1793
01:20:18,070 --> 01:20:20,330
but there's an answer to
a question that's there.

1794
01:20:20,330 --> 01:20:24,010
And, and we, we chose a continuous
AI framing because it was.

1795
01:20:24,110 --> 01:20:27,570
It's An industry, we wanted to create an
industry neutral term that a ju- uh, just

1796
01:20:27,570 --> 01:20:30,690
like you've created Agentic DevOps and,
they, and they're more or less, two, two,

1797
01:20:31,020 --> 01:20:36,310
very closely affinitized, which is great,
and, and they, they're questions with

1798
01:20:36,310 --> 01:20:40,100
an answer, with a natural answer, which
is GitHub Actions and, G- and GitHub,

1799
01:20:40,100 --> 01:20:43,790
which from a product perspect- product
development perspective is, I'm very

1800
01:20:43,790 --> 01:20:46,750
happy to have made those contributions
and where we've landed with all of that.

1801
01:20:46,850 --> 01:20:50,860
Uh, in terms of looking forward,
there's all the kind of rollout of

1802
01:20:50,860 --> 01:20:54,830
this kind of agentic workflows at scale
or agentic working in the enterprise.

1803
01:20:54,830 --> 01:20:56,210
And that's gonna take years to roll.

1804
01:20:56,240 --> 01:20:58,990
the enterprise turns slowly,
development teams turn slowly.

1805
01:20:58,990 --> 01:21:00,960
They've got their own opinions
and their own skilling.

1806
01:21:00,960 --> 01:21:04,430
As I said, this isn't, this is about
as much as I use the agentic maestro

1807
01:21:04,430 --> 01:21:08,810
thing, the Agentic maestro also listens
to the dev- developers, listens very

1808
01:21:08,810 --> 01:21:12,320
closely to them because they're the
ones up close to the coalface who

1809
01:21:12,320 --> 01:21:13,700
know how to maximize, they merge

1810
01:21:13,700 --> 01:21:13,770
the

1811
01:21:13,770 --> 01:21:14,840
Peli: PRs that are not,

1812
01:21:15,060 --> 01:21:16,980
Don: and they probably
merge the PRs or not.

1813
01:21:16,980 --> 01:21:20,470
If you're really lucky as an agentic
maestro, you get to merge PRs too.

1814
01:21:20,920 --> 01:21:22,650
It's like, it's, it's
good to have that power.

1815
01:21:22,750 --> 01:21:25,330
uh, but there's the, I'll,
I'll leave you with one thing.

1816
01:21:25,330 --> 01:21:29,170
that's a little bit further out,
uh, which is the… when we put

1817
01:21:29,170 --> 01:21:34,050
these workflows together, Previously
o often they're simulating what

1818
01:21:34,100 --> 01:21:36,930
we'd imagine a human to do, like
test improvement or test coverage.

1819
01:21:36,930 --> 01:21:40,440
They're doing one thing and you can
imagine getting someone in to do, improve

1820
01:21:40,440 --> 01:21:42,290
your tests and assigning them that job.

1821
01:21:42,390 --> 01:21:46,450
But nowadays we can actually get them
to use multiple kind of tools and

1822
01:21:46,450 --> 01:21:48,060
methodologies all at the same time.

1823
01:21:48,060 --> 01:21:51,510
And so if we look at say, performance
improvement, for example, uh, y- the-

1824
01:21:51,970 --> 01:21:55,590
this thing not only knows how to do
the profiling runs and how to write the

1825
01:21:55,590 --> 01:21:59,690
benchmarking kind of, tools and how to
do garbage collect and optimization.

1826
01:22:00,100 --> 01:22:02,130
It can also go read the
assembly code, right?

1827
01:22:02,160 --> 01:22:04,840
You know, which no- which n- we
none of us can do, Read right?

1828
01:22:04,840 --> 01:22:06,880
We can't interpret that and
it makes good sense of that.

1829
01:22:06,980 --> 01:22:12,990
And so, um, you can set up, if it's, if
you really needed to squeeze that last

1830
01:22:12,990 --> 01:22:18,320
2 or 3 or 5% out of, uh, a performance
out of, some say Go-based system, and

1831
01:22:18,320 --> 01:22:21,220
this might apply to say, GitHub or
something, then you could actually

1832
01:22:21,220 --> 01:22:26,460
set the agents also optimizing, adding
new optimizations to the go compiler.

1833
01:22:26,560 --> 01:22:26,930
Okay?

1834
01:22:27,110 --> 01:22:30,050
Like it can take us a private copy
of the Go compiler and make a, uh, a

1835
01:22:30,050 --> 01:22:34,460
kind of maker, si, you know, make its
own bots, compiler patches to that to

1836
01:22:34,460 --> 01:22:36,060
actually improve the register allocation.

1837
01:22:36,160 --> 01:22:40,780
Uh, and that's a multi-skilling thing
where you could never find a single.

1838
01:22:41,080 --> 01:22:44,250
Person who could-- had all
those skills across the board.

1839
01:22:44,350 --> 01:22:44,460
Right.

1840
01:22:44,460 --> 01:22:46,630
Ta- uh, Performance optimization-
The agent … it's full

1841
01:22:46,630 --> 01:22:48,110
of those kind of problems.

1842
01:22:48,110 --> 01:22:48,410
Right?

1843
01:22:48,800 --> 01:22:52,160
So, you know, the people who actually know
how to make their .NET or Java garbage

1844
01:22:52,160 --> 01:22:55,750
collectors, the memory hierarchies and
use them re- uh, so that everything flows

1845
01:22:55,750 --> 01:22:57,340
really, really nicely in those systems.

1846
01:22:57,680 --> 01:22:58,470
They're really rare.

1847
01:22:58,510 --> 01:23:01,840
The people who can do that, the agents
kind of know how to do that kind of work

1848
01:23:01,840 --> 01:23:04,090
where you can ki- you can encode code it.

1849
01:23:04,120 --> 01:23:08,930
So these kind of multi-skilling
flows, are super interesting.

1850
01:23:08,930 --> 01:23:14,200
Uh, I think, that, are beyond, they're,
they're a bit beyond what we, the frontier

1851
01:23:14,200 --> 01:23:16,520
of what we imagine these AI systems doing.

1852
01:23:16,520 --> 01:23:20,000
'cause they're kind of like little
teams of people or little teams of, of

1853
01:23:20,060 --> 01:23:22,690
cooperating kind of agents all taking.

1854
01:23:22,690 --> 01:23:24,940
a… It's not just critique
or different roles.

1855
01:23:24,940 --> 01:23:28,270
It's actually entirely different
compatible sort of skills, which kind

1856
01:23:28,270 --> 01:23:29,660
of compose together really nicely.

1857
01:23:29,760 --> 01:23:29,930
yeah.

1858
01:23:30,030 --> 01:23:30,740
So it's one idea.

1859
01:23:31,040 --> 01:23:31,360
Bret: All right.

1860
01:23:31,360 --> 01:23:34,650
I think the tagline for this,
this is, uh, besides that,

1861
01:23:34,870 --> 01:23:36,720
is another potential tagline.

1862
01:23:36,720 --> 01:23:38,520
'cause it, it sounds like I've
got five different options

1863
01:23:38,520 --> 01:23:40,290
for the title or the tagline.

1864
01:23:40,680 --> 01:23:44,670
It could be, uh, GitHub Actions
is the OpenClaw for grownups.

1865
01:23:44,970 --> 01:23:45,270
Dude.

1866
01:23:45,510 --> 01:23:48,020
I gotta get, I gotta think of Yeah …all
the buzzwords I gotta put in there.

1867
01:23:48,150 --> 01:23:48,210
Yeah.

1868
01:23:48,420 --> 01:23:48,480
Peli: I

1869
01:23:48,480 --> 01:23:49,510
gotta have the word agent somewhere.

1870
01:23:49,510 --> 01:23:49,600
in there.

1871
01:23:49,600 --> 01:23:50,390
That, that's your show.

1872
01:23:50,450 --> 01:23:50,540
Yeah.

1873
01:23:50,960 --> 01:23:51,830
your show.

1874
01:23:52,300 --> 01:23:53,060
Don: Absolutely.

1875
01:23:53,460 --> 01:23:53,740
it.

1876
01:23:53,840 --> 01:23:54,370
You know,

1877
01:23:54,710 --> 01:23:57,070
Bret: it's got cron, it's got A-
absolutely, we can, we've got memory.

1878
01:23:57,070 --> 01:23:58,100
It's got, yeah.

1879
01:23:58,100 --> 01:23:59,290
We can, it can learn over time.

1880
01:23:59,290 --> 01:23:59,350
Yeah.

1881
01:23:59,350 --> 01:23:59,690
The addition

1882
01:23:59,690 --> 01:24:02,960
Don: of the addition of memory to
these GitHub agentic workflows, Yeah

1883
01:24:02,960 --> 01:24:04,990
… makes huge, huge difference to that.

1884
01:24:04,990 --> 01:24:06,580
'cause now they can do research.

1885
01:24:06,580 --> 01:24:09,020
The first task they do when they
haven't done it before is they can

1886
01:24:09,020 --> 01:24:12,420
go research your code base and, and
actually work out how to do all that.

1887
01:24:12,420 --> 01:24:16,740
All those, uh, depth kind of
engineering things, uh, that, and

1888
01:24:16,740 --> 01:24:19,450
kind of keep their own private notes
on that, and update those notes.

1889
01:24:19,450 --> 01:24:19,880
Amazing.

1890
01:24:19,990 --> 01:24:20,900
Peli: We have action cache.

1891
01:24:21,130 --> 01:24:27,070
I mean we mount memory on action
caches on repo on comments in the wiki.

1892
01:24:27,170 --> 01:24:28,770
Plenty of places to store.

1893
01:24:28,870 --> 01:24:32,420
Uh, and then, you know, you've got
these long-running, We have a practical

1894
01:24:32,420 --> 01:24:33,870
Auto Loop, which is the auto loop.

1895
01:24:34,090 --> 01:24:38,300
Think of the auto researcher ref loop,
not just days, weeks, month, because

1896
01:24:38,300 --> 01:24:40,760
it's mounted on top of an action run.

1897
01:24:41,100 --> 01:24:44,090
Saves its data into a branch,
then action restarts on a

1898
01:24:44,090 --> 01:24:45,480
branch, and then it keeps going.

1899
01:24:45,580 --> 01:24:48,940
So if you think about all your inner loop
that are doing route for, let's say a

1900
01:24:48,940 --> 01:24:50,680
day now you have the outer loop of that.

1901
01:24:50,680 --> 01:24:50,770
Yeah.

1902
01:24:51,070 --> 01:24:51,900
That's gonna run for a month.

1903
01:24:52,380 --> 01:24:55,130
So you can point it at uh, an
entire code base and say, convert

1904
01:24:55,130 --> 01:24:56,270
this stuff into something else.

1905
01:24:56,570 --> 01:24:56,810
Don: Yeah.

1906
01:24:56,810 --> 01:24:58,650
But, uh, Wow OpenClaw for grownups.

1907
01:24:58,650 --> 01:24:58,790
Yeah.

1908
01:24:58,790 --> 01:25:01,020
No, automation is an incredible thing.

1909
01:25:01,020 --> 01:25:02,110
Very empowering.

1910
01:25:02,110 --> 01:25:06,890
And, uh, it's just, yeah, a lot you
can do with it, but make it safe.

1911
01:25:07,050 --> 01:25:10,940
Make a guardrail- And, yeah,
we're very happy to help provide

1912
01:25:10,940 --> 01:25:14,570
a basis for doing this, uh, at
scale in the enterprise and with

1913
01:25:14,690 --> 01:25:19,160
Peli: Um, we are very, we are a weird
open source project, but we've closed,

1914
01:25:19,370 --> 01:25:22,510
we take, uh, bugs as, uh, as specs.

1915
01:25:22,610 --> 01:25:24,030
but we've closed so far.

1916
01:25:24,400 --> 01:25:28,920
See the number today, 633
community bugs since we shipped.

1917
01:25:29,020 --> 01:25:32,320
so if you're using agentic Workflows
and you find something, uh,

1918
01:25:32,320 --> 01:25:36,270
run our agents on your workflow
and tell it to file an issue.

1919
01:25:36,370 --> 01:25:41,180
And, uh, yeah, we've been, uh,
running as fast as we can to

1920
01:25:41,180 --> 01:25:43,310
answer the needs of practitioners.

1921
01:25:43,410 --> 01:25:47,070
Most of the bugs we get are from
professional CI/CD engineers.

1922
01:25:47,540 --> 01:25:51,750
And this is, these are really deep GitHub
action feature that we didn't know.

1923
01:25:51,850 --> 01:25:51,910
Bret: Yeah.

1924
01:25:51,910 --> 01:25:52,020
Right.

1925
01:25:52,020 --> 01:25:52,300
Okay.

1926
01:25:52,300 --> 01:25:53,200
Yeah, so- All right.

1927
01:25:53,200 --> 01:25:53,450
Right.

1928
01:25:53,450 --> 01:25:56,660
Like the, the esoteric
edge cases of, uh, yes.

1929
01:25:56,660 --> 01:26:00,220
Of GitHub Actions when someone knows
every little nook and cranny of the Yeah.

1930
01:26:00,320 --> 01:26:03,640
Peli: Arc Runner on GHS
with something, something.

1931
01:26:03,800 --> 01:26:07,380
Uh, we've been looking at that and
you know, there's this thing where

1932
01:26:07,480 --> 01:26:11,140
the sandbox is closed by design,
but we also have the hooks for the

1933
01:26:11,140 --> 01:26:15,100
pros, for the people who know to
go and plug in the stuff they need.

1934
01:26:15,100 --> 01:26:20,270
GitHub apps, custom jobs, custom steps
uh, Uhashicorp step to get your secrets.

1935
01:26:20,270 --> 01:26:23,420
All this stuff is basically, you
know, we, we wanna fully leverage

1936
01:26:23,420 --> 01:26:25,610
the platform and we want the problem.

1937
01:26:25,610 --> 01:26:29,070
When you do a box that doesn't have the
escape hatch for the pros that then people

1938
01:26:29,100 --> 01:26:32,660
turn up that, you know, you need to get
stuff done, you turn up the security.

1939
01:26:32,800 --> 01:26:33,580
So we designed it.

1940
01:26:33,580 --> 01:26:33,640
Yeah.

1941
01:26:33,640 --> 01:26:33,730
bad

1942
01:26:33,730 --> 01:26:34,100
Bret: habits.

1943
01:26:34,320 --> 01:26:34,650
Yeah.

1944
01:26:34,950 --> 01:26:38,760
Peli: designed it so that you don't
have to turn up security to get into

1945
01:26:38,760 --> 01:26:45,390
these enterprise scenarios that are,
very complex CI/CD with multiple apps,

1946
01:26:45,390 --> 01:26:49,740
multiple security, multiple tokens, and
th- things become, you know, the sample

1947
01:26:49,740 --> 01:26:54,200
we see on the landing page becomes way
more, you know, when you start factoring

1948
01:26:54,200 --> 01:27:00,080
in real life CI/CD constraints, Then it
becomes, but that is because we build on

1949
01:27:00,080 --> 01:27:01,790
top of action and we're just an increment.

1950
01:27:02,090 --> 01:27:03,980
We inherit all that goodness.

1951
01:27:04,110 --> 01:27:05,220
We're in all the ecosystem.

1952
01:27:05,320 --> 01:27:07,460
So, I mean, the call for
action is try it out.

1953
01:27:07,790 --> 01:27:08,990
There's something you don't like.

1954
01:27:09,090 --> 01:27:14,450
We've been running as fast and
responding to, uh, we only have

1955
01:27:14,450 --> 01:27:18,610
a backlog of 30 issues and we've
closed 630, so we're, we're on it.

1956
01:27:18,710 --> 01:27:19,160
Bret: That's awesome.

1957
01:27:19,160 --> 01:27:19,350
Peli: Yeah.

1958
01:27:19,450 --> 01:27:19,870
Bret: Well, yeah.

1959
01:27:20,320 --> 01:27:21,110
Your agents are on it.

1960
01:27:21,420 --> 01:27:22,220
You're on top of the

1961
01:27:22,220 --> 01:27:22,960
Peli: agents.

1962
01:27:23,120 --> 01:27:24,680
No, the, we don't take PRs.

1963
01:27:24,980 --> 01:27:26,960
People run the agent on their side.

1964
01:27:27,460 --> 01:27:30,680
There's a full, ana- uh,
full investigation on their

1965
01:27:30,680 --> 01:27:31,750
run with their secrets.

1966
01:27:32,250 --> 01:27:35,430
Then it's anonymized and then
the agent files a generic issue.

1967
01:27:35,730 --> 01:27:36,250
Nice.

1968
01:27:36,350 --> 01:27:39,230
And that's just built in also it's
a crazy way to do all software.

1969
01:27:39,330 --> 01:27:42,630
Bret: You're right, It's built in,
it's built in, a feedback loop.

1970
01:27:43,040 --> 01:27:44,350
all right, so this has been awesome.

1971
01:27:44,350 --> 01:27:46,100
I'm so glad to have both of you here.

1972
01:27:46,140 --> 01:27:47,940
I'm very excited about the future of this.

1973
01:27:47,940 --> 01:27:50,970
I've been a heads down GitHub Actions
guy for a long time, and I feel

1974
01:27:50,970 --> 01:27:52,250
like This is my whole new thing.

1975
01:27:52,510 --> 01:27:55,360
And this episode has convince me that
I should have spent a lot more time

1976
01:27:55,360 --> 01:27:57,090
the last three months on it than I did.

1977
01:27:57,480 --> 01:28:00,440
you've convinced me and I'm excited
to get into the weeds of this.

1978
01:28:00,470 --> 01:28:01,970
Where can people find both of you,

1979
01:28:02,080 --> 01:28:05,350
Don: certainly if you, if you wanna
contribute ideas, uh, to the design of

1980
01:28:05,350 --> 01:28:08,940
GitHub agentic workflows or even just
feedback on using it or examples of using

1981
01:28:08,940 --> 01:28:11,500
it, there's the, the repo You can make
your pitch give us good feedback and

1982
01:28:11,500 --> 01:28:15,210
find us, make a pitch for a new feature,
a new coding agent or some new thing

1983
01:28:15,210 --> 01:28:16,990
you, you think sh- should be supported.

1984
01:28:17,290 --> 01:28:20,280
And it's all open source so you can
kind of, uh, s- see it all and work it

1985
01:28:20,280 --> 01:28:22,800
all out even before it, it gets to us.

1986
01:28:22,900 --> 01:28:23,880
and ship on Mondays.

1987
01:28:23,980 --> 01:28:25,060
Peli: You ship on Mondays?

1988
01:28:25,060 --> 01:28:25,180
Yeah.

1989
01:28:25,250 --> 01:28:25,500
You, c- and,

1990
01:28:25,500 --> 01:28:26,150
and

1991
01:28:26,150 --> 01:28:28,030
Don: uh, for me, you
can find me on LinkedIn.

1992
01:28:28,030 --> 01:28:30,780
I do a lot of posting on LinkedIn
and my blog as well, which,

1993
01:28:30,920 --> 01:28:35,540
Peli: yeah, I'm mostly on GitHub and
you can find me on LinkedIn, but Nice.

1994
01:28:35,840 --> 01:28:39,670
Please file an issue with your idea
why you think we should have it.

1995
01:28:39,970 --> 01:28:44,180
With the agentic plan, you know, burn
some opus tokens on making your claim.

1996
01:28:44,280 --> 01:28:46,480
a lot of them were one-shot in BB Kitcode.

1997
01:28:46,580 --> 01:28:46,850
Yeah.

1998
01:28:47,080 --> 01:28:47,350
Awesome.

1999
01:28:47,350 --> 01:28:50,860
And the turnaround, we've got some
three-hour turnaround sometimes if you're

2000
01:28:50,860 --> 01:28:54,650
at the right moment in time between
filing the issue and getting a release.

2001
01:28:54,950 --> 01:28:56,170
this is agentic speed.

2002
01:28:56,270 --> 01:28:56,580
Bret: Yeah.

2003
01:28:56,680 --> 01:29:00,100
Uh, I'm, I've been playing around
with GitHub mobile more and more.

2004
01:29:00,130 --> 01:29:04,570
And so it, it, the fact that you, that's
your workflow, that's your process

2005
01:29:04,670 --> 01:29:07,740
is convincing me that I need to lean
into more of that because I, I, it's

2006
01:29:07,740 --> 01:29:10,040
a, it's a habit that I don't have yet.

2007
01:29:10,040 --> 01:29:12,450
Like, I don't have the muscle
memory to go, I have an idea.

2008
01:29:12,450 --> 01:29:14,010
Let me jump into my GitHub app.

2009
01:29:14,070 --> 01:29:14,760
so I need to break that.

2010
01:29:14,760 --> 01:29:20,900
for everyone listening, it's
github.com/github/gh-aw, Obviously

2011
01:29:20,900 --> 01:29:22,380
there's a bunch of websites.

2012
01:29:22,380 --> 01:29:25,230
You can go to githubnext.com
to see all the exciting stuff

2013
01:29:25,690 --> 01:29:26,980
coming out of the research.

2014
01:29:27,010 --> 01:29:29,980
Uh, basically just a, a bunch of, it's
a whole list of smart people that I

2015
01:29:29,980 --> 01:29:32,730
basically wanna invite on, all, of
them on the show for some things.

2016
01:29:32,730 --> 01:29:35,010
I think I, I just saw the presentation
Love it …from AI Engineer

2017
01:29:35,010 --> 01:29:39,520
Linden The, n- yeah, the the new,
team-based agentic harness mindset.

2018
01:29:39,560 --> 01:29:41,980
I am like, that is my next, I am.

2019
01:29:41,980 --> 01:29:46,220
So I saw that demo and thought this
is exactly what I've been missing

2020
01:29:46,220 --> 01:29:49,080
and what my teams probably want,
and, a replacement for Slack,

2021
01:29:49,570 --> 01:29:50,550
and like all these other things.

2022
01:29:50,550 --> 01:29:51,740
So I'm very excited about that one.

2023
01:29:51,840 --> 01:29:54,200
and I was like, immediately
went to sign up for the beta.

2024
01:29:54,200 --> 01:29:58,060
I don't know if she realized that there
might be a whole lot of signups, but

2025
01:29:58,060 --> 01:30:00,760
when I shared it out, I got a bunch of
responses from people going, oh, yes.

2026
01:30:00,860 --> 01:30:02,780
You know, not just yes,
but hell yes on that.

2027
01:30:02,780 --> 01:30:08,410
So I'm excited to see how these two areas
merge and I can have the prompt cr uh,

2028
01:30:08,410 --> 01:30:12,930
crowdsourced from my humans and like
we perfect the ai- output of possibly

2029
01:30:12,930 --> 01:30:16,340
how these GitHub action workflows
are gonna be created by an agent.

2030
01:30:16,340 --> 01:30:17,910
Don: Bret, Thank you for having us on.

2031
01:30:17,910 --> 01:30:18,880
It's been a lot of fun

2032
01:30:19,030 --> 01:30:21,370
Bret AI July 2025: Thanks for joining
us, and I'll see you in the next episode.