TRANSCRIPTION 🔗
1
00:00:00,064 --> 00:00:03,024
Bret: This is DevOps and Docker
Talk, and I'm your host, Bret Fisher.
2
00:00:03,524 --> 00:00:09,024
In this episode, we're gonna get
into the weeds with agents and CI.
3
00:00:09,064 --> 00:00:13,790
I've got the team from Semaphore on the
show, Marcos and Marko, who joined the
4
00:00:13,790 --> 00:00:16,461
show to really just talk about their
transformation over the last year.
5
00:00:16,470 --> 00:00:20,100
Now, if you've been around a minute
with this show, I had Semaphore on
6
00:00:20,100 --> 00:00:24,240
probably five years ago, four to
five years ago, and we talked about
7
00:00:24,260 --> 00:00:29,130
sort of typical CI/CD stuff, and this
time it was completely different.
8
00:00:29,180 --> 00:00:33,830
just in the last year, they have
completely transformed their product
9
00:00:33,840 --> 00:00:37,220
and their tooling to be AI first,
and we get into what that really
10
00:00:37,220 --> 00:00:39,630
means, and it's not… In this case,
I don't feel like it's a buzzword.
11
00:00:39,630 --> 00:00:43,157
I think it truly is an optimized
way for interacting with your
12
00:00:43,157 --> 00:00:44,917
CI from your agent harnesses.
13
00:00:45,217 --> 00:00:49,837
And two, they open sourced their
platform, so you can now run it yourself
14
00:00:49,837 --> 00:00:51,657
if you don't wanna pay them to run it.
15
00:00:51,704 --> 00:00:57,444
So they offer the typical options of
running their open source CI or running
16
00:00:57,444 --> 00:01:00,524
it on their platform, paying for runners
and having them run their, their runners.
17
00:01:00,524 --> 00:01:03,440
They actually, just recently did a
benchmark comparing themselves to a lot
18
00:01:03,440 --> 00:01:07,040
of the other CIs and saw that they were
cheaper and faster than the others.
19
00:01:07,040 --> 00:01:11,980
we dug into exactly what it means to
just drive your CI completely from an
20
00:01:11,980 --> 00:01:15,300
agent harness and not just sort of in
the contrived way that I've been doing
21
00:01:15,300 --> 00:01:19,420
it, which is, you know, telling it, "Hey,
can you go check on that CI job? Hey, can
22
00:01:19,420 --> 00:01:24,050
you make sure that that git push commit
thing happened?" And, like, all the…
23
00:01:24,060 --> 00:01:29,354
Like, they've sort of rethought it from,
I would say, first principles of Why does
24
00:01:29,524 --> 00:01:34,104
CI running, or why do tests or workflows
that I need to run around my code, like
25
00:01:34,114 --> 00:01:37,904
linting, testing, and all the different
forms of that testing, security scanning,
26
00:01:37,904 --> 00:01:41,654
like why do I need to wait on a git
push in order to kick those things off?
27
00:01:41,664 --> 00:01:46,684
What would happen if my agent could know
everything about my CI just from starting
28
00:01:46,684 --> 00:01:51,584
up, have all the tools and MCPs and skills
and command lines, have everything it
29
00:01:51,584 --> 00:01:56,654
Load full transcript
needs on every project, and then when I
start a feature build, it knows that it
30
00:01:56,654 --> 00:02:01,424
can spin up testing and all the other
various workflows I might have without
31
00:02:01,424 --> 00:02:03,334
needing to make a commit or pushing code.
32
00:02:03,350 --> 00:02:06,230
And they have multiple features
that sort of come together in
33
00:02:06,230 --> 00:02:07,630
something they call test boxes.
34
00:02:07,630 --> 00:02:11,930
And so we get into a lot of that because
that, to me, has been my vision of where
35
00:02:11,930 --> 00:02:18,050
I think CI/CD is going for agents, where
they are free to run CI anytime they want.
36
00:02:18,050 --> 00:02:22,360
They don't necessarily have to run it
through the PR typical workflow just
37
00:02:22,370 --> 00:02:27,220
to be able to run tests and run lints
and do scans and make sure that their
38
00:02:27,220 --> 00:02:31,650
generated code is gonna work, and that
feels like what they've just built.
39
00:02:31,760 --> 00:02:35,120
I went through their backlog of blogs,
and for the last six months or more, they
40
00:02:35,130 --> 00:02:39,030
have been posting very regularly about
their updates to their open source CLI
41
00:02:39,040 --> 00:02:43,454
tooling, to all the dozens of skills that
they've made for their CI, everything
42
00:02:43,454 --> 00:02:47,280
from helping it automate the fixing of
flaky tests to essentially onboarding
43
00:02:47,890 --> 00:02:51,920
yourself to their entire platform by just
installing the Claude Code plugin or the
44
00:02:51,920 --> 00:02:56,444
Codex plugin and just saying, "Hey, set me
up on Semaphore, build me some workflows,
45
00:02:56,484 --> 00:02:58,184
and keep working until they're green."
46
00:02:58,207 --> 00:03:00,497
Which is a pretty, I don't know.
47
00:03:00,530 --> 00:03:04,510
That sounds like a demo type of prompt,
but it seems like that's what they've
48
00:03:04,520 --> 00:03:08,680
built and that they've designed this
whole platform and their skills and
49
00:03:08,680 --> 00:03:13,117
their workflows and MCP tooling around
just letting the agent cook and not
50
00:03:13,127 --> 00:03:17,227
having to worry about whether it's
figuring out CI or whether it needs to
51
00:03:17,227 --> 00:03:21,397
go check CI again or whether it needs to
fix that thing that it pushed into CI.
52
00:03:21,677 --> 00:03:24,297
And it just feels like it, they're
streamlining the whole process,
53
00:03:24,297 --> 00:03:25,277
and I'm excited to get into it.
54
00:03:25,287 --> 00:03:29,627
So let's jump in to Marcos and Marko
tell us their journey to building
55
00:03:29,627 --> 00:03:34,264
an open source AI native CI/CD
platform all right, let's get the
56
00:03:34,274 --> 00:03:36,284
titles and locations out of the way.
57
00:03:36,294 --> 00:03:38,324
Marcos, what do you do
and where are you from?
58
00:03:38,824 --> 00:03:42,854
Marcos: Well, I do software, but
I'm software engineer in Semaphore,
59
00:03:42,894 --> 00:03:45,254
and I'm here from Natal, Brazil.
60
00:03:45,754 --> 00:03:46,374
Bret: Nice.
61
00:03:46,794 --> 00:03:47,334
Brazil.
62
00:03:47,864 --> 00:03:49,514
Marko, same question
63
00:03:50,014 --> 00:03:50,394
Marko: Yeah.
64
00:03:50,394 --> 00:03:53,854
So I'm, responsible for the
product at Semaphore, and I'm
65
00:03:53,854 --> 00:03:55,704
calling in from Slovenia, Europe
66
00:03:55,863 --> 00:03:56,413
Bret: nice.
67
00:03:56,413 --> 00:04:01,233
and you're now alumni of the DevOps
and Docker Talks show, so welcome back.
68
00:04:01,283 --> 00:04:06,513
We talked a lot last time around
sort of how Semaphore was optimizing
69
00:04:06,573 --> 00:04:10,163
CI/CD, and we were f- we were talking
a lot about, local testing back then.
70
00:04:10,173 --> 00:04:11,773
I mean, this was pre-AI.
71
00:04:11,783 --> 00:04:15,863
It was almost like, is there anything
relevant in that conversation anymore?
72
00:04:16,073 --> 00:04:18,883
Because everything has changed,
especially when it comes to automation,
73
00:04:19,253 --> 00:04:23,453
and I'm excited to get into it because
you all recently went, well, in the
74
00:04:23,453 --> 00:04:26,123
last year, like the rest of us, you
went through a journey as a company
75
00:04:26,133 --> 00:04:30,263
as well as individual developers and
figuring out what agents can do for you.
76
00:04:30,483 --> 00:04:36,833
Marko, tell us about the journey
of, like, how did you start s- first
77
00:04:36,833 --> 00:04:41,733
thinking about CI as something that
needed to sort of be reinvented?
78
00:04:41,733 --> 00:04:43,773
I don't know if that's the ri- maybe
that's a strong word, reinventing.
79
00:04:44,123 --> 00:04:48,203
But, I feel like there's so much
opportunity for changing the way that
80
00:04:48,703 --> 00:04:52,343
automation and CI testing works now
that it… how did you come about this?
81
00:04:52,843 --> 00:04:53,753
What's that origin story?
82
00:04:54,755 --> 00:04:56,525
Oh, hey Thanks for listening.
83
00:04:56,545 --> 00:05:00,075
And I got a couple of quick announcements
about stuff that I've created.
84
00:05:00,085 --> 00:05:02,475
A lot of this is free stuff
you can get right now.
85
00:05:02,825 --> 00:05:06,435
Just in case you've not seen some
of my other videos or stuff I've
86
00:05:06,435 --> 00:05:11,405
been doing on my newsletter, I've
created a GitHub security scanner
87
00:05:11,735 --> 00:05:13,485
that is an open source project.
88
00:05:13,665 --> 00:05:18,425
It's called GASA, GitHub Actions
Security Assessment, and it's
89
00:05:18,425 --> 00:05:20,065
expanded to even have more rules.
90
00:05:20,065 --> 00:05:23,955
I think I'm about at now a dozen
rules on it that scans your repos,
91
00:05:23,985 --> 00:05:27,875
including the entire organization,
and the focus of it isn't to replace
92
00:05:27,895 --> 00:05:32,255
other linters like the GitHub Actions
Linter or Sysmor or other scanners
93
00:05:32,255 -->
00:05:34,125
like Poutine that are all really great.
94
00:05:34,335 --> 00:05:38,085
It's meant to supplement those and
solve my own problems as well as my
95
00:05:38,085 --> 00:05:40,405
clients' problems, and I just thought
I would open source it to give it
96
00:05:40,405 --> 00:05:43,965
to everyone, and I've spent a lot of
time trying to make it a solid tool.
97
00:05:44,035 --> 00:05:48,475
Actually just spent some today, time
today in Fable Model building out
98
00:05:48,485 --> 00:05:52,145
some better testing frameworks to
test it against real organization
99
00:05:52,145 --> 00:05:54,435
repos as a part of its PR testing.
100
00:05:54,435 --> 00:05:58,905
But the whole purpose of this is
to scan its personal repos or even
101
00:05:58,905 --> 00:06:02,805
repos you might wanna use, or mostly
your organizational repos all at one
102
00:06:02,805 --> 00:06:08,065
time to give you this report on your
stance of GitHub Action security.
103
00:06:08,215 --> 00:06:11,595
And it looks not just at the
workflows, but also the repo
104
00:06:11,595 --> 00:06:12,865
settings on the back end.
105
00:06:13,325 --> 00:06:17,195
And this is based on my half decade
of consulting around GitHub Actions,
106
00:06:17,475 --> 00:06:18,875
and there's no agenda there.
107
00:06:18,875 --> 00:06:19,815
It's completely free.
108
00:06:20,065 --> 00:06:23,955
I've actually been using it to implement
and upgrade the security of my own repos
109
00:06:23,975 --> 00:06:27,855
to make sure that I'm not as vulnerable to
some of the recent supply chain attacks.
110
00:06:28,055 --> 00:06:31,835
So the whole reason of doing it was people
were asking me to help them lock down
111
00:06:31,835 --> 00:06:35,165
their supply chain against the attacks
we've seen over the last year on GitHub
112
00:06:35,165 --> 00:06:39,085
Actions, particularly around open source
repos, or even using other people's
113
00:06:39,085 --> 00:06:41,275
GitHub actions in a more secure way.
114
00:06:41,305 --> 00:06:45,275
It will guide you through how you
can change and improve the security.
115
00:06:45,275 --> 00:06:48,135
It won't actually change all the
settings for you, because I'm not
116
00:06:48,135 --> 00:06:51,215
quite comfortable with it going in
blast radius of your entire org.
117
00:06:51,545 --> 00:06:52,435
But go check that out.
118
00:06:52,475 --> 00:06:55,075
It's on my GitHub, and the
link is in the show notes.
119
00:06:55,575 --> 00:07:01,805
Next, I had in July a GitHub
Actions security workshop for free
120
00:07:02,075 --> 00:07:03,715
because Chainguard sponsored me.
121
00:07:03,715 --> 00:07:07,175
I was actually gonna charge for this
workshop 'cause it was hours worth
122
00:07:07,185 --> 00:07:12,765
of hands-on learning around how to
use your own agent or your own hands
123
00:07:12,765 --> 00:07:16,955
to actually lock down your GitHub to
make sure that you're not vulnerable.
124
00:07:16,975 --> 00:07:21,915
So it's walking through what this GASA
tool does, but also why these settings
125
00:07:21,915 --> 00:07:24,655
need to be locked down, why you need to
care about those particular settings,
126
00:07:24,655 --> 00:07:28,645
and what happens when you maybe do one
thing in a workflow and another thing
127
00:07:28,645 --> 00:07:32,305
in a security setting in your admin
settings, and those two things conflict
128
00:07:32,305 --> 00:07:37,015
and cause a security potential risk that
GitHub isn't probably warning you about.
129
00:07:37,295 --> 00:07:40,295
So it's meant to supplement all the
current stuff out there that really
130
00:07:40,295 --> 00:07:44,085
just focuses on the workflows, but
don't actually also consider the
131
00:07:44,385 --> 00:07:47,925
admin settings of your workflows
which are behind the scenes, right?
132
00:07:48,015 --> 00:07:50,775
People can't see those settings, and
they're not usually easy to find.
133
00:07:50,775 --> 00:07:54,045
So anyway, you can go look at that
workshop also free on my website.
134
00:07:54,045 --> 00:07:56,423
Thanks to Chainguard for doing that
and making it- Free for everyone.
135
00:07:56,813 --> 00:08:02,953
And finally, if you didn't know about
my Agentic DevOps Guild, we have been
136
00:08:02,953 --> 00:08:06,973
meeting weekly since March, where we--
I think we're, like, twenty-five weeks
137
00:08:07,303 --> 00:08:13,123
of weekly meetings, and we're now over
fifty teams of people in there solving
138
00:08:13,123 --> 00:08:17,603
problems for their team on how we're gonna
manage agents, how we're gonna manage
139
00:08:17,893 --> 00:08:23,253
the security and the sandboxing of our
AI for platform engineering use, DevOps,
140
00:08:23,533 --> 00:08:25,913
anything related to cloud and Kubernetes.
141
00:08:25,913 --> 00:08:29,393
If you're somewhere in an ops role,
this was the program designed for you.
142
00:08:29,403 --> 00:08:32,633
It's not just meetups, but
I'm also creating courses.
143
00:08:32,633 --> 00:08:37,683
I've got two courses in the works
right now where videos drop on a sort
144
00:08:37,683 --> 00:08:41,853
of a random basis, but based on what
comes out of our meetings, I turn that
145
00:08:41,863 --> 00:08:44,493
into lessons for everyone to adopt.
146
00:08:44,753 --> 00:08:48,263
And also that comes with workshops
that we do, including this Gaza
147
00:08:48,263 --> 00:08:49,623
workshop that was last month.
148
00:08:49,993 --> 00:08:52,143
And those workshops are
meant for the guild.
149
00:08:52,143 --> 00:08:54,243
The guild will always get
those for free, even if someone
150
00:08:54,243 --> 00:08:55,473
doesn't sponsor the workshop.
151
00:08:55,863 --> 00:08:59,843
But go check that out also on my
website, and I hope that helps solve
152
00:08:59,853 --> 00:09:01,653
some of your AI adoption problems.
153
00:09:02,103 --> 00:09:02,783
Back to the episode.
154
00:09:03,753 --> 00:09:04,023
Marko: Yeah.
155
00:09:04,023 --> 00:09:09,403
So,
so, s- with all this automation
and, the, the CI is becoming even
156
00:09:09,403 --> 00:09:12,753
more important for the folks,
especially that the whole development
157
00:09:12,753 --> 00:09:16,223
cycle is, getting shorter and
there's a faster turnaround time.
158
00:09:16,723 --> 00:09:21,793
So, we started thinking, how can we
enable our customers to kind of get
159
00:09:21,793 --> 00:09:25,343
on that train even faster and leverage
the whole technology that is emerging?
160
00:09:25,843 --> 00:09:29,153
And of course, when we started
thinking about it, the first
161
00:09:29,153 --> 00:09:30,403
thing that we did, what is it?
162
00:09:30,593 --> 00:09:34,783
Like la- last quarter of last
year, we created an MCP server
163
00:09:35,073 --> 00:09:38,183
because that was the thing back
then, If, everybody was doing that.
164
00:09:38,263 --> 00:09:40,013
Bret: Yeah, it's crazy how early MCP was.
165
00:09:40,023 --> 00:09:43,233
Like, it was pre-skills, I
think pre-agent files really.
166
00:09:43,263 --> 00:09:43,543
Yeah.
167
00:09:44,005 --> 00:09:44,325
Marko: Yeah.
168
00:09:44,725 --> 00:09:45,515
Yeah, for sure.
169
00:09:45,855 --> 00:09:47,295
But that was the thing back then.
170
00:09:47,355 --> 00:09:50,215
It's kind of like everybody was like,
you know, "Hey, that's the way how we
171
00:09:50,215 --> 00:09:51,795
are going to connect to different tools."
172
00:09:52,215 --> 00:09:56,385
And I think I believe it still is,
but as we were starting to develop our
173
00:09:56,385 --> 00:10:01,415
internal processes around agents and
scale our agentic use, we have realized
174
00:10:01,415 --> 00:10:04,995
that it's maybe something different,
and it's not one or the other thing,
175
00:10:05,235 --> 00:10:08,955
but it's actually both things or all
things because all the developers,
176
00:10:08,955 --> 00:10:12,375
they have a special preferences,
and they have their good reasons why
177
00:10:12,405 --> 00:10:14,825
they prefer one, one or the other.
178
00:10:15,325 --> 00:10:19,605
But working with our customers, what
became very obvious is that, we have this
179
00:10:19,655 --> 00:10:23,015
new surface that the developers are using.
180
00:10:23,015 --> 00:10:26,235
It's not it's not before you had like,
I don't know, five or six windows
181
00:10:26,245 --> 00:10:29,225
that you were switching through in
order to do your work as a developer.
182
00:10:29,725 --> 00:10:33,845
And nowadays it was becoming more,
more clear that either you upload
183
00:10:33,885 --> 00:10:39,605
code or codecs or, pick a, a, a coding
agent, it's becoming your only surface.
184
00:10:39,675 --> 00:10:42,975
And you want to be able to, to
talk to it and that everything
185
00:10:42,975 --> 00:10:44,035
needs to be done, there.
186
00:10:44,035 --> 00:10:47,545
And then we were fine looking into
"Hey, what is the way, how can we
187
00:10:47,545 --> 00:10:52,635
interface with the coding agent of our
customer's choice?" Yeah, in a very
188
00:10:52,635 --> 00:10:56,245
efficient way and give it everything
that, that the developer needs to do the
189
00:10:56,245 --> 00:10:57,945
work, that the developer needs to do.
190
00:10:57,965 --> 00:11:02,045
So, in essence, Semaphore is all
about developers' productivity.
191
00:11:02,515 --> 00:11:05,745
And then for us this was like, "Hey,
okay, we need to get in like we need to
192
00:11:05,745 --> 00:11:10,065
create an interface that feels native
for this new surface coding agent," and
193
00:11:10,065 --> 00:11:12,685
that's how the whole Sem AI emerged.
194
00:11:13,185 --> 00:11:15,655
And it's, yeah, Semaphore for AI.
195
00:11:15,705 --> 00:11:17,485
The that is all about it.
196
00:11:17,945 --> 00:11:21,725
And then, as we were developing our
own hands-on experience, we were
197
00:11:21,755 --> 00:11:24,795
kind of thinking like, "Hey, how
do you connect efficiently with the
198
00:11:24,795 --> 00:11:29,425
coding agent?" And then, Marcus was
our head engineer on that project.
199
00:11:29,695 --> 00:11:34,095
He came up with the concept of "Hey,
let's create the CLI." But the CLI
200
00:11:34,095 --> 00:11:38,275
is not only commands, but it's a
mix of the commands and MCP server
201
00:11:38,285 --> 00:11:42,595
that is a local server, so that the
developers can choose their preference.
202
00:11:42,595 --> 00:11:47,145
But also agents themselves can also like,
use whatever they prefer at the same, at
203
00:11:47,145 --> 00:11:51,885
the time, because we also noticed that,
for certain tasks, developer, agents
204
00:11:51,885 --> 00:11:54,815
have the preferences for some reason.
205
00:11:55,015 --> 00:11:58,585
So it's that, that was one of, one
of, many, confirmations that it's
206
00:11:58,585 --> 00:12:00,165
not one or the other, but it's both.
207
00:12:00,665 --> 00:12:04,115
And then we try to make it as, as
optimal for the agents as possible.
208
00:12:04,235 --> 00:12:09,375
So everything is you know, JSON
native and all the commands are very
209
00:12:09,375 --> 00:12:13,245
well documented, not for the humans
to read that, but for the agents to
210
00:12:13,255 --> 00:12:17,385
be able to use them and understand
the use case and a lot of examples.
211
00:12:17,885 --> 00:12:23,595
But from the product perspective,
my kind of vision for this interface
212
00:12:23,605 --> 00:12:29,305
was that, hey, let's enable people
to have the natural language in a
213
00:12:29,305 --> 00:12:33,725
conversation with the CI and then
lower their bar- that barrier of entry.
214
00:12:34,145 --> 00:12:37,825
Because in order to deploy a, a kind
of you know, put in production a
215
00:12:37,835 --> 00:12:42,045
CI/CD tool as you… I can imagine all
the viewers know it's a lot of work.
216
00:12:42,545 --> 00:12:47,045
You had to learn the platform,
develop a com- a, a configuration,
217
00:12:47,295 --> 00:12:48,665
make sure to test everything.
218
00:12:48,965 -->
00:12:49,355
Bret: Yeah.
219
00:12:49,725 --> 00:12:51,155
Learn how the files work, yeah
220
00:12:51,313 --> 00:12:52,143
Marko: Yeah, learn.
221
00:12:52,657 --> 00:12:54,387
Marcos: The first try was SemaBot, right?
222
00:12:55,393 --> 00:12:56,463
Marko: Exactly, yeah.
223
00:12:56,963 --> 00:13:00,353
And yeah, so that, that was kind of
like also in the, in that during the
224
00:13:00,363 --> 00:13:03,793
journey we had the OpenClaw became
a very big thing, and then we did
225
00:13:03,793 --> 00:13:05,963
our own version of OpenClaw for CI.
226
00:13:06,463 --> 00:13:08,673
But there are a couple of things
from the product perspective
227
00:13:08,673 --> 00:13:10,053
that, that was very important.
228
00:13:10,113 --> 00:13:14,193
First one is that, since we are
open source but we are also like
229
00:13:14,193 --> 00:13:17,703
serving a lot of enterprises,
we are all very big on security.
230
00:13:17,713 --> 00:13:20,623
And then if you're in an enterprise
environment, you cannot just like, put
231
00:13:20,623 --> 00:13:24,583
in a AI and put your own LLM and say
to the customers, "Hey, this is going
232
00:13:24,583 --> 00:13:26,293
to be you." That, that will not work.
233
00:13:26,293 --> 00:13:30,573
So, so we said, "Hey, let's create a
solution where our, where the user, the
234
00:13:30,573 --> 00:13:35,543
developer can select the LLM that is, in
the enterprise environment, it is approved
235
00:13:35,603 --> 00:13:39,883
by the enterprise and they're paying
for subscription, all the legal work in
236
00:13:39,903 --> 00:13:44,573
is worked out. And then just create an
interface that can, that will empower
237
00:13:44,573 --> 00:13:48,923
that LLM to do more for developers."
So that, that was one thing that, that
238
00:13:48,953 --> 00:13:50,813
was like a vision for the project.
239
00:13:50,843 --> 00:13:55,083
The other thing was how can we
create, create interface for 100% of
240
00:13:55,083 --> 00:13:59,243
functionality of Semaphore so that
the developer can complete all of
241
00:13:59,243 --> 00:14:03,793
the time stay in that coding agent
of choice, let's say, Claude Code.
242
00:14:04,283 --> 00:14:07,339
And then never leave and,
and do everything through it.
243
00:14:07,679 --> 00:14:10,049
And then how can we create
experience where you… I don't
244
00:14:10,059 --> 00:14:13,159
have to learn anything except
high-level things, of course.
245
00:14:13,159 --> 00:14:16,203
Yeah, I need to conceptually
know what the CI and CD does.
246
00:14:16,243 --> 00:14:17,723
But how… I don't have to
learn the configuration.
247
00:14:19,353 --> 00:14:24,753
And that was described in a way,
hey, we know the CI/CD inside out.
248
00:14:24,813 --> 00:14:27,973
We have a lot of engineers on
our team that are like, building
249
00:14:27,973 --> 00:14:29,873
CI/CD tools for like a decade.
250
00:14:30,373 --> 00:14:33,983
Let's put all that inter- expertise,
package it somehow so that the
251
00:14:33,983 --> 00:14:36,873
other people can leverage it
instead of them becoming experts.
252
00:14:37,373 --> 00:14:40,723
And that's what we have discovered
that the good combination of a right,
253
00:14:40,913 --> 00:14:46,813
a CLI with the, as I said, commands
plus, MCP and set of skills and lot of
254
00:14:46,813 --> 00:14:49,653
examples, are actually doing the thing.
255
00:14:49,943 --> 00:14:53,613
And then actually, and getting us to
that, that, experience, that developer
256
00:14:53,613 --> 00:14:57,443
experience that I can just naturally
in natural language say "Hey, this
257
00:14:57,443 --> 00:15:01,313
is what… This is the outcome that
I want." And then the, then my coding
258
00:15:01,313 --> 00:15:06,453
agent will be able to interact with
the, with Semaphore in this case and
259
00:15:06,453 --> 00:15:08,033
do everything that needs to be done.
260
00:15:08,533 --> 00:15:08,953
Bret: Yeah.
261
00:15:09,453 --> 00:15:13,983
It's funny that in a world where we're
gonna end up with these ad- advanced
262
00:15:14,003 --> 00:15:18,323
AIs that do a lot of the, grunt work
for us, that the sweet spot seems
263
00:15:18,323 --> 00:15:20,693
right now like to stay at the CLI.
264
00:15:21,103 --> 00:15:25,023
And if you'd asked me like- 10 years
ago or even five years ago, "Hey,
265
00:15:25,023 --> 00:15:27,513
if you can imagine a future with AI,
do you think we'll be spending more
266
00:15:27,513 --> 00:15:28,933
time or less time in the terminal?"
267
00:15:29,433 --> 00:15:30,913
I would've said probably less, right?
268
00:15:30,913 --> 00:15:32,513
We're gonna have AIs do all that, right?
269
00:15:32,913 --> 00:15:36,613
But being such a, a terminal fan, like
one of… I think actually literally
270
00:15:36,623 --> 00:15:40,143
the most popular page on my website
is my s- my shell setup page, which I
271
00:15:40,213 --> 00:15:42,513
detail, like, all the different things.
272
00:15:42,513 --> 00:15:45,243
It's a lot easier nowadays, but 10
years ago, for those of you kiddies
273
00:15:45,243 --> 00:15:48,233
out there listening, we used to
have to do things like we have to,
274
00:15:48,253 --> 00:15:50,873
we'd have to install specific fonts,
which, you kind of still have to do.
275
00:15:51,523 --> 00:15:53,673
But we'd have to worry about
italics, like italicize didn't
276
00:15:53,673 --> 00:15:56,483
work by default, so you'd have to
customize that stuff in your editor.
277
00:15:56,483 --> 00:15:59,773
You'd have to do all sorts of very
particular settings and configurations
278
00:15:59,773 --> 00:16:04,503
just to get a, a VIM to work right in a
shell with all of the settings you wanted.
279
00:16:04,553 --> 00:16:08,293
We had to do a lot more a decade ago than
we did do nowadays to set up your shell.
280
00:16:08,293 --> 00:16:10,893
But I still get a ton of traffic
just when they, they just wanna
281
00:16:10,893 --> 00:16:13,603
know the fonts I'm using, the themes
I'm using,
the, the GUI I used.
282
00:16:13,633 --> 00:16:17,393
And then the, now my most popular
video this year is my video of talking
283
00:16:17,393 --> 00:16:20,903
about leaving Ghostty Terminal, which
has been for a couple years the big
284
00:16:20,903 --> 00:16:25,833
thing for terminals, leaving that for,
Supacode, which is a g- a lib Ghostty.
285
00:16:25,833 --> 00:16:29,833
It's actually a Ghostty project, but it
adds a bunch of agent management features
286
00:16:29,843 --> 00:16:31,023
so that you can manage your product.
287
00:16:31,033 --> 00:16:34,093
So those of you watching, if you're
interested in agents in the shell, go look
288
00:16:34,093 --> 00:16:35,873
at my YouTube later, but not right now.
289
00:16:35,873 --> 00:16:36,963
We're talking about CI right now.
290
00:16:37,133 --> 00:16:40,403
But I'm… It's hilarious that, the,
the terminal people are getting their,
291
00:16:40,503 --> 00:16:41,813
they're getting everything right now.
292
00:16:41,823 --> 00:16:44,383
All the best stuff right now I
feel like is in the terminal.
293
00:16:44,633 --> 00:16:45,973
All the best coding agents.
294
00:16:45,993 --> 00:16:48,763
I'm not… I'm trying to use the GUIs,
but I don't feel like they're as good.
295
00:16:49,193 --> 00:16:52,823
And the fact that I can now… it took,
to me, the way I'm looking at this
296
00:16:52,833 --> 00:16:57,443
is it took agents or it took AI for
us to start looking at platforms like
297
00:16:57,443 --> 00:17:02,823
yours in a way of, we, a lot of, we
all had CLIs for these tools, but you
298
00:17:02,833 --> 00:17:04,573
had to really memorize the commands.
299
00:17:05,073 --> 00:17:07,653
I always feel like I don't know all
the commands, and I'm constantly
300
00:17:07,653 --> 00:17:08,583
like, "Is there a command for this?
301
00:17:08,583 --> 00:17:10,663
Am I searching for a command?" you
s- look at me look at the help.
302
00:17:10,673 --> 00:17:13,883
Can I even do this in the… and
so I think as humans, we have this
303
00:17:13,883 --> 00:17:17,423
ability where we learn the command
line tool once, let's say Docker.
304
00:17:17,773 --> 00:17:20,903
But then Docker adds a bunch of
new features five years later,
305
00:17:20,903 --> 00:17:24,403
and no one uses them in the CLI
because we all already learned it.
306
00:17:24,773 --> 00:17:30,013
And when I talk to people about this, they
seem to agree that w- we don't feel like
307
00:17:30,023 --> 00:17:35,043
we're really experts in the CLI, the full
power of the CLI of a lot of these tools.
308
00:17:35,523 --> 00:17:38,993
And now I just feel like that's, that
problem has been completely erased.
309
00:17:39,143 --> 00:17:42,433
Like y- like the way that I'm
looking at your documentation, I'm
310
00:17:42,433 --> 00:17:44,643
realizing I, I could pick this up.
311
00:17:45,023 --> 00:17:49,593
I could use the CI, and the only thing
I have to be good at is understanding
312
00:17:49,653 --> 00:17:55,023
CI concepts like the, the basics
of CI and automation and how you
313
00:17:55,043 --> 00:17:56,933
manage a Git repo with workflows.
314
00:17:56,963 --> 00:17:59,763
But I feel like the only thing I
really have to understand is, how
315
00:17:59,763 --> 00:18:02,293
to prompt my intent to the agent.
316
00:18:02,713 --> 00:18:06,033
And I feel like your demos on your
website, the, the little ones that
317
00:18:06,033 --> 00:18:10,233
say "Set up my CI for me," they… A
lot of websites, they feel contrived.
318
00:18:10,503 --> 00:18:13,913
We used to have these sort of made-up
demos on the front page, but that's
319
00:18:13,923 --> 00:18:16,993
literally probably what I would
type to the CI is, "Could you please
320
00:18:16,993 --> 00:18:18,433
just set up Semaphore for me?"
321
00:18:18,433 --> 00:18:22,923
Because I don't know the shell, I don't
know the CLI, and I'm not even familiar
322
00:18:22,923 --> 00:18:27,983
yet with the GUI, so I don't know
what possible options are for the GUI.
323
00:18:28,013 --> 00:18:32,783
And that's just a, it's a cool target
to, to approach of like, our onboarding
324
00:18:32,793 --> 00:18:36,443
literally needs to be someone who's
switching from some other CI and
325
00:18:37,153 --> 00:18:41,233
doesn't know anything about our
platform, but the agent can make them
326
00:18:41,713 --> 00:18:43,673
extremely productive in the first hour.
327
00:18:43,943 --> 00:18:46,773
And is that something that you are
constantly testing and building out?
328
00:18:46,773 --> 00:18:48,543
Looks like it was, like,
a, a major feature for you.
329
00:18:49,043 --> 00:18:49,783
Marko: Yeah, absolutely.
330
00:18:50,053 --> 00:18:50,503
Absolutely.
331
00:18:50,813 --> 00:18:54,793
that barrier of entry, how, b- this
is becoming extremely important.
332
00:18:55,123 --> 00:18:58,553
Like with all the code being generated
by agents, like if you do not have a
333
00:18:58,553 --> 00:19:03,763
great CI that can test and make sure
that you're like shipping something that,
334
00:19:03,793 --> 00:19:07,533
that, has no regressions and, and it's
high quality, that's extremely important.
335
00:19:07,543 --> 00:19:10,923
So it's very important that it's very
easy for people to get on board with
336
00:19:10,923 --> 00:19:15,813
the CI/CD, and then also to give them
freedom to easily switch from one
337
00:19:16,013 --> 00:19:17,833
to another, whatever is the reason.
338
00:19:18,313 --> 00:19:21,403
And then, those barriers were
very difficult and hard before.
339
00:19:21,743 --> 00:19:25,856
These were like in a really project
that DevOps organizations were planning
340
00:19:25,856 --> 00:19:30,096
on them, they, like asking for budgets
and all like, and so it was very hard.
341
00:19:30,466 --> 00:19:34,046
And now it's, it just we have a tool
that is like very good at reading
342
00:19:34,046 --> 00:19:38,486
documentation, which we as humans were
really bad at, writing documentation,
343
00:19:38,486 --> 00:19:40,036
especially reading the documentation.
344
00:19:40,168 --> 00:19:40,828
Bret: it's our last
345
00:19:40,926 --> 00:19:41,696
Marko: Exactly the last…
346
00:19:41,738 --> 00:19:42,568
Bret: hope is documentation.
347
00:19:42,708 --> 00:19:42,928
Yeah.
348
00:19:42,938 --> 00:19:44,788
How far can I get without documentation?
349
00:19:45,288 --> 00:19:45,648
Marko: Yeah.
350
00:19:45,818 --> 00:19:49,318
And then nowadays we have s- we have a
tool that is very diligent in reading
351
00:19:49,318 --> 00:19:53,878
the documentation, especially if
you format it in the right way, and
352
00:19:53,878 --> 00:19:59,258
then it's, it's "Hey, I have this,
please, cop- look at this like Jenkins
353
00:19:59,268 --> 00:20:01,798
configuration and make it, in Semaphore."
354
00:20:01,948 --> 00:20:06,218
And then it starts to, to run
because it, it knows like a lot of
355
00:20:06,218 --> 00:20:09,038
details about Jenkins and all the
Groovy script and everything, and
356
00:20:09,038 --> 00:20:13,138
it knows exactly how to mimic that
in Semaphore and that's awesome.
357
00:20:13,508 --> 00:20:16,338
And then you can really quickly s-
try it out and see what you can,
358
00:20:16,838 --> 00:20:21,018
Marcos: It was even a point that
I was pestering Marko, because I
359
00:20:21,018 --> 00:20:24,408
wanted to set up a project of mine
on Semaphore, only it starts when
360
00:20:24,408 --> 00:20:25,948
I started working with the company.
361
00:20:26,368 --> 00:20:28,038
But I didn't know how to do that.
362
00:20:28,288 --> 00:20:32,328
And now we only-- we just have a comment
that you can talk with your agent
363
00:20:32,328 --> 00:20:34,158
that can do anything that you want.
364
00:20:34,478 --> 00:20:39,128
And you can set up a CI/CD from zero
to hero like we have, one series
365
00:20:39,128 --> 00:20:41,438
about that in around ten minutes.
366
00:20:41,638 --> 00:20:42,808
So it's very interesting.
367
00:20:43,308 --> 00:20:43,768
Bret: Yeah.
368
00:20:44,108 --> 00:20:46,788
in fact, I don't think I've thought
a lot about the fact that w- what
369
00:20:46,798 --> 00:20:51,058
one of the things that agents are
letting us do is switch tooling.
370
00:20:51,068 --> 00:20:53,698
I Not just switch tooling, but
now we're seeing, c- complete
371
00:20:53,698 --> 00:20:55,298
projects being rewritten in Rust,
372
00:20:55,798 --> 00:20:56,598
Marko: Yeah, that's very popular.
373
00:20:56,804 --> 00:21:00,224
Bret: a- and if we can rewrite an
application in Rust with $10,000 of
374
00:21:00,224 --> 00:21:06,254
tokens, like we can probably switch our
CIs much quicker with far fewer tokens.
375
00:21:06,754 --> 00:21:09,094
But I haven't thought about, too much
about the fact… 'Cause we had, I mean,
376
00:21:09,094 --> 00:21:14,674
we have an entire ecosystem, I feel
like, of tooling to migrate one workflow
377
00:21:14,684 --> 00:21:18,364
file, you know, migrate everything
away from Jenkins, which I approve,
378
00:21:18,664 --> 00:21:22,444
and then m- move from, we, a lot of us
had to move from Travis years ago when
379
00:21:22,444 --> 00:21:23,964
Travis changed a lot of its free…
380
00:21:24,004 --> 00:21:27,484
Remember f- a decade ago, Travis was
very free, and we were all addicted to
381
00:21:27,484 --> 00:21:29,754
the free, and then they weren't so free,
and then we had to move to something
382
00:21:29,754 --> 00:21:33,444
else, and I, and we have all these tools
at the ecosystem, or even some of the,
383
00:21:33,644 --> 00:21:36,284
we, you probably, I don't know, maybe
made some of these converters yourself.
384
00:21:36,284 --> 00:21:37,794
And it never went 100%.
385
00:21:37,814 --> 00:21:41,624
It was never, it never really
worked on the first try.
386
00:21:41,624 --> 00:21:45,124
It was more like it would convert…
It was kind of like my Dockerfile m-
387
00:21:45,134 --> 00:21:48,404
making, conversion, converting things that
would try to figure out your Dockerfile.
388
00:21:48,404 --> 00:21:51,564
It's like they get you to 80%, but
then the last 20% actually takes
389
00:21:51,564 --> 00:21:54,074
you a l- a really long time and
hours of work to troubleshoot.
390
00:21:54,574 --> 00:21:57,274
And some of the concepts are just
different, but I didn't really ever
391
00:21:57,274 --> 00:22:00,954
think about the fact that this is
almost a solved problem if you can
392
00:22:01,304 --> 00:22:04,094
give all the documentation and all
the right skills and all the right MCP
393
00:22:04,094 --> 00:22:08,784
endpoints to your agents, like this
could be a very smooth, fluid, process.
394
00:22:09,044 --> 00:22:10,074
But before, I want to get in that.
395
00:22:10,074 --> 00:22:13,914
So we're gonna get into the details
of, like, how exactly people can use
396
00:22:13,914 --> 00:22:15,894
an agent to access and manage a CI.
397
00:22:15,894 --> 00:22:19,264
There's some really interesting things
I wanna talk to you about, because I've
398
00:22:19,264 --> 00:22:22,834
been doing a deep dive into your blogs
over the last six months, and there's
399
00:22:22,834 --> 00:22:26,454
some really interesting ideas there that
I like about, that you all are able to…
400
00:22:26,654 --> 00:22:29,254
You're basically able to be a
little more nimble than some of the
401
00:22:29,254 --> 00:22:30,884
cor- the giants out there of CI.
402
00:22:30,894 --> 00:22:35,434
It seems like you're able to move quicker
and modify the platform to be more agent
403
00:22:35,484 --> 00:22:39,404
native, than maybe a lot of the other
sort of incumbents and legacy platforms.
404
00:22:39,754 --> 00:22:42,464
But first I wanna just talk
about you going open source.
405
00:22:42,494 --> 00:22:44,424
'Cause I'm a cloud native ambassador.
406
00:22:44,424 --> 00:22:48,244
I go to all the KubeCons, I'm a
part of the CNCF and the Linux
407
00:22:48,244 --> 00:22:50,764
Foundation, and now you all are, too.
408
00:22:50,774 --> 00:22:52,604
you're members of the Linux
Foundation, members of the Cloud
409
00:22:52,604 --> 00:22:53,374
Native Computing Foundation.
410
00:22:53,374 --> 00:22:55,894
You're actually listed, for those
of you out there that you know
411
00:22:55,894 --> 00:22:59,214
about the landscape, the CNCF
landscape, they're listed on the
412
00:22:59,214 --> 00:23:02,684
landscape, which is very cool, 'cause
there's a lot of CI options there.
413
00:23:02,994 --> 00:23:07,204
But I always thought of Semaphore
as this closed source tool that is
414
00:23:07,204 --> 00:23:13,224
hosted in Europe and is for teams that
are maybe burned by some of the big
415
00:23:13,224 --> 00:23:16,284
incumbent players and are looking for
something a little more their style.
416
00:23:16,524 --> 00:23:19,394
How did exactly, what,
how did this come about?
417
00:23:19,444 --> 00:23:22,104
This seems like a really big
decision to open source the platform.
418
00:23:22,604 --> 00:23:24,294
Can you tell me, can you walk
me through how that happened?
419
00:23:24,794 --> 00:23:28,754
Marko: Yeah, so, it was a very big lift
for the company, in order to open source
420
00:23:28,764 --> 00:23:31,294
your code, it kind of like there's
a lot of work that you need to do,
421
00:23:31,304 --> 00:23:35,364
make sure that everything is buttoned
up from the security perspective.
422
00:23:35,784 --> 00:23:41,494
So, so it was a big lift, but what we
wanted to do is really enable people and
423
00:23:41,834 --> 00:23:47,214
to really meet them where they are, If
you are only a cloud provider, then there
424
00:23:47,214 --> 00:23:49,424
are, there's… You're limited in a way.
425
00:23:49,524 --> 00:23:50,494
Who's your customer?
426
00:23:50,604 --> 00:23:54,214
And we also have some, some big
customers that are, you know, running
427
00:23:54,214 --> 00:23:56,804
the whole, Semaphore ecosystem on-prem.
428
00:23:57,304 --> 00:24:01,334
And we thought "Hey, let's open source
this, let's enable people to try the
429
00:24:01,334 --> 00:24:04,974
Semaphore in the way that suits, suits
them the best so that, if you want to
430
00:24:04,974 --> 00:24:09,224
run it on your own, you can. That we
can open up the platform to call for
431
00:24:09,224 --> 00:24:13,324
contributions so that people can…
also developers can collaborate with us
432
00:24:13,804 --> 00:24:18,244
much, much easier." For the developers
it's much easier to submit the PR or
433
00:24:18,244 --> 00:24:20,054
add the comment than to hop on a call.
434
00:24:20,554 --> 00:24:24,684
So, so, so it's also to get more
feedback, to get more visibility,
435
00:24:25,044 --> 00:24:31,144
and to also make it possible for
the, all the Jenkins customers.
436
00:24:31,474 --> 00:24:32,474
Sorry, sorry, Jenkins.
437
00:24:32,514 --> 00:24:35,344
But all the Jenkins customers to
try something else, because they
438
00:24:35,344 --> 00:24:36,624
are really in, in the corner.
439
00:24:37,124 --> 00:24:41,254
It's a very, sticky platform for the,
in my opinion, for the wrong reasons.
440
00:24:41,754 --> 00:24:46,594
So I think, these are all the, all
of, all the, motivations for us to
441
00:24:46,594 --> 00:24:50,954
do an open source and we are, like,
starting to see benefits of it.
442
00:24:51,314 --> 00:24:54,154
And then we as a company, we
believe that, that the, the
443
00:24:54,154 --> 00:24:55,084
open source is the way to go.
444
00:24:55,814 --> 00:24:56,214
Bret: Yeah.
445
00:24:56,304 --> 00:24:57,704
Marcos, were you a part of that effort?
446
00:24:58,204 --> 00:25:02,564
Marcos: No, I will-- I came later to
that, but I made some AI open source.
447
00:25:03,064 --> 00:25:04,474
Bret: Yeah, 'cause that
sounds like a lot of work.
448
00:25:04,974 --> 00:25:08,814
Marcos: I'm glad to be… didn't have
to be part of that work because it
449
00:25:08,814 --> 00:25:11,594
was a-- Like, I was reading about
the documentation that they had.
450
00:25:11,604 --> 00:25:15,974
It was a lot of things that we had to
solve before going public with the code
451
00:25:16,474 --> 00:25:16,794
Bret: Yeah.
452
00:25:16,794 --> 00:25:19,114
Yeah, y- it's, I mean, there's so
many factors, and I'm sitting here
453
00:25:19,114 --> 00:25:22,309
thinking, as a consultant, I s- I
see a lot of different projects, and
454
00:25:22,809 --> 00:25:26,409
a lot of the ones that are closed
source, if I was one of the major
455
00:25:26,809 --> 00:25:29,999
people on that project, I would be very
nervous about opening, opening it up.
456
00:25:30,109 --> 00:25:34,619
Not from just a security perspective,
but now suddenly all my code is v- it's
457
00:25:34,619 --> 00:25:37,509
not … It's one thing when it's a dozen
people that are, critiquing my code.
458
00:25:37,509 --> 00:25:39,599
It's another thing when it's thousands
of people critiquing my code.
459
00:25:40,529 --> 00:25:43,469
And and sometimes I'm … When
I look back at my old code, I'm
460
00:25:43,469 --> 00:25:44,979
often like, "Who wrote that?
461
00:25:45,399 --> 00:25:46,929
Oh, that was me.
462
00:25:46,989 --> 00:25:48,219
What was I s- what was I thinking?
463
00:25:48,219 --> 00:25:51,909
That was dumb." so yeah, that's gotta be
… I mean, especially when you have a project
464
00:25:51,929 --> 00:25:53,369
that's many years in the making, right?
465
00:25:53,369 --> 00:25:56,819
It's we all develop differently now
than we did a decade ago, and … our,
466
00:25:56,859 --> 00:25:59,929
our intelligence about a, a code
base is better than a decade ago.
467
00:25:59,929 --> 00:26:00,789
I mean, everything's different.
468
00:26:01,199 --> 00:26:04,629
So, when you have a long sort of a
monolith, maybe not monolith, but
469
00:26:04,679 --> 00:26:07,909
something, anything older than a decade,
I feel like at this point is almost
470
00:26:07,909 --> 00:26:11,789
like, legacy code if you don't constantly
maintain it, keep it up to date.
471
00:26:11,799 --> 00:26:12,679
And I imagine one of the,
472
00:26:12,779 --> 00:26:15,279
I'm gonna guess one of the things you
wanted to do before you open sourced
473
00:26:15,279 --> 00:26:18,629
it was to make sure that it was, like,
on the latest libraries and it was
474
00:26:18,629 --> 00:26:20,069
using some of the latest paradigms.
475
00:26:20,519 --> 00:26:23,239
So all right, so you made it open source.
476
00:26:23,739 --> 00:26:30,019
People can download this and run this
themselves, or they can just easy button
477
00:26:30,019 --> 00:26:31,829
it with you on the hosted platform.
478
00:26:32,279 --> 00:26:33,559
… Tell me about, like, the runners.
479
00:26:33,569 --> 00:26:36,249
You provide … Or what do you call
those workers or runners or what
480
00:26:36,249 --> 00:26:37,709
do you call those, those machines?
481
00:26:37,805 --> 00:26:38,585
Marko: Yeah, runners.
482
00:26:39,085 --> 00:26:42,225
Yeah, so we provide our own runners,
but people can also self-host.
483
00:26:42,515 --> 00:26:46,075
And there are a lot of customers
choosing for one or the other
484
00:26:46,085 --> 00:26:47,425
for their specific reasons.
485
00:26:47,645 --> 00:26:50,965
And then the benefit of running on
our runners is that we do the, all
486
00:26:50,965 --> 00:26:52,885
the maintenance of the runners.
487
00:26:53,385 --> 00:26:56,315
The benefit of having your own
self-hosted runners is that, you can
488
00:26:56,315 --> 00:27:00,265
create the custom environments that,
that are really bespoke for your needs.
489
00:27:00,355 --> 00:27:03,765
Maybe they are resembling
100% production environment.
490
00:27:04,265 --> 00:27:07,175
So it, it makes sense
in different use cases.
491
00:27:07,555 --> 00:27:09,545
It makes sense to, to
use one or the other.
492
00:27:09,545 --> 00:27:13,645
And we have a quite a lot of customers
that go hybrid, so some of the runners
493
00:27:13,645 --> 00:27:17,745
are self-hosted and some of the
runners are, they're using our cloud.
494
00:27:17,885 --> 00:27:22,135
So, kind of as I said, we like to, to,
to meet our customers where they are,
495
00:27:22,635 --> 00:27:26,625
and then, you provide them with the best
possible service, that we can offer.
496
00:27:27,125 --> 00:27:30,115
Bret: like I think any CI/CD team at
some point they're gonna end up needing
497
00:27:30,115 --> 00:27:33,795
to run runners on their, i- whether it's
in their data centers or on their VPCs,
498
00:27:34,315 --> 00:27:38,045
like they're gonna need something that's
closer to staging a production, especially
499
00:27:38,045 --> 00:27:39,615
if they're doing CD inside there.
500
00:27:39,625 --> 00:27:44,095
And the runner story is one of
those things where, until you're a
501
00:27:44,095 --> 00:27:49,105
seasoned CI/CD veteran, I feel like
you don't… You may not look at a
502
00:27:49,105 --> 00:27:52,705
platform to evaluate it and think,
let me spend a lot of time playing
503
00:27:52,705 -->
00:27:55,705
around with runners, deploying runners,
figuring out how to maintain runners,
504
00:27:55,735 --> 00:27:57,005
like getting the performance right.
505
00:27:57,035 --> 00:28:01,155
Let me…" There is so much to
it that, I mean, there are entire
506
00:28:01,155 --> 00:28:03,985
other CI/CD platforms where I d-
I don't even recommend they use
507
00:28:03,985 --> 00:28:06,955
their runners, that they run their
own, or that they use an outsourced
508
00:28:06,955 --> 00:28:09,975
service 'cause there's an entire
ecosystem of outsourcing your runners.
509
00:28:10,475 --> 00:28:14,405
And you had an interesting blog post
this year, actually this month, talking
510
00:28:14,405 --> 00:28:18,095
about benchmarks, and I thought this was
really interesting because a- as you're
511
00:28:18,095 --> 00:28:22,615
probably saying here, the speed of runners
is actually one of the most critical
512
00:28:23,115 --> 00:28:27,935
points to me in terms of evaluating a
CI platform because you're gonna spend
513
00:28:27,935 --> 00:28:33,365
so much time waiting, and historically
that has been infuriating for those of
514
00:28:33,365 --> 00:28:34,915
us that are managing these platforms.
515
00:28:35,355 --> 00:28:38,702
Because as someone who manages CI/CD,
like, my goal is to get these things to
516
00:28:38,702 --> 00:28:42,882
run as fast as possible, because I want
my developers to use it more and I am
517
00:28:42,902 --> 00:28:48,102
also kind of s- I'm not only the CI/CD
custodian a lot of times, I feel like I am
518
00:28:48,102 --> 00:28:51,092
the advocate for people automating more.
519
00:28:51,102 --> 00:28:55,062
I'm always looking for toil to automate
in teams, not just testing their code, but
520
00:28:55,072 --> 00:28:56,802
also all of the other maintenance tasks.
521
00:28:56,802 --> 00:29:01,202
Let's throw those into CI as well, make
it a more universal automation platform.
522
00:29:01,582 --> 00:29:06,392
But also I'm also security… I kind
of fit in that DevSecOps security
523
00:29:06,392 --> 00:29:09,202
realm where I'm wanting them to do
more linting, I want them to do more
524
00:29:09,202 --> 00:29:12,052
security evalu- I'm, like, suggesting
workflows all the time to them.
525
00:29:12,052 --> 00:29:14,052
"Hey, we should add this to the code.
Hey, we should add this to code,"
526
00:29:14,062 --> 00:29:18,152
even when I'm not a developer in the
team, because I feel like developers,
527
00:29:18,492 --> 00:29:23,852
they spend so much time being told to
focus on features and code maintenance
528
00:29:23,862 --> 00:29:27,112
of the actual platform that it's
one of those, what's the analogy?
529
00:29:27,112 --> 00:29:30,802
The, the cobbler's children have
no shoes or something like that.
530
00:29:30,812 --> 00:29:36,302
I always feel like our CI/CD platforms, if
there's not advocates for them, you end up
531
00:29:36,312 --> 00:29:41,092
in this space, and Jenkins was the perfect
example 15, t- 10 years ago, where it
532
00:29:41,092 --> 00:29:45,562
just became this hodgepodge of fragility,
and it ended up being that one person
533
00:29:45,562 --> 00:29:48,372
in the team that knew how to maintain
it and no one else could touch it.
534
00:29:48,682 --> 00:29:51,502
A- and good luck with the backups
and restores of that mess, right?
535
00:29:51,512 --> 00:29:55,132
Like it was… There was always that
precious Jenkins box under someone's
536
00:29:55,132 --> 00:29:58,302
desk that could not be restored
successfully, so you could not touch it.
537
00:29:58,802 --> 00:30:02,872
And I hope for everyone that those years
are gone and that we are at least using
538
00:30:02,922 --> 00:30:04,682
better maintained and supported platforms.
539
00:30:04,952 --> 00:30:09,120
But you do have this- a blog post
talking about how much faster, the
540
00:30:09,120 --> 00:30:12,330
runners build times, the cost of jobs.
541
00:30:12,640 --> 00:30:17,070
I'm a big advocate of AMD and
moving everything you can in your
542
00:30:17,070 --> 00:30:20,720
coding platforms to AMD because
it's so much a better value on the
543
00:30:20,720 --> 00:30:23,180
big clouds, and it's just cheaper.
544
00:30:23,460 --> 00:30:27,730
And then, and talking about some of
the competitors and how, whether or
545
00:30:27,730 --> 00:30:30,680
not they're faster or slower, they
also tend to be more expensive.
546
00:30:31,080 --> 00:30:34,880
So I… For those listening, do everything
you can to avoid running your own runners.
547
00:30:35,380 --> 00:30:40,810
Just because it's not a solved problem,
and you're probably not better at making
548
00:30:40,810 --> 00:30:45,530
them faster than the professionals
running these platforms that their
549
00:30:45,530 --> 00:30:47,120
whole job is to make them faster.
550
00:30:47,590 --> 00:30:50,770
Marko: Yeah, there's a whole team
of full-time employed engineers
551
00:30:50,770 --> 00:30:54,280
that are making sure that the, it
runs, everything is up to date.
552
00:30:54,780 --> 00:30:58,510
I don't know if you have already addressed
that in your, with in your podcast before,
553
00:30:58,530 --> 00:31:02,090
but in last, I would say three to five
months, especially in the last three
554
00:31:02,090 --> 00:31:08,090
months, there's like a huge amount of CVEs
out there that are popping up every day.
555
00:31:08,150 --> 00:31:11,750
Typically, you would do some maintenance
every month, and that would be fine.
556
00:31:11,790 --> 00:31:15,560
But here, we have every day
there are new things popping up.
557
00:31:16,060 --> 00:31:19,350
I'm glad that, probably we will end
up with a much more secure software
558
00:31:19,650 --> 00:31:22,800
in the end, but at the moment,
it's so overwhelming to, to make
559
00:31:22,800 --> 00:31:26,600
sure that all the CVEs and that
your environment is fully patched.
560
00:31:26,950 --> 00:31:30,560
Because in the-- if agents can find
security holes, that means
that other
561
00:31:30,560 --> 00:31:32,180
agents can also exploit your thing.
562
00:31:32,180 --> 00:31:35,520
So everything needs to be patched
up, and buttoned up really quickly.
563
00:31:35,960 --> 00:31:39,770
And having internal teams working
on that, I think it's very
564
00:31:39,770 --> 00:31:42,230
expensive at, twenty twenty-six.
565
00:31:42,700 --> 00:31:45,820
So it's much better to, to do
that on scale, as we are doing as
566
00:31:45,970 --> 00:31:47,560
example for all of our customers.
567
00:31:48,056 --> 00:31:48,486
Bret: Yeah.
568
00:31:48,836 --> 00:31:52,496
We have a question, "Are you guys taking
contributors? I'd love to contribute.
569
00:31:52,536 --> 00:31:54,096
If yes, how can I get started?"
570
00:31:54,596 --> 00:31:55,196
Marko: Yeah, of course.
571
00:31:55,226 --> 00:31:56,676
We, we take contributors.
572
00:31:56,726 --> 00:31:57,696
It's very easy.
573
00:31:57,866 --> 00:32:02,586
you submit the PR and then some of
our colleagues is going to review it
574
00:32:02,606 --> 00:32:07,546
and, getting the communication with
with the person submitting a, a PR.
575
00:32:07,596 --> 00:32:10,976
And then if, if everything is
looks good, then it passes all
576
00:32:10,976 --> 00:32:14,596
the C- CI, then it's merged and
it becomes a part of the product.
577
00:32:15,036 --> 00:32:18,896
Marcus can maybe talk about when we
open… When, we started with Sem
578
00:32:18,926 --> 00:32:23,006
AI, for example, immediately as open
source and as we presented it to a
579
00:32:23,006 --> 00:32:27,126
couple of our customers, they were
starting to submit PRs like week two.
580
00:32:27,496 --> 00:32:32,186
So it was very fun to, to watch how
people are thinking about it and what were
581
00:32:32,196 --> 00:32:34,596
their usage scenarios and their comments.
582
00:32:35,096 --> 00:32:35,376
Marcos: Yeah.
583
00:32:35,716 --> 00:32:39,786
We are always accepting new things
and also improvements that people
584
00:32:39,806 --> 00:32:41,246
see that we probably don't see.
585
00:32:41,736 --> 00:32:45,976
And you can just go to the repository
that we have and follow the
586
00:32:45,976 --> 00:32:50,026
contributing documentation, and we
will take a look on the pull request.
587
00:32:50,526 --> 00:32:50,956
Bret: Awesome.
588
00:32:51,446 --> 00:32:54,566
We got 37 contributors, so pretty cool.
589
00:32:54,926 --> 00:32:56,326
This is a question I
didn't think to talk about.
590
00:32:56,336 --> 00:32:59,776
Do you, are you thinking about
contributing to the CNCF?
591
00:32:59,776 --> 00:33:00,886
Are you gonna keep it for yourself?
592
00:33:01,386 --> 00:33:02,306
I don't know how to ask that question.
593
00:33:02,976 --> 00:33:06,436
Are you thinking about contributing
it, or are, because it's your platform,
594
00:33:06,436 --> 00:33:08,496
I guess it's a little, gonna be
a little hard to do that, right?
595
00:33:08,496 --> 00:33:10,776
'Cause this isn't just a,
a separate product, right?
596
00:33:11,276 --> 00:33:11,936
Marko: Exactly.
597
00:33:11,946 --> 00:33:13,776
It's a very big decision to make.
598
00:33:13,776 --> 00:33:18,106
So, so yeah, that's still under
consideration, I would say.
599
00:33:18,596 --> 00:33:19,016
Yeah.
600
00:33:19,306 --> 00:33:19,916
Still discussion, yeah.
601
00:33:20,052 --> 00:33:22,932
Bret: it's one thing to do open core, but
it's another thing when it's the whole
602
00:33:22,932 --> 00:33:24,662
platform and it's something like this.
603
00:33:24,662 --> 00:33:27,902
But yeah, we're all kind of, all
over the map with CI, and I was
604
00:33:27,902 --> 00:33:31,582
gonna talk about earlier that I
don't remember the last time…
605
00:33:31,762 --> 00:33:36,635
It-- I'd say 50% of the time I walk
in as a consultant, I w- especially
606
00:33:36,635 --> 00:33:39,932
if, we're talking enterprise here,
big companies, hundreds of developers
607
00:33:39,932 --> 00:33:43,662
if not thousands, it is very rare
for me to see them have one CI.
608
00:33:44,082 --> 00:33:47,572
And one of the reasons for that,
getting back to our agent story,
609
00:33:47,962 --> 00:33:51,612
one of the reasons for that is that
everyone's trying to get off Jenkins.
610
00:33:51,622 --> 00:33:54,692
For five years or more, everybody's
been trying to get off Jenkins.
611
00:33:55,022 --> 00:33:59,662
And the challenge always ends up, it's
like an 80/20 thing where they can get
612
00:33:59,662 --> 00:34:04,972
that first 80% off in a reasonable amount
of months, and then there's this 20%
613
00:34:04,982 --> 00:34:09,942
left that are these really complicated,
multifaceted jobs that are insanely
614
00:34:09,962 --> 00:34:13,292
complex and should have never been created
that way in the beginning to begin with.
615
00:34:13,662 --> 00:34:19,072
But they end up leaving them there because
they just, they burn out on the migration.
616
00:34:19,082 --> 00:34:21,502
And so then now you have
N plus one problems.
617
00:34:21,922 --> 00:34:25,232
And the, one of the biggest projects I
was on in the last five years, it was a,
618
00:34:25,272 --> 00:34:28,032
it was an enterprise security company,
and they had three different CIs.
619
00:34:28,422 --> 00:34:32,142
And they had the old CI, the CI
they tried to migrate to, but
620
00:34:32,142 --> 00:34:33,512
then it didn't do everything.
621
00:34:33,752 --> 00:34:34,672
I think that was Drone.
622
00:34:34,692 --> 00:34:35,862
I'll pick on Drone for a second.
623
00:34:36,202 --> 00:34:39,072
and this was five years ago or
four years ago, so this was pre-AI.
624
00:34:40,182 --> 00:34:43,942
And Drone didn't quite do everything
they wanted to do, and so then they
625
00:34:43,942 --> 00:34:46,527
decided they had to migra- migrate off of
that, and then they were still migrating
626
00:34:46,527 --> 00:34:49,587
off of Jenkins, and then there was the
new platform, and then it was just…
627
00:34:50,087 --> 00:34:52,977
talking
about the agent migration story,
we, I feel like we could do a whole
628
00:34:52,977 --> 00:34:56,137
ep- episode and then demo on it, but
it would be, it would be amazing for
629
00:34:56,137 --> 00:35:01,277
me to walk in and do a CI/CD migration
project in a month and just, end to
630
00:35:01,277 --> 00:35:05,797
end, cut it off, and now we're new and
improved all in a very short timeline.
631
00:35:05,797 --> 00:35:08,957
That, that almost feels like something
I need to start selling as a product.
632
00:35:09,007 --> 00:35:11,407
I will migrate your CI for
you in less than a month.
633
00:35:11,907 --> 00:35:12,557
Marko: Absolutely.
634
00:35:12,577 --> 00:35:12,897
Yeah.
635
00:35:12,967 --> 00:35:17,117
Yeah, that would be a very desirable
service out there, I'm quite sure.
636
00:35:17,217 --> 00:35:22,047
But it's especially, migrating from
the work, such a customizable tool
637
00:35:22,047 --> 00:35:27,567
like, like Jenkins, I don't think
it's possible to really migrate only
638
00:35:27,567 --> 00:35:31,497
kind of like a one-on-one, because
of that level of customization.
639
00:35:31,547 --> 00:35:35,567
I was involved in like maybe
at least like 10 projects in my
640
00:35:35,567 --> 00:35:37,397
career that were all on Jenkins.
641
00:35:37,437 --> 00:35:40,277
We were never able to upgrade
from Jenkins to Jenkins.
642
00:35:40,777 --> 00:35:42,327
Like from one version to another.
643
00:35:42,367 --> 00:35:46,227
Even not moving to another tool, it's
just keep staying like, you know, up to
644
00:35:46,237 --> 00:35:50,207
date and current was not possible because
at a certain moment somebody decided
645
00:35:50,207 --> 00:35:55,537
to use the plugin rightfully, but then
that plugin was not available anymore or
646
00:35:55,537 --> 00:35:57,167
not supported and then at that version.
647
00:35:57,167 --> 00:36:01,387
So it's, we ended up like, really, sitting
down and rethinking some of our flows,
648
00:36:01,437 --> 00:36:02,917
and some of the out- the pipelines.
649
00:36:02,957 --> 00:36:07,887
So I think that is necessary, but I also
think that's an opportunity and good
650
00:36:07,887 --> 00:36:11,667
thing to think through because as you
said, what we knew and what we thought
651
00:36:11,667 --> 00:36:15,757
it was right five years ago or even
three years ago, maybe it's not right
652
00:36:15,957 --> 00:36:18,077
now because of like a lot of reasons.
653
00:36:18,177 --> 00:36:21,177
And then it's good to see, take
some time and say "Huh, okay.
654
00:36:21,577 --> 00:36:25,907
If I have a blank slate, how would I
like this to work?" And now with agents,
655
00:36:26,067 --> 00:36:30,847
if I can describe that, like in, in
natural language and create like a very
656
00:36:30,847 --> 00:36:35,207
comprehensive like in documentation,
again with the hel- help of agent, that
657
00:36:35,207 --> 00:36:41,607
can be configured in I think 10, 15,
20 minutes, with the, with the agent.
658
00:36:41,617 --> 00:36:45,267
So, so, yeah, I think it's a
great time for engineering.
659
00:36:45,767 --> 00:36:49,727
Before we were spending a lot of time
in that grind of typing code, learning
660
00:36:49,727 --> 00:36:51,487
configuration, clicking buttons.
661
00:36:51,987 --> 00:36:54,807
Here we have a lot of time to
really think through some of our
662
00:36:54,807 --> 00:36:58,897
decisions, take a lot of things in
consideration, try out hypothesis.
663
00:36:59,397 --> 00:37:03,937
So, so, for our team internally,
that's really a blessing, where we
664
00:37:03,937 --> 00:37:07,917
can spend more time thinking through
and, and discussing and then the, the
665
00:37:08,067 --> 00:37:10,497
groundwork is handed over to the machines
666
00:37:11,329 --> 00:37:15,059
Marcos: Actually, one of the test
cases of Sem AI was migrating public
667
00:37:15,059 --> 00:37:20,289
repositories from Jenkins and GitHub
Actions to Semaphore, just like a
668
00:37:20,329 --> 00:37:22,709
benchmark for the usefulness of the tool.
669
00:37:23,209 --> 00:37:23,659
Bret: Yeah.
670
00:37:23,769 --> 00:37:26,819
it's almost like those are like evals
to test whether or not the agent
671
00:37:26,819 --> 00:37:28,349
can handle all that work, right?
672
00:37:28,639 --> 00:37:32,439
Marcos, I'm really, I really wanna
dig into, the work that y'all
673
00:37:32,469 --> 00:37:36,449
have been doing to sort of make
everything agent harness first.
674
00:37:36,839 --> 00:37:41,649
But I wanted to first talk about You
c- you've prov- you've created a plugin
675
00:37:41,649 --> 00:37:44,939
system for Claude Code and Codex.
676
00:37:45,289 --> 00:37:47,929
Can you talk about that real quick,
about everything that's included in
677
00:37:47,929 --> 00:37:52,739
there and why the decision was made
to do things like commands still, even
678
00:37:52,739 --> 00:37:57,199
though that feels almost like, w- is
that still cool anymore to do commands?
679
00:37:57,199 --> 00:38:00,079
I thought we were just vibe-
vibing the skills at this point.
680
00:38:00,389 --> 00:38:01,769
So, can you walk me through all that?
681
00:38:01,769 --> 00:38:05,099
'Cause it looked like you have
a lot of s- rigor there that is
682
00:38:05,109 --> 00:38:08,019
more than just someone simply
publishing a free skill, for example.
683
00:38:08,099 --> 00:38:09,069
seems like a lot more than that.
684
00:38:09,569 --> 00:38:09,999
Marcos: Yeah.
685
00:38:10,109 --> 00:38:14,419
Initially the SemAI was just
to be a CLI, so a simple CLI.
686
00:38:14,449 --> 00:38:16,029
We have the same CLI.
687
00:38:16,079 --> 00:38:20,659
It's a old one that we have, and we wanted
to develop something that was agent first.
688
00:38:21,109 --> 00:38:25,979
So naturally, we started with a tool
that was only communicating in JSON,
689
00:38:26,479 --> 00:38:30,889
so the agent could run it and pipe
it for… to anything that it wanted.
690
00:38:31,389 --> 00:38:35,799
But then we
discovered that MCP
was a thing, was booming, and we
691
00:38:36,299 --> 00:38:38,209
wanted to support MCP as well.
692
00:38:38,649 --> 00:38:43,409
But one rigor that we took, one
choice that we took was we wanted
693
00:38:43,409 --> 00:38:46,779
to have the same capabilities
of the CLI on the MCP server.
694
00:38:47,279 --> 00:38:52,249
So basically we are using Viper
and Cobra on the CLI, and we are
695
00:38:52,299 --> 00:38:56,109
basically using the commands that
we have for the CLI on the MCP.
696
00:38:56,519 --> 00:39:01,369
It's a little bit of a hacky thing, but
it's fully supported by the libraries,
697
00:39:01,409 --> 00:39:03,189
and you can check it on the repository.
698
00:39:03,609 --> 00:39:08,659
But you'll be a- always have the support
that you have on the CLI on the MCP.
699
00:39:09,009 --> 00:39:13,309
And this is important because as Marko
talked about on the start of the podcast,
700
00:39:13,699 --> 00:39:19,679
some tasks are easier to do using a CLI
and the others are easy to do using MCP.
701
00:39:20,179 --> 00:39:25,619
For example, your agent may decide to
pipe the output of SemAI into a Python
702
00:39:25,619 --> 00:39:27,749
script to extract something that it wants.
703
00:39:28,129 --> 00:39:30,679
This is something that
I see daily right now.
704
00:39:31,119 --> 00:39:35,489
And for some easy tasks, just asking
a question or something for the, to
705
00:39:35,489 --> 00:39:37,909
the platform, it normally uses MCP.
706
00:39:38,409 --> 00:39:43,649
And we decide to document and make
it easy for the agent to do the
707
00:39:43,649 --> 00:39:45,389
right thing, not do the wrong thing.
708
00:39:45,889 --> 00:39:49,379
So we have included descriptions,
examples, and skills.
709
00:39:49,669 --> 00:39:52,609
Each one has a, a, a place, right?
710
00:39:53,109 --> 00:39:57,749
So basically the descriptions is basically
what the surface level knowledge that
711
00:39:57,749 --> 00:39:59,509
the agent should have about that command.
712
00:40:00,009 --> 00:40:04,169
Each command has examples, so if the
agent is, doesn't know how to do the,
713
00:40:04,179 --> 00:40:08,129
the command, it can look through,
into the examples and basically
714
00:40:08,529 --> 00:40:10,239
figure it out from the examples.
715
00:40:10,539 --> 00:40:15,529
And the skills that most people
are kind of abandoning, the complex
716
00:40:15,529 --> 00:40:20,519
ones at least, is where we have
the cheat sheet for the agent.
717
00:40:20,979 --> 00:40:24,879
So basically it will be able
to know what is all of the step
718
00:40:24,879 --> 00:40:26,779
that it needs to do for one task.
719
00:40:27,199 --> 00:40:32,629
So for example, if I want to migrate from
Jenkins to Semaphore, it has a com- a
720
00:40:32,729 --> 00:40:37,309
skill that it can load, and it will have
all of the thi- all of the things that
721
00:40:37,309 --> 00:40:42,249
it needs to do, and also will have the
input from the developers of SemAI on how
722
00:40:42,249 --> 00:40:47,139
to do that, because we encounter a lot,
lots of particularities with that flow.
723
00:40:47,559 --> 00:40:49,139
And then we are creating skills.
724
00:40:49,519 --> 00:40:53,769
So basically when you install the plugin,
you will always have the skills available
725
00:40:53,779 --> 00:40:59,379
for you, and your agent can choose if it
will use the MCP or the S- CLI itself.
726
00:40:59,879 --> 00:41:04,359
And one nice feature of the plugin is
that it auto-updates, so you always have
727
00:41:04,359 --> 00:41:06,439
the latest version when you install it.
728
00:41:06,939 --> 00:41:07,239
Bret: Wow.
729
00:41:07,739 --> 00:41:08,409
That's a lot.
730
00:41:08,619 --> 00:41:11,969
What about the agent
makes it self-healing?
731
00:41:11,969 --> 00:41:16,939
I'm looking at this blog post on
your journey to making the SemAI.
732
00:41:16,959 --> 00:41:19,589
This blog post was what pulled me in a
little bit because there was a couple
733
00:41:19,589 --> 00:41:21,379
of things that I thought were different.
734
00:41:21,379 --> 00:41:24,809
One, yeah, you're doing the slash
commands and wha- and it talks about why
735
00:41:24,819 --> 00:41:26,799
those matter and basically determinism.
736
00:41:27,199 --> 00:41:31,829
Embedding CI/CD best practices into
the agents, which, I mean, for other
737
00:41:31,829 --> 00:41:33,559
CIs, I had to build that myself.
738
00:41:34,059 --> 00:41:37,269
Like, as a part of my courses I offered
because, 'cause it's not really built
739
00:41:37,269 --> 00:41:39,589
into a lot of these tools to, to really…
740
00:41:40,089 --> 00:41:42,119
Like to me, best practices are an opinion.
741
00:41:42,319 --> 00:41:46,409
I used to say better practices because
I always felt like, b-a- my best, I
742
00:41:46,409 --> 00:41:47,739
don't know what if I have is best.
743
00:41:47,739 --> 00:41:48,639
I mean, it's better.
744
00:41:49,049 --> 00:41:52,999
And I feel like the more opinionated you
can get, like the better that might be.
745
00:41:53,009 --> 00:41:56,559
And so sometimes best practices
can not always work for everyone.
746
00:41:56,569 --> 00:42:00,369
But it looks like you've got all of
these different areas that you focused
747
00:42:00,369 --> 00:42:03,699
on with like the caching, 'cause that's
a thing that like people often will
748
00:42:03,699 --> 00:42:06,679
screw up caching or they don't understand
caching, so they're like leaving,
749
00:42:06,999 --> 00:42:10,889
they're leaving minutes on the table
because they're not getting the most
750
00:42:10,899 --> 00:42:13,019
optimum performance out of their CI.
751
00:42:13,049 --> 00:42:15,989
And I've always felt like CIs are
always trying to optimize for that and
752
00:42:15,989 --> 00:42:20,239
like they either go too far to make it
too automated with the caching setup,
753
00:42:20,239 --> 00:42:23,319
and then it becomes limiting sometimes
because it's,
you have to do it this way
754
00:42:23,319 --> 00:42:27,449
in order to use the cache or it doesn't
work, or you have to become like an
755
00:42:27,459 --> 00:42:31,549
expert in how their caching works and
how your tools would need caching and
756
00:42:31,549 --> 00:42:35,259
which tools you have that need caching,
and how to avoid cache poisoning.
757
00:42:35,259 --> 00:42:37,789
And like there's just, even just
in that world, I feel like there's
758
00:42:37,789 --> 00:42:39,729
someone's job dedicated to that, right?
759
00:42:39,749 --> 00:42:41,189
And y- to be an expert.
760
00:42:41,479 --> 00:42:44,819
But you have this other list of like
the, the testing reports, the failure
761
00:42:44,819 --> 00:42:49,309
diagnosis, the artifact management, like
how do we get these artifacts out of this
762
00:42:49,319 --> 00:42:50,919
build and into where they need to live?
763
00:42:51,419 --> 00:42:54,279
And so you've got all that
built in there into the skills.
764
00:42:54,539 --> 00:42:56,579
Then you've got self-healing pipelines.
765
00:42:56,689 --> 00:42:59,669
Tell me about… I mean, you literally
have this quote in here of, "Tell the
766
00:42:59,669 --> 00:43:03,509
agent, 'Work until the pipeline is
green.'" Which I will point out is
767
00:43:03,519 --> 00:43:06,939
not the thing that I would normally
actually type, because that is an
768
00:43:06,949 --> 00:43:10,289
outcome, and I'm often dumb with agents.
769
00:43:10,639 --> 00:43:11,699
I need like agent therapy.
770
00:43:11,699 --> 00:43:15,869
I'm often dumb, and I'm like giving
them the detailed step-by-step because
771
00:43:15,869 --> 00:43:21,279
that's what we had to do two years ago,
versus I'm, this is an outcome, right?
772
00:43:21,279 --> 00:43:25,589
This is like what I would tell the
junior engineer who understands CI, but
773
00:43:25,589 --> 00:43:26,669
they're like, "What is my job here?"
774
00:43:26,669 --> 00:43:29,079
I'm like, "Your job is to work
until the pipeline is green."
775
00:43:31,269 --> 00:43:33,779
And, and so I'm guessing
you test this, right?
776
00:43:33,779 --> 00:43:34,869
Like this isn't just a demo.
777
00:43:34,869 --> 00:43:36,069
This is something that you verify.
778
00:43:36,069 --> 00:43:36,679
Tell me about that.
779
00:43:37,179 --> 00:43:39,969
Marcos: Yeah, basically the comments,
the comments, the skills that we
780
00:43:39,969 --> 00:43:43,089
put is basically we compile all
of the knowledge that we have.
781
00:43:43,089 --> 00:43:48,459
We got-- We have lots of customers and we,
years of experience on the platform, and
782
00:43:48,459 --> 00:43:52,219
we try to make it available for the agent.
783
00:43:52,439 --> 00:43:55,389
But the, this, about this quote
of just working until it's green
784
00:43:55,519 --> 00:43:57,449
is something that I do daily.
785
00:43:57,829 --> 00:44:02,719
And yes, I test that because it's
basically SemAI has tools, commands
786
00:44:02,719 --> 00:44:07,409
that the agent can run that will watch
the pipeline until it has a result.
787
00:44:07,829 --> 00:44:11,199
It will extract the errors to
the agent, and we will also be--
788
00:44:11,319 --> 00:44:16,399
you also can basically go back
in time and see errors as well.
789
00:44:16,719 --> 00:44:21,649
So for example, I can start my day just
asking my agent how the pipeline, how the
790
00:44:21,649 --> 00:44:27,249
CI is going, and can go over all of the,
the recent pull requests that we have.
791
00:44:27,659 --> 00:44:32,619
It can extract the errors, and this
part of extracting the error is the
792
00:44:32,619 --> 00:44:35,699
difference between my agent gaming
the system and just deleting the
793
00:44:35,699 --> 00:44:39,179
test and it actually fixing the test.
794
00:44:39,509 --> 00:44:44,409
So in these skills, we talk about
adversarial reviews of the agents, and
795
00:44:44,409 --> 00:44:46,299
how to use SemAI to extract the error.
796
00:44:46,799 --> 00:44:51,299
And in general, it's a little bit of
magic as we, we are having with, agents.
797
00:44:51,679 --> 00:44:56,559
So it's, it just works most of the time,
ninety-nine percent of the time, because
798
00:44:56,689 --> 00:44:58,909
it will only be able to see the error.
799
00:44:59,269 --> 00:45:03,039
It will have the error, clearly
labeled on its front, like it
800
00:45:03,039 --> 00:45:04,479
will just so see the error.
801
00:45:04,979 --> 00:45:09,599
And most of the time, it will just
look because it tries, it can try on
802
00:45:09,599 --> 00:45:11,429
the CI itself, it can push a commit.
803
00:45:11,919 --> 00:45:14,359
But if you want to test it,
you can test it locally.
804
00:45:14,699 --> 00:45:19,099
And also, if you are not able to test
it locally because the test is too
805
00:45:19,099 --> 00:45:23,729
big or because it needs something that
only the CI has, we have the test box.
806
00:45:24,229 --> 00:45:28,609
So it's basically a job that you can
spawn, and you can SSH into it, and
807
00:45:28,609 --> 00:45:32,669
you can give it that machine because
it's ephemeral to the agent to, for
808
00:45:32,669 --> 00:45:34,289
it to do anything that it wants there.
809
00:45:34,769 --> 00:45:39,749
So it can try lots of different
solutions and then just try to get
810
00:45:39,799 --> 00:45:41,859
that to the point that the CI is green
811
00:45:42,359 --> 00:45:43,289
Bret: Let's talk about that.
812
00:45:43,339 --> 00:45:45,629
let's dig into that 'cause we're,
we're running a little long, but
813
00:45:45,629 --> 00:45:49,979
I think the Test Box scenario and
having the agents doing that work is,
814
00:45:50,219 --> 00:45:52,429
to me, that's a really cool feature.
815
00:45:52,639 --> 00:45:57,289
I have been envisioning for a
while, and I don't make CI/CDs,
816
00:45:57,289 --> 00:45:58,699
I just use them constantly.
817
00:45:59,089 --> 00:46:04,849
And I just, at the end of the day, I
kinda just want my agent
to constantly be
818
00:46:04,849 --> 00:46:07,169
running CI/CD, whatever that looks like.
819
00:46:07,169 --> 00:46:10,239
Whether that's just the testing workflows
or maybe just the linting plus testing.
820
00:46:10,739 --> 00:46:14,719
But I kinda want things to constantly be
happening without me having to tell it,
821
00:46:14,749 --> 00:46:18,129
"Okay, every time you make a change, make
a commit and push, make a commit and push,
822
00:46:18,169 --> 00:46:24,629
make a commit and push." This just feels
like a, That mindset around m- waiting
823
00:46:24,629 --> 00:46:31,739
until the git com- commit and push to do
automation, it has its place, but it feels
824
00:46:31,739 --> 00:46:36,599
like in this world we need this whole
new thing, and that thing is before that.
825
00:46:37,099 --> 00:46:39,579
And s- a lot of people, I
mean, Dagger's been trying to
826
00:46:39,579 --> 00:46:40,799
solve this problem for years.
827
00:46:40,839 --> 00:46:41,859
Shout-out to Solomon Hykes.
828
00:46:42,169 --> 00:46:44,859
there's all these tools that are
trying to make it easy for us to run
829
00:46:44,859 --> 00:46:50,169
tests locally and in, in CI, and it's
always been a challenge to recreate
830
00:46:50,519 --> 00:46:54,449
the CI-like environment on a lo- on a
random local machine that's a th- of one
831
00:46:54,449 --> 00:46:55,879
of three different operating systems.
832
00:46:56,289 --> 00:46:59,169
And that's, I think that's
always been challenging for CI.
833
00:46:59,169 --> 00:47:03,499
I mean, a lot of CIs do it still, but
this idea that my agent can just do
834
00:47:03,499 --> 00:47:06,499
this on my behalf, it can do it in
the background, it can do it on my CI
835
00:47:06,549 --> 00:47:10,259
machines, but it has nothing really to
do with me committing code, like, that
836
00:47:10,279 --> 00:47:13,049
is pretty cool, and I wanna hear it.
837
00:47:13,139 --> 00:47:15,679
And what, I don't know anything
about how you do it, so I wanna
838
00:47:15,679 --> 00:47:17,259
dive in for maybe our last topic.
839
00:47:17,759 --> 00:47:20,949
Marko: So this is also one of the
transformation in the agentic way of
840
00:47:20,949 --> 00:47:25,069
working, because previously when you
were like, typing the code, then you
841
00:47:25,069 --> 00:47:29,379
know, you make a change, you run a couple
of tests on your own machine, and then
842
00:47:29,379 --> 00:47:30,809
you're, "Aha, okay, I'm happy with it.
843
00:47:31,139 --> 00:47:34,719
Let's push this and let's run
around the whole CI." But nowadays,
844
00:47:34,739 --> 00:47:38,639
that's not exactly… Nowadays,
it's not your job anymore.
845
00:47:38,649 --> 00:47:42,119
Your job is to orchestrate two,
three, five, four, five, how many,
846
00:47:42,169 --> 00:47:45,129
like what, how, like depending on
the context that you can keep in your
847
00:47:45,139 --> 00:47:47,439
head, agents that are doing things.
848
00:47:47,499 --> 00:47:49,229
And then
how do you mimic that thing?
849
00:47:49,239 --> 00:47:52,049
It's very hard to have a
local machine that is powerful
850
00:47:52,049 --> 00:47:54,089
enough to run all those things.
851
00:47:54,089 --> 00:47:59,529
So here there's like a really nice way
to scale it, to use your… A benefit
852
00:47:59,549 --> 00:48:03,929
is that you're using a CI box, so that
it's exactly the same as the CI box.
853
00:48:03,929 --> 00:48:07,729
That's an additional benefit, but
it's, it becomes scalable and it keeps
854
00:48:07,729 --> 00:48:11,889
you in that same loop as, as before
ju- just on a different scale because
855
00:48:11,899 --> 00:48:15,479
you, you have like a different you
know, automation tool at your disposal
856
00:48:16,029 --> 00:48:16,319
Bret: All right.
857
00:48:16,329 --> 00:48:20,419
So you're checking out a Git repo
and like I think a lot of us that
858
00:48:20,419 --> 00:48:24,799
are getting a little more advanced
in agent-based workflows, you have a
859
00:48:24,799 --> 00:48:28,589
local clone and then you use either
your harness or maybe yourself, you
860
00:48:28,589 --> 00:48:33,729
create gert- Git worktrees for, d- th-
the each one of the agents has its own
861
00:48:33,729 --> 00:48:35,619
safe space essentially locally, right?
862
00:48:35,969 --> 00:48:40,519
And then you're, you have this plugin,
you have the Semaphore plugin in
863
00:48:40,529 --> 00:48:42,019
your Claude Code or in your Codex.
864
00:48:42,019 --> 00:48:43,689
I think those are the two
you support right now, right?
865
00:48:44,189 --> 00:48:51,879
And in that includes, does that, that
plugin includes MCP skills, commands,
866
00:48:51,899 --> 00:48:55,169
and a CLI, or I guess it's using
the CLI to do some of the stuff?
867
00:48:55,189 --> 00:48:55,469
Okay.
868
00:48:55,799 --> 00:48:57,589
I wanna make sure I, include
everything it is, it has.
869
00:48:57,599 --> 00:49:01,239
So it has all this stuff and I guess
those skills are essentially like
870
00:49:01,289 --> 00:49:04,559
providing it the documentation as
well as the best practices and then
871
00:49:04,729 --> 00:49:07,369
how do you get it to behave this way?
872
00:49:07,379 --> 00:49:09,549
Are you prompting it
specifically around Semaphore?
873
00:49:10,259 --> 00:49:14,079
Or when you ask it to do a feature,
are you just saying, "I'd like
874
00:49:14,079 --> 00:49:15,219
you to work on this feature.
875
00:49:15,299 --> 00:49:16,929
I need you to implement this thing.
876
00:49:16,959 --> 00:49:19,179
Maybe I'm telling you about it or
maybe I'm pointing you to my ticket
877
00:49:19,189 --> 00:49:21,499
or wherever, where the, where
you need to get the information.
878
00:49:21,909 --> 00:49:25,019
And then while you're doing it, I
need you to do these CI things."
879
00:49:25,019 --> 00:49:27,469
Like how exactly, what does
the prompting look like there?
880
00:49:27,529 --> 00:49:31,359
I'm thinking about our audio listeners
and like they're trying to imagine working
881
00:49:31,359 --> 00:49:34,849
through this workflow and how do they get
Semaphore to do the coolest stuff here.
882
00:49:35,349 --> 00:49:35,709
Marcos: Yeah.
883
00:49:36,009 --> 00:49:39,949
For Semaphore doing the cool stuff
is just install SemAI because it has
884
00:49:39,949 --> 00:49:43,659
already the prompts and everything
that you for your agent to do.
885
00:49:43,959 --> 00:49:46,879
So for example, if you are using
SemAI, SemAI, you can just ask the
886
00:49:46,879 --> 00:49:52,119
agent to create a entire organization
or project, set up the Git connection
887
00:49:52,229 --> 00:49:54,309
to you so it can do everything there.
888
00:49:54,529 --> 00:49:59,029
And there is my part is like I have
some documentation on some CLAUDE.md
889
00:49:59,909 --> 00:50:05,379
and other documentation on my
repository to instruct it on how to
890
00:50:05,399 --> 00:50:10,879
do stuff and how to, basically give
one machine like one machine to one
891
00:50:10,879 --> 00:50:12,369
agent, and it can do anything there.
892
00:50:12,859 --> 00:50:17,839
I can be pretty relaxed to, some point
with the sandboxing there because I am
893
00:50:17,839 --> 00:50:22,929
just letting that sub-agent reuse that
machine so my machine is not impacted.
894
00:50:23,429 --> 00:50:29,569
And also, from the side of Semaphore,
I can put secrets on the CI and for
895
00:50:29,569 --> 00:50:33,349
example, I can create two, two projects
and one be production and the other
896
00:50:33,349 --> 00:50:35,829
one being just a staging, for example.
897
00:50:36,159 --> 00:50:39,939
And I can give each agent access
to some project because I can
898
00:50:39,949 --> 00:50:41,549
create multiple service accounts.
899
00:50:41,939 --> 00:50:45,279
This is the way that Semaphore
does the, the things, and I can
900
00:50:45,489 --> 00:50:49,089
put different roles and different
permissions to each service account.
901
00:50:49,589 --> 00:50:54,489
So I, I also had, one month ago,
persistent agents that could do things
902
00:50:54,489 --> 00:50:58,419
for me, and they were working with
the CI, but because they had specific
903
00:50:58,419 --> 00:51:03,459
permissions, they could not go outside
of the sandbox that they had on the CI
904
00:51:03,459 --> 00:51:05,219
analysis that they were doing for me.
905
00:51:05,719 --> 00:51:09,519
So for example, I just ask it
for the SemAI to give me the,
906
00:51:09,539 --> 00:51:10,779
the status of the project.
907
00:51:11,279 --> 00:51:14,129
So I have a 33% pass rate because of IQ.
908
00:51:14,629 --> 00:51:19,629
and it's telling me some flaky tests
that it detected and what I can do.
909
00:51:20,039 --> 00:51:22,859
So for example, I also have
two requests that are red.
910
00:51:23,179 --> 00:51:25,779
I can ask it to run and fix this stuff.
911
00:51:26,089 --> 00:51:30,669
So while we are talking,
please fix the red stuff.
912
00:51:31,169 --> 00:51:31,859
And
this is it.
913
00:51:32,359 --> 00:51:36,529
Bret: So in your initial prompt for
the feature, did you just add into the,
914
00:51:36,609 --> 00:51:40,439
what presumably was the big prompt on
get started on this particular feature,
915
00:51:40,729 --> 00:51:46,419
you just said, "And while you're at
it, run Semaphore test boxes until
916
00:51:46,419 --> 00:51:48,249
everything's green and then commit"?
917
00:51:48,299 --> 00:51:50,089
Is that kind of how you would do it?
918
00:51:50,589 --> 00:51:51,889
Marcos: Yes, I would do that.
919
00:51:51,919 --> 00:51:55,799
And because the plugin is installed,
it will probably just use the SemAI.
920
00:51:56,069 --> 00:52:00,719
It may be sometimes, it will request
for permission, because of the
921
00:52:00,719 --> 00:52:02,489
auto-classifier and stuff like that.
922
00:52:02,489 --> 00:52:05,999
But once you get permission
to it, it will just use.
923
00:52:06,319 --> 00:52:12,729
And also, on the CLI and on the MCP,
I don't see many people doing that,
924
00:52:12,779 --> 00:52:17,339
but we put some flags that you have to
pass so that the auto-classifier will
925
00:52:17,839 --> 00:52:21,769
be more likely to prevent that command
if you didn't give permission to it.
926
00:52:22,219 --> 00:52:27,759
So basically that, "Hey, I want-- I
really want to delete this stuff." So
927
00:52:27,769 --> 00:52:32,059
normally from the test that I did, the
auto-classifier holds the agent back
928
00:52:32,069 --> 00:52:34,279
if I don't really tell it to delete it.
929
00:52:34,749 --> 00:52:39,249
So we are trying to improve the
tool every day because we use that.
930
00:52:39,946 --> 00:52:43,726
Bret: Do you update your Claude file
or your agents file in the repo to
931
00:52:43,726 --> 00:52:49,446
say something like, every time you
make an edit run, run a Semaphore test
932
00:52:49,456 --> 00:52:52,946
box to validate so that, that way you
don't even… it's just sort of part
933
00:52:52,946 --> 00:52:57,176
of the initial prompt initially so
that it always defaults to testing.
934
00:52:57,176 --> 00:53:00,026
'Cause that feels, this feels like the
kind of thing that I'd always want running
935
00:53:00,026 --> 00:53:03,276
for every feature that my app is building.
936
00:53:03,326 --> 00:53:07,476
I would just-- I don't wanna ever have
to say, "You also need to remember to run
937
00:53:07,476 --> 00:53:09,336
some testing before we make a commit."
938
00:53:09,836 --> 00:53:15,436
Marcos: So yes, I have put that on
the claude.md, so just use SemAI.
939
00:53:15,626 --> 00:53:18,326
This is like for SemAI is
the only rule that I have.
940
00:53:18,326 --> 00:53:22,476
So use SemAI to test
stuff and r- sees the CI.
941
00:53:22,926 --> 00:53:26,526
And the other one is use sub-agents
to not pollute the main context.
942
00:53:27,026 --> 00:53:28,466
Just basically that.
943
00:53:28,966 --> 00:53:29,606
Bret: Yeah.
944
00:53:29,606 --> 00:53:33,186
I mean, I'm putting this in perspective
to what I currently have to do, right?
945
00:53:33,186 --> 00:53:37,806
'Cause I'm not using Semaphore yet,
and I'm… I have to do things like I
946
00:53:37,806 --> 00:53:41,376
want it to, I want it to at least lint,
or like you said, basic unit testing.
947
00:53:41,616 --> 00:53:44,776
And so then I have to make Makefiles
or Docker Compose files or something
948
00:53:44,776 --> 00:53:47,476
else that's gonna run locally, and
then I have to put in my agents
949
00:53:47,476 --> 00:53:49,046
file for each project a bunch of…
950
00:53:49,056 --> 00:53:49,826
a list of things.
951
00:53:49,976 --> 00:53:53,446
"Please always run these linters
before," or, "Please determine
952
00:53:53,446 --> 00:53:55,766
which are the best linters for this
language, and please always run them.
953
00:53:55,766 --> 00:54:01,166
And also run this thing to make s- if you
edit ever a g- a, a workflow file, go run
954
00:54:01,166 --> 00:54:02,676
these linters on this thing." I have to…
955
00:54:02,766 --> 00:54:06,126
I mean, I have a list of things
that I always want it to do on every
956
00:54:06,126 --> 00:54:11,456
change, and those are all really just
CI things that I can't run locally,
957
00:54:11,826 --> 00:54:15,006
so I have to give it… I have to
make sure those tools are installed.
958
00:54:15,316 --> 00:54:17,386
I have to give it permission to
install those tools if I think they're
959
00:54:17,386 --> 00:54:20,136
not gonna be installed, which means
my machine has to be very custom.
960
00:54:20,146 --> 00:54:22,296
It's just not… It is not ideal.
961
00:54:22,316 --> 00:54:25,216
I feel like I'm kind of working
around the limitations of my own CI.
962
00:54:25,216 --> 00:54:29,916
And, a- and I was trying to… And
I, and this is sort of me asking
963
00:54:29,966 --> 00:54:32,816
you all the questions of the things
that I really wish my CI did.
964
00:54:33,316 --> 00:54:33,766
So,
965
00:54:33,946 --> 00:54:34,456
Marcos: Yeah
966
00:54:34,956 --> 00:54:35,276
Bret: okay.
967
00:54:35,276 --> 00:54:37,846
So we've got this thing r-
creating these test boxes.
968
00:54:38,236 --> 00:54:43,316
are… I'm just curious, Are you able
to… I'm assuming Docker is on there,
969
00:54:43,316 --> 00:54:46,856
so then I can spin up Kubernetes clusters
and I can do all those things, right?
970
00:54:46,856 --> 00:54:48,356
That's all n- very normal runner stuff.
971
00:54:48,640 --> 00:54:49,120
Marko: Absolutely.
972
00:54:49,340 --> 00:54:49,620
Yep.
973
00:54:50,006 --> 00:54:53,326
Yeah, and then everything that you
described, Bret, it's like what this
974
00:54:53,326 --> 00:54:56,386
is what we typically see that our
customers put in their pipeline.
975
00:54:56,832 --> 00:54:57,262
Bret: Yeah
976
00:54:57,486 --> 00:54:59,486
Marko: And it's even better than
it, when it is part of the pi-
977
00:54:59,706 --> 00:55:03,336
pipeline, Because if you tell it to
the agent, then it's it will do it,
978
00:55:03,666 --> 00:55:05,606
but it's not really algorithmic.
979
00:55:06,106 --> 00:55:08,506
When you put it in a pl-
pipeline, it has to be done
980
00:55:08,516 --> 00:55:09,776
because then that's algorithmic.
981
00:55:09,956 --> 00:55:12,236
That's not probabilistic
like a, like an LLM.
982
00:55:12,266 --> 00:55:15,546
They are great, getting better
every day, but still, this
983
00:55:15,566 --> 00:55:17,026
is, for sure it will happen
984
00:55:18,106 --> 00:55:22,026
Marcos: we said, most of the s-
stuff that is, it's doing on its
985
00:55:22,026 --> 00:55:24,056
own is described on the skills.
986
00:55:24,453 --> 00:55:27,903
We have the guide on how to debug
pipelines, on how to deploy,
987
00:55:28,223 --> 00:55:33,053
fixing flake tests, even migrating
from GitHub actions to Semaphore.
988
00:55:33,963 --> 00:55:39,053
So we have lots of skills of
curated procedures that we do.
989
00:55:39,483 --> 00:55:44,353
So on Semaphore, we deploy like at least
three times a day to production, and we
990
00:55:44,353 --> 00:55:49,173
have some spiky days, some bursty days
that is more than 20 deploys on that day.
991
00:55:49,633 --> 00:55:53,543
So we have some knowledge
on how to do stuff on that.
992
00:55:53,633 --> 00:55:55,863
So for example, you can read the skills.
993
00:55:55,883 --> 00:56:01,313
I, invite the audience to also go and
view what is going on with the skills.
994
00:56:01,593 --> 00:56:05,213
We have a list of commands and steps
that you should do, for the agent to do
995
00:56:05,653 --> 00:56:07,793
and how to get the result that you want.
996
00:56:07,923 --> 00:56:11,803
So that's why you can just ask
it to do stuff because it already
997
00:56:12,063 --> 00:56:15,323
has the documentation, the
step-by-step on how to do that
998
00:56:15,823 --> 00:56:16,283
Bret: Yeah.
999
00:56:16,323 --> 00:56:19,323
And so for those not watching, the,
I mean, we're staring at the open
1000
00:56:19,323 --> 00:56:23,303
source SemAI repo, and there's what
looks like dozens of skills there.
1001
00:56:23,633 --> 00:56:27,463
I mean, for my GitHub Actions, I
only have one giant skill, so now I'm
1002
00:56:27,463 --> 00:56:28,873
realizing I'm already doing it wrong.
1003
00:56:29,303 --> 00:56:31,753
Because this is like,
these are very specific.
1004
00:56:31,753 --> 00:56:34,643
They're very detailed, but they're
also very specific, and why not?
1005
00:56:34,653 --> 00:56:36,373
Like, why shove it all into one skill?
1006
00:56:36,383 --> 00:56:36,533
Why
1007
00:56:36,605 --> 00:56:36,865
Marcos: Yeah.
1008
00:56:37,215 --> 00:56:41,955
And we also have the golden
goose here, is the sem-ai watch.
1009
00:56:42,445 --> 00:56:44,145
So this is what enables the loop.
1010
00:56:44,495 --> 00:56:47,245
basically it's a comb that
will check your pipeline.
1011
00:56:47,315 --> 00:56:48,425
It's simple.
1012
00:56:48,465 --> 00:56:51,435
It's very simple, but sometimes
the simple thing is what we miss.
1013
00:56:51,935 --> 00:56:54,955
So basically it will
watch your pipeline run.
1014
00:56:55,455 --> 00:56:58,765
When it's green, it will get back
to your agent, so that way he
1015
00:56:58,765 --> 00:57:04,005
can loop without needing to have
the AI harness loop it by itself
1016
00:57:04,505 --> 00:57:07,655
Bret: So in this scenario, the main
agent would probably start a sub-agent
1017
00:57:07,665 --> 00:57:11,165
with this watch and the command in the
background if it's doing it right, and
1018
00:57:11,165 --> 00:57:15,105
then it-- this would allow my agent
to work on consecutive features one
1019
00:57:15,105 --> 00:57:19,485
after the other, or decide stopping
points, I guess, if I gave it some more
1020
00:57:19,485 --> 00:57:23,155
detailed instruction to stop halfway
or whatever and test that particular
1021
00:57:23,155 --> 00:57:25,495
function, run it through the unit test.
1022
00:57:25,495 --> 00:57:28,655
Yeah, that's kinda how I imagine
that we would get these agents to do
1023
00:57:28,655 --> 00:57:30,395
is it's just this automatic thing.
1024
00:57:30,395 --> 00:57:31,615
It's happening in the background.
1025
00:57:31,625 --> 00:57:35,995
I'm not fully aware of how many
times it's run tests or whether I
1026
00:57:35,995 --> 00:57:38,925
need to go look at a page and stare
at it waiting for the green button.
1027
00:57:39,395 --> 00:57:41,855
And I am curious, a quick question.
1028
00:57:42,115 --> 00:57:43,385
You talked about the profiles.
1029
00:57:43,395 --> 00:57:46,165
This is a subtle little
security conversation.
1030
00:57:46,225 --> 00:57:49,425
you talked about the patches that
you're creating for each… What's that?
1031
00:57:49,925 --> 00:57:52,475
Marcos: Roles, the service accounts
basically on the Semaphore.
1032
00:57:52,525 --> 00:57:55,515
Bret: Yeah, these different accounts
essentially, like how does that
1033
00:57:55,525 --> 00:57:57,705
work in your harness specifically?
1034
00:57:57,705 --> 00:57:59,825
Okay, because I feel like this
is really important for people to
1035
00:57:59,825 --> 00:58:05,615
understand that ideally, we wanna
isolate our, and sandbox our agents
1036
00:58:05,635 --> 00:58:07,355
and give them only the keys they need.
1037
00:58:07,725 --> 00:58:13,425
And right now, one of my biggest
frustrations is we didn't… Like the
1038
00:58:13,425 --> 00:58:18,425
personal access tokens and the, and all
the ways we provide our local CLIs and
1039
00:58:18,425 --> 00:58:23,665
tooling to access ser- services on the
internet, like it wasn't really designed
1040
00:58:23,675 --> 00:58:28,425
for a world where we have disposable,
always running agents, and like we need
1041
00:58:28,425 --> 00:58:31,765
these things to be ephemeral, and they
need-- They might need to be short-lived,
1042
00:58:31,775 --> 00:58:35,315
or they need to be managed in a way
where I can make more than one, right?
1043
00:58:35,315 --> 00:58:38,295
'Cause I
remember when Docker it was
a long time before Docker Hub had
1044
00:58:38,325 --> 00:58:39,905
anything other than your password, right?
1045
00:58:39,905 --> 00:58:43,805
And then when `they made it, when they
created PATs, they were very limited.
1046
00:58:44,045 --> 00:58:46,865
They have read or read-write
across everything.
1047
00:58:47,005 --> 00:58:48,875
It's sort of like a giant blast radius.
1048
00:58:49,195 --> 00:58:53,805
And now, when we're in this agent world, I
think what you just described is probably
1049
00:58:53,805 --> 00:58:58,908
what, what I'm imagining is I really
want this agent on this work tree to
1050
00:58:58,908 --> 00:59:00,918
really have the smallest access possible.
1051
00:59:00,988 --> 00:59:03,078
I don't want them to
have admin on my repo.
1052
00:59:03,258 --> 00:59:08,038
I only maybe need them to be able to git
and git commit and push, and then run CI.
1053
00:59:08,418 --> 00:59:12,798
And I only want them to run the
CI for that particular repo, and
1054
00:59:13,278 --> 00:59:16,958
I need to give them that key, but
how do I give, how do I do that?
1055
00:59:17,458 --> 00:59:19,178
Or how do you do that, I guess,
specifically, 'cause this is
1056
00:59:19,178 --> 00:59:20,138
maybe not a Semaphore thing.
1057
00:59:20,138 --> 00:59:21,238
It's maybe more how you work.
1058
00:59:21,598 --> 00:59:25,838
How do you do that without
it being, like how the other
1059
00:59:25,838 --> 00:59:27,088
agents on your machine get that?
1060
00:59:27,098 --> 00:59:29,848
Are you just doing environment
variables per shell?
1061
00:59:29,848 --> 00:59:30,998
Is that kind of how you're doing it?
1062
00:59:30,998 --> 00:59:32,828
Or how do you approach that?
1063
00:59:33,328 --> 00:59:33,678
Marcos: Yeah.
1064
00:59:33,808 --> 00:59:38,428
What I would say, is that you should,
for example, start way more restrict
1065
00:59:38,428 --> 00:59:43,428
than what I'm doing currently, because
this is in scenario, I have lots of
1066
00:59:43,928 --> 00:59:49,058
tools, tool calls that I have already
whitelisted for my agents, right?
1067
00:59:49,378 --> 00:59:52,128
So you should start small,
don't whitelist anything or
1068
00:59:52,168 --> 00:59:54,548
just do some basis, basic stuff.
1069
00:59:55,048 --> 01:00:00,948
On Semaphore, you can create service
accounts and you can define roles for each
1070
01:00:00,948 --> 01:00:03,158
service account on each project, right?
1071
01:00:03,508 --> 01:00:07,788
I can put it as a reader or
as a admin or anything that I
1072
01:00:07,788 --> 01:00:09,518
want on my specific project.
1073
01:00:10,018 --> 01:00:13,658
You can basically control more,
most of this stuff this way.
1074
01:00:13,658 --> 01:00:17,308
So if you want a agent that
just wants to read the stuff
1075
01:00:17,308 --> 01:00:19,208
on Semaphore, you can put it.
1076
01:00:19,628 --> 01:00:23,508
Most of the time you are okay with
that because you just want to read the,
1077
01:00:23,518 --> 01:00:28,618
the pipeline status and then you, if
you are using GitOps, you are going
1078
01:00:28,628 --> 01:00:33,608
to commit another thing on GitHub
and it will spawn a-another pipeline.
1079
01:00:34,108 --> 01:00:37,398
And you can basically, like the
idea here is you can mix and match
1080
01:00:37,498 --> 01:00:42,238
permissions and each service account
will have a token, and that token
1081
01:00:42,288 --> 01:00:43,768
is used to authenticate on SemAI.
1082
01:00:44,988 --> 01:00:52,108
what I currently have, right, is basically
I have one token that has less privilege
1083
01:00:52,148 --> 01:00:56,508
than-- it cannot basically hit anything
that would damage my account or my
1084
01:00:56,508 --> 01:00:57,818
Semaphore, organization in this case.
1085
01:00:58,318 --> 01:01:01,848
because it was like with time and I put
more restrict-re-restrictions on it.
1086
01:01:02,348 --> 01:01:05,538
but basically I just let it run,
because it cannot do anything.
1087
01:01:05,618 --> 01:01:07,478
It cannot deploy to production.
1088
01:01:07,478 --> 01:01:11,028
It cannot mess with any
production related projects.
1089
01:01:11,498 --> 01:01:13,608
It can only do stuff on staging.
1090
01:01:14,108 --> 01:01:20,148
And for example, staging is something
that I think is more important in this
1091
01:01:20,148 --> 01:01:24,618
case because I can let it have more
reign over the staging environment.
1092
01:01:25,028 --> 01:01:28,948
It can do more stuff, but I am
always the gate to production when
1093
01:01:28,958 --> 01:01:30,458
we is talking about deployment.
1094
01:01:30,958 --> 01:01:31,338
Bret: Yeah.
1095
01:01:31,838 --> 01:01:33,858
Is this coming in through an
environment variable on your machine?
1096
01:01:33,868 --> 01:01:35,098
Is that how this is in your shell?
1097
01:01:35,598 --> 01:01:36,348
Marcos: This one, yeah.
1098
01:01:36,468 --> 01:01:40,238
This, this whole shell is
just one environment variable
1099
01:01:40,238 --> 01:01:42,038
of the token, basically.
1100
01:01:42,100 --> 01:01:42,440
Bret: Yeah.
1101
01:01:42,810 --> 01:01:44,460
I just had a live show last week.
1102
01:01:44,470 --> 01:01:50,010
We talked about nono, nono.sh, and
that's a sandboxing technology, and I
1103
01:01:50,010 --> 01:01:51,990
had one of the founders on the show.
1104
01:01:52,420 --> 01:01:54,260
And I'm a big fan of that.
1105
01:01:54,340 --> 01:01:57,640
So I'm just, for those listening, like
if you're trying to think about how
1106
01:01:57,640 --> 01:02:01,700
to isolate your tokens as well as your
agents, and Claude Code has this sandbox
1107
01:02:01,700 --> 01:02:05,110
thing built in, but the, the one of the
cool things that nono can do is you can
1108
01:02:05,110 --> 01:02:09,850
actually build out profiles that could
include these kind of secrets, these
1109
01:02:09,850 --> 01:02:14,580
kind of paths for things like Semaphore,
as well as egress rules that say like
1110
01:02:14,580 --> 01:02:20,220
you can only access package managers and
GitHub and then Semaphore, and that these
1111
01:02:20,220 --> 01:02:21,560
are only things you can do or whatever.
1112
01:02:21,840 --> 01:02:25,120
And then you, when you start
up Claude Code, this is kind
1113
01:02:25,120 --> 01:02:26,030
of the thing I'm wanting to do.
1114
01:02:26,040 --> 01:02:29,280
I haven't actually set it up yet 'cause
I'm using nono today, but to wrap
1115
01:02:29,350 --> 01:02:33,560
my Claude Code and my OpenCode and
my Py, my three favorite harnesses.
1116
01:02:33,850 --> 01:02:38,470
And I have found that Claude Code's
built-in sandboxing, I find it lacking.
1117
01:02:38,740 --> 01:02:42,880
So, the nono that wraps around it allows
me to give it file paths and environment
1118
01:02:42,880 --> 01:02:48,440
variables and secrets securely outside
of my harness, and then I can just sort
1119
01:02:48,440 --> 01:02:52,650
of feed all that in so that I don't
have to copy and paste things into
1120
01:02:52,650 --> 01:02:54,290
each shell every time I'm running them.
1121
01:02:54,290 --> 01:02:58,660
So I don't know if that's a pro tip, but
you can use nono in this way to probably
1122
01:02:58,660 --> 01:03:01,200
make a lot of this CI/CD stuff easier.
1123
01:03:01,260 --> 01:03:05,690
Like for me, it's like my default
GitHub command line has my key, which
1124
01:03:05,690 --> 01:03:09,460
has every repo that I have access to,
and I don't want my agents to do that.
1125
01:03:09,820 --> 01:03:13,160
Similar for like any of my
artifact storage or my CI.
1126
01:03:13,430 --> 01:03:16,330
Like I kinda want, especially as
a consultant, like I have access
1127
01:03:16,330 --> 01:03:18,910
to lots of different things, and I
don't want the blast radius there.
1128
01:03:19,350 --> 01:03:23,660
So I get nervous, and I end up using
nono to try to basically it removes
1129
01:03:23,660 --> 01:03:26,410
a lot of those secrets and removes
access to those things, so it doesn't
1130
01:03:26,410 --> 01:03:28,760
have access to my default tokens.
1131
01:03:29,230 --> 01:03:33,040
Shout out to the Agentic DevOps Guild
that I'm running because they're
1132
01:03:33,040 --> 01:03:35,640
the ones that we kinda worked out
a lot of these ideas around how to
1133
01:03:35,640 --> 01:03:37,710
use these for DevOps specifically.
1134
01:03:37,710 --> 01:03:41,140
Like, how do I avoi- avoid
Terraform, the agent having access
1135
01:03:41,140 --> 01:03:41,730
to GitHub, Terraform, Docker Hub,
1136
01:03:44,410 --> 01:03:48,250
Semaphore, AWS, and every key that
I have for the company, how do I
1137
01:03:48,250 --> 01:03:51,050
avoid it from nuking everything
every time I spin up Claude Code?
1138
01:03:51,060 --> 01:03:54,370
'Cause I know a lot of us are
dangerously skipping permissions.
1139
01:03:54,390 --> 01:03:57,730
I know I am, and that's bec- that's
because we're tired of hitting the
1140
01:03:57,730 --> 01:04:02,040
Yes key and, the, the, the Enter key
for every yes and request it has.
1141
01:04:02,050 --> 01:04:05,670
So if we're all doing that, like my pro
tip, if you haven't heard that episode
1142
01:04:05,690 --> 01:04:07,910
before this one, is to go check out nono.
1143
01:04:07,960 --> 01:04:08,670
Add that to Semaphore.
1144
01:04:09,140 --> 01:04:11,000
Now you're gonna, now you're
gonna level up your DevOps.
1145
01:04:11,000 --> 01:04:13,620
You're gonna… You're now DevSecOps
because you're using a sandbox.
1146
01:04:13,620 --> 01:04:16,910
That's maybe not the entire requirements
for being DevSecOps, but that's an
1147
01:04:16,910 --> 01:04:20,410
important step that I think a lot
of us aren't comfortable with yet.
1148
01:04:20,410 --> 01:04:21,690
But nono's really easy to use.
1149
01:04:21,690 --> 01:04:23,350
I feel like I'm doing an ad for
them in the middle of this show.
1150
01:04:23,660 --> 01:04:26,630
And they'll work great with Semaphore
for isolating some of these workflows.
1151
01:04:27,130 --> 01:04:27,660
Very cool.
1152
01:04:28,025 --> 01:04:30,805
Marcos: You can also do something
interesting with Semaphore because you
1153
01:04:30,805 --> 01:04:33,025
can put the secrets on the project itself.
1154
01:04:33,525 --> 01:04:39,005
So for example, we can put the GitHub
access tokens and other platforms
1155
01:04:39,005 --> 01:04:41,285
that you are using on the Semaphore.
1156
01:04:41,695 --> 01:04:45,685
And you can, for example, if you really
want to be very, very secure about that,
1157
01:04:46,035 --> 01:04:51,555
you can create one project per agent
or per activity that you are doing.
1158
01:04:51,925 --> 01:04:54,405
So you can basically put
the secrets on Semaphore.
1159
01:04:54,575 --> 01:04:58,395
It will only have access to those
secrets because you can restrict it to
1160
01:04:58,395 --> 01:05:02,945
only working on the ephemeral machine
and it can only have access to that.
1161
01:05:03,415 --> 01:05:08,745
And that with the service account will
be basically a really good sandbox to
1162
01:05:08,745 --> 01:05:10,715
keep your agent in check basically.
1163
01:05:11,215 --> 01:05:14,255
And also for deployments, you
can put a lot of s- specific,
1164
01:05:14,345 --> 01:05:15,735
preconditions on Semaphore.
1165
01:05:16,235 --> 01:05:20,695
So if you are deploying from Semaphore,
you can prevent rogue agents from
1166
01:05:20,695 --> 01:05:22,365
deploying stuff that it, they shouldn't.
1167
01:05:22,865 --> 01:05:23,395
Bret: Very cool.
1168
01:05:23,895 --> 01:05:29,195
Yeah, CI/CD is turning out to be
a really, a really good place for
1169
01:05:29,365 --> 01:05:33,815
attackers to find weak spots in all
of our, in all of our automation.
1170
01:05:33,815 --> 01:05:37,445
And I feel like, prompt injection,
between prompt injection and just general
1171
01:05:37,445 --> 01:05:41,005
supply chain attacks, I feel like we're,
our CI is constantly under threat.
1172
01:05:41,005 --> 01:05:46,085
So anytime I can level up the
security or run, like security audits.
1173
01:05:46,095 --> 01:05:46,876
Is Yeah, that's cool.
1174
01:05:46,876 --> 01:05:48,526
I mean, The fact that
it's built in is nice.
1175
01:05:48,546 --> 01:05:50,906
The fact that I don't have to go
find a different solution to plug
1176
01:05:50,906 --> 01:05:54,196
into my CI that makes this kind of
possible is probably where everyone
1177
01:05:54,196 --> 01:05:55,626
else is gonna go eventually too.
1178
01:05:55,626 --> 01:05:59,246
I think this is, as, as… I
just wanna stay in my harness.
1179
01:05:59,456 --> 01:06:03,886
I don't wanna have to wait any more than,
than the prompts require me to wr- wait.
1180
01:06:04,256 --> 01:06:05,586
I wanna reduce the number of steps.
1181
01:06:05,586 --> 01:06:10,386
I feel like in a lot of ways,
agents are helping us untangle the
1182
01:06:10,386 --> 01:06:13,456
last 10 to 15 years of complexity.
1183
01:06:13,516 --> 01:06:16,626
As someone in the cloud native ecosystem
and has been teaching Docker and
1184
01:06:16,636 --> 01:06:19,906
Kubernetes for a decade, to me, those
tools were, especially Kubernetes,
1185
01:06:19,936 --> 01:06:21,706
was always primarily for operators.
1186
01:06:21,976 --> 01:06:25,026
It wasn't meant to be
a tool for developers.
1187
01:06:25,056 --> 01:06:28,636
And also, a lot of the teams I would
work with would have a couple people
1188
01:06:28,636 --> 01:06:32,346
that were well-versed in CI/CD, but not
everyone needed to be a CI/CD expert.
1189
01:06:32,846 --> 01:06:36,206
But something happened over the last
decade where, I don't know if we got
1190
01:06:36,206 --> 01:06:40,376
distracted by the name full stack
developer or if the, if somehow people
1191
01:06:40,376 --> 01:06:44,646
got confused about what DevOps really
is, but somehow at some point, the
1192
01:06:44,646 --> 01:06:49,016
industry decided that developers are
responsible for everything, which I feel
1193
01:06:49,016 --> 01:06:53,816
like is a sad state of things where you
can't just focus on being an excellent
1194
01:06:53,826 --> 01:06:57,696
front-end developer or an excellent
architect of applications, but now you
1195
01:06:57,706 --> 01:07:02,956
have to know the CI, all the sa- CI/CD
paradigms, all of the operations, all
1196
01:07:02,956 --> 01:07:04,406
of the cloud infrastructure management.
1197
01:07:04,446 --> 01:07:07,596
You have to know Terraform, you have
to know Kubernetes and Docker, and
1198
01:07:07,596 --> 01:07:10,496
oh yeah, and you also gotta be this
great developer, and I always felt
1199
01:07:10,496 --> 01:07:12,216
like that was completely unrealistic.
1200
01:07:12,436 --> 01:07:17,946
And now what I'm seeing is, as me and my
friends talk about how we talk about a
1201
01:07:17,946 --> 01:07:20,246
couple of things that aren't necessarily
great, like one, when was the last time
1202
01:07:20,246 --> 01:07:22,296
we actually
looked at code besides this?
1203
01:07:22,346 --> 01:07:23,986
When was the last time I
actually wrote some code?
1204
01:07:24,216 --> 01:07:24,956
It's been a while.
1205
01:07:25,136 --> 01:07:27,916
And two, when was the last time
I actually learned something
1206
01:07:27,916 --> 01:07:30,536
new without the agent involved?
1207
01:07:30,586 --> 01:07:33,526
Like, when was the last time
that I learned a new tool just by
1208
01:07:33,536 --> 01:07:36,966
hand, or learned a new language,
or learned a new platform?
1209
01:07:37,166 --> 01:07:40,436
It's actually, we were talking about
it, that we're actually worried
1210
01:07:40,436 --> 01:07:45,116
that these agents are making us so
nimble and it's real- It's, to me,
1211
01:07:45,116 --> 01:07:48,966
I think it's just a sign that we all
love this tech, but it was too much.
1212
01:07:48,966 --> 01:07:50,536
There was too much we had to know.
1213
01:07:50,536 --> 01:07:52,486
There was too much we
all felt we had to learn.
1214
01:07:52,876 --> 01:07:57,856
And this is allowing us to do things
like be an expert in CI/CD, be an
1215
01:07:57,856 --> 01:08:01,976
expert in automation, be an expert
in Kubernetes without having to
1216
01:08:01,976 --> 01:08:03,236
know the Kubernetes command line.
1217
01:08:03,666 --> 01:08:05,456
And I don't know where this is all going.
1218
01:08:05,466 --> 01:08:08,096
I don't know if in three years we're
gonna wish that we hadn't done all
1219
01:08:08,096 --> 01:08:12,946
this and, we- we're all Luddites in our
understanding of command line tools, but
1220
01:08:13,366 --> 01:08:17,576
I am enjoying very much not having to
remember all the command line nuances
1221
01:08:17,586 --> 01:08:21,896
of the dozens of command line tools I
think I have to use in a given week.
1222
01:08:22,256 --> 01:08:23,786
It's just, it's gotten crazy.
1223
01:08:23,786 --> 01:08:26,774
Like 15, 20 years ago
we had very few tools.
1224
01:08:27,104 --> 01:08:28,774
We were all in one editor.
1225
01:08:28,994 --> 01:08:30,854
On Windows side, we
barely even had terminals.
1226
01:08:31,224 --> 01:08:36,904
So I'm just sort of soaking, basking
in the amazement of this moment where
1227
01:08:37,304 --> 01:08:39,944
agents are allowing us to implement
these things very quickly and to
1228
01:08:39,954 --> 01:08:42,094
get very productive pretty quickly.
1229
01:08:42,374 --> 01:08:45,954
I just hope they don't go away because…
Or that we don't lose access to them.
1230
01:08:45,954 --> 01:08:48,304
While we were talking before
the show about, you gotta have
1231
01:08:48,304 --> 01:08:49,464
a backup subscription now.
1232
01:08:49,464 --> 01:08:52,424
Your agents have to be redundant,
so Claude Code isn't… We had
1233
01:08:52,424 --> 01:08:53,704
a Claude Code outage yesterday.
1234
01:08:53,704 --> 01:08:55,774
Claude Code can't be
your only subscription.
1235
01:08:55,774 --> 01:08:58,424
You have to have some other backup
plan, whether
that's OpenRouter or
1236
01:08:59,054 --> 01:09:02,634
ChatGPT or, yeah, or Kimi or whatever
your favorite subscription is.
1237
01:09:02,634 --> 01:09:03,464
You gotta have something there.
1238
01:09:03,964 --> 01:09:04,264
Okay.
1239
01:09:04,654 --> 01:09:06,084
This has been really cool to dive in.
1240
01:09:06,114 --> 01:09:09,284
I've been really interested in how
this all works, and I do think between
1241
01:09:09,284 --> 01:09:14,084
your advanced plugin model and the
agent, having all those different
1242
01:09:14,084 --> 01:09:17,404
skills and all the different commands,
as well as the fact that it is sort
1243
01:09:17,404 --> 01:09:20,364
of empowered with the test boxes, I
think that's a really cool advantage.
1244
01:09:20,684 --> 01:09:24,014
'Cause someone earlier was asking,
"What are your differences between
1245
01:09:24,014 --> 01:09:27,624
you and other CI/CD platforms?" And
you're welcome to answer that question.
1246
01:09:27,624 --> 01:09:30,454
We always get the comparison
question in a show about a product.
1247
01:09:30,814 --> 01:09:33,484
But I will just answer that question
myself and say, as someone who operates
1248
01:09:33,484 --> 01:09:38,174
CI/CDs, those two things, like the
plugin model for those harnesses and
1249
01:09:38,574 --> 01:09:43,134
the test boxes to me with, in just
one hour of looking at Semaphore are,
1250
01:09:43,284 --> 01:09:46,684
to me, the advantages there, as well
as it just being open source and I
1251
01:09:46,684 --> 01:09:48,254
can run it myself if I feel like it.
1252
01:09:48,704 --> 01:09:53,084
l- like that, that seems like a,
th- the true futuristic agent n-
1253
01:09:53,384 --> 01:09:55,564
agent native way to operate a CI.
1254
01:09:55,564 --> 01:09:58,134
So I don't know if you
can say it better, but
1255
01:09:58,634 --> 01:10:00,724
Marko: No, you, no, you did a great job.
1256
01:10:00,764 --> 01:10:01,564
Thank you so much.
1257
01:10:01,664 --> 01:10:01,854
Yeah
1258
01:10:01,862 --> 01:10:03,402
Bret: But yeah, you can hire me anytime.
1259
01:10:03,452 --> 01:10:04,202
I'll be your DevRel.
1260
01:10:04,502 --> 01:10:07,232
but yeah, this is r- it's amazing
to see how far you all have come
1261
01:10:07,232 --> 01:10:08,572
in a year since you open sourced.
1262
01:10:08,582 --> 01:10:11,682
This is just a lot that you've
done, and it's pretty impressive.
1263
01:10:12,122 --> 01:10:13,772
So what's next?
1264
01:10:13,812 --> 01:10:14,472
What are you working on?
1265
01:10:14,942 --> 01:10:17,032
What are the hot tips for
the rest of this year?
1266
01:10:17,532 --> 01:10:17,972
Marko: Yeah.
1267
01:10:17,982 --> 01:10:23,572
So, so now that we have this interface to
the agents, we want to get to the basics.
1268
01:10:24,072 --> 01:10:28,232
And it's again, focused on, like really
doubling down on developers' productivity
1269
01:10:28,352 --> 01:10:29,702
and what does that mean for us?
1270
01:10:30,202 --> 01:10:34,712
First of all, if something fails in
my workflow, in my pipeline, I want
1271
01:10:34,722 --> 01:10:39,462
to be able to give as much as possible
information, the right information
1272
01:10:39,482 --> 01:10:43,162
to the agent so that the agent can
figure out what it is all about,
1273
01:10:43,192 --> 01:10:45,272
and hopefully fix it on its own.
1274
01:10:45,692 --> 01:10:48,012
So that I don't, as a developer,
I don't have to intervene.
1275
01:10:48,502 --> 01:10:51,812
And there is like a lot of nuances
like, what are, what's the format?
1276
01:10:51,942 --> 01:10:54,622
What's the amount of information
that you need to deliver to the
1277
01:10:54,622 --> 01:10:56,132
agent in order to get this right?
1278
01:10:56,172 --> 01:11:00,472
So that's the number one thing,
because I want to get to green state
1279
01:11:00,502 --> 01:11:02,472
as Marcus showed as soon as possible.
1280
01:11:02,972 --> 01:11:08,302
The other thing is that if everything is
green and everything is okay, I want to be
1281
01:11:08,312 --> 01:11:14,152
able to monitor how is my pipeline working
over the time and give enough data to my
1282
01:11:14,152 --> 01:11:17,682
agent so that it can optimize my pipeline,
because that's also very important.
1283
01:11:17,832 --> 01:11:22,582
We all know that as your test suite
grows, the duration of your pipeline
1284
01:11:22,582 --> 01:11:27,192
grows as well, and it's always healthy
to refactor or optimize your pipeline in
1285
01:11:27,192 --> 01:11:31,372
order to keep it under 10 minutes, under
15 minutes, whatever is your target.
1286
01:11:31,782 --> 01:11:35,822
And then last but not least is
like, enterprises, they need a
1287
01:11:35,822 --> 01:11:38,252
lot of governance and security.
1288
01:11:38,552 --> 01:11:41,502
We want to introduce some additional
services that can complement
1289
01:11:41,542 --> 01:11:45,572
the SLSA compliance, that's
the word that I'm looking for.
1290
01:11:45,692 --> 01:11:48,962
So that, that is also, these things can
be done with Semaphore right now, but
1291
01:11:48,962 --> 01:11:52,612
we want to make them out of the, like
working out of the box without additional
1292
01:11:52,612 --> 01:11:54,192
configuration and thinking through.
1293
01:11:54,192 --> 01:11:58,762
So it's how can we make developers more
productive with the focus on CI/CD.
1294
01:11:58,862 --> 01:12:00,612
And there are like s-
a lot of small things.
1295
01:12:00,652 --> 01:12:04,892
For example, like a next thing that we are
going to roll out is ability to just run,
1296
01:12:04,902 --> 01:12:07,602
rerun a single job in the whole pipeline.
1297
01:12:08,102 --> 01:12:11,882
Because a lot of the times, currently
you can rerun only the whole block.
1298
01:12:12,382 --> 01:12:15,472
But for a lot of the
times, only one job fails.
1299
01:12:15,602 --> 01:12:19,582
And you really want to be able to really
effectively just rerun that job to s- to
1300
01:12:19,582 --> 01:12:23,302
make sure that, hey, maybe it was like a
little bit of flakiness or something like
1301
01:12:23,302 --> 01:12:28,292
that, but you want to be able to push it
forward quickly and keep on developing.
1302
01:12:28,792 --> 01:12:29,752
That's, that's the goal.
1303
01:12:29,820 --> 01:12:32,570
Bret: Especially if that was
the f- the two-minute run
1304
01:12:32,580 --> 01:12:34,560
versus the, the 20-minute run
1305
01:12:36,066 --> 01:12:36,486
Marko: Absolutely.
1306
01:12:36,546 --> 01:12:37,056
Absolutely
1307
01:12:37,556 --> 01:12:38,836
Bret: You mentioned SLSA.
1308
01:12:38,966 --> 01:12:39,996
This is what you're talking about, right?
1309
01:12:39,996 --> 01:12:40,626
S-L-S-A?
1310
01:12:41,126 --> 01:12:41,276
Marko: Mm-hmm.
1311
01:12:41,546 --> 01:12:41,826
Yes.
1312
01:12:42,170 --> 01:12:42,260
Bret: Yeah.
1313
01:12:42,610 --> 01:12:46,265
So for those of you listening that are
not aware, I'm actually a big fan of SLSA.
1314
01:12:46,265 --> 01:12:47,995
we actually talked about it recently.
1315
01:12:47,995 --> 01:12:51,265
I've mentioned it more than a few times
over the years because I was really
1316
01:12:51,265 --> 01:12:55,745
paying attention to it even before version
one came out, and this is, this to me
1317
01:12:55,755 --> 01:12:58,855
is, the supply chain checklist bible.
1318
01:12:58,875 --> 01:13:04,825
I look at this as if I am not sure
around whether my supply chain is up
1319
01:13:04,825 --> 01:13:09,435
to snuff in terms of its security and
me doing all the right things as a
1320
01:13:09,875 --> 01:13:13,475
DevOps professional, it's not like we
have formal education where I come out
1321
01:13:13,475 --> 01:13:17,635
of school and I know the exact minimum
things that I should be doing in an
1322
01:13:17,645 --> 01:13:19,285
enterprise to keep my software safe.
1323
01:13:19,665 --> 01:13:23,885
And so SLSA to me is one of the
closest things to help me with that.
1324
01:13:24,185 --> 01:13:28,955
It will give me very prescriptive
step-by-step approaches to make sure
1325
01:13:28,955 --> 01:13:33,005
that I'm building securely, that I'm
providing providence with my code,
1326
01:13:33,015 --> 01:13:35,785
essentially built into the artifacts,
that I'm building the artifacts with
1327
01:13:35,785 --> 01:13:39,375
all the details that I need in order to
understand how and where they were built,
1328
01:13:39,375 --> 01:13:42,435
and when they were built, and by who,
and with all the right, all the different
1329
01:13:42,445 --> 01:13:44,105
details that I need to put in there.
1330
01:13:44,515 --> 01:13:48,435
And then making sure that I'm sort
of doing the best practices in the
1331
01:13:48,435 --> 01:13:52,225
industry, like linting code, or I'm not
sure if linting is actually on there.
1332
01:13:52,225 --> 01:13:56,865
But like doing some of the minimum sort
of… So the, the stuff that honestly,
1333
01:13:56,865 --> 01:14:01,195
quite frankly, if we, if
every open
source repo in particular was doing
1334
01:14:01,195 --> 01:14:04,315
that over the last year, we probably
wouldn't had near as many attacks on
1335
01:14:04,335 --> 01:14:07,485
repos that we've seen in the headlines
over the last year, because almost
1336
01:14:07,495 --> 01:14:13,785
all of that is due to a lack of teams
properly securing their workflows
1337
01:14:13,835 --> 01:14:17,385
and their behaviors and approaching
things from a DevSecOps mindset.
1338
01:14:17,675 --> 01:14:20,555
And if they had been following the
SLSA, which has levels, so, you can
1339
01:14:20,555 --> 01:14:24,115
start at a sort of beginner level, and
you can sort of promote and graduate as
1340
01:14:24,115 --> 01:14:27,025
you get better and more strict, 'cause
some of these things are easy and some
1341
01:14:27,025 --> 01:14:30,395
of these things, the steps by, steps
that it makes you do are gonna require
1342
01:14:30,395 --> 01:14:32,575
your, more than just your own effort.
1343
01:14:32,885 --> 01:14:36,865
And so I'm always trying to s- on,
especially on the re- open source
1344
01:14:37,265 --> 01:14:40,385
repos where they're public and I care
more about them, I'm always trying
1345
01:14:40,385 --> 01:14:44,335
to, level up my SLSA, from w- I can't
remember the levels or whatever.
1346
01:14:44,625 --> 01:14:47,505
But level one, level two, I'm not sure
if they're numbered, but I'm always
1347
01:14:47,505 --> 01:14:50,465
wanting to try to level those up to get
my software a little more hardened, a
1348
01:14:50,465 --> 01:14:52,445
little more, a little safer essentially.
1349
01:14:52,445 --> 01:14:53,355
And so go check that out.
1350
01:14:53,355 --> 01:14:58,655
That's S-L-S-A.dev, and they are part of
the OpenSSF, which is also a great thing
1351
01:14:58,655 --> 01:15:02,815
for everyone to check out for security,
and they have other stuff beside…
1352
01:15:02,945 --> 01:15:06,325
All, all the software security stuff is
over at OpenSSF, so go check that out.
1353
01:15:06,825 --> 01:15:09,145
That's another part of the cloud native
ecosystem, so I feel like I have to
1354
01:15:09,145 --> 01:15:12,155
plug it ' cause it's, it's cool stuff.
1355
01:15:12,655 --> 01:15:13,145
awesome.
1356
01:15:13,195 --> 01:15:14,335
Well, I'm looking forward to it.
1357
01:15:14,345 --> 01:15:15,385
How can people get started?
1358
01:15:15,415 --> 01:15:16,765
Okay, so you have the repo.
1359
01:15:17,155 --> 01:15:18,345
they can go over to,
1360
01:15:18,425 --> 01:15:20,165
Marko: Just go over to semaphore.ai.
1361
01:15:20,665 --> 01:15:20,865
Yeah.
1362
01:15:21,365 --> 01:15:22,995
And yeah, .io, exactly.
1363
01:15:24,185 --> 01:15:28,725
And, and, there is it's a simple command
that, they can run and install it.
1364
01:15:28,765 --> 01:15:32,455
it gets installed and then they can
say, "Hey, create my account." It will
1365
01:15:32,495 --> 01:15:34,305
prompt you to create your organization.
1366
01:15:34,305 --> 01:15:36,935
So the whole onboarding
is also part of the SemAI.
1367
01:15:37,885 --> 01:15:38,295
Bret: Right.
1368
01:15:38,505 --> 01:15:41,735
yeah, 'cause on your getting started
guide, it's funny, this is this is just
1369
01:15:41,735 --> 01:15:45,085
the indication of the cloud native or the,
agent native world that we live in now.
1370
01:15:45,365 --> 01:15:50,465
it's not download our CLI and
create an account and go do this
1371
01:15:50,485 --> 01:15:53,765
and read our repo for, or read our
documentation for how to use a CLI.
1372
01:15:53,795 --> 01:15:59,335
It's pl- install this plugin into Claude
Code or Codex and then ask it to get
1373
01:15:59,335 --> 01:16:02,805
started with Semaphore or whatever
the com- whatever the prompt is.
1374
01:16:03,155 --> 01:16:07,615
And, I am so… I feel like this is one
of my last resistances against the a-
1375
01:16:08,025 --> 01:16:11,965
the agents, is I'm so not used to that
workflow of just get the tool, install
1376
01:16:11,965 --> 01:16:13,645
it, create my account, do the things.
1377
01:16:13,685 --> 01:16:15,255
I don't want it… And
tell me when it's all done.
1378
01:16:15,275 --> 01:16:20,015
I'm just, I d- I s- I still am so
programmed to look for the command in
1379
01:16:20,015 --> 01:16:24,165
Brew, find which Brew package I need to
install, read the getting started guide,
1380
01:16:24,235 --> 01:16:26,115
create my account and all that stuff.
1381
01:16:26,115 --> 01:16:30,065
So it's cool that you're making--
You're not just making the use
1382
01:16:30,065 --> 01:16:33,035
of a CI AI native, you're making
the setup and c- account stuff AI
1383
01:16:33,075 --> 01:16:34,065
native too, so that's really cool.
1384
01:16:34,565 --> 01:16:37,105
All right, Marcos, Marko, thank
you so much for being here.
1385
01:16:37,605 --> 01:16:37,835
Marko: Yeah.
1386
01:16:38,025 --> 01:16:38,615
Thank you for having
1387
01:16:38,711 --> 01:16:42,141
Bret: you around the, we'll see
you around the agents, around…
1388
01:16:42,451 --> 01:16:44,501
We'll be chatting with our agents
are just gonna do this from now on.
1389
01:16:44,501 --> 01:16:47,001
Actually at some point this podcast is
just gonna be a bunch of agents talk.
1390
01:16:47,221 --> 01:16:48,641
Your agents will join my agents.
1391
01:16:48,641 --> 01:16:51,041
We will, they will live stream and
tell everybody about it, and then
1392
01:16:51,451 --> 01:16:54,101
our jobs will be to sit around
the pool and drink margaritas
1393
01:16:54,601 --> 01:16:54,901
Marko: Yeah.
1394
01:16:54,911 --> 01:16:55,631
Well, okay.
1395
01:16:55,851 --> 01:16:56,271
Sounds good.
1396
01:16:56,295 --> 01:16:56,665
Bret: All right.
1397
01:16:56,825 --> 01:16:57,325
Sounds good.
1398
01:16:57,445 --> 01:16:58,545
That's the utopia future.
1399
01:16:58,555 --> 01:16:59,885
The other future I don't wanna talk about.
1400
01:17:01,535 --> 01:17:03,235
Bret (2): Thanks for watching, and
I'll see you in the next episode