สำเนาบทสนทนา 🔗
WEBVTT
00:00:00.000 --> 00:00:03.500
When you valet your car, you don't hand over
00:00:03.500 --> 00:00:07.480
your key ring. You hand over the valet key. It
00:00:07.480 --> 00:00:10.099
starts the car, but it won't open the trunk.
00:00:10.439 --> 00:00:14.000
That's how an AI agent should get into your cloud.
00:00:14.439 --> 00:00:19.000
My guest today builds AI agents for cloud infrastructure.
00:00:19.339 --> 00:00:23.399
Early on, one of his agents hit a snag on a simple
00:00:23.399 --> 00:00:27.559
Terraform change and decided the fix was to run
00:00:27.559 --> 00:00:32.820
apply on staging. Resources got recreated. He
00:00:32.820 --> 00:00:37.119
calls that his wake-up call. Now his agents
00:00:37.119 --> 00:00:41.060
start with read-only access. Every change goes
00:00:41.060 --> 00:00:45.820
through a PR and trust grows from there. I'm
00:00:45.820 --> 00:00:49.228
Brian Teller from Teller's Tech and this is Ship It Weekly.
00:01:06.077 --> 00:01:08.400
Welcome back to Ship It Weekly, where
00:01:08.400 --> 00:01:11.299
I filter the noise and focus on what matters.
00:01:11.689 --> 00:01:14.069
when you are the one running infrastructure and
00:01:14.069 --> 00:01:17.750
owning reliability. Most weeks, it's a quick
00:01:17.750 --> 00:01:22.609
DevOps, SRE, platform, cloud, and security news
00:01:22.609 --> 00:01:26.549
recap. In between those, I do conversation episodes
00:01:26.549 --> 00:01:29.810
with people building and operating these systems.
00:01:30.430 --> 00:01:34.409
Ship It Weekly turns one this month. To celebrate
00:01:34.409 --> 00:01:37.349
through mid-December, you are getting two conversations
00:01:37.349 --> 00:01:42.209
a week, Mondays and Wednesdays, with the news every Friday.
00:01:42.519 --> 00:01:46.920
Then we're back to our usual rhythm. Today, I'm
00:01:46.920 --> 00:01:50.719
joined by Davlet, CEO and co-founder of Cloudgeni,
00:01:50.719 --> 00:01:54.319
which builds AI agents for cloud infrastructure.
00:01:54.739 --> 00:01:58.299
Davlet got into this after grinding through SOC
00:01:58.299 --> 00:02:02.640
2 and ISO compliance prep, mostly by hand, and
00:02:02.640 --> 00:02:05.659
asking what it would take for agents to fix those
00:02:05.659 --> 00:02:08.780
problems reliably without making things worse.
00:02:09.310 --> 00:02:12.169
We talk about the Terraform mistake that made
00:02:12.169 --> 00:02:15.490
guardrails the first problem to solve. Using
00:02:15.490 --> 00:02:18.949
agents to review infrastructure PRs against your
00:02:18.949 --> 00:02:23.050
own standards. Why running your own agents can
00:02:23.050 --> 00:02:26.849
cost a lot more than your AI subscription suggests.
00:02:27.409 --> 00:02:30.930
And where a platform team should start without
00:02:30.930 --> 00:02:34.330
building a monster. All right, let's jump in.
00:02:38.669 --> 00:02:41.090
Davlet, thank you for joining me. Yeah, thanks,
00:02:41.270 --> 00:02:44.389
Brian, for inviting us to your studio. Great
00:02:44.389 --> 00:02:47.909
to be here. I appreciate it. So I guess let's
00:02:47.909 --> 00:02:51.879
start with compliance, or at least like the risky
00:02:51.879 --> 00:02:56.080
parts. So everyone wants to do AI agents, right?
00:02:56.159 --> 00:02:58.400
That's the big thing right now with cloud and
00:02:58.400 --> 00:03:01.159
infrastructure work. What needs to be solved
00:03:01.159 --> 00:03:03.780
before that's a good idea? Yeah, that's a great
00:03:03.780 --> 00:03:07.620
question, Brian. And this is basically the question
00:03:07.620 --> 00:03:12.349
I had in my mind two years ago out of another
00:03:12.349 --> 00:03:15.229
project I was working before that where I had
00:03:15.229 --> 00:03:18.810
to go through, you know, still pretty much manual
00:03:18.810 --> 00:03:21.590
process of going through compliance requirements
00:03:21.590 --> 00:03:27.030
such as SOC 2 and ISO 27001 to be able to prepare
00:03:27.030 --> 00:03:31.650
our company to service a Fortune 500 customer.
00:03:31.870 --> 00:03:35.699
And even though some... the AI models were already
00:03:35.699 --> 00:03:41.000
available there, GPT 3.5 and similar, they still
00:03:41.000 --> 00:03:44.879
haven't been sufficient enough to help you in
00:03:44.879 --> 00:03:47.819
a meaningful way. So that's where my question
00:03:47.819 --> 00:03:50.780
popped in my mind. What do we need to build?
00:03:50.939 --> 00:03:56.460
What kind of harness or agentic workflows, guardrails,
00:03:56.520 --> 00:04:00.240
what do we need to help the agents, first of
00:04:00.240 --> 00:04:03.449
all, to be able to identify and solve compliance
00:04:03.449 --> 00:04:06.729
problems and solve them reliably. And in the
00:04:06.729 --> 00:04:08.969
process of using agents, that we actually don't
00:04:08.969 --> 00:04:11.550
make things worse as well. And this is the second
00:04:11.550 --> 00:04:13.770
part that was interesting as well, because as
00:04:13.770 --> 00:04:18.290
we started using agents more and more, they became
00:04:18.290 --> 00:04:21.589
generally more useful, but they also became another
00:04:21.589 --> 00:04:26.029
layer of potential vulnerabilities and breaches.
00:04:26.430 --> 00:04:30.029
So this is, it kind of brought both solution,
00:04:30.209 --> 00:04:33.240
but also... new set of problems to have to think
00:04:33.240 --> 00:04:39.180
about and solve it. So concretely, what was the
00:04:39.180 --> 00:04:41.720
first thing that we had in mind? Okay, let's
00:04:41.720 --> 00:04:45.120
use all the data that we already have. There
00:04:45.120 --> 00:04:47.839
is a lot of tools, open source commercial tools
00:04:47.839 --> 00:04:52.120
that already scan all kinds of your assets from
00:04:52.120 --> 00:04:54.980
infrastructure to your application code to maybe
00:04:54.980 --> 00:04:59.550
organizational policies for security and compliance
00:04:59.550 --> 00:05:02.430
findings. Everyone has them, thousands of alerts
00:05:02.430 --> 00:05:06.550
Load full transcript
in their hyperscalers and dashboards everywhere.
00:05:07.110 --> 00:05:09.769
What do we do with those, right? Takes a lot
00:05:09.769 --> 00:05:12.029
of time to plan for, takes a lot of time to go
00:05:12.029 --> 00:05:14.550
through the backlog. And this was our thought.
00:05:14.649 --> 00:05:17.529
Okay, how about we build an agentic workflow that
00:05:17.529 --> 00:05:20.870
will use this deterministic data and understanding
00:05:20.870 --> 00:05:24.319
of your... code, of your infrastructure as a
00:05:24.319 --> 00:05:27.139
code, of your policies, and find the correct
00:05:27.139 --> 00:05:30.620
solution to those findings. And we'll bring it
00:05:30.620 --> 00:05:34.720
to you in the best possible way. And this goal
00:05:34.720 --> 00:05:38.920
of finding the solution brought us finding out,
00:05:39.079 --> 00:05:42.959
okay, how can we build those agents in a way
00:05:42.959 --> 00:05:45.360
that you can use them in production? Because
00:05:45.360 --> 00:05:48.420
we also encountered a lot of problems while doing
00:05:48.420 --> 00:05:53.490
that. Yeah, some real stories. Luckily, our agents
00:05:53.490 --> 00:05:56.889
didn't cause any production problems because
00:05:56.889 --> 00:05:59.250
we were careful and we were thinking about in
00:05:59.250 --> 00:06:01.730
advance before, you know, you had those headlines
00:06:01.730 --> 00:06:04.649
where people were deleting their database and
00:06:04.649 --> 00:06:06.449
things like that. But we had a similar incident
00:06:06.449 --> 00:06:11.550
where an agent in the agentic mode, and that
00:06:11.550 --> 00:06:14.410
is important to note, right? Because there is
00:06:14.410 --> 00:06:16.370
different ways you can use AI agents. You can
00:06:16.370 --> 00:06:19.120
use them as just helpers or just... You know,
00:06:19.120 --> 00:06:20.920
you ask a question, you get the code back, or
00:06:20.920 --> 00:06:24.319
you actually allow them to do the whole work
00:06:24.319 --> 00:06:27.779
in iterative loop, right, until it actually completes
00:06:27.779 --> 00:06:30.560
the goal that you set to it. And this is what
00:06:30.560 --> 00:06:33.480
we call fully, you know, or not fully, but at
00:06:33.480 --> 00:06:36.720
least autonomous mode or agentic mode. And in
00:06:36.720 --> 00:06:39.579
this sort of mode, there is a higher risk that
00:06:39.579 --> 00:06:42.480
things can go sideways. And so one example of
00:06:42.480 --> 00:06:46.639
what happened is that agent was solving, you
00:06:46.639 --> 00:06:49.930
know, simple... Simple Terraform problem, right?
00:06:49.970 --> 00:06:51.870
Updating Terraform to update some environments.
00:06:52.290 --> 00:06:55.449
Things didn't work exactly like it was supposed
00:06:55.449 --> 00:06:58.110
to. And they decided that, okay, maybe applying
00:06:58.110 --> 00:07:02.089
the changes would be a good idea, right? And
00:07:02.089 --> 00:07:04.629
that was, you know, two years ago when we were
00:07:04.629 --> 00:07:07.350
still early on on the guardrails. So it went
00:07:07.350 --> 00:07:11.550
on and applied some changes on staging environment
00:07:11.550 --> 00:07:16.269
that led to some resources being recreated, because naturally
00:07:16.269 --> 00:07:19.649
apply can be destructive, right? So this was
00:07:19.649 --> 00:07:23.529
the sort of wake-up call for me, where I understood,
00:07:23.610 --> 00:07:25.970
okay, this is the more important problem that
00:07:25.970 --> 00:07:28.129
we need to solve first before we can actually
00:07:28.129 --> 00:07:30.449
allow agents to touch our infrastructure. And
00:07:30.449 --> 00:07:33.610
so that's where we changed our platform and started
00:07:33.610 --> 00:07:36.490
building all these guardrails on top of it as
00:07:36.490 --> 00:07:39.389
the first class citizens. So what kind of guardrails?
00:07:39.449 --> 00:07:42.310
Are we talking like OPA, Kyverno? Or are we talking
00:07:42.310 --> 00:07:45.279
like pre prompts? Are we talking validating output?
00:07:45.439 --> 00:07:47.279
What kind of guardrails do you have in place?
00:07:47.560 --> 00:07:52.259
All of it. And we have a good open source guide,
00:07:52.459 --> 00:07:55.339
infrastructure agents guide on GitHub that goes
00:07:55.339 --> 00:07:57.180
through all the levels. But basically there is
00:07:57.180 --> 00:08:02.379
around 13 or let's say 10 realistic levels that
00:08:02.379 --> 00:08:06.620
you need to think through when using or designing.
00:08:07.459 --> 00:08:10.399
Your system for agentic use, right? And it all
00:08:10.399 --> 00:08:13.019
starts with architecture, right? How do you architect
00:08:13.019 --> 00:08:17.100
actually the whole way of the system, the agentic
00:08:17.100 --> 00:08:20.000
system, right? And in those levels, you decide,
00:08:20.180 --> 00:08:23.360
okay, what kind of agents you're going to use
00:08:23.360 --> 00:08:26.199
and how do you want to orchestrate them? Where
00:08:26.199 --> 00:08:27.819
are you going to source them? Because that also
00:08:27.819 --> 00:08:32.019
brings certain kinds of questions about... you
00:08:32.019 --> 00:08:34.980
know, data privacy, you know, depending on where
00:08:34.980 --> 00:08:37.200
you source them. Then where do you get your tools,
00:08:37.379 --> 00:08:41.519
CLIs, skills? How do you actually enable agent
00:08:41.519 --> 00:08:45.220
to do those things that we want them to be able
00:08:45.220 --> 00:08:48.820
to do? How these agents run, right? Sandbox environments
00:08:48.820 --> 00:08:52.279
or running versus running on your desktop. Of
00:08:52.279 --> 00:08:55.259
course, you know, by definition, if you run it
00:08:55.259 --> 00:08:58.080
in sandbox, it's going to be safer. There is
00:08:58.080 --> 00:09:02.049
less likely that it will find some SSH file
00:09:02.049 --> 00:09:05.710
on your desktop to gain keys to your systems
00:09:05.710 --> 00:09:07.690
that you didn't intend to give, right? So these
00:09:07.690 -->
00:09:10.750
kind of problems at the, you know, that wouldn't
00:09:10.750 --> 00:09:13.470
be possible if you just run on sandbox environments.
00:09:13.809 --> 00:09:16.169
And then credential management, right? How do
00:09:16.169 --> 00:09:19.730
you actually provide it access to the things
00:09:19.730 --> 00:09:22.549
you wanted to have access, right? You know, if
00:09:22.549 --> 00:09:24.690
you want to solve with cloud infrastructure problems,
00:09:24.850 --> 00:09:27.809
how do you give it permissions, right? What kind
00:09:27.809 --> 00:09:30.389
of... Is it like a service principal? Are you
00:09:30.389 --> 00:09:34.509
creating a special user for it? And what kind
00:09:34.509 --> 00:09:36.929
of permissions you give it? And if you limit
00:09:36.929 --> 00:09:40.570
it, by definition, it might not be able to apply
00:09:40.570 --> 00:09:44.330
at all. Again, at every decision, you need to
00:09:44.330 --> 00:09:47.169
kind of think through what is intended here and
00:09:47.169 --> 00:09:50.110
can it do unintended. If you give it access to
00:09:50.110 --> 00:09:54.129
workflows on GitHub, it will be able to apply
00:09:54.129 --> 00:09:56.850
still. It will find some way to do things that
00:09:56.850 --> 00:09:59.509
you didn't maybe intend it to do. So at architectural
00:09:59.509 --> 00:10:02.090
level it shouldn't be possible even to do things
00:10:02.090 --> 00:10:04.730
you don't want it to do. And then of course, you
00:10:04.730 --> 00:10:07.409
know, what things you mentioned, policies and like
00:10:07.409 --> 00:10:10.409
OPA, and those are those are also important part
00:10:10.409 --> 00:10:14.710
of it. So it sounds like the foundational stuff
00:10:14.710 --> 00:10:16.490
that we've been talking about for years, even
00:10:16.490 --> 00:10:19.169
pre-AI, just making sure that's in place,
00:10:19.169 --> 00:10:22.570
time-bound access, least privilege. Yeah, yeah. Okay.
00:10:22.570 --> 00:10:25.669
And that's a fun part of it, right? I think this
00:10:25.669 --> 00:10:29.110
is where good engineering is becoming even more
00:10:29.110 --> 00:10:32.750
important. I can't remember who said it, but
00:10:32.750 --> 00:10:36.309
it was recently a notable quote that, you know,
00:10:36.330 --> 00:10:38.649
if you have a well-engineered system, right,
00:10:38.690 --> 00:10:41.250
with good access, you already have a process
00:10:41.250 --> 00:10:43.789
of onboarding junior engineers, right, where
00:10:43.789 --> 00:10:46.289
they start with little permissions and then you
00:10:46.289 --> 00:10:50.850
can grow from there as their autonomy increases,
00:10:51.090 --> 00:10:53.799
you know. Same principles can be applied to introducing
00:10:53.799 --> 00:10:56.000
a new teammate, right? In this case, it's AI.
00:10:56.860 --> 00:10:59.840
And of course, if you have not mature processes,
00:11:00.059 --> 00:11:03.860
then AI can make those things even worse. And
00:11:03.860 --> 00:11:07.929
I think, you know, the
platform teams will probably
00:11:07.929 --> 00:11:10.789
suffer even more if they don't have a tight grip
00:11:10.789 --> 00:11:14.669
and control already on those kinds of systems.
00:11:14.990 --> 00:11:18.049
Do you think that agents should get direct cloud
00:11:18.049 --> 00:11:21.830
credentials or like in practice? If they are
00:11:21.830 --> 00:11:25.730
time-bound and least privileged? Yes. In practice you create
00:11:25.730 --> 00:11:30.649
a special service principal or whatever way
00:11:30.649 --> 00:11:33.429
that is designed for your cloud of choice, but
00:11:33.429 --> 00:11:37.629
you limit the access to read-only if you want
00:11:37.629 --> 00:11:41.169
to make sure that it doesn't change your infrastructure
00:11:41.169 --> 00:11:44.250
directly. It should be able to see it. And then
00:11:44.250 --> 00:11:47.389
your changes, the way how you suggest your changes,
00:11:47.409 --> 00:11:49.929
should go through your typical change management
00:11:49.929 --> 00:11:53.669
process, right? Through your Git repo, CI/CD pipelines,
00:11:53.669 --> 00:11:57.570
tests, build tests. But it is useful when agent
00:11:57.570 --> 00:12:01.590
understands and sees the data, right? What kind
00:12:01.590 --> 00:12:05.570
of context should that agent have? Yeah. So typically
00:12:05.570 --> 00:12:10.190
what we see works best is it needs a context
00:12:10.190 --> 00:12:13.769
of your, how your infrastructure is structured,
00:12:13.769 --> 00:12:16.230
right? The architecture, all the resources, all
00:12:16.230 --> 00:12:18.730
the dependencies between them. That is important
00:12:18.730 --> 00:12:21.990
information. Your infrastructure is a code, so
00:12:21.990 --> 00:12:24.710
then it can start mapping between what what is
00:12:24.710 --> 00:12:26.450
in the cloud and what is infrastructure as a
00:12:26.450 --> 00:12:28.769
code. There is also very, you know, you can build
00:12:28.769 --> 00:12:32.110
a lot of useful workflows around it, you know,
00:12:32.110 --> 00:12:36.820
drift detection, migrating from ClickOps to IaC
00:12:36.820 --> 00:12:40.120
is a very useful one. Observability data is very
00:12:40.120 --> 00:12:43.960
useful so that it can help you with finding problems
00:12:43.960 --> 00:12:47.549
or potentially alerting you. Security findings,
00:12:47.929 --> 00:12:50.909
right? Whether it comes from AWS Security Hub
00:12:50.909 --> 00:12:54.769
or Azure Defender or some third-party tools,
00:12:54.970 --> 00:12:59.049
it's good when we have some more deterministic
00:12:59.049 --> 00:13:02.090
or some tools that already scan you for security.
00:13:02.309 --> 00:13:06.070
So then agents have it in the context. And cost
00:13:06.070 --> 00:13:08.470
data is also useful if you are interested to
00:13:08.470 --> 00:13:10.269
understand how infrastructure looks like from
00:13:10.269 --> 00:13:12.529
the cost perspective and FinOps. Do you have
00:13:12.529 --> 00:13:14.669
a human in the loop? Is it only in that approval
00:13:14.669 --> 00:13:17.090
right before merge or do you have a human early
00:13:17.090 --> 00:13:19.750
on in the loop? What's your thoughts on that? Yeah.
00:13:19.750 --> 00:13:23.309
So the way we think about it is that humans kind
00:13:23.309 --> 00:13:26.389
of get into this kind of agentic systems in two
00:13:26.389 --> 00:13:29.610
places. In the beginning, when you need to set
00:13:29.610 --> 00:13:32.490
it up, right? This is where you kind of need to
00:13:32.490 --> 00:13:35.230
put some upfront effort to make sure, again, you
00:13:35.230 --> 00:13:37.850
know, correct credentials, current access, and setting
00:13:37.850 --> 00:13:40.029
up workflows. What, what do you want agents to
00:13:40.029 --> 00:13:42.789
do? What kind of things, right? Do you want it to
00:13:42.789 --> 00:13:45.129
do daily security checks and then creating patches
00:13:45.129 --> 00:13:48.409
for you? Or do you want to rather come in and
00:13:48.409 --> 00:13:53.049
ask it to do things as you wish on the need-by-need
00:13:53.049 --> 00:13:55.090
basis, right? So it's like sort of inception
00:13:55.090 --> 00:13:59.830
point. And then at the very end, after the agent's
00:13:59.830 --> 00:14:03.330
done all the useful work, that would be the reviewer
00:14:03.330 --> 00:14:07.330
role where you kind of need to verify the quality
00:14:07.330 --> 00:14:09.889
of the work and if it matches the intent you
00:14:09.889 --> 00:14:13.250
had. So are you using agents at all or are you
00:14:13.250 --> 00:14:16.909
finding customers using agents to help review
00:14:16.909 --> 00:14:20.570
Terraform or OpenTofu plans? And how does that
00:14:20.570 --> 00:14:23.450
work in practice? Absolutely. And this is an
00:14:23.450 --> 00:14:26.029
interesting thing because agents are also good
00:14:26.029 --> 00:14:28.629
at reviewing or sometimes even better than humans
00:14:28.629 --> 00:14:32.409
in some cases. And agents can review other agents'
00:14:32.570 --> 00:14:35.889
work. So in a way, we can use agents to help
00:14:35.889 --> 00:14:37.789
us to review work, right? Because it will be
00:14:37.789 --> 00:14:40.740
very tedious if we have to go through each line
00:14:40.740 --> 00:14:43.919
that produces pages. So as part of agentic work,
00:14:43.980 --> 00:14:47.919
we also offer automatic PR reviews, agentic PR
00:14:47.919 --> 00:14:50.600
reviews that review both agentic and human work
00:14:50.600 --> 00:14:54.379
the same way. And the benefit, you know, these
00:14:54.379 --> 00:14:57.779
agentic reviewers, they can have the same context
00:14:57.779 --> 00:14:59.919
that you already provided. So they understand
00:14:59.919 --> 00:15:02.940
how your infrastructure is structured, what are
00:15:02.940 --> 00:15:04.980
the, you know, what are your compliance issues
00:15:04.980 --> 00:15:10.720
so that it can, for example, notify you if you
00:15:10.720 --> 00:15:14.720
make a change. Maybe it's a good change from
00:15:14.720
--> 00:15:17.700
a security point of view, but it will block access
00:15:17.700 --> 00:15:22.820
of your applications to it. For example, if you
00:15:22.820 --> 00:15:25.960
put a private network, for example, around your
00:15:25.960 --> 00:15:29.220
key vault, you might risk losing access yourself.
00:15:29.980 --> 00:15:33.259
So then PR reviewer should not only find the
00:15:33.259 --> 00:15:35.379
obvious errors, but also this kind of operational
00:15:35.379 --> 00:15:39.870
impact. Am I going to have downtime or lose access
00:15:39.870 --> 00:15:43.870
or will this recreate? So the PR reviewers can
00:15:43.870 --> 00:15:47.250
have this knowledge if you give it the right
00:15:47.250 --> 00:15:49.429
context and set it up correctly. And you can
00:15:49.429 --> 00:15:53.190
also enrich it with your custom policies or your
00:15:53.190 --> 00:15:56.309
organizational policies. Even if you have some
00:15:56.309 --> 00:16:00.389
rules about styling or how your modules need
00:16:00.389 --> 00:16:02.730
to be structured, all this knowledge you can
00:16:02.730 --> 00:16:06.809
include. And then PR reviewers will help you
00:16:06.809 --> 00:16:10.330
to keep your infrastructure standardized. This
00:16:10.330 --> 00:16:15.389
is also one of the key findings and key features
00:16:15.389 --> 00:16:17.710
that were important for one of our enterprise
00:16:17.710 --> 00:16:20.830
customers, for example. What about drift? So
00:16:20.830 --> 00:16:23.889
I know we deal with drift a lot with IaC. Maybe
00:16:23.889 --> 00:16:26.110
not a lot. I guess it depends on the organization.
00:16:26.470 --> 00:16:28.750
But if you're dealing with drift, you maybe don't
00:16:28.750 --> 00:16:31.350
have the same context that you would for a PR
00:16:31.350 --> 00:16:33.669
review. So how does the agent deal with that?
00:16:33.730 --> 00:16:37.700
And how do you reconcile that agent doing that
00:16:37.700 --> 00:16:39.360
work on the drift? Or do you not have agents
00:16:39.360 --> 00:16:43.059
do drift work? Yeah, yeah. No, we do. So this
00:16:43.059 --> 00:16:45.419
was one of the problems that we saw in the beginning,
00:16:45.539 --> 00:16:48.039
right? We early on decided, okay, we don't want
00:16:48.039 --> 00:16:50.360
agents, for example, touch infra directly, right?
00:16:50.399 --> 00:16:54.269
Like this was for us a no-no. So how do we avoid
00:16:54.269 --> 00:16:57.669
it? How do we circumvent it? We want to create
00:16:57.669 --> 00:17:01.450
PRs and do changes to infra through CI/CD pipelines.
00:17:02.269 --> 00:17:05.329
But to be able to do that, you need to have no
00:17:05.329 --> 00:17:07.170
drift or minimal drift, right? You need to be
00:17:07.170 --> 00:17:09.029
able to trust your infrastructure as a code.
00:17:09.170 --> 00:17:11.450
So we built the whole workflow to be basically
00:17:11.450 --> 00:17:15.190
to detect drift and use, again, agents to reconcile
00:17:15.190 --> 00:17:17.809
everything that you have in
your cloud back to
00:17:17.809 --> 00:17:20.970
your infrastructure as a code. Because, of course,
00:17:20.990 --> 00:17:23.779
the best way, you just have your infrastructure
00:17:23.779 --> 00:17:26.859
code and from it you create your cloud infrastructure
00:17:26.859 --> 00:17:32.470
and that's it. In reality, I have not... I haven't
00:17:32.470 --> 00:17:34.650
worked with an organization yet that would be
00:17:34.650 --> 00:17:37.950
able to do that, right? Unfortunately. So, and,
00:17:38.049 --> 00:17:40.390
you know, we work with great organizations and
00:17:40.390 --> 00:17:42.529
it's not that it's completely messed, but, you
00:17:42.529 --> 00:17:44.509
know, there's still some important changes in
00:17:44.509 --> 00:17:46.269
the cloud that you would not be able to just
00:17:46.269 --> 00:17:49.730
overwrite with apply as purists might want to
00:17:49.730 --> 00:17:54.450
do. So to account for real-world scenario, we
00:17:54.450 --> 00:17:57.490
build this scanner that would find these drifts
00:17:57.490 --> 00:18:00.819
and then use agents to help clean it up. And
00:18:00.819 --> 00:18:02.940
bring everything into infrastructure as a code
00:18:02.940 --> 00:18:05.119
to following your standards. So having that,
00:18:05.220 --> 00:18:08.559
if you have that kind of information, then dealing
00:18:08.559 --> 00:18:11.960
with drift is pretty simple. And you can build
00:18:11.960 --> 00:18:14.759
it yourself, this kind of logic as well. We've
00:18:14.759 --> 00:18:16.819
talked about drift. We've talked about the policy.
00:18:17.119 --> 00:18:19.200
What we haven't really touched on is the financial
00:18:19.200 --> 00:18:22.559
aspect of running agents in cloud. How do companies
00:18:22.559 --> 00:18:25.160
deal with the financial implications of running
00:18:25.160 --> 00:18:28.140
models? It's saving them time. It's maybe helping
00:18:28.140 --> 00:18:31.230
with velocity. But how do they manage that cost
00:18:31.230 --> 00:18:33.170
when it comes to running agentic workflows?
00:18:33.650 --> 00:18:37.109
Yeah, I think right now the conversations are
00:18:37.109 --> 00:18:39.869
pretty early, right? But they are starting to
00:18:39.869 --> 00:18:42.930
show up. Until now, I think the mandate was very
00:18:42.930 --> 00:18:45.410
simple. Just use tokens as much as possible.
00:18:45.869 --> 00:18:48.190
Let's play with this. You know, people don't
00:18:48.190 --> 00:18:51.390
know how to use it still. Let's not put any limitations.
00:18:51.950 --> 00:18:55.109
Let's just, you know, enable it at organization
00:18:55.109 --> 00:18:59.049
level. Unlimited usage oftentimes. And let's
00:18:59.049 --> 00:19:01.430
see what happens. So what happens is that, of
00:19:01.430 --> 00:19:04.250
course, you will end up in a situation where
00:19:04.250 --> 00:19:08.210
people use a lot, right? I think there was a
00:19:08.210 --> 00:19:11.990
new case where someone just built a non-work-related
00:19:11.990 --> 00:19:18.069
game and used maybe a few hundred thousand, or not a few, tens of thousands
00:19:18.069 --> 00:19:21.089
of dollars worth of tokens. Or other cases where
00:19:21.089 --> 00:19:23.829
the budget was used up within three months, the
00:19:23.829 --> 00:19:26.210
budget that was planned to be used for a year.
00:19:26.569 --> 00:19:29.509
And I think also the disconnect also comes from
00:19:29.509 --> 00:19:34.369
that we may be used to using AI from, you know,
00:19:34.410 --> 00:19:38.250
ChatGPT, from OpenAI, from Claude Code, through
00:19:38.250 --> 00:19:40.990
subscriptions, and they subsidize them a lot.
00:19:41.369 --> 00:19:44.069
So if you use, let's say, if you build your own
00:19:44.069 --> 00:19:47.490
agentic systems and you use tokens through API,
00:19:47.849 --> 00:19:52.160
you might see that you will have to pay 10, 20
00:19:52.160 --> 00:19:55.680
times more in real cost, you know, for the same
00:19:55.680 --> 00:19:59.660
capabilities. So this makes it, of course, challenging
00:19:59.660 --> 00:20:02.920
to invest into this kind of your own, building
00:20:02.920 --> 00:20:05.799
your own workflows. And this is also something
00:20:05.799 --> 00:20:09.869
that we, you know, we also offer subsidized in
00:20:09.869 --> 00:20:13.289
a way service right now, right? But for companies
00:20:13.289 --> 00:20:15.690
to be able to build, let's say, same capabilities
00:20:15.690 --> 00:20:18.109
internally, they want similar to Cloudgeni,
00:20:18.250 --> 00:20:22.049
they would have to pay 10 to 20 times more to
00:20:22.049 --> 00:20:25.309
enable the same things. So I think understanding,
00:20:25.529 --> 00:20:28.190
I think first question is understanding where
00:20:28.190 --> 00:20:31.390
the cost comes from, right? To be able to track
00:20:31.390 --> 00:20:34.569
it. Right now, there is no control, basically,
00:20:34.710 --> 00:20:37.529
right? There is no... And this also goes back
00:20:37.529 --> 00:20:40.410
to the security and policies part. There is no
00:20:40.410 --> 00:20:45.680
audits on the agentic usage, right? No one knows
00:20:45.680 --> 00:20:49.799
where is it used, what is it used for, what kind
00:20:49.799 --> 00:20:52.059
of prompts people are running. Are there maybe
00:20:52.059 --> 00:20:55.230
agentic... loops that just got stuck and using
00:20:55.230 --> 00:20:59.650
money for no reason, right? It's, it can easily
00:20:59.650 --> 00:21:04.049
just, you know, pull a non-existent API server
00:21:04.049 --> 00:21:06.670
every day and use tokens and no one will know.
00:21:06.670 --> 00:21:10.150
It is a completely new area, I think. And I think
00:21:10.150 --> 00:21:14.319
the FinOps for AI is going to be big, a big problem
00:21:14.319 --> 00:21:16.519
and big area to explore, just like we have in
00:21:16.519 --> 00:21:18.460
cloud infrastructure, right? Orphan resources,
00:21:18.920 --> 00:21:21.019
you know, orphaned agents that don't have purpose
00:21:21.019 -->
00:21:23.660
anymore. We will see a lot of that. So I think
00:21:23.660 --> 00:21:26.819
first step is actually having, if you're going
00:21:26.819 --> 00:21:29.440
to build systems, agentic systems internally,
00:21:29.759 --> 00:21:34.329
you need to design it with a way where you can
00:21:34.329 --> 00:21:37.970
audit and see both costs, but also, you know,
00:21:37.970 --> 00:21:40.869
the types of prompts and what they do and how
00:21:40.869 --> 00:21:44.690
they do and measure some impact as well. So
00:21:44.690 --> 00:21:48.509
this is what we also started realizing also with
00:21:48.509 --> 00:21:52.309
Cloudgeni. And we started, we pull out the
00:21:52.309 --> 00:21:54.990
foundation that, because in Cloudgeni, you
00:21:54.990 --> 00:21:59.259
already have audits and control. So we pull out
00:21:59.259 --> 00:22:01.400
this part and we made it open source, basically.
00:22:01.539 --> 00:22:04.900
The foundation, the agentic builder, so to speak,
00:22:05.000 --> 00:22:08.839
as an open source that anyone can use to build
00:22:08.839 --> 00:22:11.329
their own workflows. Could be infrastructure
00:22:11.329 --> 00:22:14.490
related, could be completely other area of business.
00:22:14.750 --> 00:22:20.569
How do you correlate the spend in the API calls
00:22:20.569 --> 00:22:24.930
for the agent versus the production output that
00:22:24.930 --> 00:22:26.690
you're getting from that? How do you correlate
00:22:26.690 --> 00:22:30.650
those two? Yeah, right now it is more vibes,
00:22:30.690 --> 00:22:33.950
I would say, than hard data, definitely. But
00:22:33.950 --> 00:22:38.150
we can see it already in our own case, for example,
00:22:38.210 --> 00:22:41.990
right? We are able to achieve quite a lot with
00:22:41.990 --> 00:22:46.049
extremely small engineering teams. We don't basically
00:22:46.049 --> 00:22:50.849
have a dedicated DevOps engineer. We run everything
00:22:50.849 --> 00:22:54.430
using our own product. And our engineering is
00:22:54.430 --> 00:22:58.509
also quite thin. So what we focus on rather is
00:22:58.509 --> 00:23:01.750
investing into those agentic workflows and automate
00:23:01.750 --> 00:23:05.309
more and more engineering that way. And from
00:23:05.309 --> 00:23:09.609
that point of view, it is still... cheaper or
00:23:09.609 --> 00:23:13.349
there's less cost than hiring a person. And it
00:23:13.349 --> 00:23:16.869
also less overhead in terms of knowledge sharing,
00:23:17.049 --> 00:23:19.730
in terms of alignment. We would want to scale
00:23:19.730 --> 00:23:21.450
though, definitely with more people as well,
00:23:21.470 --> 00:23:24.630
but it also puts a certain requirement on engineers
00:23:24.630 --> 00:23:29.089
to, instead of focusing on one-to-one work,
00:23:29.289 --> 00:23:33.690
rather than focusing on, again, building agentic
00:23:33.690 --> 00:23:36.710
capabilities and orchestrating agents to do their
00:23:36.710 --> 00:23:39.880
work, right? So this is what we are focusing
00:23:39.880 --> 00:23:43.180
on. And from that perspective, the economic impact
00:23:43.180 --> 00:23:46.240
is visible, right? But we are yet to see it on
00:23:46.240 --> 00:23:49.460
a larger scale, I would say, like in larger companies
00:23:49.460 --> 00:23:53.700
where it can be more difficult to do so, these
00:23:53.700 --> 00:23:56.500
kind of transformations. Okay, so given the cost
00:23:56.500 --> 00:24:00.079
aspect, given the guardrails, given the models,
00:24:00.319 --> 00:24:02.599
is there anything that you wouldn't trust an
00:24:02.599 --> 00:24:05.119
AI agent with yet that you think it's maybe
00:24:05.119 --> 00:24:07.839
not great at solving, maybe a person whether it
00:24:07.839 --> 00:24:09.660
doesn't have the context or just a person is
00:24:09.660 --> 00:24:12.519
better suited for solving it? Yeah, I still wouldn't
00:24:12.519 --> 00:24:14.859
trust it to run, to make changes directly in infrastructure,
00:24:14.859 --> 00:24:19.039
right? Like, unless I'm okay that this might, this
00:24:19.039 --> 00:24:22.279
might, this is a staging environment or test environment
00:24:22.279 --> 00:24:26.119
that is meant to, to be, right? And this is what
00:24:26.119 --> 00:24:29.039
we are trying to solve as well in some limited
00:24:29.039 --> 00:24:31.759
way. Can we let agent run, for example, our own
00:24:31.759 --> 00:24:35.359
infrastructure without our oversight? And we have
00:24:35.359 --> 00:24:39.039
internal experiments where we start looking into
00:24:39.039 --> 00:24:42.200
it. But when it comes to, let's say, customer
00:24:42.200 --> 00:24:48.680
data, privacy, losing data, losing access, where
00:24:48.680 --> 00:24:52.819
there is big consequences. I would still want
00:24:52.819 --> 00:24:55.880
to have some sort of guardrails before it, even
00:24:55.880 --> 00:24:58.039
if it's semi-autonomous, but I would want to
00:24:58.039 --> 00:25:01.380
have some deterministic blocks. For example,
00:25:01.539 --> 00:25:04.500
it could be as simple as deletion block on the
00:25:04.500 --> 00:25:06.420
resource group, for example, for production resources,
00:25:06.799 --> 00:25:09.960
right? So it wouldn't be possible to delete it
00:25:09.960 --> 00:25:12.940
without someone approving it, like a person,
00:25:13.079 --> 00:25:15.960
a human engineer. Yes, but putting termination
00:25:15.960 --> 00:25:19.329
protection on an instance as an example. At
00:25:19.329 --> 00:25:21.710
the API level. So even if the agent did try to
00:25:21.710 --> 00:25:24.210
run it, it couldn't without removing that protection.
00:25:24.470 --> 00:25:27.509
Yeah. So I think at some point, if you have enough
00:25:27.509 --> 00:25:29.769
of those guardrails and if you have sufficient
00:25:29.769 --> 00:25:33.549
enough checks where you prove, you know, error-proof
00:25:33.549 --> 00:25:37.789
your systems, basically, I do think there
00:25:37.789 --> 00:25:40.690
will come a moment and maybe not very far in
00:25:40.690 --> 00:25:43.809
the future where you would see fully autonomous
00:25:43.809 --> 00:25:47.990
agentic systems, right? Where no human actually
00:25:47.990 --> 00:25:50.589
looks at it. It will, you know, human will look
00:25:50.589 --> 00:25:52.970
at dashboard to see like, okay, everything is
00:25:52.970 --> 00:25:57.289
healthy. And this is what we're also building
00:25:57.289 --> 00:26:01.089
for, right? Starting from smaller scale. Yeah.
00:26:01.150 --> 00:26:03.930
Are you building like a risk scoring system where
00:26:03.930 --> 00:26:07.450
like, if I'm adding tags to a resource, like
00:26:07.450 --> 00:26:10.089
very low risk, right? I can probably let an agent
00:26:10.089 --> 00:26:13.990
do that. But if I'm modifying VPC route table,
00:26:14.130 --> 00:26:16.710
might not want an agent touching that without
00:26:16.710 --> 00:26:19.730
a human. Like, are you? scoring that and how
00:26:19.730 --> 00:26:22.519
do you score that like where we'd have a confidence
00:26:22.519 --> 00:26:24.480
level for an agent to be able to make those changes
00:26:24.480 --> 00:26:28.099
or not make those changes? Yeah, I think the way
00:26:28.099 --> 00:26:30.579
we do it, let's say in Cloudgeni, we built, we
00:26:30.579 --> 00:26:34.079
separated by workflows, right? Certain workflows
00:26:34.079 --> 00:26:37.180
are more okay to do automatic, like for example
00:26:37.180 --> 00:26:40.019
import. This is very safe action, there is no impact,
00:26:40.019 --> 00:26:42.559
right? So we just let agents do completely. Security
00:26:42.559 --> 00:26:44.779
and compliance patches, they're a little bit more
00:26:45.450 --> 00:26:47.609
you know, where you need more control, right?
00:26:47.670 --> 00:26:50.309
And this is where we put a little bit more guardrails
00:26:50.309 --> 00:26:53.329
and checks. So I think for now we do it sort
00:26:53.329 --> 00:26:57.970
of semi-manually. So for now it's up to a person,
00:26:57.990 --> 00:26:59.630
I think, to decide because even tagging could
00:26:59.630 --> 00:27:03.569
be quite important, right? Let's say someone...
00:27:03.569 --> 00:27:04.990
If it's FinOps related or something, yeah. ...doesn't
00:27:04.990 --> 00:27:07.769
want to mess it up as well. So for one team,
00:27:07.809 --> 00:27:09.690
it could be like, ah, that's no problem. For
00:27:09.690 --> 00:27:11.410
another team, it's like, no, no, we need to make
00:27:11.410 --> 00:27:13.289
sure it's right. So I think you kind of need
00:27:13.289 --> 00:27:19.059
this kind of flexibility. In that, for the future,
00:27:19.140 --> 00:27:22.539
for that future product, basically where we think
00:27:22.539 --> 00:27:25.720
agent will be able to run everything, we're sort
00:27:25.720 --> 00:27:28.740
of starting from the foundation of it. But of
00:27:28.740 --> 00:27:31.640
course, it will only be applied only to either
00:27:31.640 --> 00:27:35.380
non-production environments or let's say small
00:27:35.380 --> 00:27:38.259
startup environments, right? Where the impact
00:27:38.259 --> 00:27:41.299
is lower. And from there, we will just, yeah.
00:27:41.579 --> 00:27:43.579
If we know it's ephemeral, it doesn't really
00:27:43.579 --> 00:27:46.490
matter as much, right? Yeah. And there we just
00:27:46.490 --> 00:27:49.589
let agent go crazy and see where it breaks. And
00:27:49.589 --> 00:27:53.250
then we can build, okay, this is where it breaks.
00:27:53.309 --> 00:27:55.210
This is where it can cause problems. Let's build
00:27:55.210 --> 00:27:58.680
a wall around this so that it will not do it.
00:27:58.960 --> 00:28:02.119
Yeah, no, that makes sense. Okay, so if a cloud
00:28:02.119 --> 00:28:05.259
or platform team wanted to start experimenting
00:28:05.259 --> 00:28:07.960
with agents, where should they start without
00:28:07.960 --> 00:28:10.559
creating like a monster? If they're very new,
00:28:10.680 --> 00:28:12.279
they've used agents, maybe like they've used
00:28:12.279 --> 00:28:14.380
Cursor, they've used Claude locally, but they
00:28:14.380 --> 00:28:16.259
actually want to start using it in their IaC.
00:28:16.680 --> 00:28:18.579
They want to start using it in their PR workflows.
00:28:19.359 --> 00:28:22.849
Where should they start? Yeah. I can share how
00:28:22.849 --> 00:28:25.390
we see, for example, some of our customers start,
00:28:25.529 --> 00:28:28.029
right? There's different levels of adoption of
00:28:28.029 --> 00:28:29.990
AI in general, and that can also correlate to
00:28:29.990 --> 00:28:33.539
their willingness to adoption of it. All right.
00:28:33.599 --> 00:28:35.579
If, let's say, they come from a background where
00:28:35.579 --> 00:28:38.579
they just use it as an autocomplete or, you know,
00:28:38.640 --> 00:28:40.700
some ChatGPT interface, so they haven't done
00:28:40.700 --> 00:28:45.000
a lot of agentic, maybe safer way to adopt something
00:28:45.000 --> 00:28:48.220
more agentic would be PR reviews, right? You
00:28:48.220 --> 00:28:51.079
just enable connected to a couple of your environments,
00:28:51.220 --> 00:28:53.299
a couple of your repositories, and start getting
00:28:53.299 --> 00:28:56.599
reviews into your infrastructure's code, which
00:28:56.599 --> 00:28:59.319
you can still write yourself or use agents to
00:28:59.319 --> 00:29:01.559
help you. And that kind of builds trust. You
00:29:01.559 --> 00:29:04.440
can start to see, okay, where agents can help
00:29:04.440 --> 00:29:07.579
you, where there may be the limitations, where
00:29:07.579 --> 00:29:10.019
they need a little bit more context. And next
00:29:10.019 --> 00:29:13.279
step would be, well, now I can maybe let agents
00:29:13.279 --> 00:29:16.460
to write infrastructure as a code for me, right?
00:29:16.559 --> 00:29:19.279
Larger pieces of it, maybe larger refactorings.
00:29:19.299 --> 00:29:22.059
Maybe, okay, I need a new set of resources. Here
00:29:22.059 --> 00:29:26.599
is the template already. Agents are
great at
00:29:26.599 --> 00:29:29.740
following templates and examples. So if you need
00:29:29.740 --> 00:29:33.730
a new setup, it will do it perfectly. And after
00:29:33.730 --> 00:29:36.029
that is comfortable, I would say is automating
00:29:36.029 --> 00:29:39.089
more and more work, right? Automating compliance
00:29:39.089 --> 00:29:42.960
part, automating FinOps, right? Using agent
00:29:42.960 --> 00:29:45.980
to finding orphan resources or so. This is where
00:29:45.980 --> 00:29:50.329
it becomes more. But I think, you know, I would
00:29:50.329 --> 00:29:54.130
say it's useful to tinker with it yourself, right?
00:29:54.130 --> 00:29:57.390
Try to kind of build some, some automations yourself.
00:29:57.390 --> 00:29:59.970
And this is why we also built OpenGeni because
00:29:59.970 --> 00:30:03.630
we, I thought, you know, you can skip a few steps
00:30:03.630 --> 00:30:06.990
and start building automations yourself. But I
00:30:06.990 --> 00:30:10.240
think very quickly, if one gets very serious
00:30:10.240 --> 00:30:12.440
about building agents, they will realize that
00:30:12.440 --> 00:30:15.640
there is quite a lot of moving parts, just like
00:30:15.640 --> 00:30:19.000
in anything else, right? In engineering. So if
00:30:19.000 --> 00:30:22.140
you want to get quick results faster, there are
00:30:22.140 --> 00:30:23.920
a few open source solutions, for example, for
00:30:23.920 --> 00:30:28.490
SRE, for different things, right? And we see
00:30:28.490 --> 00:30:31.170
Cloudgeni, for example, is one type of solution,
00:30:31.269 --> 00:30:35.240
like Cloud Ops Management where you can start
00:30:35.240 --> 00:30:37.900
managing your cloud using agents already without
00:30:37.900 --> 00:30:42.079
having to invest into understanding how agents
00:30:42.079 --> 00:30:44.700
should work and all the guardrails and all the
00:30:44.700 --> 00:30:47.539
complexities. But it is still useful to understand
00:30:47.539 --> 00:30:49.579
how agents work because then you will also be
00:30:49.579 --> 00:30:52.259
able to interact with systems like Cloudgeni
00:30:52.259 --> 00:30:56.140
in a much more efficient way, even if it doesn't
00:30:56.140 --> 00:31:00.119
require you to be an expert in it. Okay, so wrapping
00:31:00.119 --> 00:31:02.460
up, what about guardrails? Where do I start with
00:31:02.460 --> 00:31:04.910
that? First guardrail, our first three guardrails.
00:31:05.029 --> 00:31:09.710
I mean, 101, you know, run it in sandbox, right?
00:31:10.029 --> 00:31:13.569
Maybe inside Docker, which I still don't consider
00:31:13.569 --> 00:31:15.970
actually a proper sandbox because it still can
00:31:15.970 --> 00:31:19.900
get out of your... So ideally, maybe somewhere
00:31:19.900 --> 00:31:21.980
in the cloud, you know, run your agent in the
00:31:21.980 --> 00:31:25.119
cloud in sandbox, give it a limited set of permissions,
00:31:25.359 --> 00:31:28.220
right? To two things you wanted to have, let's
00:31:28.220 --> 00:31:32.119
say, read access to, you know... subscription
00:31:32.119 --> 00:31:36.559
on Azure or account on AWS, read-only. And then,
00:31:36.579 --> 00:31:38.900
you know, connect maybe one infrastructure as
00:31:38.900 --> 00:31:42.900
a code repo, only previous permission of PR,
00:31:43.119 --> 00:31:47.740
creating PRs, right? So this is enough to kind
00:31:47.740 --> 00:31:50.880
of say, hey, I can't do anything now. I can ask
00:31:50.880 --> 00:31:54.460
my agent, delete this resource and it will not
00:31:54.460 --> 00:31:56.859
interpret it. Even if it will interpret it in
00:31:56.859 --> 00:32:00.500
the wrong way, it can't do anything. But most
00:32:00.500 --> 00:32:02.650
likely it will just, it will understand, okay,
00:32:02.710 --> 00:32:04.930
I'm in a sandbox, I have only read access, and
00:32:04.930 --> 00:32:08.490
I can only create PRs. It will create PR where
00:32:08.490 --> 00:32:12.910
it will remove it from Terraform instead of deleting
00:32:12.910 --> 00:32:16.849
resource outright, or it will create a script
00:32:16.849 --> 00:32:19.490
for this. And then you will be able to review
00:32:19.490 --> 00:32:22.930
it. So very simple. No matter how you prompt
00:32:22.930 --> 00:32:25.490
it, you make a mistake in the prompt, there is
00:32:25.490 --> 00:32:28.509
no chance the agent will do a mistake there.
00:32:28.769 --> 00:32:32.359
Makes sense. Anything to leave our audience with?
00:32:32.680 --> 00:32:35.980
Any last thoughts? Yeah, I think the last thing,
00:32:36.200 --> 00:32:39.200
you know, just to touch up on the human moment
00:32:39.200 --> 00:32:43.339
as well. I think people are also, you know, there
00:32:43.339 --> 00:32:45.880
is a different sentiment we see when it comes
00:32:45.880 --> 00:32:49.119
to AI in engineering. And it also changed a lot
00:32:49.119 --> 00:32:52.660
as we saw them from skepticism to, oh, wow, this
00:32:52.660 --> 00:32:55.700
actually works. And now there's like, okay, there's
00:32:55.700 --> 00:32:59.180
some concern, I think, naturally. What I see
00:32:59.180 --> 00:33:02.099
instead is that I think especially platform and
00:33:02.099 --> 00:33:04.440
DevOps people and in general, good engineers,
00:33:04.579 --> 00:33:07.539
I think we are in a very important position right
00:33:07.539 --> 00:33:10.900
now. We can decide how AI is going to be adopted
00:33:10.900 --> 00:33:13.279
at a larger scale in organizations and our teams.
00:33:13.359 --> 00:33:16.380
And instead of, let's say, ignoring it or instead
00:33:16.380 --> 00:33:18.819
of avoiding it, what we can try to think of,
00:33:18.900 --> 00:33:21.140
okay, how can we adopt it in a way that is actually
00:33:21.140 --> 00:33:23.259
useful, that is safe, that is cost efficient,
00:33:23.500 --> 00:33:26.539
that benefits us, our organization, our time.
00:33:27.259 --> 00:33:29.359
How can we actually make it useful? And I think
00:33:29.359 --> 00:33:33.200
if we kind of approach it with that
set of mindset,
00:33:33.519 --> 00:33:36.519
we can find new kind of fun with it as well.
00:33:36.599 --> 00:33:38.400
Because it's actually, you know, it is still
00:33:38.400 --> 00:33:40.180
building systems, but it's maybe a little bit
00:33:40.180 --> 00:33:43.059
different kind of systems, right? Maybe not what
00:33:43.059 --> 00:33:46.500
we're used to, but it is still, you know, a lot
00:33:46.500 --> 00:33:48.940
of thinking that requires. It's not just a mindless
00:33:48.940 --> 00:33:52.700
vibe coding that many people maybe think of when
00:33:52.700 --> 00:33:56.910
they hear AI. If you go deep enough, there is
00:33:56.910 --> 00:34:00.190
a lot of technical challenges, and they need
00:34:00.190 --> 00:34:05.470
to be solved today before we see more outages,
00:34:05.490 --> 00:34:09.650
problems, and costs and impacts on people. So
00:34:09.650 --> 00:34:12.670
I think, yeah, that's what I would leave people.
00:34:12.789 --> 00:34:16.590
Be curious. Try to figure out, okay, what can
00:34:16.590 --> 00:34:19.590
I do with this, and how can I use it, and what's
00:34:19.590 --> 00:34:22.869
required from me to enable it at its full capacity.
00:34:23.110 --> 00:34:26.260
I think that will be fun. Yeah, well said. Where
00:34:26.260 --> 00:34:29.159
can people find out more about you and Cloudgeni?
00:34:29.159 --> 00:34:33.300
So I'm on LinkedIn, Davlet Dzhakishev. It's
00:34:33.300 --> 00:34:37.239
a hard name, but maybe people can read it. It's
00:34:37.239 --> 00:34:42.570
a cloud, Cloudgeni.ai, our commercial product. OpenGeni.ai
00:34:42.570 --> 00:34:46.849
is the open source, MIT-licensed foundation
00:34:46.849 --> 00:34:50.230
of it. And yeah, I think on LinkedIn is the best.
00:34:50.530 --> 00:34:54.090
I have X and other stuff, but it's not very...
00:34:54.409 --> 00:34:58.349
big yet. Give me a follow if you want to help.
00:34:58.449 --> 00:35:02.630
It's Davlet Cloudgeni. So yeah. Awesome. I'll put links
00:35:02.630 --> 00:35:04.949
for all those in the show notes as well. Davlet,
00:35:04.989 --> 00:35:06.369
thank you so much for coming on. Really appreciate
00:35:06.369 --> 00:35:08.969
your time. Awesome. It was great chatting with
00:35:08.969 --> 00:35:12.070
you, Brian. Great conversation. Thank you. That
00:35:12.070 --> 00:35:15.269
was my conversation with Davlet from Cloudgeni.
00:35:15.590 --> 00:35:18.969
The biggest takeaway for me is that a safe agent
00:35:18.969 --> 00:35:23.190
is mostly a permissions problem, not a prompting
00:35:23.190 --> 00:35:26.679
problem. After that Terraform apply, his fix
00:35:26.679 --> 00:35:30.000
wasn't a better prompt. It was guardrails at
00:35:30.000 --> 00:35:33.400
the architecture level. So the agent can't do
00:35:33.400 --> 00:35:36.519
whatever you don't want, however you word the
00:35:36.519 --> 00:35:40.340
request. His starter kit is simple. Run the agent
00:35:40.340 --> 00:35:44.539
in a real sandbox, ideally in the cloud. In his
00:35:44.539 --> 00:35:48.409
book, Docker doesn't count. Give it read-only
00:35:48.409 --> 00:35:52.969
cloud access, one IaC repo, and... permission
00:35:52.969 --> 00:35:57.329
to open PRs. Ask it to delete a resource, and
00:35:57.329 --> 00:36:01.349
the worst it can do is open a PR. That's the
00:36:01.349 --> 00:36:05.170
valet key. His on-ramp is agentic PR reviews
00:36:05.170 --> 00:36:09.630
first, then agents writing IaC from your templates,
00:36:09.829 --> 00:36:14.510
then automation. And a good reviewer catches
00:36:14.510 --> 00:36:18.989
more than typos. His example was a private network
00:36:18.989 --> 00:36:23.039
around a key vault. Great for security, until
00:36:23.039 --> 00:36:27.360
your own apps lose access. On cost, Davlet points
00:36:27.360 --> 00:36:31.059
out that AI subscriptions are heavily subsidized
00:36:31.059 --> 00:36:35.179
and running your own agents on API tokens can
00:36:35.179 --> 00:36:39.219
cost 10 to 20 times more. He thinks FinOps for
00:36:39.219 --> 00:36:42.760
AI is going to be big, orphaned agents and all.
00:36:42.900 --> 00:36:46.380
You can't manage what you can't audit. You can
00:36:46.380 --> 00:36:50.280
find Davlet on LinkedIn and X, plus Cloudgeni,
00:36:50.500 --> 00:36:53.280
OpenGeni, and his Infrastructure Agents Guide
00:36:53.280 --> 00:36:56.099
on GitHub through the links in the show notes.
00:36:56.440 --> 00:36:59.579
Follow or subscribe to Ship It Weekly wherever
00:36:59.579 --> 00:37:03.639
you listen and find previous episodes at shipitweekly.fm.
00:37:03.639 --> 00:37:07.039
Thanks for listening and I'll see you Friday
00:37:07.039 --> 00:37:07.840
for the news.