The Detection Factory (Loop Engineering in Practice)

WEBVTT
00:00:12.400 --> 00:00:20.239
Welcome back to Detection Dispatch, the show where we treat detection engineering like the real engineering discipline that it's always deserved to be.
00:00:20.480 --> 00:00:43.920
I'm your host, Alex Hurtado, back from the desert, and excited to feature today's guest, who's been quietly building something that I feel like I haven't stopped thinking about since he described it to me as a four-station AI-assisted detectionist code pipeline running inside Stargate, the largest obviously AI data center on the planet.
00:00:44.000 --> 00:00:47.759
And he calls it Detection Factory.
00:00:48.079 --> 00:00:56.079
And it's like simple, but yet also like a little terrifying because you can't detect what you can't identify.
00:00:56.159 --> 00:01:02.560
And obviously, you can't respond at machine speed unless everything is as code.
00:01:02.799 --> 00:01:05.040
So welcome to Detection Dispatch.
00:01:05.200 --> 00:01:12.719
I am so glad to have you here after all of the technical difficulties that we just had with my new Sunny EBZ camera.
00:01:13.200 --> 00:01:21.120
Well, well, well, Alex has a new camera now, and Alex absolutely wanted to have that camera on this podcast.
00:01:21.280 --> 00:01:27.680
So I guess it has been, you know, a technical challenge to get that back on.
00:01:27.760 --> 00:01:29.680
But yeah, I think I think we are good.
00:01:30.000 --> 00:01:31.760
I think we are absolutely golden.
00:01:32.480 --> 00:01:35.200
We gotta up-level, we gotta keep up-leveling here.
00:01:35.359 --> 00:01:37.120
We gotta continuously improve.
00:01:37.280 --> 00:01:45.200
That's what this like lean manufacturing Six Sigma concept is when it comes to detection, the detection factory that we're gonna be talking about tonight.
00:01:45.519 --> 00:01:46.879
Yep, absolutely.
00:01:47.040 --> 00:01:48.959
That is what we do one by one.
00:01:49.120 --> 00:01:50.959
We upgrade, we keep upgrading.
00:01:51.359 --> 00:02:01.680
So, Tez, for those that do not know you, can you tell us a little bit about you and how you got into the space and uh and a little bit about what you're working on right now?
00:02:02.000 --> 00:02:03.439
All right, yeah, absolutely.
00:02:03.599 --> 00:02:08.319
Uh so you know, I started off in cybersecurity a long, long time back.
00:02:08.479 --> 00:02:22.879
Uh, I was like super interested in my underrad about this small project uh which was at Videos Tigger, which is the art of like modifying pixels in a video in such a way that you encode information into the video.
00:02:23.039 --> 00:02:25.520
And I kind of created a new algorithm for it.
00:02:25.599 --> 00:02:29.280
I wrote a paper for it, and that was my you know stuff.
00:02:29.599 --> 00:02:30.080
Did you patent?
00:02:30.159 --> 00:02:30.960
Did you patent it?
00:02:31.039 --> 00:02:31.919
I gotta ask.
00:02:32.240 --> 00:02:33.520
I did not patent it.
00:02:33.680 --> 00:02:36.560
Oh yeah, I did write a paper about it.
00:02:36.639 --> 00:02:41.199
So but I will patent the detection factory though.
00:02:41.360 --> 00:02:42.319
Uh so uh-huh.
00:02:42.560 --> 00:02:44.719
You should you should copyright that term, detection factory.
00:02:45.039 --> 00:02:46.080
Absolutely, absolutely.
00:02:46.240 --> 00:02:50.960
Well, I have I have actually filed a patent uh for the detection factory, by the way.
00:02:51.199 --> 00:02:53.120
I think I forgot to mention it to you.
00:02:53.199 --> 00:02:55.759
I think this is uh I am not kidding.
00:02:56.240 --> 00:02:59.759
Oh shit, amazing, amazing.
00:02:59.919 --> 00:03:06.080
I will give you your copyrights uh titles once that gets approved, and I'll link that in the show notes.
00:03:06.400 --> 00:03:07.520
Absolutely, absolutely.
00:03:07.759 --> 00:03:08.800
Can do, can do.
00:03:08.960 --> 00:03:14.879
Uh all right, so um, you know, after that I published a bunch of papers there.
00:03:15.120 --> 00:03:19.360
Uh I did my master's from Carnegie Millen, cybersecurity.
00:03:19.599 --> 00:03:21.439
Very nice, nice plug.
00:03:22.080 --> 00:03:22.639
Absolutely.
00:03:22.719 --> 00:03:24.479
I mean, I have to, I spent a lot of money.
00:03:24.560 --> 00:03:24.719
Okay.
00:03:24.960 --> 00:03:29.280
Right, but yeah, I mean it is uh the the US, yeah.
00:03:29.360 --> 00:03:32.159
It's very expensive to go to go to university here.
00:03:32.719 --> 00:03:36.080
I will absolutely talk about it every single chance that I get.
00:03:36.240 --> 00:03:39.280
So please be ready to hear about it.
00:03:39.520 --> 00:03:42.240
Uh multiple times in this podcast.
00:03:42.879 --> 00:03:48.719
Almost as much time, almost as much times as patent holders like to tell you that they have patents.
00:03:49.680 --> 00:03:52.240
Oh yeah, yeah, probably, probably, yeah.
00:03:52.560 --> 00:03:53.520
True that.
00:03:54.000 --> 00:04:10.560
Um, yeah, and after that, I worked at Salesforce for a bit, and then uh N-phase energy was where I you know got my first uh cybersecurity, I guess, exposure to like everything, everything because I was the only uh engineer, security engineer in the US time zone.
00:04:10.879 --> 00:04:17.199
So I had you know broad ownership of like incident response, uh PKI over there.
00:04:17.519 --> 00:04:23.519
Uh I got to like rebuild some parts of the PKI, I got to implement you know different things over there.
00:04:23.600 --> 00:04:32.000
So I think that was my first uh ownership of like you know uh organizational security, this thing.
00:04:32.240 --> 00:04:37.279
As an IC, and yeah, I'll I'll tell you that experience was really amazing.
00:04:37.439 --> 00:04:53.680
So you it seems like you're like a root CA, a certificate authority, uh like custodian for critical energy, infra for Salesforce, for for and then now at a startup, you're you're uh you you moved into the startup world now.
00:04:54.000 --> 00:04:54.399
Yes.
00:04:54.560 --> 00:05:06.160
Umfest was also you know on the border of what you would say a startup to enterprise kind of it is a public company, but uh we still operated mostly like a startup, I would say.
00:05:06.319 --> 00:05:11.519
Uh not in terms of the maturity, but in terms of how things were managed.
00:05:11.680 --> 00:05:14.879
Like you were the owner of you know a lot of things there.
00:05:15.279 --> 00:05:19.519
So uh I guess I kind of carried that forward into Crusoe.
00:05:19.839 --> 00:05:32.800
Crusoe is building you know some of the biggest data centers, like you said, in Abilene, uh Texas, Wyoming, this, that, uh, you know, OpenAI, Oracle, like all of these big names that you hear.
00:05:32.959 --> 00:05:37.199
Uh Crusoe probably has a contact with them for building a data center.
00:05:37.360 --> 00:05:46.720
So this is you're you I mean you're really you're becoming kind of like the single point of coverage for critical infrastructure.
00:05:46.800 --> 00:05:48.959
Like that's a story in an in and of its own.
00:05:49.120 --> 00:05:59.360
And I I think that sets us up very nicely for how you're bringing this like identity PKI lens into detection engineering and response.
00:05:59.759 --> 00:06:01.279
Absolutely, absolutely.
00:06:01.519 --> 00:06:11.199
Uh so I think right now, uh, like I work on you know this cutting edge of AI and I see attacks coming at machine speed.
00:06:11.360 --> 00:06:18.399
I see uh, you know, models that are already getting so good that we are almost there.
00:06:18.480 --> 00:06:23.519
Like it is, you know, you don't really need a human to like orchestrate these attacks.
00:06:23.759 --> 00:06:30.480
And the I guess the thing about you know, there is a fundamental difference between like attackers and defenders, right?
00:06:30.720 --> 00:06:36.079
Attackers can just give a separate laptop to any agent and be like, go to town.
00:06:36.319 --> 00:06:38.000
A defender can't really do that.
00:06:38.160 --> 00:06:44.639
I can't give my laptop to any agent and be like, hey man, you know, set up defenses for me.
00:06:44.879 --> 00:06:46.639
That's just not an option.
00:06:46.800 --> 00:06:52.480
Like, no, no, not without a lot of permissions, giving it permission.
00:06:52.720 --> 00:06:54.560
Yeah, yeah, that's true.
00:06:56.079 --> 00:06:58.800
That's a very unfair disadvantage there.
00:06:59.279 --> 00:07:11.279
Uh and I I I will say though, machines, this machine speed, it's is feels a little backwards to me because I mean, haven't we always been doing things with machines?
00:07:11.439 --> 00:07:15.199
Like machine speed doesn't seem like it's fast enough.
00:07:15.439 --> 00:07:25.839
Like I feel like it's it's in reality, AI uh the things that you can do with AI as your enabler is so much faster than the the amount of time machine speed takes.
00:07:26.240 --> 00:07:28.560
Yeah, yeah, I do agree with that.
00:07:28.879 --> 00:07:33.360
So, you know, it is it is a new frontier.
00:07:33.519 --> 00:07:45.600
I mean, even uh even when you go back, right, it is it was not so difficult as such if you look at like you know, hey man, there is this vulnerability in this particular package, you know, upgrade that package.
00:07:45.759 --> 00:07:48.079
It's it's pretty much as simple as that.
00:07:48.240 --> 00:07:57.759
But you know, I know it, uh, how many days, weeks, months, years it takes for some vulnerabilities to get patched.
00:07:57.920 --> 00:08:11.600
So I think we we always knew that hey, this threat exists, that you know, we should be patching stuff more often, we should be detecting, you know, in a better way, we should be doing XYZ things.
00:08:11.680 --> 00:08:23.759
But I think now it is at a point where if you don't act on it today, if you don't do something about it today, yeah, I mean it's going to be a tough world going ahead.
00:08:24.160 --> 00:08:24.800
So yeah.
00:08:25.120 --> 00:08:26.800
It it I you're absolutely right.
00:08:26.959 --> 00:08:50.960
Everyone is talking about keeping up, like patching the patching at the speed of AI, but no one is talking enough about building detections at the speed of AI as well, or like or and hence the concept of this factory, which is like what you seem to be working on right now, this detection factory, I think, really undersells the the power of it.
00:08:51.120 --> 00:09:00.559
Uh so I love how you described your pipeline to me before as the factory with like these four stations.
00:09:00.720 --> 00:09:03.200
Can you walk us through what they are?
00:09:03.360 --> 00:09:05.759
Like, what does each station actually do?
00:09:05.919 --> 00:09:07.679
Tell us a little bit about that.
00:09:08.559 --> 00:09:09.279
Absolutely.
00:09:09.440 --> 00:09:10.960
Uh, can do, can do.
00:09:11.120 --> 00:09:23.039
So, first of all, I want to say that this is all a part of like loop engineering, which I feel is you know a frontier that all engineers, most engineers, if not all, should be working on.
00:09:23.200 --> 00:09:32.480
Like, you need to be creating pathways for agents to uh in a verifiable way, in a trustworthy way, go about doing their job.
00:09:32.639 --> 00:09:35.679
Because, like I said, you can't hand over your laptop to them.
00:09:35.919 --> 00:09:43.120
You have to have these pathways in place so that they can work at machine speed to you know come up with things.
00:09:43.360 --> 00:09:52.159
So uh I would like to you know give you a brief about like how I envision uh detection security, detection response program to be, right?
00:09:52.480 --> 00:10:01.600
Uh firstly, everything has to be as code, your detections have to be as code, your workflows have to be as code, your infrastructure has to be as code.
00:10:01.759 --> 00:10:02.320
Everything.
00:10:02.559 --> 00:10:07.279
Everything has to be managed through code so that an agent at least has access to it.
00:10:07.519 --> 00:10:28.720
And when a uh zero-day vulnerability does get discovered, uh, before somebody burns it, you have to, you know, your AI agent should tell you that, hey, there is this detection that I came up with for this thing that was noticed just minutes back, or like, you know, that I discovered, and there is this particular detection that I have written for it.
00:10:28.879 --> 00:10:37.120
That detection should go into production in a matter of minutes with a human approval, uh, you know, a human read-through.
00:10:37.360 --> 00:10:41.519
Then it should uh you know come up with a workflow as code.
00:10:41.600 --> 00:10:44.720
That here is this workflow that I have come up with.
00:10:44.879 --> 00:10:50.000
Uh approved and I suggest changes according to the context and stuff.
00:10:50.159 --> 00:10:59.519
Um, once you give it that, it puts in the workflow in place, which is like a you know temporary blocker against like any attacks which are coming at machine speed.
00:10:59.600 --> 00:11:14.159
But it basically means that within minutes, your system our systems are at least detecting things, and there are at least there are at least workflows in place to mitigate any uh ill effects of this zero date that I was just discovered.
00:11:14.399 --> 00:11:22.480
And then after that, you can go on to hey, I'm going to patch this super quick, which I mean, you know, this is the real world.
00:11:23.679 --> 00:11:25.519
Uh that does take some time.
00:11:25.600 --> 00:11:30.159
So, you know, coming back to I guess the detection factory, how does that work?
00:11:30.399 --> 00:11:36.720
So uh there are four different stations that are there in this detection factory.
00:11:36.879 --> 00:11:43.600
The first one is a tuner, which is basically something that tunes it, tunes a rule, backtests it, stuff like that.
00:11:43.840 --> 00:11:55.440
Um is uh Herald, which is another model, which kind of reviews and uh suggests changes according to some specifications that I've given it.
00:11:55.679 --> 00:11:59.600
Uh, the third one is a Warden station.
00:11:59.840 --> 00:12:04.159
So uh, you know, I kind of imagine all of this like a car pipeline.
00:12:04.320 --> 00:12:20.320
Like you have the car chassis being made in station one, then you have the car's uh seats being put in in station two, you have the car being painted in station three, and station four basically says, hey, this car is ready to be shipped out or like sold.
00:12:20.480 --> 00:12:21.279
I don't know.
00:12:21.600 --> 00:12:29.279
Uh so in a similar manner, you have Warden, which you know has like some guardrails that it looks for.
00:12:29.440 --> 00:12:40.879
Uh, and then number four is your porter, which uh the last station actually creates a MR for it, which then you can review or like you know, approve, deny, stuff like that.
00:12:40.960 --> 00:12:48.159
Yeah, it actually is is constantly piping in new content to generate and new detection content to generate.
00:12:48.320 --> 00:12:51.360
And what what would you say is like the the what triggers it?
00:12:51.519 --> 00:12:57.360
What is the uh how does it know when to kick off that assembly line, would you say?
00:12:57.919 --> 00:13:04.320
Okay, so uh there are a bunch of like ideas that you know uh most of us keep having, right?
00:13:04.399 --> 00:13:11.679
Like we have ideas that hey, this particular bucket needs to be monitored, or this particular like thing needs to be monitored, this particular whatever.
00:13:11.840 --> 00:13:20.000
You just note them down in like a database or something, uh any table or something, uh wherever you are running this particular factory.
00:13:20.240 --> 00:13:34.240
And for me, I uh I we could go through the you know exact technological components later on, but uh I keep it in a database, and that database can be updated by an AI agent or a human being.
00:13:34.399 --> 00:13:44.480
Right now, it is a human being who is typing, hey, I need this particular detection, but I do envision a future where you know this is a own backlog.
00:13:44.879 --> 00:13:45.919
Correct, correct, correct.
00:13:46.320 --> 00:13:46.960
Yeah.
00:13:47.679 --> 00:13:58.240
Someday I do hope that you know an AI agent would create an idea as well by itself, and then that gets propagated through this detection factory.
00:13:58.639 --> 00:14:00.879
No, I think that's you've hit it key.
00:14:01.120 --> 00:14:22.879
It is so important to uh keep this going through like these different phases of the detection building process, and obviously feed it all everything that it needs to be doing across each phase, make it so purpose-built for a world where it should be very cleanly read from an AI agent.
00:14:23.120 --> 00:14:30.320
And it it they because they they are doing it's obviously at a much larger scale, and they can do it in a repetitive loop.
00:14:30.639 --> 00:14:31.039
Exactly.
00:14:31.200 --> 00:14:31.440
Yeah.
00:14:31.600 --> 00:14:31.919
Yes.
00:14:32.240 --> 00:14:44.720
But what's not happening, I think today, uh, and I see this in threat hunting all the time and in detection engineers, is this carrying of knowledge across these different phase phases.
00:14:44.879 --> 00:14:54.159
All of that is kind of done on a procedure basis, lives in the practitioners' minds and heads, and it's hard to be replicated, right?
00:14:54.399 --> 00:15:06.559
For example, you inherit, I've seen this time and time again, where I inherit this detection library that I have to maintain from people that were here that predate me 10 years ago, even.
00:15:06.799 --> 00:15:09.440
And I have to continue that, I have to continue it going.
00:15:09.600 --> 00:15:11.679
And maybe those rules were built like five years ago.
00:15:11.840 --> 00:15:12.879
Do they still apply?
00:15:13.120 --> 00:15:14.159
Nobody knows.
00:15:14.320 --> 00:15:41.519
Um, but it if you can if you like have a means to document this knowledge, like this MD file per se, for every reason why a rule was created, and uh all of the different approaches that have been tested when you were even testing it, this information travels with you so that when your team looks different next year, you know, you have all of you know exactly the history, the documented and it creates this loop.
00:15:41.600 --> 00:15:55.200
This I love how you call it loop engineering because that's that's what we should be optimizing for and building around is how can you continue this context and this knowledge from phase to phase and loop?
00:15:55.360 --> 00:15:58.080
Um, where where does that where should that live?
00:15:58.240 --> 00:16:04.320
Like how do you integrate this feedback into all of your stations and all of your production alerts?
00:16:04.480 --> 00:16:05.759
How how does that even work?
00:16:06.159 --> 00:16:12.320
Charlotte, uh so you can use any workflow, you know, you can even use your own source for that matter.
00:16:12.559 --> 00:16:16.320
Uh I use this open source tool called TraceCat.
00:16:16.960 --> 00:16:23.279
And um essentially there are these stations are essentially different agents that are running.
00:16:23.679 --> 00:16:26.559
And these rules are Python rules.
00:16:26.960 --> 00:16:32.399
And there is a factory log that I attach to every single rule at the very beginning.
00:16:32.639 --> 00:16:40.000
So whenever the uh idea comes into the tuner, tuner basically is your station one, which you know does some backtesting.
00:16:40.080 --> 00:16:45.039
It it actually writes the rule in the first place, it creates this acting log at the beginning.
00:16:45.200 --> 00:17:02.320
It uh it has some very strict backtesting um like metrics that I've given it that you have to have at least one hit in like the last three months or six months something so that you know that you know this will actually be deploying a role that came back with thousands of results, right?
00:17:02.480 --> 00:17:02.639
No.
00:17:03.200 --> 00:17:04.079
Exactly, exactly.
00:17:04.240 --> 00:17:11.039
If you do see that there are thousands of alerts, then I want uh tuning as well to go into the same rule.
00:17:11.200 --> 00:17:16.480
Yeah, I want to say that hey, this particular service account gives me you know thousands of alerts.
00:17:16.720 --> 00:17:20.400
I want to uh allow listed at the get-go.
00:17:20.640 --> 00:17:32.720
Yeah, so uh all of that is like pruned down to you know, uh, it should not be triggering more than like five times a week, but it should at least have one detection in like the last six months or something.
00:17:32.880 --> 00:17:33.039
Yeah.
00:17:33.279 --> 00:17:37.440
And then there is also the concept of you know uh interchangeability.
00:17:37.519 --> 00:17:44.079
Like, for example, if you're detecting for like one particular bucket, but there is no log for that particular sensitive bucket.
00:17:44.240 --> 00:17:48.880
But if you have logs coming for some other bucket, then you already know the format of the log.
00:17:48.960 --> 00:17:55.119
You can just change those names out and you can still say that hey, this rule will work.
00:17:55.359 --> 00:18:00.000
So you know, uh, you can do those types of backtests for yeah.
00:18:00.160 --> 00:18:01.039
Oh, absolutely.
00:18:01.200 --> 00:18:04.960
You could do like 15-day retro hunts, like there's this syntax.
00:18:05.759 --> 00:18:15.279
Not only does the syntax is it a syntax and schema validation, but it's like also how uh how many uh how many alerts per day would this generate?
00:18:15.440 --> 00:18:22.640
Like how and out of those, out of that, how many come with actually unique endpoints associated with that as well?
00:18:22.880 --> 00:18:30.640
And um, and that's that's that's really good information to know as you're getting ready to like have this pre-deployment checklist before you push it off.
00:18:30.960 --> 00:18:31.359
Yep.
00:18:31.519 --> 00:18:33.119
And all of this is like written.
00:18:33.200 --> 00:18:39.599
It is in a standardized format, it is written in that factory log before you know it even gets moved to the next station.
00:18:39.920 --> 00:18:42.400
So like when you go through a rule, right?
00:18:42.640 --> 00:18:49.359
You have uh this entire log of like, hey, this this particular rule was generated on this date.
00:18:49.440 --> 00:18:52.880
It passed through this station, this station did XYZ things on it.
00:18:53.039 --> 00:18:59.119
You know, uh the station uh listed this, tuned this, uh did these back tests.
00:18:59.279 --> 00:19:00.880
It I gotta know, I gotta know.
00:19:00.960 --> 00:19:04.400
Before it reaches that next station, what is what is your criteria?
00:19:04.559 --> 00:19:11.039
Like how what is the number of alerts per day it should be, or volume per per day?
00:19:11.200 --> 00:19:15.119
Like what is your good uh sniff test before you move it on?
00:19:15.839 --> 00:19:22.000
Uh I would say anything more than five alerts a week is probably okay.
00:19:22.240 --> 00:19:27.519
I'm I am so missed what other people think about this too, because I it's different for a lot of people.
00:19:27.920 --> 00:19:28.400
It is different.
00:19:29.039 --> 00:19:32.000
I've had people say, I I don't want to be notified of this.
00:19:32.079 --> 00:19:35.440
Uh oh, I only want to be notified like what, like once, twice.
00:19:35.680 --> 00:19:40.960
And then that's like, okay, well, if it's too precise, you might be catching, you might never catch anything.
00:19:41.119 --> 00:19:47.359
Like, uh I I agree, but like, you know, what happens is if you keep seeing a rule, right?
00:19:47.440 --> 00:19:53.200
If you if you keep seeing an alert that comes like more than five times a week, you get used to it.
00:19:53.279 --> 00:19:58.160
You get so used to that alert title and that everything slip through the cracks.
00:19:58.640 --> 00:20:00.400
So it will slip through the cracks.
00:20:00.559 --> 00:20:08.880
It's like you know, whenever a rule triggers, whenever an alert comes, you actually want someone to are you going to press a button?
00:20:09.200 --> 00:20:12.559
Yeah, unless unless they're gonna get lost in the sauce.
00:20:12.799 --> 00:20:13.839
Yeah, that's true.
00:20:14.000 --> 00:20:14.319
Okay.
00:20:14.480 --> 00:20:15.839
Well, that's that's good to know.
00:20:15.920 --> 00:20:18.480
Um, five alerts per day before the next station.
00:20:18.559 --> 00:20:22.000
Then what then what's that that's the next what's the next station after that?
00:20:22.480 --> 00:20:26.799
Uh the next station is basically any station with a different model.
00:20:27.119 --> 00:20:39.759
And it has a bunch of you know uh good practices that I've given it that it has to like station one has to have done these bunch of things that you know that make an alert a good alert.
00:20:39.839 --> 00:20:46.319
Like those standards can, I guess, vary from org to org, like whatever your org has for your detections.
00:20:46.640 --> 00:21:02.480
You can include those standards in there that hey, I want my alert to look like this, I want the function to look like this, I want the you know, I want these particular functions to be a separate uh function instead of like in the main block, or like whatever, whatever you want.
00:21:02.880 --> 00:21:06.559
And this kind of you know also takes care of most hallucinations.
00:21:06.720 --> 00:21:14.799
Like I have I don't think I've seen a single hallucination make it out of the detection factory in the last I don't know, months.
00:21:14.880 --> 00:21:20.400
Like after I added in this station because Claude is hallucinating almost every day for me.
00:21:20.720 --> 00:21:21.200
Really?
00:21:21.440 --> 00:21:26.720
See, Claude hallucinates, but then you have you know codex that like catches those hallucinations.
00:21:26.960 --> 00:21:27.839
Oh, look at that.
00:21:28.480 --> 00:21:31.599
That's why you have you know codex, please sponsor this podcast.
00:21:32.000 --> 00:21:33.440
Yes, clearly.
00:21:34.400 --> 00:21:39.359
So I should I should uh like to change to codex then because we use Claude here, I or at least I do.
00:21:39.680 --> 00:21:48.960
Whichever you say, I am ready to say that that is the best model, and that you should use only that model in the detection factory for its optimal working, you know.
00:21:49.440 --> 00:21:50.400
Okay, all right.
00:21:50.559 --> 00:21:52.799
And so no hallucinations on codex.
00:21:53.119 --> 00:21:55.279
Um and no, that's why you have a different model.
00:21:55.519 --> 00:21:57.359
I'm not saying no hallucinations on codex.
00:21:57.519 --> 00:21:59.759
That's why if station one is codex, station two has to.
00:22:00.079 --> 00:23:12.960
be Claude if station one is Claude Station two has to be codex like they catch each other's hallucinations like they catch each other's okay I like that yeah look at that different model different model different different different uh different ai providers at different stations yep yep yep hi okay uh codex is lower lower token consumption so maybe you're on to something there true true true you can also use an open source model I think uh you know deep seek v3 is uh v4 is pretty good provides that on managed inference another plug I am oh my god but yeah you can you can use any model doesn't matter as long as it is decently capable as long as it is a different model as long as it's not crock I mean I I don't want to go there okay so you do your your bag testing do you you uh I didn't hear you say similarity matching do you do like how does it do a reference of does this new detection already kind of exist to a degree or is it worthwhile modifying another existing detection to just add this additional filter?
00:23:13.519 --> 00:23:15.839
That is carried out in station three by Warden.
00:23:16.160 --> 00:23:58.640
Warden is basically you know it looks at the existing code repo it has access to you know the code repo it looks at what detections are already there it says you know hey this already exists or like you know make uh some changes to this instead of writing your own new detection altogether there are also like linting uh you know there is this uh I guess testing like you have these test events and then you have the Python file so you can or like the YAML file you can like run those tests in there apply like see the baseline the baseline from that from normal operation before choosing what what are what is the outlier what what is anomalous.
00:23:59.039 --> 00:26:51.920
Correct exactly so like Warden is like you know a guard at the end which like makes sure that uh your nails are trimmed which makes sure your nature yeah a little code rabbit situation yeah exact you could say that yeah yeah yeah yeah yeah another another I know Jesus why am I mentioning literally I made it a thing to not mention any products on this podcast and we just named four but that's okay because they're all AI models and and GitHub plugins but I I do make it a thing to not plug uh to not plug for example an AI like some of these new AI saw I just came back from spending seven days in the desert I'm I'm just fatigued from how much sim not sim people are out there now selling and so I I will not speak on on um any products themselves I'll only speak to approaches that some take or some don't take yes yeah yeah the whole reason that I watch your podcast is because you don't excessively plug in some particulars you know you don't have like these exterior motives that oh I'm selling your product so yes also like you know I think uh I see I should mention this uh it is probably you know this is a full circle moment for me why I've been watching your podcast for I don't know since when it started and I've been like seeing all of these experts come in talk like you know I had the podcast on in the background stuff like that oh my god that's so cool yeah I'm on the loop engineering it's so on brand I love it but I appreciate that I appreciate that I uh long time long time uh planning uh podcast planning in the running I never even knew that I I would love doing this so much and yeah folks like you is what keep keep me very motivated to keep to keep doing that because sometimes you know you just want to be like you know what how about like fuck everything and how about I just go move to to Germany and start a little bakery shop like well a couple of startups make it big and you happen to make it they do oh oh for sure they do I mean your dream could be a reality sooner than you think so yeah fingers crossed fingers no I I I'm I'm very certain I was very so I I took some time off between where I was running the podcast before um and then I was not I was not allowed to bring the podcast with me and as my own independent podcast um and therefore hence Alex's version here we are um but I I I I was very selective on where what I wanted my next thing to be uh and I took some time off to get married and then uh and yeah like and especially was just so there's just so many, so many resolution opportunities out there.
00:26:52.160 --> 00:26:52.480
Thank you.
00:26:52.640 --> 00:26:53.680
Thank you so much.
00:26:53.920 --> 00:27:21.680
Um and yeah but before kind of like a marriage like you you gotta really you gotta really think through about what you want your where where you want to spend like what eight hours of your day every single day like you really got to love what you're doing and it's got the work's got to be motivating and and so I yeah I I'm I'm really happy where I landed awesome and it sounds like you're also very happy building this factory over at Crusoe.
00:27:21.920 --> 00:27:56.720
Absolutely yeah I I happen to you know yeah I I don't know I mean I just thank my stars that I happen to come in a field which I absolutely love but I you're so lucky too because not everyone is so open to using let letting you use AI across critical processes and I consider detection engineering um insanely critical right and there's a lot of people that maybe are not our age not to not to be mean to people that are older but some people are just they will just never get on the AI bandwagon and they're just like no against it against it against it.
00:27:56.799 --> 00:28:16.480
Actually my partner he's one of them like he hates AI and I'm like you can't think this way but and so um it's the fact that this organization is letting you bolt on AI to this literally factory process is amazing because you're just like pushing the bounds of what it can do and you're seeing success.
00:28:16.720 --> 00:28:20.960
And a lot of people are trying to bolt on LLMs onto detection engineering right now.
00:28:21.039 --> 00:28:23.359
So you've you've done the work you've seen how it works.
00:28:23.519 --> 00:28:34.160
Where would you say AI genuinely is helpful and where has it burned you okay so AI is a necessity.
00:28:34.319 --> 00:28:41.359
I would not even you know I like we are past the point where you know oh do you like AI do you use AI do not use AI?
00:28:41.519 --> 00:30:36.799
AI is a necessity like you cannot live without it you will get fall behind fall behind get uh it not have a job I mean there is uh there's a lot of more other things that could happen to your company forget you as an individual yeah uh you know people are not using AI uh in you know a faster way because I think you know this whole this whole thing uh I guess I'm taking a tangent but oh there will be job losses and this and that I mean probably but then you know the economy always finds a way to bounce back so uh there will always be things that we keep doing because you know I went on a hike uh just a couple of weeks back and uh it was a backpacking trip across Yosemite four days I went there I stayed there in like a you know random camp somewhere in the middle of nowhere I ate food$20 and it was it was bad food but like I was super hungry so hey in a backpacking trip somebody's giving you food absolutely worth it but you know uh the point of telling all of this is uh what was I doing like you know 400 years ago if you try explaining this activity to someone that hey just walk in a forest with no you know network no connectivity to human civilization and you pay someone to you know stay in a a pretty shitty uh camp and you know uh sit by a campfire and you pay someone for giving you some random food and then walk for like 40 miles or something in four days and you I mean it is uh people would call you a fool and yet this is something that you know a backpacking trip is something that is like it's just such a nice experience.
00:30:37.119 --> 00:30:45.920
It keeps you so grounded I I mean I can't imagine how anyone would go see Yosemite and then be like yeah let's put a data center here.
00:30:46.799 --> 00:31:31.359
Yeah that's I guess there are other places to put a data center in I hope they don't touch Yosemite but you know my point being that uh the economy finds a way to uh you know create jobs to create uh you know more things that people want that's true that's true they you'll just you'll just be doing something else something better just doing something else you know yeah so I I mean that is a moot point for me that hey we there will be job losses this that I mean yeah probably I mean yes in you know some sectors yes but the economy will find a way so anyways uh I guess coming back to AI is a necessity and your question being are were you burnt where is it most impactful in your pipeline?
00:31:31.519 --> 00:31:42.160
Because it clearly it seems like you're doing a lot like advanced things all the way to the very very like almost end of the spectrum where you're using it agentically throughout all of it.
00:31:42.480 --> 00:31:42.960
Yep.
00:31:43.200 --> 00:31:43.599
Yeah.
00:31:43.839 --> 00:32:34.960
And I mean uh the next step is going to be you know workflows as code uh already some um vendors support that some don't which is you know I mean I just want to like make this a public announcement that if you do not support workflows as code you are going to be left behind no one will buy your product i it is not an if but a when you have workflows as code okay can you decode that because it it means like the ability for me to tell to tell the product what to do uh pretty much so workflows as code is basically let's say you know let's take a pretty basic example that if you say that if this person does XYZ things then I want you to do ABC then I want you to take ABC actions.
00:32:36.240 --> 00:32:56.720
If that thing can be encoded in right now you can do it through GUI for like you know a bunch of like sour platforms with this that uh the day you can bring it in code and the day you can connect your AI agent to it you can create a similar workflow factory for your workflows as well.
00:32:57.039 --> 00:33:16.640
That your AI agent can write up a workflow for you as code you just you know probably if it requires like a secret if it requires like some kind of a you know service account created for that particular workflow you create those things but other than that everything comes to your return as code and then you just you know continue from there.
00:33:16.880 --> 00:33:32.160
So I think that is the next step for uh you know my thing where I guess that is how AI will be super helpful in mitigating against these attacks that will be coming at machine speed.
00:33:32.480 --> 00:33:37.119
Uh and I guess your other question was like the burn.
00:33:37.279 --> 00:33:55.279
When when is it not where does it not helpful where do you not trust it yet where do you constantly have to go back and babysit it because it's just not quite getting it yeah uh I will I'd say I'll give an example for this right uh so I it's the easiest to understand to an example.
00:33:55.440 --> 00:37:13.199
So for example uh this detection factor itself right it created a detection for this big query uh data set that we have that if somebody does something you send me an alert for a particular thing so amazing uh you know detection wrote it perfectly went through all stations uh done well uh it did not take into account that there are two different log formats like there are two different versions for that particular event so uh Google has a v1 and a v2 and this uh entire thing was written for v2 and then there is also a v1 that shows up once in a while but then the agent was just you know backtesting for v2 types of uh events so uh it wrote its own tests and it said that hey if I see even a single of this event the back test thing gets a chick tick mark we move to the next agent and essentially what it did was it checked the backtest it saw a v2 event log it said yes this uh code works this detection works shipped it and shipped it as in uh you know uh it created a MR and then basically like you have to have that visibility into I guess the understanding is not yet there that hey there are more than just one there are you know other things to consider so I guess that is one thing where AI like there are still you know loopholes that it just completely misses it just completely ignores parts of the problem I would say but other than that I think it is in a decent place already like with you know standardization with it's pathways defined pathways I think it gets to a goal pretty well at the end of the day what you want is a verifiable a trustworthy detection so I think uh you know uh 95% 98% of the times it is there that last two percent is what you are there to catch so that's why you know human in the loop is important at the very end of that so yeah that is that is so key uh I it's in I got a little tripped up so it it used v2 was that not the latest version was it like was it supposed to use v3 uh no v2 is the latest version and then v1 uh it also sends those logs for certain types of things which deprecated log format but they still have to keep sending it so uh yeah it's just uh quirk I guess ah interesting in and not any way to yeah I how did you tell your factory to correct that for the future and incorporate like this loop engineering learning uh I basically asked it to you know write this in its memory that you have to check for all types of logs into its uh pre-launch checklist in its uh pre-condition checklist that uh it wrote a bunch of more tests I guess and then an overlap chat yeah okay okay you just have to make sure that you know all bases are covered your quality gates need to be on par.
00:37:13.599 --> 00:37:45.599
Yep yeah okay well uh I I'm having a hard time with uh Claudia so I'm learning a new information model I'll share mine where AI burns um and everyone's got their own information model like the way whatever sim you're using insert here I won't mention any products but what they're going to normalize to a certain standard right that's just how it goes and um if it's OCSF fantastic which it is the case that is the case here at Nebuloc.
00:37:46.159 --> 00:38:45.920
But and I'm trying to like make basically learn it uh in a way with having been used to another uh another schema uh that I use for two almost three years and so it's a little bit of a learning curve and I thought that I'm like oh this is gonna be no no problem at all I'm gonna use Claude to just like help me churn like burn through uh not only tokens but also just like lower the learning curve for me to get used to it and get used to it and um it's uh I it's it's not as effective as I uh as I thought it would be like there's a lot of things that's missing here and there and before I we have some we run a very similar detection factory here at Nebulog um and and as I'm pushing these new detections into our into our library uh it's going through these quality gates and it's catching a lot of things that I'm doing wrong.
00:38:46.159 --> 00:38:50.400
And I'm like wait what Claude like I'm literally telling you to reference this information model.
00:38:50.480 --> 00:39:29.679
Like why are you embarrassing me and showing all these little things it's not getting right like endpoint underscore ID it's actually a a period dot ID not not an underscore and I'm like why did it like you should know this like you should you should absolutely not let me push anything through and now of course it's showing up in my PRs like all of these little mistakes I'm I'm doing and I'm committing and I'm like damn I then that means I really got to really kind of sit sit behind Claude and just really micromanage and um yeah because it's it's just not helping me out right now learning a good information model.
00:39:30.000 --> 00:39:38.639
Um so that's that's my poor experience but that's also I I I think yeah that also takes time to learn.
00:39:39.039 --> 00:39:44.639
Price edition two I would say you know I had the exact same issue uh price and two with a different model.
00:39:44.800 --> 00:40:01.440
I think it will solve uh at least a bunch of your issues okay a different model I'm using I'm using sonnet I'm using sonnet okay uh but but yeah I it's it's pretty it's pretty it's pretty good for literally everything else.
00:40:01.760 --> 00:40:05.199
Like I've been doing also some testing on I'm building a new framework.
00:40:05.519 --> 00:40:26.239
Uh it's my first time I've I announced that uh I'm building a new framework on what when it's time to graduate a det uh a hunt uh a previous threat hunt your team has built and which ones should be promoted into ongoing schedule detections and what gates it should go through if you will.
00:40:26.880 --> 00:40:34.239
And so I'm looking at all of our previous hunts and kind of what they have in common and these kind of quality things it should pass.
00:40:34.559 --> 00:41:08.320
I'm also looking at false positives as well but um that that it's pretty it's very good at analyzing large like uh all of I pointed it to our detection library or our uh detection factory or hunt factory and it's looking at all of this amazingly well documented well journaled hunts and why they've run these hunts in the past and what results they got and it's doing a fantastic job at suggesting like what these conditions are and what quality gates should be and what and thus me like what framework I should use.
00:41:08.880 --> 00:41:11.199
So I will say it's it is very good at that.
00:41:11.360 --> 00:41:14.800
So that was like my bad example good example.
00:41:15.199 --> 00:41:49.280
Yeah yeah nice nice nice yeah so you talked about human review so where how how does that look like to you um where the human in the loop is that like where are you reviewing if it's the hand holding part uh where am I reviewing if okay so this entire thing right it sits at these are like workflows of its own that are running in my SOAR platform and uh these agents are like interesting you're doing all of this in SOAR.
00:41:49.840 --> 00:41:51.519
You can do it anywhere you want.
00:41:52.719 --> 00:42:20.800
All that you need is like a you know hook that kind of wakes up these agents every like let's say five minutes or something asks them to check a table or a database of some sort and the database should have um like detections you know on there like uh if I am station one I go to the database I see what all ideas are in queue for station one I just pick one of them I put a lock on that and I start working on it.
00:42:21.039 --> 00:42:54.719
Once I'm done I put it back in a database I update the pointer to hey now this needs to be put up by station two and that's it I go and sit back and then similarly station two gets called on by its own hooks and you know all of these uh stations are kind of connected to my um you know Slack on like your teams or whatever and then you're when they finish a task they reply on the thread of that particular detection so every idea gets its own thread inside a channel.
00:42:55.039 --> 00:44:03.679
Interesting in that particular thread uh it will tell you that hey this is what I did uh and if you have anything you know any comments that you leave on it uh it will pick it back up on the next it's like bi directional then it's put it's it's capturing everything you you just send as a Slack channel it's like your tools are coming to you where we already live in Slack we're all in Slack exactly exactly yeah so is it it's its own channel then it is its own channel yes okay and oh yeah you can also like increase the number of stations like if I see that hey station two is being bottlenecked it is there are a lot of things sitting in the queue waiting for station twos interesting and they can you know start running so yeah interesting that'd be that'd be cool to have a little bit uh I'm gonna have to bring that up to our death team uh our detection engineering and their hunting team uh to visualize like where things are in the queue as we're all like because we're all piping in detections like I'm piping in a bunch of macOS detections like we've got Josh um he is uh turning all of our like Previous detections into another into another uh more updated format.
00:44:03.760 --> 00:44:05.440
And so he's he's adding it.
00:44:05.519 --> 00:44:08.719
And then we've got like two other guys too that are adding things.
00:44:08.800 --> 00:44:12.000
So it'd be cool to kind of see like where things go in each station.
00:44:12.079 --> 00:44:13.679
I'm gonna bring that up to the guys.
00:44:14.079 --> 00:44:14.480
Yeah.
00:44:14.800 --> 00:44:15.920
So it's in Slack.
00:44:16.000 --> 00:44:16.480
That's very good.
00:44:16.559 --> 00:44:20.079
I I think everyone's jumping onto the Slack bandwagon.
00:44:20.239 --> 00:44:25.440
Uh Claude, now you're able to uh bring bring some commands into Slack.
00:44:25.519 --> 00:44:30.079
Everything is now having a little bit of a slack uh Slack plugin or bot.
00:44:30.719 --> 00:44:31.199
Yeah.
00:44:31.519 --> 00:44:33.039
Slack is good.
00:44:33.199 --> 00:44:38.079
Slack does its work, like you know, without errors.
00:44:38.239 --> 00:44:40.000
That's all that I like about it.
00:44:40.159 --> 00:44:42.239
I have used Teams before.
00:44:42.800 --> 00:44:44.159
I mean, yeah.
00:44:44.480 --> 00:44:44.880
I'm sorry.
00:44:44.960 --> 00:44:45.679
I'm sorry to hear that.
00:44:45.760 --> 00:44:47.119
I've never used Microsoft Teams.
00:44:47.280 --> 00:44:47.599
Yeah.
00:44:47.920 --> 00:44:49.840
Oh, lucky you, lucky you.
00:44:50.079 --> 00:44:54.960
Yeah, I've only I went, no, but I have used Lotus Notes and I I'm aging myself a little bit.
00:44:55.119 --> 00:44:55.920
I'm aging myself.
00:44:56.000 --> 00:45:07.679
Uh you're probably younger than me, but I uh I there was once upon a time when I uh worked at IBN early in my career and they used Lotus Notes, which was like I forgot.
00:45:09.519 --> 00:45:10.480
Have you heard of this?
00:45:10.800 --> 00:45:11.920
I have not noticed this.
00:45:12.159 --> 00:45:13.199
Yeah, yeah, no.
00:45:13.280 --> 00:45:19.840
Yeah, that it it's it was uh it uh was I forgot what the messenger was called, but it seriously looked like AIM.
00:45:20.000 --> 00:45:20.960
Do you remember AIM?
00:45:21.199 --> 00:45:22.559
AOL instant messenger.
00:45:22.719 --> 00:45:23.440
I'm dating myself.
00:45:23.599 --> 00:45:25.679
Okay, I'm gonna edit that part out.
00:45:26.000 --> 00:45:29.840
Anyway, um, PKI, identity for everything.
00:45:30.000 --> 00:45:30.880
Moving on.
00:45:31.119 --> 00:45:37.360
Um, you have an interesting uh perspective on detection engineering from a PKI uh angle.
00:45:37.519 --> 00:45:41.119
And that's something that I've I've actually never talked about on the podcast.
00:45:41.280 --> 00:45:48.960
You're a detection engineer that runs also, in addition to your detection factory, a PKI, an internal PKI program.
00:45:49.360 --> 00:45:53.920
Uh what is like why, and also why what is that connection?
00:45:54.239 --> 00:45:54.719
All right.
00:45:54.880 --> 00:46:05.920
Uh so PKI essentially has uh public infrastructure, is the way of giving you know uh identity at something's birth.
00:46:06.159 --> 00:46:18.559
So whenever you have you know a human or you have a service account or you have an AI agent now or you have a GitLab runner or you have a workload, Kubernetes spots, whatever.
00:46:18.719 --> 00:46:20.400
I don't care what it is.
00:46:20.719 --> 00:46:28.719
Anything that comes into existence can and should be given uh an identity at the very beginning.
00:46:28.800 --> 00:46:33.039
That hey, you come from this place, here is your identity.
00:46:33.280 --> 00:46:39.599
Uh so it especially is useful when you know you have your own infrastructure, for example.
00:46:39.840 --> 00:46:48.480
You do uh when you have your own products, when you have your own, you know, custom built, I guess, internal websites, when you have internal apps.
00:46:48.719 --> 00:46:51.119
So many people are building a lot of things in-house now.
00:46:51.280 --> 00:46:51.760
Yeah.
00:46:52.400 --> 00:46:56.320
So all of these things can and should use PKI.
00:46:56.480 --> 00:47:03.199
So you should have like a root PKI uh so all of this, you know, comes from my working at N phase.
00:47:03.360 --> 00:47:09.280
Like we had all of these microinverters out in the field, solar uh inverters.
00:47:09.599 --> 00:47:18.880
And essentially they had to communicate back to us so that you know we could get their data, we could uh send out updates, things like that.
00:47:19.119 --> 00:47:26.480
And so for all of that, essentially we had to know if this is our microinverter or is this an imposter.
00:47:26.880 --> 00:47:36.239
So, for that, when we birthed these uh products, right, we embedded an identity in them using the PKI chain.
00:47:36.880 --> 00:47:45.519
And when these would go out in the field, they would send back something with that identity that hey, I am this, and here is my birth certificate.
00:47:45.920 --> 00:47:57.920
And then, you know, based off of that, you can then say that okay, you're allowed to this, this, this, or you're allowed to do XYZ things, you're not allowed to do ABC and so on and so forth.
00:47:58.159 --> 00:48:03.360
And I think that's where the detection engineering part comes in, right?
00:48:03.679 --> 00:48:09.599
Because if you don't know something's identity, it is really hard to detect something.
00:48:09.679 --> 00:48:14.000
Then you can say that, hey, if Alex does something, send me an alert.
00:48:14.159 --> 00:48:14.480
Sure.
00:48:14.719 --> 00:48:20.880
But you know, what if somebody else shows up, says, Hey, I'm Alex, and you know, tries to do something.
00:48:21.119 --> 00:48:26.719
So you it all goes back to how do you give an identity in the first place?
00:48:26.800 --> 00:48:28.800
And that is where PKI comes in.
00:48:28.960 --> 00:48:52.639
So, you know, I'm helping design the root PKI at Crusoe, which will then be used to have multiple levels, and you can also have you have these multiple keys, and uh there are different levels of keys, and then you is send out certificates, you can like mint certificates for certain services, you can do stuff with that.
00:48:52.800 --> 00:48:55.760
So I guess that is the whole premise.
00:48:56.239 --> 00:48:58.880
Sign everything, trust nothing.
00:48:59.039 --> 00:49:05.039
So instead of like verify, it's like uh unless it's signed, I don't I don't want to know, I don't want to hear it.
00:49:05.119 --> 00:49:06.159
I don't want you you can't get it.
00:49:06.800 --> 00:49:08.159
Exactly, exactly.
00:49:08.480 --> 00:49:10.079
Because these certificates, right?
00:49:10.239 --> 00:49:12.159
You can have them very short-lived as well.
00:49:12.400 --> 00:49:15.039
You can have like a five-minute long certificate.
00:49:15.199 --> 00:49:15.360
Yeah.
00:49:15.760 --> 00:49:20.559
So uh use a show certificate, I validate it against my route.
00:49:20.880 --> 00:49:23.440
And if it is valid, you go ahead.
00:49:23.599 --> 00:49:26.079
If not, you don't get access.
00:49:26.559 --> 00:49:31.280
Of course, you it's uh you can do this, you can over-control.
00:49:31.599 --> 00:49:37.440
I I I see this like kind of like the the New Zealand or the Australia way.
00:49:37.599 --> 00:49:38.880
Yes, they don't have any crime.
00:49:39.039 --> 00:49:49.840
Yes, it's a wonderful, like crimeless, uh, peaceful country, country, set of countries, because it's a smaller population to control.
00:49:50.400 --> 00:50:00.159
Um and if you build everything in-house, you have kind of that New Zealand problem, or not problem, I should say, uh, where you can do that.
00:50:00.239 --> 00:50:04.880
You can give a cryptographic identity to things, to workloads, to runners to host.
00:50:05.119 --> 00:50:18.000
But like if you are using SASI, if you're SASI, like you use everything SaaS, how easy it is, is it for like Salesforce to give you their crypt their like CA, their cryptographic identity?
00:50:18.559 --> 00:50:20.960
So uh that is a tricky part.
00:50:21.119 --> 00:50:26.880
So uh SaaS applications are the ones that you can't really touch with your own PKI.
00:50:27.280 --> 00:50:42.000
Uh you can have there are some applications, there are some vendors that allow you to give a certificate to them or some kind of uh you know identity that allows you to uh like send and receive encrypted communication.
00:50:42.320 --> 00:50:46.880
But in general, like it can be used for MTLS and such things.
00:50:47.119 --> 00:50:52.159
So some SaaS vendors do support that, but most of them I don't think they do.
00:50:52.320 --> 00:51:02.320
So you have to, for such things, you have to like rely on, you know, OIDC or you have to rely on other ways of managing temporary identities.
00:51:02.400 --> 00:51:09.039
Like, how do I trust if this is something that you know comes from a particular vendor or not?
00:51:09.440 --> 00:51:11.199
Yeah, that's such a great point.
00:51:11.360 --> 00:51:16.800
Uh I I'm looking at this in a in a different facet, uh, like in terms of reference objects.
00:51:16.960 --> 00:51:26.800
Um, there's many different uh set of like reference lists that multiple of your rules can call on, um, particularly for benign processes.
00:51:26.960 --> 00:51:38.320
So, like, let's say I want to be notified every time this happens, but exclude all of these processes that it's okay for um like real versions of Jam to do this, right?
00:51:38.639 --> 00:51:45.920
Like real versions of CrowdStrike to do this, and like or anything like admin IT related, and you want to put that into a list.
00:51:46.239 --> 00:51:57.440
Right now we're we're calling on a lot of these lists um through from a lot of rules, but we're just using it as like text in the name.
00:51:57.679 --> 00:52:06.559
So it's like, okay, it's okay that um, you know, or it's not okay that uh who am I.exe ran, right?
00:52:06.880 --> 00:52:17.199
Uh and but that's if you're basing it off of the name and it's not signed that yes, that is a Microsoft application that signed that, like it it we we're kind of we don't really know.
00:52:17.440 --> 00:52:23.679
It could be another process, yeah, named who am I, and but not actually signed.
00:52:23.840 --> 00:52:25.840
And so it's a little bit, it's a little brittle.
00:52:25.920 --> 00:52:26.320
Yeah.
00:52:26.639 --> 00:52:28.400
It is, yes, absolutely.
00:52:28.639 --> 00:52:40.880
Uh you can't really trust anything based off of you know a name or like a static token, or like you know, if I have this particular thing, then I'm valid to do this.
00:52:41.199 --> 00:52:52.320
So I think that's why I am so bullish, you can say, on PK, because that way you can hand out very temporary certificates at birth.
00:52:52.480 --> 00:52:58.960
You can, you know, check for things uh that nobody can replicate, pretty much, right?
00:52:59.119 --> 00:53:06.239
There is always a source of truth, and that is what you use to model your PK on.
00:53:06.480 --> 00:53:11.280
Uh, it really helps if you have your own products, it really helps if you have your own infrastructure.
00:53:12.559 --> 00:53:18.880
But even if you don't, even if you don't, you still have you know your own uh clusters that you're running, for example.
00:53:19.039 --> 00:53:24.480
You still have your own uh internal websites that you have for like, I don't know, whatever purpose.
00:53:24.719 --> 00:53:34.079
You still have your own internal uh vaults, you still have your uh there have to be services that you operate internally for a lot of things.
00:53:34.239 --> 00:53:36.480
Your endpoint, your laptop, right?
00:53:36.639 --> 00:53:42.880
Your laptop can be issued an identity that hey, you are a uh laptop of this particular company.
00:53:43.119 --> 00:53:53.360
That identity can be used to sign that uh ops certificate, which then you know, operational certificate, which then does stuff for you.
00:53:54.159 --> 00:53:59.760
And like it also think about it as also a translation for all of these things, right?
00:53:59.840 --> 00:54:08.559
So, for example, if my laptop is saying that I am this, this, this, this is where I came from, this is who I belong to, whatever, this is my IP address.
00:54:08.800 --> 00:54:13.199
It has all of this information in your you know original birth certificate.
00:54:13.519 --> 00:54:30.000
Then the I guess the translator is the one that takes that, it hands out another certificate, which is the actual operational certificate, which only has information that your identity number is this, your you are valid till this, and you can do XYZ things.
00:54:30.159 --> 00:54:32.159
That is the operational certificate.
00:54:32.239 --> 00:54:45.440
So that when that particular identity goes to something else, it doesn't have to give its whole history of, you know, oh, I am this person and this uh IP address and I belong to this or whatever.
00:54:45.679 --> 00:55:00.639
It can just show that uh translation of I hold this other thing, which must have which must mean that I have this original with me, and based off of this, you give me access to something.
00:55:00.800 --> 00:55:03.599
And then you can have detections on top of all of this.
00:55:04.400 --> 00:55:09.760
If you know something is off, give me another same manner.
00:55:10.079 --> 00:55:13.760
I love that you're bringing that nuance into how we're thinking about this.
00:55:13.920 --> 00:55:30.400
I feel like most detection engineers treat identity as another team's problem, someone else's problem, uh, which we still take into account, the logs of identity, like octologs per se.
00:55:31.119 --> 00:55:37.280
But it uh yeah, this is a whole different like door it opened.
00:55:37.440 --> 00:55:43.920
That are the the rules that we're building from processes that aren't who they say they are, workloads are who they say they are.
00:55:44.079 --> 00:55:45.840
That just adds more enforcement.
00:55:46.000 --> 00:55:53.199
Uh, but I I just I can't imagine like the cost of building detections on on top of an unverified entity.
00:55:53.360 --> 00:55:55.360
I just can't even picture that.
00:55:56.000 --> 00:55:56.400
Yeah.
00:55:56.639 --> 00:56:06.000
Not not to instill fear, but it yeah, that's that's like you're moving, you're moving in the you're moving, you think you're moving in the right direction, but maybe not.
00:56:06.159 --> 00:56:06.800
Yeah.
00:56:07.440 --> 00:56:13.920
I think most detection programs assume the fact that you know this identity problem will be solved.
00:56:14.639 --> 00:56:28.800
And most detection programs assume that if this person says, I'm this person must be this person, or you have like something in place to make sure that this person hopefully is who they say.
00:56:29.280 --> 00:56:31.199
Is who they say they are, yeah.
00:56:31.519 --> 00:56:39.199
So but I think PK is uh, you know, very, very, very old concept, but I think it is the AAA, yes.
00:56:39.360 --> 00:56:40.400
The uh what is it?
00:56:40.480 --> 00:56:46.719
The authoriz uh authoris no authorization, authentication, and accounting.
00:56:46.960 --> 00:56:47.199
Yes.
00:56:47.519 --> 00:56:48.719
I know what you're talking about.
00:56:48.880 --> 00:56:54.079
Yes, authentication, authorization, or was it audit?
00:56:54.800 --> 00:56:56.400
I I think it's just three.
00:56:56.880 --> 00:56:57.840
Should it be the fourth?
00:56:58.639 --> 00:56:59.599
Yeah, why not?
00:56:59.840 --> 00:57:03.199
Yeah, well that that's that gave me a lot to think about, Tehas.
00:57:03.519 --> 00:57:08.719
I and I think that that is the through line that I definitely want people to sit with.
00:57:09.039 --> 00:57:14.559
Uh, you can't detect what you can't identify and confirm.
00:57:15.119 --> 00:57:16.320
That's signing.
00:57:16.480 --> 00:57:21.679
Uh, and you can't automate uh what is hard to trust.
00:57:21.920 --> 00:57:24.559
Uh I think that's what it comes down to it.
00:57:24.719 --> 00:57:29.199
A lot of us skip that part because it's not very glamorous to do that.
00:57:29.360 --> 00:57:34.320
And I've never think thought about PKA testing, uh, identity testing and and review gates.
00:57:34.480 --> 00:57:37.840
Not it doesn't, it's you tell me is it is it fun to do that?
00:57:38.000 --> 00:57:48.880
I feel like uh when I remember doing this in the practice portion of the CISSP, it was it was like what a Windows 7.
00:57:49.039 --> 00:57:52.639
It looked, it kind of looked like an active directory kind of a module GUI.
00:57:53.039 --> 00:57:56.559
Uh is it still like that, or did they modernize that a little bit?
00:57:56.880 --> 00:57:59.840
Uh you can get things done through CLA.
00:58:00.079 --> 00:58:01.679
Oh, perfect, headless.
00:58:01.760 --> 00:58:06.559
No, better, easier, faster, and you can have your cloud do it.
00:58:06.639 --> 00:58:09.440
So, you know, you don't have to type code.
00:58:09.599 --> 00:58:10.320
So yeah.
00:58:11.679 --> 00:58:20.800
Before it was like um like uploading certificates to like it gave you like two buttons, an import and output like you know what an export.
00:58:21.039 --> 00:58:21.760
Yeah, yeah.
00:58:22.000 --> 00:58:24.239
That was that was old, old school way of doing it.
00:58:24.400 --> 00:58:26.880
But the thing too that you're so much fun.
00:58:27.760 --> 00:58:32.159
Yeah, that you have to you have to have both both approaches, right?
00:58:32.239 --> 00:58:37.760
Everything that's a that's a button should be able to have an API to do it from the command line or headless.
00:58:38.000 --> 00:58:38.159
Absolutely.
00:58:39.280 --> 00:58:41.760
Nowadays I don't even stop at API.
00:58:41.840 --> 00:58:43.440
Like, I need an MCP connection.
00:58:43.679 --> 00:58:45.519
You need an oh wow, okay.
00:58:45.760 --> 00:58:47.599
No MCP, then no talking.
00:58:47.920 --> 00:58:48.639
I love that.
00:58:48.880 --> 00:58:51.920
Yeah, no, this is like you know, still five, ten years old.
00:58:52.159 --> 00:58:53.840
Yeah, that is that is the new way.
00:58:53.920 --> 00:58:56.000
Uh have your MCP talk to my MCP.
00:58:56.239 --> 00:58:56.800
Exactly.
00:58:56.880 --> 00:58:57.280
Yeah.
00:58:58.000 --> 00:59:01.199
Run the podcast, have your MCP ask the questions.
00:59:01.519 --> 00:59:05.280
Oh no, but then it ru ruins the human touch of it.
00:59:05.760 --> 00:59:06.400
That is true.
00:59:06.480 --> 00:59:07.039
That is true.
00:59:07.280 --> 00:59:08.559
The vibes, the vibes.
00:59:08.639 --> 00:59:10.000
Um, so uh thank you.
00:59:10.159 --> 00:59:11.760
The vibes have been fantastic.
00:59:11.920 --> 00:59:18.159
Uh I appreciate you for working with me through the technical issues.
00:59:18.320 --> 00:59:21.840
I feel like they were worth it because this this camera setup is so good.
00:59:22.000 --> 00:59:23.599
It's my first video ever.
00:59:23.760 --> 00:59:25.760
Um awesome.
00:59:26.320 --> 00:59:39.920
Thank you for coming on and uh actually showing your work because I think that this is a lot of folks like to gatekeep, not they don't want to show the the the burns they they don't want to show, uh they don't want to build in public.
00:59:40.079 --> 00:59:42.400
And that's something I really try to promote here.
00:59:42.559 --> 00:59:47.440
Uh, to you know, just show us what you're doing, show us what works, share it with others.
00:59:47.599 --> 00:59:52.320
Uh, the burns and you know, not the burns as well, like the not just the wind.
00:59:52.800 --> 00:59:53.440
Yeah.
00:59:53.840 --> 00:59:54.480
Yeah.
00:59:54.880 --> 00:59:56.559
Awesome, awesome, cool, cool.
00:59:56.639 --> 00:59:58.400
Uh, thank you so much for having me.
00:59:58.559 --> 01:00:01.039
It has been an absolute pleasure talking to you.
01:00:01.360 --> 01:00:02.079
Likewise.
01:00:02.320 --> 01:00:04.559
This has been Detection Dispatch.
01:00:04.719 --> 01:00:05.599
We'll see you next time.