Sysmon for AI Visibility feat. Anton Ovrutksy

WEBVTT
00:00:12.000 --> 00:00:21.440
Welcome back to Detection Dispatch, the show where we go find the people actually pushing the limits of detection engineering and threat hunting and all things in between.
00:00:21.679 --> 00:00:26.719
Today we're talking about a blind spot that most of us don't even know we even have yet.
00:00:26.960 --> 00:00:49.359
Coding agents, claude code, codex cursor, whatever you're running now, it seems to be living at the endpoint level, spawning processes, uh in touching credential stores, installing packages, reading files, and your EDR can see the process tree, but not really have an idea of what the agent was actually asked to do or why.
00:00:49.840 --> 00:00:52.799
The gap is super real, very new on everybody's mind.
00:00:52.880 --> 00:00:57.520
And today's guest built a tool specifically to address that.
00:00:57.840 --> 00:01:02.960
Joining me today is Anton Ovritsky on the on the pod.
00:01:03.200 --> 00:01:08.560
He's been in the space literally since the Sysmon days and quite literally built the Sysmon for AI telemetry.
00:01:08.799 --> 00:01:11.280
Anton, thank you so much for coming on the show.
00:01:11.519 --> 00:01:12.239
How are you?
00:01:13.200 --> 00:01:13.519
I'm good.
00:01:13.599 --> 00:01:13.920
I'm good.
00:01:14.000 --> 00:01:14.560
How are you doing?
00:01:14.640 --> 00:01:16.159
How's everyone uh hanging out?
00:01:16.480 --> 00:01:17.280
Uh good.
00:01:17.680 --> 00:01:23.280
I I gotta say, I have I'm still buzzing from this new gig that I'm finally able to share.
00:01:23.760 --> 00:01:28.319
I've been keeping it for quite uh on a low, low kilo profile for quite some time.
00:01:28.400 --> 00:01:32.079
And finally it's it's F FB official, LinkedIn official.
00:01:33.040 --> 00:01:34.000
Congratulations, yeah.
00:01:34.159 --> 00:01:34.560
That's awesome.
00:01:34.640 --> 00:01:39.439
I'm I'm so honored to be like on the episode where it gets announced, and it wasn't planned or anything.
00:01:39.519 --> 00:01:39.920
So yeah.
00:01:40.239 --> 00:01:41.040
It totally wasn't.
00:01:41.120 --> 00:01:43.280
We've been, I feel like we've been rescheduling this pod.
00:01:43.840 --> 00:01:49.200
You you just went on vacation, which absolutely take every to everyone listening, take this as your reminder to take your PTO.
00:01:50.400 --> 00:01:51.280
Take your PTO.
00:01:51.519 --> 00:01:51.920
Yeah, yeah.
00:01:52.079 --> 00:01:52.879
Take care of yourselves.
00:01:53.040 --> 00:01:53.439
Uh-huh.
00:01:53.599 --> 00:01:54.319
Take care of yourself.
00:01:54.400 --> 00:01:55.599
Did you go anywhere fun?
00:01:56.400 --> 00:01:58.319
No, I I'm I'm a homebody.
00:01:58.400 --> 00:02:00.319
I don't I don't enjoy leaving the house.
00:02:00.400 --> 00:02:02.959
You know, I just like hanging out at home with my wife and cats.
00:02:03.599 --> 00:02:04.480
We love a staycation.
00:02:04.640 --> 00:02:05.439
We love a staycation.
00:02:05.599 --> 00:02:06.560
Yeah, that's that's my jam.
00:02:06.799 --> 00:02:07.439
Yeah.
00:02:08.319 --> 00:02:15.280
Well, I hope you try and tried anything new locally because I'm I'm literally still discovering my city every single day.
00:02:16.240 --> 00:02:16.960
Yeah, yeah.
00:02:17.039 --> 00:02:20.000
I uh I actually live in like a very small town in Ontario.
00:02:20.159 --> 00:02:22.479
Shout out to anyone who lives in Oxford County, Ontario.
00:02:22.639 --> 00:02:25.280
I'm in Ingersoll, so it's a very like small town.
00:02:25.360 --> 00:02:36.479
There's like two coffee shops, so there's like not a whole lot to discover, but there are some like awesome farmers markets around that I got to that I got to like visit that I usually don't get a chance to because they're like nine to five, right?
00:02:36.560 --> 00:02:39.759
So it's uh fresh produce always delicious.
00:02:40.000 --> 00:02:42.639
They should well they don't they have them on the weekends?
00:02:42.960 --> 00:02:48.800
Some of them, some of them close at like Friday, you know, like they want to spend time with their families on the weekends too, I guess.
00:02:48.879 --> 00:02:53.599
So you know, if you're working full time, it's hard to get to, but PTO, perfect time.
00:02:53.919 --> 00:02:54.400
Yeah.
00:02:54.639 --> 00:02:58.800
Well, of course, we gotta know what you're working on over there in Ontario.
00:02:59.199 --> 00:03:01.280
Look, it seems like you've been cooking.
00:03:01.680 --> 00:03:03.280
I've been trying, yeah, for sure.
00:03:03.439 --> 00:03:08.080
Been been busy with clocking up like more sources of telemetry.
00:03:08.159 --> 00:03:12.000
Uh that's where like the new project ATIN came into being.
00:03:12.319 --> 00:03:13.520
I hate picking names for this stuff.
00:03:13.599 --> 00:03:15.599
Everything with like agent is taken already.
00:03:16.240 --> 00:03:29.280
I had I had a project that was named like Agent Fence, I think that's what I was gonna call it originally, and then you do like the Googling and the research, and it's like everyone has their their own little vibe coded tool for agent something.
00:03:29.360 --> 00:03:44.960
But yeah, I think it's uh a tool that's like I've been craving because I found that as I've been like building my own projects and things like that, I start pulling all these dependencies down and I start pulling skills down, and I start like authenticating Claude to like every system that I could find, you know.
00:03:45.120 --> 00:03:50.719
And it what when I'm kind of like wrapping up at the end of the session, I'm like, what did I just give it creds to?
00:03:51.039 --> 00:03:59.919
You know, and like what what package did I just pull down and and like what am I even doing here as a security professional without even you know thinking about this kind of stuff twice?
00:04:00.080 --> 00:04:02.240
I'm just like I I need to make this work, damn it.
00:04:02.319 --> 00:04:05.919
And it doesn't matter where my creds are going and like who has access to them.
00:04:06.080 --> 00:04:10.879
And uh and yeah, I just found that there was like so little the visibility around it, right?
00:04:10.960 --> 00:04:16.079
And you and you look at the logs and it shows you like you know your standard command line stuff, but you you have no idea.
00:04:16.160 --> 00:04:17.759
Like, did it access a credential?
00:04:17.839 --> 00:04:19.199
Did it send anything anywhere?
00:04:19.360 --> 00:04:25.040
Like whether the skill that I just installed, like you see people liking it on Twitter, does that mean that it's safe?
00:04:25.199 --> 00:04:27.920
Uh, you know, like what commands is it running?
00:04:28.000 --> 00:04:31.920
So it's all these questions I I wanted to have answers to, right?
00:04:32.000 --> 00:04:32.800
To have telemetry to.
00:04:33.040 --> 00:04:38.720
I always found that like if you're building like a detection or a hunt or something, like it always starts from telemetry sources, right?
00:04:38.800 --> 00:04:43.519
You can't like do a cool beaconing thread hunt without network data, right?
00:04:43.600 --> 00:04:54.079
So it's uh I think it's cool that these days you can, you know, instead of just saying, hey, I don't have the telemetry for it, you can like roll up your sleeves and vibe code something that gives you the telemetry for it.
00:04:54.160 --> 00:04:55.519
Uh so it's pretty amazing.
00:04:55.839 --> 00:05:03.279
So you so you built a tool with Claude to look for Claude behavior.
00:05:03.680 --> 00:05:08.000
Yeah, yeah, it was Claude and Codecs I use because I've been trying to get a little bit more into Codecs.
00:05:08.079 --> 00:05:13.199
I've been so like Claude piled lately that uh I've heard people say, like, yeah, Codecs is good, so I've been trying it as well.
00:05:13.279 --> 00:05:14.639
And yeah, it performs awesome.
00:05:14.800 --> 00:05:19.199
So sometimes I have the two like piggybacking, you know, like hey, Claude made changes, review them.
00:05:19.519 --> 00:05:21.439
And yeah, yeah, I used Claude for this.
00:05:21.600 --> 00:05:30.480
Obviously, like I'm not a not a developer by trade, uh, you know, but I understand endpoint telemetry, you know, I understand that part part, and I know kind of what I want.
00:05:30.560 --> 00:05:33.680
Um, and I know enough about Windows internals to make this kind of work.
00:05:33.839 --> 00:05:38.639
Uh and yeah, I think like the the little pieces that it has is is fairly unique.
00:05:38.720 --> 00:05:43.120
I know there's been some tools out there that have been you know released, and I think there's gonna be more.
00:05:43.680 --> 00:05:49.519
Oh, I bet right, because if if I'm having like these issues, I'm sure smarter people than me are having them too.
00:05:49.600 --> 00:05:51.759
And um I yeah, I think it's a gap.
00:05:51.839 --> 00:05:56.959
Uh I think it's a huge gap because like you have no idea what the user prompted, right?
00:05:57.040 --> 00:05:58.319
If you're just running EDR.
00:05:58.560 --> 00:06:03.120
And I think like to step back a second, not everyone even has EDR, right?
00:06:03.279 --> 00:06:05.120
Like if they even have that, yeah.
00:06:05.360 --> 00:06:09.759
So you're you're stuck with Sysmon at that point, which you have actually you're you're being super humble.
00:06:09.839 --> 00:06:12.000
You have a very interesting Sysmon story.
00:06:12.959 --> 00:06:18.879
Yeah, I started working with Sysmon, I forget, I think it was like 2014 or 2012 or something.
00:06:18.959 --> 00:06:22.000
I I'm pretty old now, so it was uh and it's pretty crazy.
00:06:22.160 --> 00:06:28.800
It feels like just yesterday that that I like like I still remember like the install screen in Sysmon and my first time installing Sysmon.
00:06:29.040 --> 00:06:32.639
It feels so fresh to me, but it was so like long ago at this point.
00:06:32.879 --> 00:06:34.160
It's pretty crazy how time works.
00:06:34.319 --> 00:06:36.160
Uh well it has it hasn't changed much.
00:06:36.959 --> 00:06:48.399
Well, yeah, I guess like other than new like events and stuff, but yeah, the general idea hasn't changed, and I think like that like paradigm shift of like holy crap, I I could actually like see what the system is doing now.
00:06:48.480 --> 00:06:49.360
You know, you know what I mean.
00:06:49.439 --> 00:06:53.600
I could see like a file being created, I could see a network connection being made.
00:06:54.399 --> 00:07:07.199
Like that paradigm, like compared to the visibility that I had with, I think at the time that I first started using Sysmon, it was just I had antivirus and it gave me a command line, but it didn't give me like the parents, right?
00:07:07.759 --> 00:07:08.959
Where it all started.
00:07:09.279 --> 00:07:10.000
Yeah, yeah.
00:07:10.079 --> 00:07:14.800
So yeah, I started working with Sysmon and I've been like in love with it uh for for decades.
00:07:14.879 --> 00:07:23.199
And I still am, you know, it's just it doesn't have um like if you don't have fancy tools like this, I think Sysmon is still your best bet, right?
00:07:23.360 --> 00:07:29.600
To get into this kind of stuff, but um but yeah, yeah, for like the agentic stuff, right?
00:07:29.680 --> 00:07:36.959
Like for the for the prompts and the credential access piece that that the new uh tool that I released covers, like Sysmon's not very good for that.
00:07:37.120 --> 00:07:44.240
Um, you know, it doesn't have like the capability to tell you, hey, someone touched your AWS key on on your endpoint or something like that.
00:07:44.319 --> 00:07:54.879
And I think like that whole aspect of creds living on your endpoint had we I think we like skipped over all that, even like in the cloud era, right?
00:07:54.959 --> 00:08:02.800
Like I've written about this like in my previous employers' blogs and things like that, where like I really think we have like a blind spot there as defenders, right?
00:08:02.879 --> 00:08:07.040
Like you don't know where your creds are going, right?
00:08:07.120 --> 00:08:11.360
Or or like where they're being used, and and that was like before this whole agentic era.
00:08:11.839 --> 00:08:13.040
So yeah.
00:08:14.160 --> 00:08:15.759
It's no, it's it's true.
00:08:16.000 --> 00:08:17.360
It's it's pretty spot on.
00:08:17.439 --> 00:08:38.559
Even I I I'm getting tripped up in deciphering like multiple uh touches at the credential store in like a short period of time and distinguishing when an agent does that versus an info stealer because they do both similar things, and it's getting tripped, and uh it's hard to get get specific on on what is what at this point.
00:08:38.799 --> 00:08:40.399
Um yeah, yeah, yeah.
00:08:40.559 --> 00:08:41.200
Yeah, exactly.
00:08:41.279 --> 00:08:41.440
Right.
00:08:41.519 --> 00:08:44.480
You think of like an info stealer running an endpoint, right?
00:08:44.720 --> 00:08:47.279
And we see that that all the time on my day job, right?
00:08:47.440 --> 00:08:50.720
We see info stealers targeting Mac Windows, right?
00:08:50.799 --> 00:09:01.600
That they don't discriminate, but there's like very little telemetry to show you, like, hey, this info steeler, you know, like stealer.exe touched AWS creds, you know, dot text, right?
00:09:01.679 --> 00:09:10.879
That there's no you need to have like SACL auditing in Windows and Linux, you need like audit D configurations and all that, and it's like very difficult to configure, right?
00:09:11.039 --> 00:09:14.240
So you need some kind of tool to like auto-discover that for you.
00:09:14.320 --> 00:09:20.240
So that's why I tried to build in um into A and it actually like finds what creds you're using and audits those.
00:09:20.320 --> 00:09:25.679
So you know, like, hey, like this was the prompt that led to this particular event.
00:09:25.759 --> 00:09:28.480
Like it has like a session idea that ties all those together.
00:09:28.879 --> 00:09:31.360
So I think it's like a really rich telemetry for hunting, right?
00:09:31.440 --> 00:09:33.039
It's like a Sysmon GUID.
00:09:33.200 --> 00:09:35.120
So yeah, but yeah.
00:09:35.440 --> 00:09:39.519
I was gonna ask, do you need is Sysmon a requirement to for this or no?
00:09:39.840 --> 00:09:41.840
No, no, you don't need Sysmon for it.
00:09:41.919 --> 00:09:45.279
I think uh if you if you have Sysmon, it it it'll help, right?
00:09:45.360 --> 00:09:49.039
Obviously, like Sysmon's great, um, but you don't need it for for this particular tool.
00:09:49.120 --> 00:09:57.279
Uh it only audits from the agent, so it won't log like every process on your host, uh like Sysmon does.
00:09:57.360 --> 00:10:04.399
It'll only log stuff spawned from like Claude or Codex or whatever you enrol in the tool, so it's not like spamming your event log.
00:10:04.799 --> 00:10:05.360
At least I hope.
00:10:05.440 --> 00:10:07.200
I haven't done too many testings with it.
00:10:07.279 --> 00:10:11.679
So if someone actually runs this sucker, please let me know if it's noisy or not.
00:10:11.759 --> 00:10:16.799
Uh, but I did try my best to keep it from like blowing up your system with event logs.
00:10:17.279 --> 00:10:31.600
Well, so I when I read your blog, uh, which was very timely, uh and I think when we originally talked about creating a podcast, I don't even think the topic we agreed wasn't was not what we're gonna what we're gonna be getting into today.
00:10:32.480 --> 00:10:35.679
Yeah, it was completely different because you're building all sorts of tools.
00:10:36.000 --> 00:10:38.879
But I I think this is very timely because you're absolutely right.
00:10:39.039 --> 00:10:47.600
Like this coding agents are genuinely the new blind spot that is missing from your and EDR telemetry and Sysmon.
00:10:47.919 --> 00:10:50.000
So you wrote it extremely well.
00:10:50.080 --> 00:10:55.519
Uh I think you said it it's almost like an entirely new OS running on top of the endpoint.
00:10:56.480 --> 00:11:06.559
Um so what did you find was actually missing from that standard EDR or Sysmon process information when agents are now in the picture?
00:11:06.960 --> 00:11:07.360
Yeah, yeah.
00:11:07.440 --> 00:11:08.559
I think, yeah, that's a great question.
00:11:08.639 --> 00:11:12.320
I it it may maybe that's a like a little bit of a hyperbolic example.
00:11:12.399 --> 00:11:16.879
I don't know, but I it it almost feels like a different OS running on your host, right?
00:11:16.960 --> 00:11:23.679
It's like this especially like I read some of the stuff that people are doing with agents where they have like multi-agent workflows, right?
00:11:23.759 --> 00:11:25.759
And and like a loop that runs for days.
00:11:25.840 --> 00:11:31.200
And like I don't really do anything that fancy, but if you have something like that running on your host, right?
00:11:31.279 --> 00:11:33.679
Like what does the telemetry for that look like?
00:11:33.919 --> 00:11:39.679
Like what does like a seven-day bender from an agent, you know, look like in the logs, I wonder, right?
00:11:39.759 --> 00:11:42.080
Like, and like what did it do in those seven days?
00:11:42.159 --> 00:11:45.919
And how how do you know that it did what you asked it to do?
00:11:46.159 --> 00:11:49.039
And how do you know that it didn't like go nuts on day three, right?
00:11:49.120 --> 00:11:52.320
Unless you're actually like monitoring for for that stuff.
00:11:52.480 --> 00:12:08.240
And I I think like as an industry, we focus so much on like identities, and I think that it is maybe maybe not the wrong, maybe wrong is like too strong of a word, but I feel like we haven't focused a lot on creds that those identities are built on, right?
00:12:08.480 --> 00:12:17.840
Like we focus on like the user, but we don't do a really good job of like tracking how that user moves from like endpoint to cloud and then back to endpoint, right?
00:12:17.919 --> 00:12:20.559
Like we don't have a good way of like querying that.
00:12:20.720 --> 00:12:28.240
And and if you ever worked with like a sim and you tried to track an incident that spanned that kind of vector, right?
00:12:28.320 --> 00:12:29.919
It it's awkward, it's clumsy.
00:12:30.000 --> 00:12:31.120
Like the data looks different, right?
00:12:31.200 --> 00:12:35.120
You're like, yeah, the C Smack the Sysmon logs don't look like the Cloud Trail logs, right?
00:12:35.360 --> 00:12:36.240
No, none at all.
00:12:36.559 --> 00:12:36.799
Right?
00:12:36.879 --> 00:12:43.200
There's nothing like tying those events together, and I feel like that dynamic is replicated with an agent on the endpoint, right?
00:12:43.279 --> 00:12:48.240
Like the transcripts that Claude produces or that codex produces don't look like Windows logs.
00:12:48.480 --> 00:12:51.039
No, and you know, like, yeah, right.
00:12:51.519 --> 00:12:58.480
And if you look at like a process tree, you'll see like Claude spawning node, and but you don't know like why it did that, right?
00:12:58.799 --> 00:13:00.399
You you have you have no idea.
00:13:00.480 --> 00:13:04.720
You you just like was this part of Claude like automatically doing something?
00:13:04.879 --> 00:13:05.759
Was it part of a loop?
00:13:05.840 --> 00:13:07.600
Did someone like prompt it to do that?
00:13:07.840 --> 00:13:09.840
Yeah, um, yeah, no, it's true.
00:13:10.080 --> 00:13:14.399
Claude to command line to code tells us absolutely almost nothing to do with, right?
00:13:15.120 --> 00:13:15.600
Yeah, yeah.
00:13:15.679 --> 00:13:16.960
And and we see that so much, right?
00:13:17.039 --> 00:13:26.080
Like, I look at this kind of telemetry all day, and and it's very hard to like draw a difference between what the agent did versus what it was asked to do.
00:13:26.320 --> 00:13:31.279
And I think that gap is worth like wrapping some telemetry around, right?
00:13:31.360 --> 00:13:40.159
Like to figure out like what did Anton asked Cloud to do while it sent his AWS creds to to like Digital Ocean or whatever, right?
00:13:40.240 --> 00:13:45.759
Like, um, but yeah, I and I don't feel like we're like monitoring that enough.
00:13:45.840 --> 00:13:53.200
Like there's very little um like general credential access type events out there for for security products, right?
00:13:53.279 --> 00:14:02.000
You don't know um, you know, like who's touching your Kubernetes secrets right on your endpoint when a dev uses kubecuddle, right?
00:14:02.080 --> 00:14:04.960
Like we we just don't have good monitoring around that, right?
00:14:05.039 --> 00:14:23.600
And I think we instead of like going back and building that monitoring around it and then using agents, we just started using the agents right away, and now we have like a compounding set of problems where we didn't have good visibility before, and now there's like something autonomous happening on our endpoint, and we still don't have visibility into that.
00:14:23.679 --> 00:14:28.080
So I feel like we're kind of like you know, building on top of quicksand a little bit here.
00:14:28.480 --> 00:14:34.240
Anton, we're in the find out phase in the F around and find out.
00:14:34.480 --> 00:14:39.840
People are realizing that AI is causing just as many many problems as it is allegedly solving.
00:14:40.159 --> 00:14:40.720
Yeah, right.
00:14:40.799 --> 00:14:54.720
Like, how would you even like I read the you know the like the hugging face incident that I'm sure like every blue team read, but like if you're not a frontier AI lab, like how would you find out that a developer's creds were compromised through a supply chain attack, right?
00:14:54.799 --> 00:14:59.039
To to piece that together, you would need so many different telemetry strands, right?
00:14:59.120 --> 00:15:06.799
You need like the endpoint data, then you gotta dig through the transcripts to figure out like what the prompt was, and then you gotta look at the network event, right?
00:15:06.879 --> 00:15:08.159
And then that's probably gone, right?
00:15:08.240 --> 00:15:09.919
You can't forensicate that.
00:15:10.000 --> 00:15:12.159
Uh and yeah, like working backwards from that.
00:15:12.240 --> 00:15:21.759
I I yeah, I feel sorry for defenders these days that have to like figure out how to how to like work work backwards from that because the telemetry is just not there.
00:15:22.080 --> 00:15:22.960
Well, until now.
00:15:23.200 --> 00:15:23.679
Until now.
00:15:23.919 --> 00:15:25.759
Until now, yeah, until you're running like A.
00:15:26.320 --> 00:15:26.799
Yeah, yeah.
00:15:27.759 --> 00:15:38.080
Well, it which is fascinating because what I'm hearing is you're effectively modeling intent versus action, potentially, with what is what this new telemetry is uncovering.
00:15:38.159 --> 00:15:44.639
That's the core idea behind uh A10, which by the way, uh we missed out.
00:15:44.879 --> 00:16:01.679
You're right about the naming, because we missed out personally on the opportunity to create def D E A F because Nebuloc and the Death team, they just released this agentic detection engineering uh framework, and it it's a it's a DEF, but I'm like, it could have been DEF.
00:16:02.799 --> 00:16:03.519
That's awesome.
00:16:03.600 --> 00:16:05.360
Yeah, I didn't I didn't think of that.
00:16:05.440 --> 00:16:05.679
Damn it.
00:16:05.759 --> 00:16:08.320
I think I missed out, yeah.
00:16:08.480 --> 00:16:09.840
See, I suck at naming this stuff.
00:16:10.080 --> 00:16:16.799
I'm okay at like ideating and building it and testing it out, uh, but but actually like naming it and marketing it.
00:16:17.919 --> 00:16:20.159
Well that's that's the important that that's the important stuff.
00:16:20.240 --> 00:16:22.240
The the whole model, the whole model.
00:16:22.399 --> 00:16:33.200
So so what uh the model's trying to prompt, right, and the tool call intent to what actually executed on the host aka the action.
00:16:33.360 --> 00:16:42.080
So why was that pairing the right way to look at it, the right abstraction versus just logging more processed data?
00:16:43.120 --> 00:16:52.399
I think yeah, I I designed it that way to give like if you're hunting through this data, I think it's a good like pivot point to the Taunton, right?
00:16:52.480 --> 00:16:58.799
Like what was performed by the coding agent that wasn't requested, right?
00:16:58.960 --> 00:17:02.159
I think that's uh an interesting like starting point, right?
00:17:02.240 --> 00:17:05.119
And I feel that's just there as part of the schema for you.
00:17:05.440 --> 00:17:09.119
So you don't need to do any fancy like sim work with that.
00:17:09.200 --> 00:17:15.359
You know, you don't need to translate events or you don't need to normalize anything, you can just search for you know when this field equals no.
00:17:15.440 --> 00:17:27.119
I I forget the exact uh field or what it's called, but yeah, I tried to include like whether we could attribute a tool call that the agent did to a user request or not.
00:17:27.279 --> 00:17:27.519
Uh-huh.
00:17:27.839 --> 00:17:32.240
And I don't know, like to be honest, I don't know how that would work at scale, right?
00:17:32.319 --> 00:17:38.880
Because I don't have you know access to like 10,000 dev machines to actually run this, but I felt that that was pretty important, right?
00:17:38.960 --> 00:17:44.240
You you probably like want to know if Claude did something that the user didn't ask for, right?
00:17:44.400 --> 00:17:49.279
And uh whether that comes into like a prompt injection, right?
00:17:49.359 --> 00:17:54.720
If you're like I I think in my like blog, I just used and and actually Claude like gave me a hard time with this.
00:17:54.799 --> 00:17:56.079
It wouldn't, it wouldn't run it for me.
00:17:56.240 --> 00:18:02.000
Like I just had like a markdown file with like a little HTML embedded there and that just said like hey, run this instead.
00:18:02.400 --> 00:18:03.519
And it was so benign, right?
00:18:03.599 --> 00:18:04.960
It's just it's just for a demo.
00:18:05.119 --> 00:18:09.519
I didn't want to go too nuts, but Claude was like, no, this is prompt injection, I'm not running it.
00:18:09.759 --> 00:18:24.079
Um so so that's why I baked in those kinds of fields in there, so that you can kind of like easily sweep your data for that, because I felt like the like there's a lot of events in here, and the scheme is like pretty complex, and the events are are beefy.
00:18:24.240 --> 00:18:25.039
There's there's a lot of it.
00:18:25.119 --> 00:18:32.240
So I wanted to include some fields in there that made it so that you can filter out stuff very, very easily.
00:18:32.559 --> 00:18:35.359
Uh so that's kind of like my thought process there.
00:18:35.680 --> 00:18:53.680
Have you gotten to the point where it makes it um maybe not, I don't think easy is the right word, but it it makes it much more clear, or you could derive what was done by human, a human prompt versus maybe maybe a non-human prompt or a malicious one.
00:18:54.400 --> 00:18:56.319
Yeah, yeah, I think, yeah, that's a great question.
00:18:56.400 --> 00:19:09.359
So I think with the event types that it omits, it should be really easy because you should be able to correlate the user prompt, because I have a field for user prompt that sucks in the transcript that codex and and uh cloud leaves behind.
00:19:09.440 --> 00:19:14.960
So you can actually filter all the events that don't have that field in them, right?
00:19:15.039 --> 00:19:21.039
So you can just say like give me everything that you saw that wasn't tied to a user prompt.
00:19:21.279 --> 00:19:24.960
So yeah, I merely actually like like I've only run this in my lab, right?
00:19:25.039 --> 00:19:29.440
With with like uh the dev scenarios that I tried to cover in the blog.
00:19:29.599 --> 00:19:35.200
Yeah, but I haven't actually run this with like you know, like a real world dev workflow.
00:19:35.759 --> 00:19:38.319
Funchers customers, you gotta get on this, right?
00:19:38.400 --> 00:19:39.119
Yeah, yeah, yeah.
00:19:39.279 --> 00:19:40.720
I mean, this is like my own project.
00:19:40.799 --> 00:19:45.200
I I don't I think you know, I think as a company we're probably gonna build something like this at some point.
00:19:45.279 --> 00:19:48.079
I I don't know, I don't want to speak to like timelines or something.
00:19:48.400 --> 00:19:52.400
But yeah, this is like you know, something that you play around with in your lab, right?
00:19:52.480 --> 00:19:57.599
I I probably wouldn't go around and deploying this to like 10,000 endpoints because I like I said, I'm not a dev.
00:19:58.000 --> 00:20:09.200
This was 100% vibe coded, but I thought it was an interesting experiment because I don't think there's anything out there that ties in like this transcript portion to the cred access portion.
00:20:09.839 --> 00:20:16.079
I think those two things when combined are pretty unique, so that you can see like the full chain of it.
00:20:16.160 --> 00:20:20.400
You can see like what the user prompted for, where the creds accessed or or not.
00:20:20.559 --> 00:20:30.799
And I think other tools don't have that cred access piece, and I think that's super, super important because without it, most systems are not instrumented with that kind of stuff.
00:20:30.960 --> 00:20:43.920
Like I don't know for if you're listening to this podcast and you're like an instant responder, like a sim engineer, I don't know how many times have you seen like SACL auditing events, you know, like a 4663 in an environment like auditing files, right?
00:20:44.000 --> 00:20:45.359
Like very rare.
00:20:45.680 --> 00:20:47.759
I've seen it maybe a handful of times.
00:20:47.920 --> 00:20:50.880
Uh and and even something like a file share, right?
00:20:50.960 --> 00:20:53.519
Like file shares always have creds, right?
00:20:53.680 --> 00:21:00.000
So I think that yeah, that that little piece of like the credential auditing piece, I think that's a big like differentiator for for this tool.
00:21:00.319 --> 00:21:01.119
Absolutely.
00:21:01.200 --> 00:21:09.839
Uh though I've in the Mac world, I feel like they wouldn't expose so so much, so much of the minutiae with it, right?
00:21:10.559 --> 00:21:19.839
Yeah, I think like the with the EFS standpoint security framework, I think it exposes something, but yeah, I'm not really a Mac person, so I I I'm not like an expert in that.
00:21:20.000 --> 00:21:30.799
Um but I think there is a way, and I know there is on Linux, so this so there should be on Mac as well, but like in my little Windows world for sure, like I haven't seen too many environments with with SACL auditing.
00:21:30.880 --> 00:21:52.000
And I and I've tried to blog about it um my last DEF CON workshop in 2023 or four, what was all about SACL auditing and how to find like you know, Cloud Creds, like your Azure token, and not the Azure token that is sent through OAuth, like not that token, like the the CLI token, like if you use like AZ login, right?
00:21:52.079 --> 00:21:53.680
That there's a token that lives on the disk, right?
00:21:53.759 --> 00:21:55.039
And Steelers left to take it.
00:21:55.359 --> 00:21:56.720
And then oh, that's right.
00:21:57.039 --> 00:21:59.680
Yeah, and and same with like AWS, and same with Kubernetes.
00:22:00.240 --> 00:22:03.279
Same with uh like a bunch of other cloud CLI tools, right?
00:22:03.359 --> 00:22:09.920
That that that authenticate to the cloud and and the endpoint holds all those creds and there's no extra syspawn event, right?
00:22:10.079 --> 00:22:10.799
When they're accessed.
00:22:10.880 --> 00:22:26.000
So I think no, and and and if we think of most dev, they prefer the Mac OS world and their shipping product or they're shipping some real like sensitive things via the cloud CLI directly from their Mac.
00:22:26.400 --> 00:22:32.240
Yeah, and and so it's it's absolutely it's so important to capture uh to capture that.
00:22:32.720 --> 00:22:33.680
Yeah, yeah, 100%.
00:22:33.920 --> 00:22:48.240
Yeah, I think like yeah, like I said, creds are like I just don't think we do a great job of auditing creds in general, be it on like Linux, be it on Windows, and uh linking like the process part to the cred access part, I think is super, super difficult.
00:22:49.279 --> 00:22:51.200
But until now just not enough, yeah.
00:22:51.359 --> 00:22:52.640
Until now, yeah, exactly, right?
00:22:52.720 --> 00:22:53.680
Yeah, yeah, until now.
00:22:53.759 --> 00:22:54.000
Yeah.
00:22:54.400 --> 00:22:57.200
Let's talk about when when this can go wrong.
00:22:57.359 --> 00:23:07.039
Uh I'm talking uh I'm thinking supply chains, I'm thinking prompt injection, uh network even uh if if people are still doing that.
00:23:07.200 --> 00:23:22.720
I I can you believe I heard a wild take about like that if you configure your your cloud routing correctly, uh technically NDR should not really be existing, which is a wild take.
00:23:23.200 --> 00:23:27.440
But uh but that's yeah, but but everything happens on the network.
00:23:27.599 --> 00:23:32.079
Uh yeah, and uh access to the public internet will never not happen.
00:23:33.039 --> 00:23:39.599
Uh so yeah, that's I I I I invited this person to debate on the podcast several times, but they don't want to.
00:23:39.839 --> 00:23:44.960
I'm like, okay, well keep keep saying that behind behind unpodcast closed doors, but okay.
00:23:45.279 --> 00:23:46.079
That's hilarious.
00:23:46.160 --> 00:23:48.160
Yeah, but maybe one day because I want to hear more.
00:23:48.240 --> 00:23:49.839
I wanna hear more about this take.
00:23:49.920 --> 00:23:51.039
Yeah, yeah.
00:23:51.279 --> 00:23:53.920
Um, and identity too.
00:23:54.000 --> 00:23:56.160
Like if you uh it was it was paired with that.
00:23:56.240 --> 00:23:59.519
I think I guess I should I should state that to save Grace.
00:23:59.759 --> 00:24:30.559
Uh they said that if you configure the proper controls as well, detection engineering is shouldn't it doesn't is not needed uh because you you configure the right access to the right to the right resource at just the right time, and then uh you can figure the right, you know, fire the right traffic and and therefore no this reactive state of DE is is becomes obsolete and like yeah, well that zero zero trust never never I mean we're zero pro zero trust has been implemented for now a decade and yeah, yeah, here we are.
00:24:30.880 --> 00:24:31.519
Yeah, yeah.
00:24:31.599 --> 00:24:39.359
The the profession still stays how would you like I guess my argument against that is how would you know without the DE function?
00:24:40.079 --> 00:24:44.480
Like how would you know that you configured everything properly if you're not monitoring it afterwards, right?
00:24:44.559 --> 00:24:47.759
Like how would you like Yeah, quite literally.
00:24:48.079 --> 00:24:48.559
What would you do?
00:24:48.640 --> 00:24:49.839
Like, oh I'm done.
00:24:50.000 --> 00:24:53.279
I'm just gonna trust myself that everything is configured 100%.
00:24:53.839 --> 00:24:57.920
Um no logging, no detections, I'm just gonna trust it.
00:24:58.319 --> 00:24:59.039
I don't know.
00:24:59.359 --> 00:25:00.720
Yeah, no, literally.
00:25:00.880 --> 00:25:07.200
Uh and and then on a the the last take, I'm bringing it on to the podcast, whatever.
00:25:07.359 --> 00:25:16.720
The last take she said was um that in the in like decades of incident response, nothing was ever like identified by sim.
00:25:17.119 --> 00:25:18.640
Like sim barely ever.
00:25:19.119 --> 00:25:31.920
It's always because a user a user reports something, or uh it's it's obviously external, like an ext external breach is reported, or a user starts seeing something suspicious, but it was not because of a trigger or because a rule hit.
00:25:32.400 --> 00:25:32.720
Wow.
00:25:32.880 --> 00:25:33.759
Okay, interesting.
00:25:33.920 --> 00:25:34.319
I don't know.
00:25:34.400 --> 00:25:37.519
Like they I would love to show them our internal slack at work.
00:25:37.599 --> 00:25:44.640
We get sim alerts all the time that kickoff incidents that stop ransomware and like all the time.
00:25:44.960 --> 00:25:45.440
I don't know.
00:25:45.680 --> 00:25:48.000
Yeah, I'll be happy to share a redacted screenshot.
00:25:48.079 --> 00:25:51.920
If you go to the Hunter's blog, you can find a bunch of incidents that a sim kicked off.
00:25:52.079 --> 00:25:52.559
But I don't know.
00:25:52.799 --> 00:25:54.799
Yeah, obfuscated, of course.
00:25:55.599 --> 00:25:56.400
Well, yeah, yeah.
00:25:56.559 --> 00:25:57.920
I mean, uh yeah, yeah.
00:25:58.160 --> 00:25:59.279
Yeah, it's an interesting take.
00:25:59.359 --> 00:26:06.559
I mean, I get the I get the the fact that like sims traditionally, especially ones that aren't like a managed sim, right?
00:26:06.640 --> 00:26:14.799
If you're running a sim like in your own environment, I I get the the critique that they're not as like accurate, you know, as they could be.
00:26:15.039 --> 00:26:18.960
I think that's like a known thing about sims, right?
00:26:19.039 --> 00:26:19.759
Is that they're noisy.
00:26:19.839 --> 00:26:24.000
But the fact to say that like no incident ever kicked off of the sim ever, I don't I don't know.
00:26:24.079 --> 00:26:25.039
I don't I don't agree with that.
00:26:25.119 --> 00:26:26.640
I don't think that's right.
00:26:26.720 --> 00:26:27.839
You heard it here.
00:26:28.000 --> 00:26:32.079
Uh um friend who starts with an L and ends with an Izzy.
00:26:34.559 --> 00:26:35.680
Um so funny.
00:26:35.839 --> 00:26:36.720
She's still a friend.
00:26:36.960 --> 00:26:38.480
Um okay, supply chain.
00:26:38.640 --> 00:26:46.720
Supply chains, um, and when an agent installs a package, right, and it's compromised, then credentials get, you know, touch that nobody asks for.
00:26:47.039 --> 00:26:51.039
What does that look like with with your tool with your new telemetry?
00:26:51.200 --> 00:26:54.960
Uh, what's that one field that now can make it huntable?
00:26:55.519 --> 00:26:56.240
That's a good question.
00:26:56.319 --> 00:26:58.960
So I I cover that scenario uh in the blog.
00:26:59.279 --> 00:27:01.359
And yeah, there's a whole bunch of fields actually.
00:27:01.440 --> 00:27:03.440
There's like you get the prompt, right?
00:27:03.519 --> 00:27:24.559
The user prompt that that says like, hey, install this package, so you can trace it back to to its you know, like true origin, so you could figure out whether it was like a prompt that led to this or whether it was uh you know a script that called a package or something like that, like an environmental script that sets up the environment and it pulled you know the wrong version or something like that.
00:27:24.720 --> 00:27:38.480
Um you also get like the cred access event, so you see that you know this particular credential, whatever the compromise package happened to be going after, you know, was touched by node or whatever the prop parent process is.
00:27:38.799 --> 00:27:50.319
You also get like the network event, so you can see like where that cred went, and that all has like a session ID, so you can tie those events together, and I think that's the like that's the like money shot.
00:27:50.400 --> 00:27:56.480
So you can see like, hey, Anton asked for this, and that credential was touched, and it went over here.
00:27:56.640 --> 00:27:56.960
Uh-huh.
00:27:57.200 --> 00:27:59.839
Or conversely, you can see Anton never asked for this.
00:28:00.079 --> 00:28:00.400
Never did.
00:28:00.799 --> 00:28:04.000
Claude did it anyway, and that cred went somewhere else, kind of thing.
00:28:04.079 --> 00:28:12.319
And I think if you combine that with like other data, especially like S bombs, like I used to think S bombs were kind of weird, you know, like why would someone need this?
00:28:12.400 --> 00:28:14.079
And now I'm like, okay, now I understand.
00:28:15.119 --> 00:28:15.759
Yeah, right.
00:28:15.920 --> 00:28:19.039
Just stupid me, but uh, you know, now I get it.
00:28:19.119 --> 00:28:24.319
Um, so if you combine that with you know S bomb data and uh the name is escaping me.
00:28:24.400 --> 00:28:30.960
I I think that there's a tool recently um released that does that kind of thing that does like the package like uh hierarchy.
00:28:31.119 --> 00:28:32.559
It'll it'll come to me at some point.
00:28:32.640 --> 00:28:36.720
Um the package hierarchy, and it's not the right tree.
00:28:36.880 --> 00:28:37.200
No.
00:28:37.759 --> 00:28:42.960
No, I I think it's the same folks who released uh the tool that you messaged me about yesterday.
00:28:43.519 --> 00:28:44.880
Oh the perplexity guides.
00:28:45.039 --> 00:28:46.960
Yes, but yeah, Adele, yeah.
00:28:47.039 --> 00:28:49.920
I only know his Twitter handle, he's a brilliant guy.
00:28:50.000 --> 00:28:57.440
He I think he released a tool that does like um like pack like scoping, it'll like scan your endpoint and tell you like what packages.
00:28:57.680 --> 00:29:03.519
If you combine like that telemetry with the telemetry that like ATM gives you, that you're you're you're cooking now.
00:29:04.079 --> 00:29:05.839
You're a fro, you're a force to reckon with.
00:29:06.079 --> 00:29:06.880
Yeah, right.
00:29:07.119 --> 00:29:23.759
And I think like with that amount of data, right, if you had that, like if an incident responder had that, then you will be able to like piece the incident together because without it, right, if you don't have the prompt, you know, like if that transcript rolled over, right, on the disk or something like that, now now you're missing that piece, right?
00:29:23.839 --> 00:29:33.279
So if you just see like node coming out from Claude, and then you see like Claude making a network connection, uh, you know, is that X filled?
00:29:33.359 --> 00:29:34.960
I I don't know, or is that legitimate?
00:29:35.039 --> 00:29:36.480
Yeah, it's hard to say.
00:29:37.519 --> 00:29:38.960
It's almost like a certificate.
00:29:39.680 --> 00:29:42.240
It's almost like the same like certificate signing.
00:29:42.319 --> 00:29:43.680
Like, did Anton do this?
00:29:43.839 --> 00:29:45.200
Did did he sign this?
00:29:45.279 --> 00:29:47.359
Is it is it a confirmed legitimate certificate?
00:29:47.440 --> 00:29:50.799
Is it is it an expired certificate, or is it not fair?
00:29:51.279 --> 00:29:56.000
Yeah, yeah, you get like the third degree for for certain, but yeah, no, it's it's it's literally that, right?
00:29:56.079 --> 00:30:00.240
I think like the the and I think that's the piece that EDR won't won't tell you, right?
00:30:00.319 --> 00:30:05.359
It won't it won't give you the prompt, it won't tell you that this credential was touched, it won't tell you where that credential went.
00:30:05.680 --> 00:30:19.440
So just so now you can piece together like the whole the the whole thing and it yeah, it tries to bring a little bit more like uh yeah, like telemetry, you know, it's all it's all it's all about that because uh yeah, by default, it's just not there, at least from what I found, right?
00:30:19.519 --> 00:30:22.480
I mean, and at least not in a real-time way, right?
00:30:22.559 --> 00:30:29.519
You can go digging through your transcript and try to link it, but without without a tool like this, I think you'll be you'll you'll be having a hard time.
00:30:29.759 --> 00:30:42.720
Well, this is a hell of a step up from whatever the first version of like Claude Otel data, what what the hell that was, where it was just token usage, uh the the session name.
00:30:42.799 --> 00:30:44.240
It was like, what am I supposed to do with that?
00:30:44.319 --> 00:30:55.519
I feel like I've I've I thought it sounded like a broken record because I've said this so many times, but now I I keep hearing that it's now a little bit better, but I I can't imagine it's anything like compared to this.
00:30:56.079 --> 00:31:04.880
Yeah, and and I don't think like I I have a little bit of familiar familiarity with OTEL from my previous job, and it's not really designed for security use cases, I don't think, right?
00:31:04.960 --> 00:31:13.200
It's more designed for like um monitoring and then and all that and uh availability and stuff like that, uh like observability, not so much the security part.
00:31:13.440 --> 00:31:30.960
Um yeah, I I feel like I I think that this is definitely like a best effort type tool, and I'm really looking forward to people trying it, you know, and letting me know if it finds stuff or doesn't find stuff, and uh maybe it'll inspire people to vibe code their own that that are like more comprehensive and better.
00:31:31.039 --> 00:31:33.119
But or contribute to your repo.
00:31:33.359 --> 00:31:35.359
Are you are you accepting pull requests?
00:31:35.680 --> 00:31:36.799
Yeah, yeah, contribute.
00:31:36.880 --> 00:31:45.519
Yeah, I'm I'm not like uh like I said, I'm not a dev by trade, so I might fumble here and there with your pull requests if it's a little bit complex for my P brain.
00:31:45.599 --> 00:31:46.480
Uh so bear with me.
00:31:46.559 --> 00:31:50.559
But yeah, definitely feel free to contribute, fork it, make your own versions.
00:31:50.720 --> 00:31:52.640
Um, especially on the Mac side.
00:31:52.799 --> 00:31:53.759
I'm not a Mac expert.
00:31:53.839 --> 00:32:01.519
I had a hard time getting like the Mac like developer account, uh, and I couldn't even get this to run on my Mac without disabling like everything.
00:32:01.680 --> 00:32:07.119
And it's like my personal Mac, so I was like, I probably don't want to like disable like all the security stuff that it comes with.
00:32:07.440 --> 00:32:14.240
Not gonna lie, I did try it myself and I couldn't figure it out because uh because it is on a nebula computer, so sorry, sorry, uh Damien and Cindy.
00:32:14.880 --> 00:32:16.960
Yeah, no, that's good that you're doing that.
00:32:17.119 --> 00:32:20.319
That uh that that you're keeping like the internal security going there.
00:32:20.400 --> 00:32:26.640
Uh but yeah, definitely like yeah, if someone is good of good with Macs and knows how to like bundle this or come out of the colour.
00:32:26.720 --> 00:32:27.680
Oh, if you have the loop team.
00:32:28.240 --> 00:32:30.480
Oh, yeah, I love the shout out to you.
00:32:30.720 --> 00:32:31.599
Yeah, yeah, yeah.
00:32:31.680 --> 00:32:32.319
I love Datadog.
00:32:32.480 --> 00:32:35.839
So yeah, definitely um, yeah, definitely, yeah, to take a look at it.
00:32:35.920 --> 00:32:41.039
I think Datadog has their own kind of product for this too, or or something like that, that audits agent.
00:32:41.119 --> 00:32:53.279
So yeah, I feel like every everyone's gonna have like their own flavor of this, and whether it's like open source, whether it comes as part of a sim, whether it ships as like part of like an XDR endpoint, something.
00:32:53.440 --> 00:32:57.279
I I think we're all gonna have to get used to dealing with this kind of telemetry.
00:32:57.359 --> 00:33:04.480
And I think this is just a way to get it for free, you know, just to like play with it, uh get used to it, just see what Claude gives you, right?
00:33:04.559 --> 00:33:11.119
I I think if you're like a defender and you're not studying Claude telemetry a little bit at this point, you may be a good time to start.
00:33:11.279 --> 00:33:11.599
Oh, yeah.
00:33:12.079 --> 00:33:14.720
Like it's it's everywhere, whether you want to accept it or not.
00:33:15.200 --> 00:33:16.880
Oh, a hundred percent.
00:33:16.960 --> 00:33:22.960
Even people that shouldn't be using it are are using it, and there's there's no way back now, there's no going back now.
00:33:23.039 --> 00:33:28.000
Now that it's a it's like literally quite any one uh vibe code apps uh now.
00:33:28.160 --> 00:33:38.480
There's so much slop out there, and people are realizing that the that it doesn't have the proper auth, and you know that it that it rightfully needs, and it it's it's getting out of hand.
00:33:38.720 --> 00:33:45.920
Uh but you and there may be iterations of this, there may be new pro new ways of this, but you it they will never be the first.
00:33:46.000 --> 00:33:47.680
And if you're not first, you're last.
00:33:47.839 --> 00:33:51.039
So yeah, yeah, yeah, definitely.
00:33:51.279 --> 00:33:58.640
Uh the one on your blog, the what really scared me though was the prompt injection, like when it could go wrong.
00:33:58.799 --> 00:34:02.240
Uh that was the probably the most exciting one.
00:34:02.319 --> 00:34:07.680
Oh well, scary, but also exciting the when the instructions never come from the user themselves.
00:34:07.839 --> 00:34:08.239
Yeah.
00:34:08.480 --> 00:34:18.559
Uh and I and you were talking about this earlier, is now you can distinguish a user asked for this versus the page told the agent to do this because of because of that transcript and the creds.
00:34:19.119 --> 00:34:19.840
Yeah, yeah.
00:34:20.480 --> 00:34:23.599
I think that the concept even applies to like a malicious skill too.
00:34:23.679 --> 00:34:26.480
I I know like skills are all the rage these days, right?
00:34:26.559 --> 00:34:33.519
Like everyone's installing skills and uh looking at skills from like a marketplace, and like you don't know what it does unless you look at it, right?
00:34:33.679 --> 00:34:39.280
It's it's it's like you're running code and everyone tells you like don't uh don't run untrusted code on your endpoint.
00:34:39.440 --> 00:34:44.800
So I think we found almost like a loophole where it's like I'm not running it, you know, Claude's running it for me.
00:34:44.960 --> 00:34:51.280
So yeah, it's even like a an additional layer of of danger now where you might not even know.
00:34:51.360 --> 00:34:54.960
You could just tell Claude, like, hey, install this skill pack or something like that.
00:34:55.199 --> 00:34:57.119
And a skill could like sneak in there.
00:34:57.199 --> 00:35:00.559
That um I I did find that Cloud has pretty good guardrails for this stuff.
00:35:00.639 --> 00:35:06.320
So I don't want to make it sound like you can just uh you know, like install like any skill out there and you're compromised, you know.
00:35:06.400 --> 00:35:10.639
No, I don't think that's the case, but we we do like I don't know.
00:35:10.800 --> 00:35:16.880
Like there's some maybe, maybe there's some um I'm a little bit, I'm a little bit, I'll take some blame for this.
00:35:16.960 --> 00:35:20.960
Like I have so I'm kind of a hoarder, a digital hoarder.
00:35:21.039 --> 00:35:30.000
I'm not luckily I'm not a physical hoarder, but I'm a digital hoarder when it comes to like uh bookmarking all these GitHub repos and that I I want to eventually do.
00:35:30.239 --> 00:35:34.239
And and within it in the last year, it's all about clot skills.
00:35:34.320 --> 00:35:39.119
Like I want the best cloud skills for literally every use case in my life, both personal and and work.
00:35:39.280 --> 00:35:48.800
And so I've I I'm finding myself uh GH pip installing a lot of GitHub repos without even really like looking through them first.
00:35:49.039 --> 00:35:57.599
And uh flash forward dev popper now, you know, since 2024 to now, there's malicious GitHubs uh just living everywhere, all over the place.
00:35:57.679 --> 00:36:00.400
Yeah, uh, so uh I I don't know.
00:36:00.480 --> 00:36:09.199
I if I haven't seen really clo Claude trip at all over uh pip installing every repo that I want out there, and I do it a lot.
00:36:09.519 --> 00:36:11.199
Yeah, same, same, yeah.
00:36:11.280 --> 00:36:17.119
And that was like, yeah, like like I mentioned when we started this podcast, how like I I scared myself, you know.
00:36:17.280 --> 00:36:19.599
I was like, why am I installing it?
00:36:19.840 --> 00:36:22.559
And luckily, like I I work with like you know, like a local lab, right?
00:36:22.639 --> 00:36:34.559
So it's not you know the end of the world if those creds get popped or whatever, but still I'm feeding Claude all these types of creds and tokens, and like you said, installing skills and and like those skills could call script, right?
00:36:34.639 --> 00:36:47.519
That can call like another script, and you're chusting on so many nodes in that trust chain going right, and that on the other end of it from where you're using it, there's just so little telemetry, right?
00:36:47.599 --> 00:36:50.800
You like I tried to look at Claude under Procmon, right?
00:36:50.880 --> 00:36:57.760
Like I'm probably like dating myself now by by like saying Procmon, but like I was curious, right?
00:36:58.079 --> 00:36:58.719
I wanted to know.
00:36:58.960 --> 00:36:59.920
Procmon's great.
00:37:00.079 --> 00:37:02.480
I want to know, like, hey, what's Claude doing on the system, right?
00:37:02.559 --> 00:37:06.079
Like when I click co-work, is it launching a different process?
00:37:06.159 --> 00:37:13.519
And and when you look at what it's doing, it's so like loud on on Windows and so convoluted, you can't tell what's what.
00:37:13.599 --> 00:37:17.679
You know, it's writing JSON files, it's spawning like weird command lines.
00:37:17.760 --> 00:37:25.039
So it's hard to like detection engineer your way around Claude without additional telemetry, I find.
00:37:25.199 --> 00:37:33.199
So yeah, I think we all need something to to give us more visibility into like you know, what skill that Alex installed this week, right?
00:37:33.280 --> 00:37:34.079
Like, how would you know?
00:37:34.239 --> 00:37:41.119
Like what what audit trail is there to to like you know, read that back to you.
00:37:41.440 --> 00:37:46.400
Uh and and even further than that, like how would you know what those skills did, right?
00:37:46.480 --> 00:37:47.599
Like, yeah.
00:37:48.159 --> 00:37:58.639
Yeah, I just feel like we're we're we're flying blind and like yeah, I feel like we're gonna get bombarded with like different EDR vendors releasing their own solutions to this problem.
00:37:58.719 --> 00:38:05.519
Yeah, I think that's the next wave of this, but why didn't they you'll be you gotta ask, why didn't they get ahead of it earlier?
00:38:06.480 --> 00:38:07.679
I yeah, I don't know.
00:38:07.760 --> 00:38:08.400
It's a good question.
00:38:08.480 --> 00:38:17.599
You know, like if you've ever worked for for like a product org, you might know how like not super fast some product orgs move, right?
00:38:17.679 --> 00:38:18.960
By the time decisions are made.
00:38:19.199 --> 00:38:24.559
Yeah, it's hard to I think like individually it's easy to iterate quickly these days.
00:38:24.880 --> 00:38:27.760
No, organizationally, it might be difficult, yeah.
00:38:28.000 --> 00:38:29.679
I think it's this innovator's dilemma.
00:38:29.760 --> 00:38:44.400
Like CrowdStrike used to be so incredible at releasing what the people wanted fast, but now they're so busy wanting to build their own sim, and they're like the less love goes to the EDR product.
00:38:44.480 --> 00:38:47.280
Like that that was their their core all along.
00:38:48.000 --> 00:38:52.639
And identity and everything, like the whole minutiae, the whole platform now, platformization.
00:38:52.880 --> 00:38:56.000
Um, yeah, I call it the innovator's dilemma.
00:38:56.320 --> 00:38:56.880
Interesting.
00:38:57.039 --> 00:39:00.639
I wonder if it's like um this might be like a spicy take.
00:39:00.719 --> 00:39:08.079
I wonder if it's just a like looking at the problem the wrong way, like it's not seen as like an endpoint problem, it's seen as like an AI problem.
00:39:08.639 --> 00:39:15.119
I see it's own its own domain, but but where else would it run besides the end if not the endpoint?
00:39:15.519 --> 00:39:16.320
I I don't know.
00:39:16.400 --> 00:39:28.159
And then I think we made that same mistake when we did like the whole cloud security, you know, like uh yeah, because you hear a lot about cloud security, right?
00:39:28.320 --> 00:39:37.679
But I mean and I gave a I think I gave a couple talks on this or webinar on this year years ago, where it's I think the endpoint has a huge role to play in cloud security, right?
00:39:37.840 --> 00:39:40.559
Because that's where you access your cloud data from, right?
00:39:40.639 --> 00:39:51.840
Like, like obviously you need to understand like Azure logs and AWS logs and GCP logs like 100%, but how the endpoint interacts with those, I I think that's we're missing that piece.
00:39:52.079 --> 00:39:55.519
And we see it as a cloud security problem, not an endpoint security problem.
00:39:55.760 --> 00:40:03.519
And I feel like we're doing the same now with this AI stuff where we don't really see it as like a you know, quote unquote boring EDR problem.
00:40:03.840 --> 00:40:17.440
We see it as an AI problem, and I think from like a product perspective, if you're like, hey, I'm gonna like wrap a solution around this, it you you probably run the risk of like shipping a new EDR feature rather than introducing like a new product category.
00:40:17.679 --> 00:40:28.960
And and I'm pretty sure that's where we're kind of stuck as like an industry and and and from a like a product standpoint, but you know, maybe a wrong because no, no, no, that that that's a fantastic take.
00:40:29.119 --> 00:40:41.920
I I just I just keep thinking about how uh the under prom uh overpromise, under delivered of the XDR, because it seemed like maybe the XDR would have been would have been what would have solved it here, the that it touches it at all, I guess.
00:40:42.079 --> 00:40:50.400
But no, I guess uh I wouldn't be surprised if there's now a new AI um now a bunch of new startups trying to go after the AI market category.
00:40:51.199 --> 00:40:52.320
Oh, a hundred percent, yeah.
00:40:52.400 --> 00:41:08.559
Yeah, and um, you know, like I'm I wouldn't be surprised to see a whole bunch of yeah tools that act like this and give you very similar telemetry like this, but I I think the key piece for us to watch out for is like hunters, as boot teamers, as defenders, is the whole like chain.
00:41:08.719 --> 00:41:23.199
Like, can you see the whole thing, or does it just give you visibility into this one narrow aperture, and now you have to kind of do your best to link that together with your EDR logs and then link that together to your cloud logs and stuff like that?
00:41:23.280 --> 00:41:26.800
Because then I think then we're only compounding the problem, right?
00:41:26.880 --> 00:41:30.719
Because now we're now instead of looking at one data source, we're looking at three, right?
00:41:30.880 --> 00:41:32.480
It's always like convoluted that way.
00:41:32.639 --> 00:41:34.239
So yeah, it's gonna be interesting.
00:41:34.320 --> 00:41:34.960
It's gonna be interesting.
00:41:35.760 --> 00:41:42.559
Sorry, how how does this um how does this extend to m like an M and MCP calls it all?
00:41:42.719 --> 00:41:45.119
Or is that more does that have to have its own wrapper?
00:41:45.840 --> 00:41:47.599
Uh no, there's no MCP here.
00:41:47.679 --> 00:41:55.920
I I think there's probably other tools that do this via like the API layer and the MCP layer where they like intercept something.
00:41:56.159 --> 00:42:01.119
Yeah, that's not the route this tool uses, but it may be valid to do it that way.
00:42:01.199 --> 00:42:07.360
Uh maybe I just don't know enough about like Claude internals or codex internals to come up with a better solution.
00:42:07.440 --> 00:42:11.440
So I just went straight for like the kernel layer to monitor all that.
00:42:11.599 --> 00:42:25.199
Um but I think there are other tools out there that act as like a like almost like an AI WAF that like inspect the requests that Claude is doing or Codex is doing and and say, like, hey, do I want to allow this or not allow this?
00:42:25.360 --> 00:42:33.920
And I originally wanted to include that, like the blocking functionality into this, but then I thought like I'd rather just make it pure telemetry.
00:42:34.079 --> 00:42:34.400
Yeah.
00:42:34.639 --> 00:42:47.119
Um I felt like that was at least for a V1, you know, maybe maybe if this gets some traction and people start using it and want like a rules engine, you know, and like a decision making aspect to it, maybe I could add that.
00:42:47.199 --> 00:42:50.320
But for now, I just wanted it to I just wanted telemetry, you know.
00:42:50.400 --> 00:42:51.840
Like I I love telemetry.
00:42:52.000 --> 00:42:53.519
I'm a telemetry nerd, right?
00:42:53.599 --> 00:42:57.519
I love logs, I want to see so that that's that that's my like goal here.
00:42:57.920 --> 00:42:59.920
Logs, logs or didn't happen.
00:43:00.239 --> 00:43:00.719
Exactly.
00:43:01.280 --> 00:43:01.920
Right?
00:43:02.159 --> 00:43:22.000
Uh so no, I mean, I think everyone should uh ASAP drop it all and get started with this, but if they didn't, let's let's say that someone wanted to do this, spin, spin something up, uh, and start hunting for this behavior without A10, I feel like I I mean how would they even do it?
00:43:22.079 --> 00:43:22.880
I feel like you can't.
00:43:23.039 --> 00:43:25.519
What would they need to turn on to start logging?
00:43:26.159 --> 00:43:27.519
Uh that's a great question.
00:43:27.599 --> 00:43:32.559
Uh maybe like if you're using Splunk, maybe Splunk has like an app for Claude or something like that.
00:43:32.719 --> 00:43:33.599
Uh, I don't know.
00:43:34.000 --> 00:43:37.679
If you're pulling stuff off disk, maybe like the transcripts.
00:43:37.920 --> 00:43:39.519
Uh like yeah.
00:43:40.000 --> 00:43:47.280
If you have like I think as always with with like hunting and detection engineering, like you gotta make do with what you have, right?
00:43:47.440 --> 00:43:48.559
Um yeah.
00:43:48.960 --> 00:43:52.079
So I yeah, I would start with like the telemetry that you got.
00:43:52.159 --> 00:43:59.760
And if you're looking at process data, try to baseline it as as much as you can, like see like can I even see who's using Claude?
00:44:00.159 --> 00:45:25.039
period you know in my environment or or do I have that filtered out a hundred percent which it might be a thing you know like back in the day when I actually managed like a Sysmon deployment I was always fighting with the noise and I was editing the Sysmon config like once every 30 minutes you know and I have no idea how noisy Claude is with with Sysmon right like I don't I don't know I don't run a Sysmon environment anymore so I feel like if you're an admin right that has to balance like sim quota right you can't send like 60 billion Claud events to your simple you have to be cognizant about that right so you might have filtered it out altogether so yeah I I would probably start from like from scratch and just try to figure out you know like what what does the cloud profile look like for the users right like like does the finance person running cloud is that like are they installing packages like maybe they shouldn't be maybe they should just be using like cowork or something like that right so it's a probably not for sure right but we we know they're building those dashboards in cloud right yeah yeah and I think that's like an interesting part of this problem now where like everyone's kind of a developer right we used to say like oh man like the developer workstations are the trickiest right because they're running like VS Code and all these like tunnels that might look weird to to a blue team or right like hey why is this cloud for tunnel on this machine or whatever.
00:45:25.199 --> 00:45:45.519
But now everyone's doing this right now it's like all of a sudden you're seeing all these like Python installs on finance machines and I know pip installing all the prereqs that they need yeah and then it's like which version and which package and now we have to do like S bombs for everyone and and I bet you they don't know what the hell they're even installing.
00:45:45.599 --> 00:45:48.320
They just want they just want to get to the end goal like quickly.
00:45:48.800 --> 00:45:55.679
I I don't blame them because I I I've I found myself in that loop with BOD as well you know like like I just want this to work.
00:45:55.760 --> 00:47:46.639
Like who cares about the security warning right uh you know I think we're all we're all impatient humans right and uh always I I can't tell you how many times like um my partner will be trying to talk to me and I'll be like yes like always allow uh yeah yeah yeah yeah yeah yeah yeah yeah yeah and then I have like a Claude for my like personal finances and stuff like that and then like my code Claude and yeah oh my gosh my wife's always like are you talking to Claude again yeah I had Claude plan like almost my whole wedding yeah yeah it would be amazing at that but yeah oh yeah uh well I would advise them to to to start here uh what does the detect detections look like have you have you built anything out because I this is a good follow-up for my previous episode the AI defend framework has a lot of defenses in the detect pillar that um that you could uh probably use against this telemetry for let me pull it uh like yeah right over here there's a bunch of defense that I feel like this would this opens up just a whole world of opportunity now for uh uh detection mechanisms again against AI system use yeah yeah if if you look at the GitHub repo for for A10 there's uh there's a mapping to Dell's endpoint AI agent abuse it's like right in the middle under coverage and like as I was building this he put out this framework uh so specific to not because this won't detect you know like model poisoning or or things like that uh it's very specific to like the endpoint like where the actual like harness is running so I found that this framework that was specific to endpoint I tried to map what A10 detects to this.
00:47:47.039 --> 00:47:55.360
So so it doesn't do everything because like you you can't because otherwise I I would just be like blowing up you know everyone's computer.
00:47:55.679 --> 00:48:14.960
But I think I think like the yeah like I think that yeah this chart does a good job of good because in the blog I don't cover all these scenarios because building the the actual like wrapper to demonstrate these these scenarios was the most time consuming part of all this uh even with Claw just getting it all to work really difficult.
00:48:16.000 --> 00:48:33.360
But if you're if you're wondering like hey what does this even detect uh this is the section to look at because then you can click the the link and look at the actual framework and I I think this is a little bit more what's it called a little bit more maybe a little bit more practical than something like Atlas true.
00:48:33.679 --> 00:49:02.639
I mean there's 10 frameworks AI frameworks the new the new OWASP uh there's like it seems to be or databrex even has one nowadays it's hard it's hard to navigate not gonna lie yeah yeah and I know people love their frameworks that and like I do too but I also love like the practical part in you know what I mean like if someone tells me like hey just detect credential theft I wouldn't like how you know like what event ID do I need what query do I need like tell me more so I found that this framework was good for that.
00:49:02.960 --> 00:49:10.480
Did you use AI or did you use Claude to run the adversary uh behaviors as well against it?
00:49:10.960 --> 00:49:20.400
Yeah so so when I finished the the tool I told Claude like hey I want to have a a blog about these scenarios so build those for me.
00:49:20.639 --> 00:49:32.719
And it and it did but then it would always like it it was it was sweet right like the the only thing that wasn't clawed by code it was the blog the the blog I actually wrote you know the old school way by hand.
00:49:33.039 --> 00:50:17.519
Oh yeah how are how uh historic sucker I guess how artisanal yeah yeah and it's in the blog yeah yeah but yeah I think like the um yeah the the the man I forgot I forget what was the question sorry I lost my thought there for a second no the adversary emulation you were using it to to to like very for example run run this controlled initiator or permissive unattended yeah yeah yeah I would I would tell it to like set up those scenarios and it would try but then I would have to actually run the scenario from a different cloud session oh which had like the guardrails and I found like Opus 5 especially was was pretty sensitive to those so it wouldn't like interesting it wouldn't run my prompt injection.
00:50:18.239 --> 00:50:35.920
I think if you're maybe this is a bit of a tangent uh but if you're like in a purple team role I I would be so like you're just like kid to candy store these days with purple team scenarios that you that you can build around yeah clawed endpoint abuse.
00:50:36.239 --> 00:50:57.599
I think it's just like a a super fascinating area and and I could have spent probably more time like building the scenarios than the actual like app but which now that I have the app may maybe that's where I'll spend my efforts into actually like running like a full kind of purple team cycle and and showing the detections for that and the kind of queries that you can run and all that good stuff.
00:50:57.920 --> 00:51:02.880
Upload this to Caldera for me because they've got some pretty basic attacks over there.
00:51:04.159 --> 00:51:31.760
Yeah I found most uh like most purple team tools like that do like those bass tools right they have they have scenarios but it it's all like I I I always and and I hope they don't come after me for this but I've always found like I found that the juice isn't worth the squeeze you know what I mean like I I just set up all this like YAML and stuff and it's like running who am I for me you know like I I could do that myself I don't need uh I don't need cab there I think that's why I built like TTP runner.
00:51:32.079 --> 00:51:35.440
I don't know if you've seen I yeah no TTP runner is fantastic.
00:51:35.679 --> 00:51:56.239
Yeah yeah that that's why I built it because I I just wanted to like tell me like hey run run this thing and just have you know that now that the agent could do it uh it it just does it for me now that's amazing well I you're always welcome back to talk about any any one of these other projects like this is this is fantastic.
00:51:56.320 --> 00:53:02.159
I mean we only scratch the surface here with the novelty of it all of course uh and uh last last thing I wanted to touch on is like what's what is next for uh for 810 it do you are you gonna break in I know that mac os is still in progress uh you know what what what's the roadmap for for this I guess the purple team stuff yeah I would love to build like a an actual simulated rundown with it that looks a little bit more real realistic than than the stuff I have in the blog but um yeah I would love to have like feedback on it so if anyone actually tries it I'm sure there's bugs and stuff so I would love to like run it on more systems than just my lab and yeah I think the next step is keep iterating on this and keep seeing how like I'm I'm really curious for the next supply chain attack or whatever like I almost want to spin up a VM with this and just run it and see what it finds you know like whether it actually gives me the telemetry that I thought about honey caught your VM yeah and then with this with the this telemetry turned on yeah I might do that.
00:53:02.239 --> 00:53:02.639
There you go.
00:53:02.880 --> 00:53:05.039
Now you give me the idea for V2 of A10.
00:53:05.119 --> 00:53:05.599
There you go.
00:53:05.679 --> 00:53:35.119
Yeah but yeah it's something like that I I just want to see how it works with like a real like you know like a real world kind of like um attack scenario and whether actually gives you the telemetry that I thought it would uh you know a lab is always one thing right when when you're doing like synthetic tests and setting it up all perfectly and and timing and all that and it all works and then you're like oh cool and then you go to try it and prod and it doesn't so yeah I think that's probably the next step for it.
00:53:35.599 --> 00:53:58.320
If you're gonna go if you're gonna be at Black Hat or or Def CON I'm sure I you know you you just by asking around you always find I can like a SMB type of environment that maybe would be like pretty useful to use that has just enough endpoints that's just a little bit little a little bit of a makes it a a controllable scale.
00:53:58.639 --> 00:54:02.960
Yeah that's a good idea I might just run into somebody um so I'll I'll point them your way.
00:54:03.199 --> 00:54:20.079
Yeah please do please do yeah I I I'd love to actually run this on it and see if it even blows up it might be like too noisy for for for someone who uses claude all day you know like a heavy cloud user it might just be like nah this is not you know the the log file fills up too quickly or the telemetry is not useful.
00:54:20.639 --> 00:54:21.199
I don't know.
00:54:21.360 --> 00:54:27.920
It's hard to say without you know access to to that kind of like production uh data that I don't have in my lab.
00:54:28.079 --> 00:54:32.559
But I think even if the tool like doesn't work uh I think the idea is still cool.
00:54:32.880 --> 00:54:48.639
It is that like it gets picked up on yeah oh my gosh it absolutely has a place for it in the community and I can't thank you enough for bringing this out to us and uh we want to we want to see we want to continue seeing obviously more of it so absolutely keep doing what you do.
00:54:48.960 --> 00:54:49.920
Thank you thank you so much.
00:54:50.000 --> 00:54:56.880
Yeah definitely uh yeah hit me up if you try it if it doesn't work if it works yeah hit me up either way uh I'd love to talk shop about it.
00:54:57.199 --> 00:55:01.119
Let us know and of course all of this will be available in the show notes.
00:55:01.280 --> 00:55:05.519
Thank you so much Anton for coming on the pod I know I've been after you for thank you for quite a while.
00:55:06.719 --> 00:55:20.559
Thank you you said no if you remember you were like no I don't know well you were like I don't know maybe maybe in a few months like tell them get come back to me in a few months and but I'm glad we waited because now we've we've got this incredible tool.
00:55:20.719 --> 00:55:26.000
If you're listening please download subscribe where can we follow your newsletter blog content?
00:55:26.480 --> 00:55:46.079
Uh just Antonlovesdmb.com excellent uh and if you are and if your EDR can't see uh the process or the activity that your codex agents are running you're only obviously only getting half the story good check out a 10 and until next time keep engineering