WEBVTT
00:00:12.000 --> 00:00:20.960
Welcome back to Detection Dispatch, a show where we go beyond the alert and into the craft of the engineers, uh building the detections that actually hold up under fire.
00:00:21.280 --> 00:00:32.799
Every team I talk to right now, I feel like is either building an agent, buying a stag of agent swarms, talking about their frontier model comparisons with the release of Kimmy K.
00:00:33.840 --> 00:00:38.799
But the threat frameworks, it seems like they caught up pretty fast, right?
00:00:38.880 --> 00:00:44.880
The miter, I guess OWASP now has its own uh agentic top 10 and then Maestro as well.
00:00:45.200 --> 00:00:51.119
There's almost so many that we as detection engineers now have to keep track of.
00:00:51.600 --> 00:00:57.840
We're going to get into today something that can help us navigate it all, quite literally.
00:00:58.079 --> 00:01:00.079
Uh joining me today is Edward Lee.
00:01:00.159 --> 00:01:01.679
Um, Edward, welcome to the show.
00:01:01.840 --> 00:01:03.359
I'm so excited to have you on.
00:01:03.520 --> 00:01:08.640
I've been after you for a while, chasing the time zones, and we're finally on the same ish time zone.
00:01:08.959 --> 00:01:09.519
Yes, yes.
00:01:09.599 --> 00:01:10.719
Thank you very much, Alex.
00:01:10.879 --> 00:01:11.680
Um, hi everyone.
00:01:11.840 --> 00:01:13.040
My name is Edward Lee.
00:01:13.200 --> 00:01:17.439
I'm currently the uh CEO and founder of AI BFEN Labs.
00:01:17.519 --> 00:01:19.840
And yeah, I like what Alex introduced.
00:01:20.000 --> 00:01:27.680
I've been working in AI security field for some time, and I hope uh my work can contribute a little bit back to the community.
00:01:27.840 --> 00:01:42.879
And my eventual goal is to democratize the AI security so that everyone, including big companies, including small companies, any organizations, can adopt the best in-class AI security concepts and put them into practice.
00:01:43.120 --> 00:01:47.680
Your work is already creating impact and making its rounds around our industry.
00:01:47.840 --> 00:01:55.840
I've just saw last night, it seems like there's a bit of iteration, a similesque type of defense framework, which I want to get into.
00:01:56.000 --> 00:01:59.840
But of course, we always want to know uh how did you get into this space?
00:02:00.000 --> 00:02:03.439
What was your journey into the cybersecurity industry like?
00:02:03.680 --> 00:02:04.719
Sure thing, sure thing.
00:02:04.799 --> 00:02:05.040
Yes.
00:02:05.120 --> 00:02:09.360
So I've been in cybersecurity for about 14, 15 years from now.
00:02:09.520 --> 00:02:28.879
At the very beginning, uh, I was a security operations engineer and working on the actual tuning of the firewall rules, uh, setting up how to setting up locks, uh, how to how they get consumed into the systems, setting up the detection, the detection systems for enterprises, enterprise networks, things like that.
00:02:28.960 --> 00:02:30.800
And then detection engineer yourself.
00:02:30.960 --> 00:02:31.120
Yes.
00:02:31.280 --> 00:02:33.280
Because this is a space for detection engineers.
00:02:33.439 --> 00:02:36.159
Did you do it across like what kind of environments?
00:02:36.400 --> 00:02:41.199
Mid-size environments, 100, 500 employees, or like the large enterprise?
00:02:41.439 --> 00:02:43.759
Yeah, it was a mid-sized enterprise at the time.
00:02:43.840 --> 00:02:45.520
That's where my uh first job was.
00:02:45.680 --> 00:02:47.919
I was a security operations engineer at Box.
00:02:48.319 --> 00:02:51.840
Yeah, yeah, for a couple of months, which I learned a lot.
00:02:52.000 --> 00:03:06.080
And then, yeah, and then I moved on to the security architecture place into my next company, which is Akamai Technologies, where I served as a security solutions architect as well as enterprise security architect for almost five years.
00:03:06.240 --> 00:03:15.599
And before I move on to another space, which is cloud security, which is more interesting in a sense, but uh more things to look at uh at the same time.
00:03:15.840 --> 00:03:25.199
So the next job that I went to was the security and trust advisory at Google, uh, where I served as the post sales security advisory person.
00:03:25.360 --> 00:03:27.759
Um the GCP, the G Suite.
00:03:27.919 --> 00:03:35.520
Back then it was G Suite, now it's workspace, but back then G Suite, uh cloud identity, cloud armor, security consultations.
00:03:35.680 --> 00:03:35.919
Yeah.
00:03:36.080 --> 00:03:42.080
And that that was a good experience when you can get into the space where uh cloud security was uh was hot.
00:03:42.159 --> 00:03:42.400
Yeah.
00:03:42.560 --> 00:03:43.599
Oh, it is still hot.
00:03:43.759 --> 00:03:44.400
It is still hot.
00:03:44.639 --> 00:03:45.199
Oh, 100%.
00:03:45.599 --> 00:03:46.879
Hot hotter now than ever.
00:03:47.120 --> 00:03:56.719
Uh but it seems like you made uh sort of this rotational wave around all the the different parts of security, like you did from the firewall, from the cloud.
00:03:56.879 --> 00:03:58.800
Have you ever touched endpoint at all?
00:03:59.120 --> 00:04:00.879
And point, not in my career.
00:04:01.039 --> 00:04:06.240
That's uh something that maybe worth exploring as the as the next step.
00:04:06.319 --> 00:04:08.639
Maybe we'll we'll we'll see what happens.
00:04:08.960 --> 00:04:17.600
Well, my philosophy or my approach to my career is that at uh from the beginning, there are a lot of things that you need to understand.
00:04:17.759 --> 00:04:19.920
So you want to exactly, exactly.
00:04:20.079 --> 00:04:35.759
And to be a good expert or a good leader in this field, you need to get your hands dirty into different uh fields before you can, you know, fully understand that and then you know race up to the next level to uh lead the different fields of security.
00:04:35.839 --> 00:04:37.519
So that that's that's my thought and approach.
00:04:37.759 --> 00:04:38.079
Absolutely.
00:04:38.160 --> 00:04:46.399
I've always gotten along better with leaders and CISOs that are more on the technical side of things, that truly understand things.
00:04:46.560 --> 00:04:47.839
They've walked the walk before.
00:04:48.000 --> 00:04:51.839
Former practitioners live by the trade of it all, like the trade craft.
00:04:51.920 --> 00:04:55.759
I have found them to be the most pleasurable people to work with.
00:04:56.560 --> 00:05:09.519
At a certain level, you will realize that only if you know something in detail, yeah, that you will understand the good side of it, the best side of it, the challenge of it.
00:05:09.839 --> 00:05:18.079
And then you can you can relate to the engineers and the actual practitioners and then tell them the right thing to do.
00:05:18.480 --> 00:05:29.120
Otherwise, you will, you know, if you don't have the practical experience, it's from time to time you will give them some guidance that don't work as expected, which we don't really want.
00:05:29.279 --> 00:05:30.160
So that's true.
00:05:30.240 --> 00:05:35.600
And I also find that your team, the team that you're leading, uh just also just respects you more.
00:05:35.680 --> 00:05:40.319
I don't know if respect is the right word, but just sees you have a better working relationship, I guess.
00:05:40.399 --> 00:05:47.439
Uh, but also the caveat, the opposite side of it, is that being too in the weeds sometimes can lead through can lead to rabbit holes.
00:05:47.519 --> 00:05:55.839
So just still keeping that touch of reality with the bigger picture, too, is like this this fine line that we tend to play like push and pull with.
00:05:56.480 --> 00:05:57.439
Exactly, exactly.
00:05:57.600 --> 00:06:08.879
One of the best leaders that I work with, uh actually in my previous row at JP Morgan, I respected him a lot in the sense that uh whenever we went through uh projects altogether, right?
00:06:08.959 --> 00:06:12.160
There are things that I could probably properly check, tracked.
00:06:12.240 --> 00:06:14.160
Uh, there are things that I lost track, right?
00:06:14.240 --> 00:06:18.560
And when I reviewed that with the leader, uh he will be like, okay, you did not track this.
00:06:18.720 --> 00:06:22.480
Let me show you these are the things, these are the systems that we can check into.
00:06:22.639 --> 00:06:25.120
Hmm, let me take a look with you at this right now.
00:06:25.279 --> 00:06:26.959
Hmm, this, this, and that.
00:06:27.120 --> 00:06:33.120
Okay, now you can find A, you can find B, you can find C to get the progress of the next step.
00:06:33.279 --> 00:06:49.120
I love this style that uh no blaming and uh giving clear instructions with the solid knowledge and understanding about how system how system works within one organization and use that to lead people with by example, not by authority.
00:06:49.360 --> 00:06:50.959
That's by example.
00:06:51.199 --> 00:06:53.839
That's by example hits it right on the money.
00:06:54.000 --> 00:06:59.519
And funny you mentioned that you also came come from JP Morgan because you're now I can see it perfectly.
00:06:59.600 --> 00:07:05.600
You're cut from the same cloth as a lot of people I know that come out of JP Morgan, they have built incredible careers.
00:07:05.759 --> 00:07:10.160
Everyone I know that has done security there, I feel like they are incredibly talented.
00:07:10.240 --> 00:07:16.560
And also just it's a great place to build your cybersecurity career because JP Morgan has a lot of budget to work.
00:07:16.720 --> 00:07:17.759
You could try so many things.
00:07:17.839 --> 00:07:21.360
They're so open to trying all of these different products and systems.
00:07:21.439 --> 00:07:26.560
And I don't know, I feel like it's almost like unlimited budget to do what truly needs to be done.
00:07:26.639 --> 00:07:27.439
Or maybe I'm wrong.
00:07:27.600 --> 00:07:30.240
That's the perspective I get from the outside and who I've met.
00:07:30.480 --> 00:07:30.959
Yeah, yeah.
00:07:31.040 --> 00:07:33.279
I mean, that that's mostly correct.
00:07:33.439 --> 00:07:44.560
That uh yeah, JP Morgan did give me and my team, uh, and a lot of the cybersecurity folks within JP Morgan, a lot of rooms and a lot of budget to do a good cybersecurity.
00:07:44.720 --> 00:07:49.759
We had very good leaders that meets the uh cybersecurity and the technology space.
00:07:49.920 --> 00:07:52.800
So, you know, I I like that environment, I like that workplace a lot.
00:07:53.040 --> 00:07:53.680
Okay, love that.
00:07:53.839 --> 00:08:03.680
So, all of this journey and your path into the space now has led you to create all of these different facets and angles to uh this knowledge base you've basically built.
00:08:03.839 --> 00:08:05.759
So, tell us a little bit about what you've been working on.
00:08:06.000 --> 00:08:06.959
Sure thing, sure thing.
00:08:07.040 --> 00:08:13.519
So uh AIDFend, uh, you can find it on aidfend.net, which is the website that I show the whole framework.
00:08:13.680 --> 00:08:19.199
And you can also find it at AIDFend.dev, which is the GitHub repository for AIDFend.
00:08:19.360 --> 00:08:34.799
It's an open source AI security defense, it's an open source AI security defense knowledge phase for practitioners, for engineers to start with, to understand how exactly one can do AI security and one should do AI security.
00:08:34.960 --> 00:09:04.320
So the AI Defense itself is a framework that consists of different tactics and um 300 plus techniques, the uh practical defense techniques for AI security, and it cross map to nine frameworks at this moment, including uh OWASP, yeah, uh My Cherry Atlas, NIST Advisory Advisorial Machine Learning, yeah, NIST uh Cisco, Google, Scife, uh, Data Bricks AI security framework.
00:09:04.559 --> 00:09:09.919
These are the top and most mentioned uh AI security frameworks in the industry today.
00:09:10.080 --> 00:09:21.679
So it is cross mapped to those, so that I try to find a balance between the the practitioner's perspective versus what management level wants to see.
00:09:21.840 --> 00:09:24.480
So that's why uh that's how the cross map happened.
00:09:24.799 --> 00:09:34.960
I think that you you solved the the particular problem of there isn't like a single place to map every you know AI defense that is possible.
00:09:35.120 --> 00:09:40.399
And when there's literally so much, like you just mentioned more than five AI frameworks.
00:09:40.480 --> 00:09:44.879
So I think that that's literally the problem solving aspect of it here.
00:09:45.120 --> 00:09:52.799
When was the moment that you realized was there a story behind that or an incident, or was it just literally like death by a thousand disconnected frameworks?
00:09:52.960 --> 00:09:55.039
Yes, so that's exactly the point, right?
00:09:55.120 --> 00:09:58.240
Uh so as a security person, right?
00:09:59.200 --> 00:10:04.879
We are always under very tight timeline limitation and that budget issue, right?
00:10:05.039 --> 00:10:09.679
So we can only focus on the things that we want to focus the most.
00:10:09.840 --> 00:10:18.240
However, what happens is that back in my previous roles within financial services companies, people say, yes, we need to do AI security.
00:10:18.399 --> 00:10:19.200
How to do that?
00:10:19.360 --> 00:10:24.000
Let's find there are some good resources or frameworks, and they go ahead and find that.
00:10:24.080 --> 00:10:34.320
And then they realize, okay, great, we need to do uh prompt injection protection, we need to do data poisoning protection, we need to do intent drifting protection.
00:10:35.360 --> 00:10:42.480
Those are the things that exist in those frameworks that tells you that, hey, you need to do this so that you can secure your AI environment.
00:10:42.639 --> 00:10:45.039
And as a security person, I was happy to see that.
00:10:45.200 --> 00:10:47.679
And then the next question, Tom, okay, great.
00:10:47.840 --> 00:10:49.360
I need to do these 10 things.
00:10:49.440 --> 00:10:49.759
How?
00:10:49.919 --> 00:10:50.960
How do I do this?
00:10:51.120 --> 00:10:51.360
Right?
00:10:51.519 --> 00:10:55.600
And that's the next question that as practitioners we have to figure out, right?
00:10:55.759 --> 00:11:14.559
So we either have to go ahead to find if some frameworks mention their solutions about it, or we go online to see if there are commercial solutions for it, or if there are uh open source resources for this, if there are guidances provided by some institutions or community that help us to do that.
00:11:14.720 --> 00:11:24.799
Now, all of these are, but I myself, I find it extremely difficult to keep all of these in one place so that I it's easier for me to do my job, right?
00:11:24.879 --> 00:11:29.679
I, as a practitioner, I want to know okay, things that I want to know about prompt injection.
00:11:29.759 --> 00:11:30.559
Here's a goal.
00:11:30.720 --> 00:11:35.840
Here are the technical guidance, the technical steps that I can do or I should be doing.
00:11:35.919 --> 00:11:40.240
And if I don't have time to do it, here's the list of open source resources.
00:11:40.320 --> 00:11:46.559
And if I have more budget, here are the commercial solutions that can solve this problem, right?
00:11:46.720 --> 00:11:55.519
And yeah, at the end, what compliance or regulatory or framework or framework problems that this uh this uh technique solve.
00:11:55.759 --> 00:11:57.360
That's that's the thing that I want to know.
00:11:57.440 --> 00:12:00.320
And that's the thing probably I want to show to my management.
00:12:00.480 --> 00:12:19.039
So AI defense is the attempt to collect all of these together into one single source of truth that shows that AI security uh could be separated into tactics and techniques, and practitioners can use it to actually implement these with this information given.
00:12:19.120 --> 00:12:29.440
And the management can use this information as hey, now by implementing this, it fulfills this from the uh from the frame from the other frameworks.
00:12:29.600 --> 00:12:33.200
So that's the goal of the of the AIDF and framework.
00:12:33.600 --> 00:12:37.360
As you were building it out, did you notice or were you surprised?
00:12:37.440 --> 00:12:41.840
Um, because I'm sure there is a lot of overlap between a lot of these frameworks.
00:12:42.000 --> 00:12:43.519
How did you deal with the overlap from it all?
00:12:43.759 --> 00:12:44.720
Sure thing, sure thing.
00:12:44.879 --> 00:12:49.200
Yes, there are certainly some overlaps between different uh frameworks that we're seeing.
00:12:49.360 --> 00:13:06.159
I would say that the personally, and I liked them a lot, data bricks, their AI security framework, and Cisco's AI security and safety framework, they are they are narrower in scope, but they are closer in granularity to AI defense implementation first orientation.
00:13:06.320 --> 00:13:07.679
So I like that a lot.
00:13:07.919 --> 00:13:14.639
What happens is that from implementation perspective, these frameworks tell you what you need to do, right?
00:13:14.879 --> 00:13:27.039
But so far, at least from what I can see, uh, there's no single source of knowledge base that tells you exactly what, that tells you what you need to do, and in addition, how you can do it.
00:13:27.200 --> 00:13:37.200
So AIDFEN, if you think about many security frameworks, they are the backbone or skeleton of a good AI security practice.
00:13:37.279 --> 00:13:46.159
And uh AIDFEN fills in the muscle of those in those skeletons to show you how exactly you should do good AI security.
00:13:46.399 --> 00:13:47.919
Yeah, no, this is a lot more.
00:13:48.080 --> 00:13:52.559
We were just talking about this, it's a lot more practical information across the tactics.
00:13:52.720 --> 00:13:56.799
Like it truly tells me what I need to go build detections for, right?
00:13:56.879 --> 00:14:02.960
Like you were mentioning prompt injection, the drift detection, the monitoring of the policy enforcement of it all.
00:14:03.120 --> 00:14:19.759
Whereas I've I've seen some iterations of literally almost like the same thing, though the same way you you called this the AI defense framework, probably more for the C level, C level friendly, executive friendly view of kind of cross-reference against the NIST framework.
00:14:20.399 --> 00:14:22.879
And by the way, no, I will be sharing.
00:14:23.279 --> 00:14:27.679
I just permissions don't let me share right now, but I will be sharing like both of them.
00:14:27.840 --> 00:14:33.200
Um and uh and yeah, like like what am I supposed to do with AI security posture management?
00:14:33.279 --> 00:14:41.440
Like whereas in the AI defend framework, it's a lot more robust, a lot more concrete and specific of a behavior that you can actually build for.
00:14:41.679 --> 00:14:43.600
Yep, yep, exactly, exactly.
00:14:43.759 --> 00:14:46.720
And as you uh as what Alex you described, right?
00:14:46.799 --> 00:14:50.320
Yeah, do you think it's not trying to replace any of these frameworks, right?
00:14:50.480 --> 00:14:57.279
It's just trying to tell you, uh fill in the how of what's missing in these uh in these frameworks.
00:14:57.440 --> 00:14:58.080
So yeah.
00:14:58.240 --> 00:14:58.559
Yeah.
00:14:59.200 --> 00:15:00.799
Well, I gotta ask.
00:15:00.879 --> 00:15:11.279
I mean, we obviously love all and any community open source work, but it's not always the easiest to be a maintainer of an open source initiative or repo.
00:15:11.519 --> 00:15:12.960
So how are you doing this?
00:15:13.120 --> 00:15:15.679
Is it a personal, unaffiliated initiative?
00:15:15.840 --> 00:15:18.080
What's it been like maintaining this solo?
00:15:18.639 --> 00:15:22.159
Are you getting any kind of help or support, maybe miter-backed efforts?
00:15:22.399 --> 00:15:26.000
Yeah, for now it's it's my personal effort mostly, right?
00:15:26.159 --> 00:15:32.399
I do get the uh pull requests from some of the uh community uh contributors, which I appreciate a lot.
00:15:32.559 --> 00:15:32.879
Yeah.
00:15:33.120 --> 00:15:39.919
And including those latest information on the defensive side into the AI different framework.
00:15:40.080 --> 00:15:42.320
That's something that I uh that I'm passionate about.
00:15:42.480 --> 00:15:55.759
That I'm doing almost every day, doing the research and track on the latest AI security trends and see if any of the new AI security defense techniques could be included into the AI defense framework.
00:15:55.840 --> 00:16:00.720
That's something that I am passionate about and happy to do when I wake up every morning.
00:16:00.879 --> 00:16:02.720
So that's fantastic.
00:16:02.960 --> 00:16:03.600
Same pattern.
00:16:03.759 --> 00:16:23.279
I mean, this is this is what the the community thrives off of is in MITRETAC did this also too in the early days where one person or maybe it was even a small group, but they just decided that the industry needed a shared dictionary or shared language, shared vocabulary, and you just build it and share it and just not you know not wait for permission.
00:16:23.679 --> 00:16:24.399
Actually, exactly.
00:16:24.639 --> 00:16:30.159
Actually, if you look at the uh the structure of AIDFEN framework, uh it contains tactics, right?
00:16:30.320 --> 00:16:30.879
Yep, yep.
00:16:31.039 --> 00:16:32.559
Model, harden, detect, yep.
00:16:32.720 --> 00:16:32.960
Yes.
00:16:33.120 --> 00:16:35.360
So the concept of those seven tactics.
00:16:35.519 --> 00:16:41.279
Wait, yes, as this is model, harden, detect, isolate, deceive, evict, restore, those tactics.
00:16:41.440 --> 00:16:43.039
I didn't reinvent the wheel.
00:16:43.200 --> 00:16:47.759
Those are inspired by another popular framework created by Mitre.
00:16:48.000 --> 00:16:49.679
It's called Defense Framework.
00:16:49.840 --> 00:16:53.360
And it's the defense, they spelled it as D and number three.
00:16:53.440 --> 00:16:54.480
D three.
00:16:54.559 --> 00:16:54.639
Yeah.
00:16:54.720 --> 00:17:01.200
Yeah, I and D, uh, which is a uh a defense framework that mapped to its own mitre attack framework.
00:17:01.279 --> 00:17:02.879
I I love both a lot, right?
00:17:02.960 --> 00:17:12.079
I mean, both of them have good, there are good indications of how secure in the security world, how attacks and how defense should be done uh from different perspectives.
00:17:12.319 --> 00:17:24.160
So, and one of the starting points of me creating AI Defense is that Mitre also created this amazing Atlas framework, which focuses on AI security, right?
00:17:24.319 --> 00:17:30.960
Attack is the uh attack uh techniques and uh mechanisms on the on the general uh world.
00:17:31.039 --> 00:17:36.799
And mitra Atlas is the the AI version of the attack framework, right?
00:17:36.960 --> 00:17:42.000
And so far, I haven't seen a defense version for the AI security.
00:17:42.559 --> 00:17:43.440
Atlas, yeah.
00:17:43.680 --> 00:17:44.720
Exactly, for Atlas.
00:17:45.039 --> 00:17:54.079
So uh that's where I, you know, that's when I thought about that, which is in about around June and last year, June 2025.
00:17:54.240 --> 00:17:56.400
And I think, hey, there's there's a gap here.
00:17:56.480 --> 00:18:04.799
Uh in my organization, uh, we really wanted to understand how defense technicism, how defense techniques and mechanisms should be done on our end.
00:18:04.960 --> 00:18:12.160
So, you know, we figure out, hey, why don't we make it a collection of a knowledge base or database so that we know what to do?
00:18:12.319 --> 00:18:14.480
And that's how I started this project.
00:18:14.720 --> 00:18:20.640
And then I feel that hey, this is something that I could contribute back to the uh security community.
00:18:20.799 --> 00:18:23.039
So that thing uh got started.
00:18:23.200 --> 00:18:23.759
Yes.
00:18:24.160 --> 00:18:25.119
Well, you're right.
00:18:25.200 --> 00:18:28.480
The uh miter attack has become such a household name now.
00:18:28.640 --> 00:18:33.759
We get less love for defend, let alone miter atlas.
00:18:33.920 --> 00:18:34.240
Right.
00:18:34.400 --> 00:18:35.200
Hi, yeah.
00:18:35.359 --> 00:18:35.680
Uh-huh.
00:18:36.000 --> 00:18:39.440
Yeah, I think there are a lot of reasons behind it, right?
00:18:39.599 --> 00:18:48.559
Uh, and I do think that as a security practitioner, right, I am happy to learn about the attacking mechanisms, how bad guys attack a system.
00:18:48.720 --> 00:18:49.200
That's great.
00:18:49.359 --> 00:18:53.519
Uh, but more importantly, at the same time, my management will ask about okay, great.
00:18:53.680 --> 00:18:57.680
Now you have all of these knowledge about how our systems could be attacked.
00:18:57.839 --> 00:19:00.400
Tell me how you can protect against these attacks.
00:19:00.559 --> 00:19:08.720
Then I'm like, oh yeah, we can use this tool, we can use this solution, I can create a script to detect this and that, things like that.
00:19:08.880 --> 00:19:28.160
They're all good, but I think it will be the best if we can collect all of them into one single place, and um you can keep track on all different kinds of defensive techniques, and then uh and and then you can decide um which one you should use to put into practice so that you can protect your organization better.
00:19:28.319 --> 00:19:33.279
No, no, that's the the thought of the uh that's the thought of uh AI defense framework behind the scenes.
00:19:33.359 --> 00:19:38.720
So yeah, no, it absolutely has a place, and you got on top of it pretty fast.
00:19:38.880 --> 00:19:45.440
Have you gotten into translating this defense uh detection work and operationalizing and implementing it at companies?
00:19:45.599 --> 00:19:52.799
I want to get into that because that's something that I think that is on a lot of our mind as an actual detection engineer.
00:19:53.200 --> 00:19:58.400
I think we would probably live, if I'm looking at your framework, on the detect side of the pillar.
00:19:58.720 --> 00:19:59.839
Sure, sure, sure thing.
00:19:59.920 --> 00:20:01.920
Now, if you look at uh the framework, right?
00:20:02.079 --> 00:20:09.119
Detection, yeah, detect is a tactic that consists of uh 17, if I'm not mistaken, 16 or 17 techniques.
00:20:09.279 --> 00:20:18.480
So there are definitely in AI security world, there are definitely uh detection engineering things that uh will be included, at least in the AI defense framework.
00:20:18.640 --> 00:20:25.920
I can share some of the things that I put into the framework, which I think that from detection engineering perspective is super important.
00:20:26.240 --> 00:20:28.079
Prompt level telemetry, right?
00:20:28.240 --> 00:20:38.000
You will need um structural features like injected content length related to system prompt, uh delimiter row confusion patterns, and sudden shift in prompt entropy.
00:20:38.240 --> 00:20:40.880
Those are the top things you want to look at at prompt level.
00:20:40.960 --> 00:20:45.920
And in agency system world, right, tool call graphs is super important.
00:20:46.079 --> 00:20:56.400
Uh, which tools got involved, uh, in what sequence, with what parameters, whether the per the sequence deviated from the from the agent's expected task.
00:20:56.480 --> 00:21:03.200
Those are the things that you need to focus you need to focus on when it comes to detection engineering in the agentic security world, right?
00:21:04.240 --> 00:21:05.359
And reg, right?
00:21:05.519 --> 00:21:06.079
And RAG.
00:21:06.319 --> 00:21:06.720
Exactly.
00:21:06.880 --> 00:21:10.480
Retrieval provenance is super important for RAG, right?
00:21:10.559 --> 00:21:15.440
Which document chunks actually got pulled into the context and from where, right?
00:21:15.519 --> 00:21:27.119
This is the uh these these are the things that are included in the techniques under the AI deep and detects tactics that are super important to look at in the from the context of AI AI security.
00:21:27.359 --> 00:21:34.880
Yeah, as well as the framework, the view by framework, it literally has every single defenses by every single technique.
00:21:35.039 --> 00:21:35.920
Yes, exactly.
00:21:36.160 --> 00:21:36.480
Exactly.
00:21:36.640 --> 00:21:37.279
Yes, yep.
00:21:37.519 --> 00:21:38.240
Oh, yeah.
00:21:38.400 --> 00:21:41.200
I I had not gone to that part to that tab.
00:21:41.839 --> 00:21:43.519
No worries, no worries, no worries.
00:21:43.599 --> 00:21:59.759
You will love the different things that I love about the about AI defense framework is that I try to make it as operationalized as possible so that in addition to tactics view, I also provide several views if you want to do good AI security.
00:22:00.480 --> 00:22:10.960
So, for example, in Pillar's view, you can see that the defensive mechanisms, the defensive techniques, they are split into four different pillars.
00:22:11.039 --> 00:22:18.319
They have security, infrastructure security, model, algorithm security, and application security.
00:22:18.480 --> 00:22:18.720
Right?
00:22:18.880 --> 00:22:24.160
So Pillar Vie is for is for people in different roles, right?
00:22:24.240 --> 00:22:27.680
Let's say I am in this company and I focus on infrastructure security.
00:22:27.839 --> 00:22:28.960
Okay, what do I need to do?
00:22:29.119 --> 00:22:36.960
Now, if I come to the pillars view, organize, yeah, that's that that's a list of things that I should be aware of or I should be doing.
00:22:37.440 --> 00:22:41.759
Now in addition, the the phases view, right?
00:22:42.000 --> 00:22:53.759
It is using the uh the concept of separating um separating security defense techniques into different phases when it comes to AI system deployment.
00:22:54.079 --> 00:23:14.079
So when we uh when we're deploying AI an AI system, uh we have different phases from system design scoping to model training, uh training and building, pre-deployment validation, pre-deployment validation, production operation, incident containment, and system restoration improvement.
00:23:14.160 --> 00:23:15.920
There are different phases, right?
00:23:16.160 --> 00:23:23.519
And in some companies, we have people that that are in charge of uh different phases of the AI system deployment.
00:23:23.680 --> 00:23:24.640
That's number one.
00:23:24.880 --> 00:23:36.000
And number two is that uh we can now clearly see that okay, for this system, now it's under the phase of model training and building and hardening.
00:23:36.079 --> 00:23:40.319
Okay, so here's the line that we of defense techniques that we should be looking at.
00:23:40.480 --> 00:23:46.400
And when we move to the next phase, then it's the next line of defensive techniques that we um that we should be looking at.
00:23:46.559 --> 00:23:47.920
So that's that that's a concept there.
00:23:48.160 --> 00:24:00.240
Yeah, I also I keep thinking about what you said about um you you're you're offering various options, like from free all the way to commercial, because that story will be different, right?
00:24:00.319 --> 00:24:02.559
Depending on depending on who you are.
00:24:02.720 --> 00:24:15.759
For example, if we take this like prompt injection, uh what a detection engineer needs to look for, sometimes it's gonna have to be, you know, versus uh signature versus behavioral baseline versus maybe an LLM judging another one.
00:24:15.839 --> 00:24:20.960
And that that that that means different the the cost of doing each one is gonna be very much different.
00:24:21.279 --> 00:24:21.680
Exactly.
00:24:21.759 --> 00:24:22.160
Yep.
00:24:22.400 --> 00:24:30.000
So so they can either go with more of an open source route of detection or they can fully now go buy a tool that helps with maybe something like this.
00:24:31.519 --> 00:24:31.839
Yeah.
00:24:32.240 --> 00:24:32.640
Exactly.
00:24:32.799 --> 00:24:35.359
That's that's the uh the concept behind it, right?
00:24:35.440 --> 00:24:44.480
The the goal of this framework is that organizations or teams in different size in with different budgets can do good AI security, right?
00:24:44.559 --> 00:24:51.519
So and it also depends on how uh critical that specific defense related to your environment.
00:24:51.680 --> 00:24:55.519
Maybe to your environment, a certain defense is super critical.
00:24:55.680 --> 00:25:07.279
Then my recommendation in that framework is go ahead and buy the best in class, super expensive solution if that works well for you, and I list out those uh commercial solutions for your reference.
00:25:07.440 --> 00:25:27.759
If it's not as critical as a functionality device to you, then I provide the example code as well as the open source resources that you can refer to as well as building it, building one for yourself to protect your system for to protect that uh that field of um security risk at certain basic level.
00:25:27.920 --> 00:25:33.200
So that's a thought behind uh the different information given in this AI defense framework.
00:25:33.440 --> 00:25:35.920
I want to touch back on the RAG part of it all.
00:25:36.079 --> 00:25:42.240
You said there's a lot of a potential opportunity detection engineering opportunities for the RAG pipeline specifically.
00:25:42.400 --> 00:25:46.319
Where would you tell a detection engineer to start instrumenting first?
00:25:46.480 --> 00:25:50.880
Is it the retrieval layer, the prompt lead construction, the model output?
00:25:51.200 --> 00:25:54.319
Do we really just have to wait on the until the downstream side of it?
00:25:54.480 --> 00:25:56.000
Like that's you're too late at that point.
00:25:56.240 --> 00:25:58.400
Yeah, that's that's uh that's a great question.
00:25:58.640 --> 00:26:01.279
I I personally have a strong opinion about it.
00:26:01.440 --> 00:26:04.559
So uh feel free to disagree with me if you would like to.
00:26:04.720 --> 00:26:08.880
And obviously, in security, we want to do layered security, right?
00:26:08.960 --> 00:26:13.200
There's no single silver bullet to do good security, right?
00:26:13.279 --> 00:26:14.640
We want to do defense in depth.
00:26:14.720 --> 00:26:20.799
And but I do have this strong opinion that if you can only pick one, the retrieval layer is super, super important.
00:26:21.200 --> 00:26:28.160
Almost every serious rec compromise incidents that I've seen, they enter at retrieval layer, right?
00:26:28.400 --> 00:26:40.640
So if you're not logging what retrieved and you know from from where, uh you have no way to reconstruct an incident after the fact, even though, even though you have contained the uh the impact that it is causing.
00:26:40.799 --> 00:26:49.759
But if you there if you have no way to trace back to how it happened, how it got happened, then uh it could still happen again next time, right?
00:26:50.400 --> 00:26:50.880
Yeah.
00:26:51.599 --> 00:26:58.720
Yeah, so uh, you know, definitely for rack and pipeline, you want to do good security at uh retrieval layer.
00:26:59.279 --> 00:27:00.240
At the retrieval layer.
00:27:00.480 --> 00:27:01.599
At retrieval layer, yes.
00:27:02.640 --> 00:27:07.519
Prompt construction is also is also very super important.
00:27:07.839 --> 00:27:20.160
You want to have the visibility into how retrieve content got merged with the system prompt, and because that's where the injected uh instructions actually take effect.
00:27:20.480 --> 00:27:24.319
So uh that's I would say that's the second thing to look at.
00:27:24.400 --> 00:27:35.839
But retrieval layer is super important that uh you need to know and you need to understand where uh things were retrieved from at a very at a very basic level.
00:27:35.920 --> 00:27:36.559
So yes.
00:27:37.359 --> 00:27:40.960
Yes, and of course, just by keyword searching retrieval rag.
00:27:41.599 --> 00:27:42.480
Yep, yep.
00:27:43.359 --> 00:27:46.799
We'll we'll get we'll we'll we'll get populate populated some things.
00:27:46.880 --> 00:27:55.519
I'm already seeing some harden and some detect things, permission aware retrieval, of course, always always at the identity part as well, like defense and depth.
00:27:56.319 --> 00:28:02.799
For for OWASP specifically, for so for the OWASP top 10, it seems like now Agent AI it's getting its own OWASP.
00:28:03.440 --> 00:28:15.119
Uh and so what do you think is fundamentally different about detecting an agent doing something malicious versus an LLM just generating bad text?
00:28:15.759 --> 00:28:16.480
Yeah, yeah.
00:28:16.720 --> 00:28:21.279
So uh I think it's uh it's much closer to insider threat detection than malware detection.
00:28:21.359 --> 00:28:22.240
That's my thought, right?
00:28:22.400 --> 00:28:30.240
An agent is it is closer to a legitimate credentialed user that can be socially engineered by prompt, right?
00:28:30.319 --> 00:28:31.599
By malicious prompt.
00:28:32.480 --> 00:28:44.480
And when they got socially engineered, they are led into the field of they can do malicious things within or outside of their original intention, uh, but with legitimate permission.
00:28:44.559 --> 00:28:49.680
That's the worst part of it, because they have in that in that scenario, they have the legitimate permission.
00:28:49.920 --> 00:28:50.240
Yeah.
00:28:50.400 --> 00:28:58.799
And personally, I think that actually reframes the detection engineering posture that you're looking for behavior that's technically authorized, right?
00:28:58.880 --> 00:29:02.559
It's authorized by the uh correct source.
00:29:03.200 --> 00:29:05.680
Yep, but they are contextually wrong, right?
00:29:05.839 --> 00:29:15.039
Yeah, the agent using a tool it has every right to use in a way that it doesn't match the task that it was given, right?
00:29:15.759 --> 00:29:34.000
So my working point of view is that you look for the goal drift, which does the agent's tool call sequence stay coherent with the um original stated task, or does it branch off in a way that only makes sense if instructions were injected mixed stream from a external source?
00:29:34.160 --> 00:29:36.799
So yeah, that's my thought on how this should be done.
00:29:36.880 --> 00:29:37.039
Yes.
00:29:37.279 --> 00:29:51.200
Yeah, I think this is that's the conversation I feel like that I keep hearing across podcasts is how do you distinguish a true, you were talking about insider, a true human prompted request versus a malicious one.
00:29:51.359 --> 00:29:54.240
It's the line is getting blurry between what is what.
00:29:54.319 --> 00:29:58.480
I I've been I've been watching my own Claude Code request.
00:29:58.640 --> 00:30:00.559
They touch a bunch of my credential stores.
00:30:00.720 --> 00:30:09.440
And sometimes looking back, like I can't tell if it's just the tool doing the job or if it's sometimes getting abused because they look almost the exact same.
00:30:09.759 --> 00:30:10.880
Exactly, exactly.
00:30:11.039 --> 00:30:13.440
And uh it's super hard to detect.
00:30:13.599 --> 00:30:13.759
Yeah.
00:30:13.920 --> 00:30:26.799
So it's a hard job for detection engineering people that uh to distinguish between between the a legitimate prompt versus uh what's being maliciously injected into the system.
00:30:26.960 --> 00:30:28.799
So yeah, at different level, right?
00:30:28.880 --> 00:30:37.920
Uh you want to be able to detect the malicious or the not so malicious, but the prompts that could lead to goal drifting later on.
00:30:38.000 --> 00:30:39.759
Yeah, you want to be able to detect that.
00:30:39.920 --> 00:30:53.759
And at goal level, you want to check the agents or the uh MCP service that you tell that you you authorize uh um doing your work are actually doing the things that you're expecting.
00:30:54.319 --> 00:30:54.559
Yeah.
00:30:54.799 --> 00:31:02.240
So these are things that uh uh you know, a huge amount but but valuable job for detection engineering people.
00:31:03.359 --> 00:31:20.160
Oh uh do you think they should have their own set of credentials or like its own entity, its own employee ID, so that maybe on the governance side you you can keep track of them as like users and which one's an agent, which yeah.
00:31:20.240 --> 00:31:21.680
I mean should they have their own?
00:31:22.079 --> 00:31:26.400
Very similar to 10 or 15 years ago when we introduced cloud, right?
00:31:26.480 --> 00:31:28.160
That the the the cloud environment.
00:31:28.319 --> 00:31:31.680
We used to have human being account, now we have the service account.
00:31:31.920 --> 00:31:32.480
The service account.
00:31:32.640 --> 00:31:37.200
Which is used for the uh the the the system services and APIs.
00:31:37.359 --> 00:31:51.839
So I agree with that approach that uh from tracking perspective, it will be much easier and clearer if we have different if we have categorized identity for the agents and for different AI services.
00:31:51.920 --> 00:31:53.359
I agree with that approach.
00:31:53.759 --> 00:31:54.160
Yeah.
00:31:54.400 --> 00:31:59.440
No, if we think about the data feed of it all, I guess that maybe now it's getting better.
00:31:59.519 --> 00:32:07.519
I haven't, to be honest, looked at it in a while now, and I keep hearing that it's much better, that the Claude, specifically the OTEL telemetry.
00:32:07.759 --> 00:32:13.279
But before it used to just be like token usage and session duration.
00:32:13.440 --> 00:32:20.079
It's not really that uh helpful information to build a detection per se, a rule.
00:32:20.880 --> 00:32:28.240
What would you say is probably the best visibility if someone had let's say zero visibility into their company's LLM usage?
00:32:28.400 --> 00:32:30.480
Where would you get started with logging?
00:32:31.039 --> 00:32:34.319
What hotel you do would you have them stand up?
00:32:34.720 --> 00:32:37.599
Uh what kind of variables would you be looking for?
00:32:37.920 --> 00:32:38.480
Sure thing.
00:32:38.559 --> 00:32:38.799
Yeah.
00:32:38.880 --> 00:32:40.240
Uh if I have to choose, right?
00:32:40.319 --> 00:32:47.519
Uh I would say start with the tool call logs as well as the uh uh the API call login, right?
00:32:48.000 --> 00:32:52.720
And uh this can be done at the uh at the proxy and gateway level, right?
00:32:52.880 --> 00:33:06.799
Uh whatever sits between you and your users and the model provider, you know, if you have the uh gateway and proxy setup, then you will have visibility into the tool call history, uh the tool call locks and the API call logs.
00:33:06.960 --> 00:33:16.960
Yeah, I mean uh from there it gives you the uh the request volume, which models are used, yeah, which tools got involved, you know, without touching the content at all.
00:33:17.119 --> 00:33:23.440
So I think that's a good thing that you could use that you track the tool call and API call at the proxy level.
00:33:23.599 --> 00:33:32.240
And I think that the layer in retrieval provenance login, if you're running around, that is also super important, right?
00:33:32.400 --> 00:33:37.279
And then probably tool call parameter login if you're running uh agents, right?
00:33:37.359 --> 00:33:40.559
Uh that's that's the next things that I I'll be looking at.
00:33:40.640 --> 00:33:44.960
Yeah, much practical than the token usage, token maxing of it all.
00:33:45.200 --> 00:33:48.400
That doesn't tell you anything meaningful from security perspective.
00:33:48.559 --> 00:33:51.119
So, well, it is a well, the indicator, right?
00:33:51.279 --> 00:33:57.359
You can you do see that uh if there's a search on the usage of token, that that means something.
00:33:57.680 --> 00:34:04.160
But you know, models uh behave in ways that people cannot fully uh expect, right?
00:34:04.240 --> 00:34:08.880
So there are some legit tasks that uh cause a lot of usage in tokens.
00:34:09.039 --> 00:34:13.840
You think that's weird, but if you look at it, you know, it's normal, it's just how that works.
00:34:14.000 --> 00:34:18.880
So it's an indicator, but it's not the uh most useful indicator in the security world.
00:34:18.960 --> 00:34:19.760
That's what that's with.
00:34:20.079 --> 00:34:23.599
Yeah, that I've seen some good research from Trail of Bits.
00:34:23.760 --> 00:34:33.360
Uh, this would only apply if you if you have an MCP, since MCPs has now become more of the default way that agents are or talking to each other, to tools.
00:34:33.519 --> 00:34:41.360
And a trail of bits has like a like a like an MCP wrapper that will track all of the input-output requests into the MCP.
00:34:41.599 --> 00:34:45.280
Yeah, which but again, not that's not the only way agents can be called.
00:34:45.679 --> 00:34:46.079
That's true.
00:34:46.239 --> 00:34:46.719
That's true.
00:34:46.880 --> 00:35:01.679
So I do think that there are several good contributors in the security world and in the security field that there are several tools that try to that try to collect as as many logs in a meaningful way in detail at different levels.
00:35:01.840 --> 00:35:07.039
So I think there are several very problem and promising open source projects for that.
00:35:07.280 --> 00:35:08.000
Okay, yeah.
00:35:08.320 --> 00:35:15.280
I love seeing that because you know detection engineering is so important that it's almost the very first step.
00:35:15.360 --> 00:35:18.480
Well, second step after uh after log collection, right?
00:35:18.639 --> 00:35:26.559
You have to collect all the signals, but everything that you need to operationalize that comes after the actual detection happens.
00:35:26.800 --> 00:35:30.719
So yeah, that's something that I myself, I'm, you know, I look forward to happening.
00:35:30.880 --> 00:35:39.679
I look forward to see more tools get into this field that people can leverage to do detection engineering and the follow-up operationalization.
00:35:39.840 --> 00:35:44.000
So yeah, no, totally like the same way, like what Sysmon was for Windows.
00:35:44.159 --> 00:35:45.599
That that's what I'm I'm looking for.
00:35:45.840 --> 00:35:47.760
Trell of Bits was like the closest thing.
00:35:47.840 --> 00:35:50.159
This is this is also a step in that direction.
00:35:50.320 --> 00:35:56.559
Uh, there doesn't seem to be like a an open source telemetry enrichment type of thing for this yet.
00:35:56.639 --> 00:36:03.280
Uh, I think everyone is still building kind of bespoke logging for vendor uh or the tool calls, like you said.
00:36:04.000 --> 00:36:07.039
Yeah, have have a lot more to explore there.
00:36:07.760 --> 00:36:09.760
Uh we're getting to the end of the episode.
00:36:10.000 --> 00:36:23.440
I want to kind of close with what how exactly our detection engineers, who's never touched security, maybe has a backlog, you know, a mile long before they even start to get into some of the some of the techniques.
00:36:23.519 --> 00:36:28.079
Uh, what would you tell them to implement first, would you say, to get their best return for their efforts?
00:36:28.320 --> 00:36:29.280
Sure thing, sure thing.
00:36:29.440 --> 00:36:40.000
So I would say that depending on what your role is, depending on which phase that you're at when it comes to the AI deployment at your in your organization, right?
00:36:40.159 --> 00:36:47.519
And depending on what your management cares about in terms of uh in terms of framework or in terms of best practices guide, right?
00:36:47.679 --> 00:37:01.360
You can always refer to the uh the detect tactics under under AI defense and cross-map that with your with your role responsibility as well as the phase that your organization or the application that you is at.
00:37:01.519 --> 00:37:38.880
And and you know, look in look at the the those techniques that you think that will to your work when it comes to implementation or operationalization, and then exploring the tools that I mentioned in those in those under those techniques, which I think that uh it will be most beneficial for you to understand the tool first, and then you can decide how important that is to our organization, and then you can decide, okay, I want to build it myself, uh, or I want to build it within my team, or oh, that's a lot of things to do, and we have the budget, so let's you know, buy a good solution for that.
00:37:39.039 --> 00:37:46.639
So, you know, just remember that the the goal of this framework is not selling you things, right?
00:37:46.719 --> 00:37:55.519
It's not it's not another framework that tells you that tells you, hey, you need to do this, but without telling you how to do that, right?
00:37:55.840 --> 00:38:21.440
The goal of this is depending on your the reality that you're at, yeah, uh it tells you the different options that you can do based on the resources, based on the budget, based on the uh the the criticality of that specific technique, uh the that specific AI security uh defense perspective in your organization.
00:38:21.599 --> 00:38:22.480
So that's a goal.
00:38:22.639 --> 00:38:37.119
So if this uh the if if the tool or if the framework could help you a little bit in doing better AI security, then I am so happy about that.
00:38:37.280 --> 00:38:38.320
That's the goal of this.
00:38:39.199 --> 00:38:49.840
Spoke as like a true engine uh detection engineer, is starting with what you have, what you prioritize, like the good key, what your business landscape looks like before trying anything out.
00:38:50.480 --> 00:38:55.039
See, I I am I'm immediately drawn to the to the rogue agent discovery.
00:38:55.360 --> 00:38:56.239
Yep, yep, yep.
00:38:56.480 --> 00:38:58.880
You should uh yeah, that that's a good part.
00:38:59.039 --> 00:39:06.320
I uh I think I spent almost two days researching on that topic and then and then worked on that that part.
00:39:06.480 --> 00:39:13.199
That's uh that's a good part that I have that I I have pretty strong memory around uh the creation of the part.
00:39:13.440 --> 00:39:13.840
So yeah.
00:39:14.320 --> 00:39:22.559
Do you do you think the role of an um an AI detection engineer will will c will start to pop up in like LinkedIn?
00:39:23.360 --> 00:39:23.760
A lot.
00:39:24.000 --> 00:39:24.880
A lot, yes.
00:39:25.039 --> 00:39:36.480
So we are now talking about uh you can see a lot of job jobs around uh AI governance or even AI security governance that are open up today, right?
00:39:37.119 --> 00:39:39.440
Yeah, which which is great, right?
00:39:39.599 --> 00:39:49.679
We uh as more and more companies and organizations are deploying AI systems and no governance is the next thing that people start looking at, right?
00:39:49.920 --> 00:39:52.719
Yeah, okay, what do we need to govern?
00:39:53.039 --> 00:39:53.199
Right?
00:39:53.360 --> 00:39:54.880
That's the next question, right?
00:39:55.039 --> 00:40:05.920
So follow up on that question, the next thing we would need to understand what to govern before we can do good governance.
00:40:06.400 --> 00:40:06.639
Yeah.
00:40:06.960 --> 00:40:13.039
Where do we get the data of understanding what to govern from detection engineering, right?
00:40:13.119 --> 00:40:29.039
So I think the role, I think the role of the the the detection engineering will be uh will be thriving a lot, simply because of the need of more clarity and more governance are needed in the AI and both both AI and the uh agentic AI world.
00:40:29.199 --> 00:40:44.400
So uh that's that's uh that's something that I I would put a lot of effort into my uh my framework as well to help the uh detection engineering people getting more resources and getting more better guidance on how to do good security.
00:40:44.639 --> 00:40:44.880
Yeah.
00:40:44.960 --> 00:40:55.840
For for my for my more senior detection engineers, do you think that their access to the way they consume this framework is it mcp friendly a ball?
00:40:56.000 --> 00:41:06.639
So like if they have a whole, let's say, you know, codified CI CD pipeline as a part of their coverage assessment and everything is API driven and a part of this like agentic framework.
00:41:06.800 --> 00:41:15.760
Is is this front is this MCP friendly where they can query back into their system and their setup where they don't have to, let's say, you know, not go to the UI.
00:41:15.840 --> 00:41:17.199
They could do it in a headless way.
00:41:17.440 --> 00:41:18.079
Yeah, yes.
00:41:18.320 --> 00:41:24.079
So this framework, AI Defense, it does have uh come with the uh the MCP service, which I built.
00:41:24.239 --> 00:41:43.280
So if you go to AIDFend.net, you will see that there's a uh on the top, there's a UMCP server and service of AIDFEN, which you can integrate with your CI CD or operation operations pipeline uh to make better use of the knowledge in the uh that that is built in in AIDFAN.
00:41:43.519 --> 00:42:06.800
Uh also on the AIDFEN.net, the the website, uh I integrate it with the the web mcp, which means that if you use your your browser's AI tool and ask your browser's AI tool on the AIDFAN.net saying, hey, I want to do this, this, and that watch in, what's in here in this in this in this AIDFEN.net website, it acts like an MCP service.
00:42:06.960 --> 00:42:17.599
So it will give you the answer back with a more structured and accurate way, uh, as how we build it in the AIDFEN.NET uh framework structure.
00:42:17.760 --> 00:42:28.960
So yeah, it has the AIDFEN has both uh the MCP service provided, which you can integrate directly, or on the website it has it is integrated with the web mcp.
00:42:29.280 --> 00:42:30.079
Fabulous, fabulous.
00:42:30.159 --> 00:42:32.079
Yeah, and it's all linked to your GitHub here, I see.
00:42:32.239 --> 00:42:32.800
Yeah, yes.
00:42:32.960 --> 00:42:34.639
Well, thank you so much, Edward.
00:42:34.800 --> 00:42:40.320
I think we've covered quite a bit in terms of your repo, your contribution, your initiative to the community.
00:42:40.480 --> 00:42:44.800
I'm so glad we were finally able to catch up on the same time zone.
00:42:45.039 --> 00:42:47.199
Actually, are you in town for Black Hat?
00:42:47.519 --> 00:42:49.519
Not this year, so uh this year.
00:42:49.840 --> 00:42:57.599
Yes, I will be traveling in the uh Asia Pacific region, speaking in different conferences in August and in September.
00:42:57.760 --> 00:43:07.679
But I surely will keep my eye on the latest AI defense security trend and uh keep working on refining my the the AI defense framework.
00:43:07.760 --> 00:43:08.719
So that that's for sure.
00:43:09.039 --> 00:43:11.440
What's your plans for V2 of this?
00:43:11.599 --> 00:43:12.800
Just continuing refinement?
00:43:12.960 --> 00:43:15.119
Are you gonna productize this maybe?
00:43:15.519 --> 00:43:19.599
That's in the plan, but I think either I productize it or not.
00:43:19.679 --> 00:43:22.719
Uh the goal is to make it more operationalized, right?
00:43:22.960 --> 00:43:23.280
Okay.
00:43:23.519 --> 00:43:26.480
So then I think that let me put it that that way.
00:43:26.639 --> 00:43:34.000
That if we if I if there's a way that I could make aitfant.net more operation operationalizable.
00:43:34.079 --> 00:43:35.119
Is that a is that a word?
00:43:35.280 --> 00:43:36.000
Operationalized.
00:43:36.079 --> 00:43:36.639
Yeah, yeah.
00:43:36.880 --> 00:43:38.079
Operationalizable.
00:43:38.159 --> 00:43:39.599
Yeah, then I'll do it.
00:43:39.760 --> 00:43:42.079
Either open source it or making it a product.
00:43:42.159 --> 00:43:42.960
I'm happy to do it.
00:43:43.039 --> 00:44:06.079
Uh, in fact, there are some projects underway that one of them is that I am trying to grouping, trying to group different use cases into um, you know, trying to put different uh AI different techniques into groups so that you could, based on your use case, you could understand faster and easier what to do if you want to implement security force in in specific use case.
00:44:06.239 --> 00:44:08.639
Let's say you're doing uh rack, right?
00:44:08.800 --> 00:44:17.760
Then one of the projects that I'm working on is okay, if you're doing rack, out of these 200 techniques, these 30 techniques are the top ones that you want to look at.
00:44:17.920 --> 00:44:19.280
So it's easier for you.
00:44:19.519 --> 00:44:30.400
It categorize things for for people in different roles or working on different projects so that you can utilize and leverage the knowledge in the uh AI Defense framework better.
00:44:30.719 --> 00:44:31.199
Fantastic.
00:44:31.280 --> 00:44:32.639
That's I'm so happy to hear that.
00:44:32.800 --> 00:44:35.199
Well, uh, that has been AI Defend.
00:44:35.360 --> 00:44:38.000
Remember not to be confused by AI defense.
00:44:38.400 --> 00:44:45.760
There are there are several names from different companies, from different people around that, which I like a lot, right?
00:44:45.840 --> 00:45:03.519
I mean, that the more that we have the awareness that AI security is important, the more people that we have people, the more people that pour in and uh counsel people into this field, uh the more the the better, the better future that we have for knowledge and the technology in this field, which is something that we would love to see.
00:45:03.760 --> 00:45:06.320
And that's the spirit of community, which is what this is about.
00:45:07.599 --> 00:45:23.519
So thank you so much for the reminder that the fastest growing attack surface in most companies right now literally has a place where it could it can all live and have a shared language that we can yeah, that we can enforce and harden and build our detections against.
00:45:23.599 --> 00:45:29.280
So thank you so much for building this out in the open and for walking us through what it actually takes to operationalize it.
00:45:29.360 --> 00:45:34.960
And I look forward to continuing to see how you continue to operationalize it or make it more operationalizable.
00:45:35.280 --> 00:45:36.159
Yeah, sounds good.
00:45:36.320 --> 00:45:36.719
Sounds good.
00:45:37.039 --> 00:45:39.199
I really appreciate your comment, Alex.
00:45:39.360 --> 00:45:43.679
And yeah, I had a great time talking with you and the uh the audience here today.
00:45:44.000 --> 00:45:50.559
Likewise, to our listeners, if you're a detection engineer and you haven't looked at your own AI or agentic tooling as an attack service yet.
00:45:50.719 --> 00:45:54.079
I think this is a really fantastic place to start and you're prompt to go do that.
00:45:54.159 --> 00:45:56.800
I'll link all of the resources in our show notes.
00:45:56.960 --> 00:46:03.039
So go actually check out what you're logging and you know, not just bookmark it on and just save it for later.
00:46:03.199 --> 00:46:04.480
Literally actually tune in.
00:46:04.639 --> 00:46:07.360
So thank you all for joining us on Dispatch today.
00:46:07.599 --> 00:46:08.719
We'll see you next time.
00:00:12.000 --> 00:00:20.960
Welcome back to Detection Dispatch, a show where we go beyond the alert and into the craft of the engineers, uh building the detections that actually hold up under fire.
00:00:21.280 --> 00:00:32.799
Every team I talk to right now, I feel like is either building an agent, buying a stag of agent swarms, talking about their frontier model comparisons with the release of Kimmy K.
00:00:33.840 --> 00:00:38.799
But the threat frameworks, it seems like they caught up pretty fast, right?
00:00:38.880 --> 00:00:44.880
The miter, I guess OWASP now has its own uh agentic top 10 and then Maestro as well.
00:00:45.200 --> 00:00:51.119
There's almost so many that we as detection engineers now have to keep track of.
00:00:51.600 --> 00:00:57.840
We're going to get into today something that can help us navigate it all, quite literally.
00:00:58.079 --> 00:01:00.079
Uh joining me today is Edward Lee.
00:01:00.159 --> 00:01:01.679
Um, Edward, welcome to the show.
00:01:01.840 --> 00:01:03.359
I'm so excited to have you on.
00:01:03.520 --> 00:01:08.640
I've been after you for a while, chasing the time zones, and we're finally on the same ish time zone.
00:01:08.959 --> 00:01:09.519
Yes, yes.
00:01:09.599 --> 00:01:10.719
Thank you very much, Alex.
00:01:10.879 --> 00:01:11.680
Um, hi everyone.
00:01:11.840 --> 00:01:13.040
My name is Edward Lee.
00:01:13.200 --> 00:01:17.439
I'm currently the uh CEO and founder of AI BFEN Labs.
00:01:17.519 --> 00:01:19.840
And yeah, I like what Alex introduced.
00:01:20.000 --> 00:01:27.680
I've been working in AI security field for some time, and I hope uh my work can contribute a little bit back to the community.
00:01:27.840 --> 00:01:42.879
And my eventual goal is to democratize the AI security so that everyone, including big companies, including small companies, any organizations, can adopt the best in-class AI security concepts and put them into practice.
00:01:43.120 --> 00:01:47.680
Your work is already creating impact and making its rounds around our industry.
00:01:47.840 --> 00:01:55.840
I've just saw last night, it seems like there's a bit of iteration, a similesque type of defense framework, which I want to get into.
00:01:56.000 --> 00:01:59.840
But of course, we always want to know uh how did you get into this space?
00:02:00.000 --> 00:02:03.439
What was your journey into the cybersecurity industry like?
00:02:03.680 --> 00:02:04.719
Sure thing, sure thing.
00:02:04.799 --> 00:02:05.040
Yes.
00:02:05.120 --> 00:02:09.360
So I've been in cybersecurity for about 14, 15 years from now.
00:02:09.520 --> 00:02:28.879
At the very beginning, uh, I was a security operations engineer and working on the actual tuning of the firewall rules, uh, setting up how to setting up locks, uh, how to how they get consumed into the systems, setting up the detection, the detection systems for enterprises, enterprise networks, things like that.
00:02:28.960 --> 00:02:30.800
And then detection engineer yourself.
00:02:30.960 --> 00:02:31.120
Yes.
00:02:31.280 --> 00:02:33.280
Because this is a space for detection engineers.
00:02:33.439 --> 00:02:36.159
Did you do it across like what kind of environments?
00:02:36.400 --> 00:02:41.199
Mid-size environments, 100, 500 employees, or like the large enterprise?
00:02:41.439 --> 00:02:43.759
Yeah, it was a mid-sized enterprise at the time.
00:02:43.840 --> 00:02:45.520
That's where my uh first job was.
00:02:45.680 --> 00:02:47.919
I was a security operations engineer at Box.
00:02:48.319 --> 00:02:51.840
Yeah, yeah, for a couple of months, which I learned a lot.
00:02:52.000 --> 00:03:06.080
And then, yeah, and then I moved on to the security architecture place into my next company, which is Akamai Technologies, where I served as a security solutions architect as well as enterprise security architect for almost five years.
00:03:06.240 --> 00:03:15.599
And before I move on to another space, which is cloud security, which is more interesting in a sense, but uh more things to look at uh at the same time.
00:03:15.840 --> 00:03:25.199
So the next job that I went to was the security and trust advisory at Google, uh, where I served as the post sales security advisory person.
00:03:25.360 --> 00:03:27.759
Um the GCP, the G Suite.
00:03:27.919 --> 00:03:35.520
Back then it was G Suite, now it's workspace, but back then G Suite, uh cloud identity, cloud armor, security consultations.
00:03:35.680 --> 00:03:35.919
Yeah.
00:03:36.080 --> 00:03:42.080
And that that was a good experience when you can get into the space where uh cloud security was uh was hot.
00:03:42.159 --> 00:03:42.400
Yeah.
00:03:42.560 --> 00:03:43.599
Oh, it is still hot.
00:03:43.759 --> 00:03:44.400
It is still hot.
00:03:44.639 --> 00:03:45.199
Oh, 100%.
00:03:45.599 --> 00:03:46.879
Hot hotter now than ever.
00:03:47.120 --> 00:03:56.719
Uh but it seems like you made uh sort of this rotational wave around all the the different parts of security, like you did from the firewall, from the cloud.
00:03:56.879 --> 00:03:58.800
Have you ever touched endpoint at all?
00:03:59.120 --> 00:04:00.879
And point, not in my career.
00:04:01.039 --> 00:04:06.240
That's uh something that maybe worth exploring as the as the next step.
00:04:06.319 --> 00:04:08.639
Maybe we'll we'll we'll see what happens.
00:04:08.960 --> 00:04:17.600
Well, my philosophy or my approach to my career is that at uh from the beginning, there are a lot of things that you need to understand.
00:04:17.759 --> 00:04:19.920
So you want to exactly, exactly.
00:04:20.079 --> 00:04:35.759
And to be a good expert or a good leader in this field, you need to get your hands dirty into different uh fields before you can, you know, fully understand that and then you know race up to the next level to uh lead the different fields of security.
00:04:35.839 --> 00:04:37.519
So that that's that's my thought and approach.
00:04:37.759 --> 00:04:38.079
Absolutely.
00:04:38.160 --> 00:04:46.399
I've always gotten along better with leaders and CISOs that are more on the technical side of things, that truly understand things.
00:04:46.560 --> 00:04:47.839
They've walked the walk before.
00:04:48.000 --> 00:04:51.839
Former practitioners live by the trade of it all, like the trade craft.
00:04:51.920 --> 00:04:55.759
I have found them to be the most pleasurable people to work with.
00:04:56.560 --> 00:05:09.519
At a certain level, you will realize that only if you know something in detail, yeah, that you will understand the good side of it, the best side of it, the challenge of it.
00:05:09.839 --> 00:05:18.079
And then you can you can relate to the engineers and the actual practitioners and then tell them the right thing to do.
00:05:18.480 --> 00:05:29.120
Otherwise, you will, you know, if you don't have the practical experience, it's from time to time you will give them some guidance that don't work as expected, which we don't really want.
00:05:29.279 --> 00:05:30.160
So that's true.
00:05:30.240 --> 00:05:35.600
And I also find that your team, the team that you're leading, uh just also just respects you more.
00:05:35.680 --> 00:05:40.319
I don't know if respect is the right word, but just sees you have a better working relationship, I guess.
00:05:40.399 --> 00:05:47.439
Uh, but also the caveat, the opposite side of it, is that being too in the weeds sometimes can lead through can lead to rabbit holes.
00:05:47.519 --> 00:05:55.839
So just still keeping that touch of reality with the bigger picture, too, is like this this fine line that we tend to play like push and pull with.
00:05:56.480 --> 00:05:57.439
Exactly, exactly.
00:05:57.600 --> 00:06:08.879
One of the best leaders that I work with, uh actually in my previous row at JP Morgan, I respected him a lot in the sense that uh whenever we went through uh projects altogether, right?
00:06:08.959 --> 00:06:12.160
There are things that I could probably properly check, tracked.
00:06:12.240 --> 00:06:14.160
Uh, there are things that I lost track, right?
00:06:14.240 --> 00:06:18.560
And when I reviewed that with the leader, uh he will be like, okay, you did not track this.
00:06:18.720 --> 00:06:22.480
Let me show you these are the things, these are the systems that we can check into.
00:06:22.639 --> 00:06:25.120
Hmm, let me take a look with you at this right now.
00:06:25.279 --> 00:06:26.959
Hmm, this, this, and that.
00:06:27.120 --> 00:06:33.120
Okay, now you can find A, you can find B, you can find C to get the progress of the next step.
00:06:33.279 --> 00:06:49.120
I love this style that uh no blaming and uh giving clear instructions with the solid knowledge and understanding about how system how system works within one organization and use that to lead people with by example, not by authority.
00:06:49.360 --> 00:06:50.959
That's by example.
00:06:51.199 --> 00:06:53.839
That's by example hits it right on the money.
00:06:54.000 --> 00:06:59.519
And funny you mentioned that you also came come from JP Morgan because you're now I can see it perfectly.
00:06:59.600 --> 00:07:05.600
You're cut from the same cloth as a lot of people I know that come out of JP Morgan, they have built incredible careers.
00:07:05.759 --> 00:07:10.160
Everyone I know that has done security there, I feel like they are incredibly talented.
00:07:10.240 --> 00:07:16.560
And also just it's a great place to build your cybersecurity career because JP Morgan has a lot of budget to work.
00:07:16.720 --> 00:07:17.759
You could try so many things.
00:07:17.839 --> 00:07:21.360
They're so open to trying all of these different products and systems.
00:07:21.439 --> 00:07:26.560
And I don't know, I feel like it's almost like unlimited budget to do what truly needs to be done.
00:07:26.639 --> 00:07:27.439
Or maybe I'm wrong.
00:07:27.600 --> 00:07:30.240
That's the perspective I get from the outside and who I've met.
00:07:30.480 --> 00:07:30.959
Yeah, yeah.
00:07:31.040 --> 00:07:33.279
I mean, that that's mostly correct.
00:07:33.439 --> 00:07:44.560
That uh yeah, JP Morgan did give me and my team, uh, and a lot of the cybersecurity folks within JP Morgan, a lot of rooms and a lot of budget to do a good cybersecurity.
00:07:44.720 --> 00:07:49.759
We had very good leaders that meets the uh cybersecurity and the technology space.
00:07:49.920 --> 00:07:52.800
So, you know, I I like that environment, I like that workplace a lot.
00:07:53.040 --> 00:07:53.680
Okay, love that.
00:07:53.839 --> 00:08:03.680
So, all of this journey and your path into the space now has led you to create all of these different facets and angles to uh this knowledge base you've basically built.
00:08:03.839 --> 00:08:05.759
So, tell us a little bit about what you've been working on.
00:08:06.000 --> 00:08:06.959
Sure thing, sure thing.
00:08:07.040 --> 00:08:13.519
So uh AIDFend, uh, you can find it on aidfend.net, which is the website that I show the whole framework.
00:08:13.680 --> 00:08:19.199
And you can also find it at AIDFend.dev, which is the GitHub repository for AIDFend.
00:08:19.360 --> 00:08:34.799
It's an open source AI security defense, it's an open source AI security defense knowledge phase for practitioners, for engineers to start with, to understand how exactly one can do AI security and one should do AI security.
00:08:34.960 --> 00:09:04.320
So the AI Defense itself is a framework that consists of different tactics and um 300 plus techniques, the uh practical defense techniques for AI security, and it cross map to nine frameworks at this moment, including uh OWASP, yeah, uh My Cherry Atlas, NIST Advisory Advisorial Machine Learning, yeah, NIST uh Cisco, Google, Scife, uh, Data Bricks AI security framework.
00:09:04.559 --> 00:09:09.919
These are the top and most mentioned uh AI security frameworks in the industry today.
00:09:10.080 --> 00:09:21.679
So it is cross mapped to those, so that I try to find a balance between the the practitioner's perspective versus what management level wants to see.
00:09:21.840 --> 00:09:24.480
So that's why uh that's how the cross map happened.
00:09:24.799 --> 00:09:34.960
I think that you you solved the the particular problem of there isn't like a single place to map every you know AI defense that is possible.
00:09:35.120 --> 00:09:40.399
And when there's literally so much, like you just mentioned more than five AI frameworks.
00:09:40.480 --> 00:09:44.879
So I think that that's literally the problem solving aspect of it here.
00:09:45.120 --> 00:09:52.799
When was the moment that you realized was there a story behind that or an incident, or was it just literally like death by a thousand disconnected frameworks?
00:09:52.960 --> 00:09:55.039
Yes, so that's exactly the point, right?
00:09:55.120 --> 00:09:58.240
Uh so as a security person, right?
00:09:59.200 --> 00:10:04.879
We are always under very tight timeline limitation and that budget issue, right?
00:10:05.039 --> 00:10:09.679
So we can only focus on the things that we want to focus the most.
00:10:09.840 --> 00:10:18.240
However, what happens is that back in my previous roles within financial services companies, people say, yes, we need to do AI security.
00:10:18.399 --> 00:10:19.200
How to do that?
00:10:19.360 --> 00:10:24.000
Let's find there are some good resources or frameworks, and they go ahead and find that.
00:10:24.080 --> 00:10:34.320
And then they realize, okay, great, we need to do uh prompt injection protection, we need to do data poisoning protection, we need to do intent drifting protection.
00:10:35.360 --> 00:10:42.480
Those are the things that exist in those frameworks that tells you that, hey, you need to do this so that you can secure your AI environment.
00:10:42.639 --> 00:10:45.039
And as a security person, I was happy to see that.
00:10:45.200 --> 00:10:47.679
And then the next question, Tom, okay, great.
00:10:47.840 --> 00:10:49.360
I need to do these 10 things.
00:10:49.440 --> 00:10:49.759
How?
00:10:49.919 --> 00:10:50.960
How do I do this?
00:10:51.120 --> 00:10:51.360
Right?
00:10:51.519 --> 00:10:55.600
And that's the next question that as practitioners we have to figure out, right?
00:10:55.759 --> 00:11:14.559
So we either have to go ahead to find if some frameworks mention their solutions about it, or we go online to see if there are commercial solutions for it, or if there are uh open source resources for this, if there are guidances provided by some institutions or community that help us to do that.
00:11:14.720 --> 00:11:24.799
Now, all of these are, but I myself, I find it extremely difficult to keep all of these in one place so that I it's easier for me to do my job, right?
00:11:24.879 --> 00:11:29.679
I, as a practitioner, I want to know okay, things that I want to know about prompt injection.
00:11:29.759 --> 00:11:30.559
Here's a goal.
00:11:30.720 --> 00:11:35.840
Here are the technical guidance, the technical steps that I can do or I should be doing.
00:11:35.919 --> 00:11:40.240
And if I don't have time to do it, here's the list of open source resources.
00:11:40.320 --> 00:11:46.559
And if I have more budget, here are the commercial solutions that can solve this problem, right?
00:11:46.720 --> 00:11:55.519
And yeah, at the end, what compliance or regulatory or framework or framework problems that this uh this uh technique solve.
00:11:55.759 --> 00:11:57.360
That's that's the thing that I want to know.
00:11:57.440 --> 00:12:00.320
And that's the thing probably I want to show to my management.
00:12:00.480 --> 00:12:19.039
So AI defense is the attempt to collect all of these together into one single source of truth that shows that AI security uh could be separated into tactics and techniques, and practitioners can use it to actually implement these with this information given.
00:12:19.120 --> 00:12:29.440
And the management can use this information as hey, now by implementing this, it fulfills this from the uh from the frame from the other frameworks.
00:12:29.600 --> 00:12:33.200
So that's the goal of the of the AIDF and framework.
00:12:33.600 --> 00:12:37.360
As you were building it out, did you notice or were you surprised?
00:12:37.440 --> 00:12:41.840
Um, because I'm sure there is a lot of overlap between a lot of these frameworks.
00:12:42.000 --> 00:12:43.519
How did you deal with the overlap from it all?
00:12:43.759 --> 00:12:44.720
Sure thing, sure thing.
00:12:44.879 --> 00:12:49.200
Yes, there are certainly some overlaps between different uh frameworks that we're seeing.
00:12:49.360 --> 00:13:06.159
I would say that the personally, and I liked them a lot, data bricks, their AI security framework, and Cisco's AI security and safety framework, they are they are narrower in scope, but they are closer in granularity to AI defense implementation first orientation.
00:13:06.320 --> 00:13:07.679
So I like that a lot.
00:13:07.919 --> 00:13:14.639
What happens is that from implementation perspective, these frameworks tell you what you need to do, right?
00:13:14.879 --> 00:13:27.039
But so far, at least from what I can see, uh, there's no single source of knowledge base that tells you exactly what, that tells you what you need to do, and in addition, how you can do it.
00:13:27.200 --> 00:13:37.200
So AIDFEN, if you think about many security frameworks, they are the backbone or skeleton of a good AI security practice.
00:13:37.279 --> 00:13:46.159
And uh AIDFEN fills in the muscle of those in those skeletons to show you how exactly you should do good AI security.
00:13:46.399 --> 00:13:47.919
Yeah, no, this is a lot more.
00:13:48.080 --> 00:13:52.559
We were just talking about this, it's a lot more practical information across the tactics.
00:13:52.720 --> 00:13:56.799
Like it truly tells me what I need to go build detections for, right?
00:13:56.879 --> 00:14:02.960
Like you were mentioning prompt injection, the drift detection, the monitoring of the policy enforcement of it all.
00:14:03.120 --> 00:14:19.759
Whereas I've I've seen some iterations of literally almost like the same thing, though the same way you you called this the AI defense framework, probably more for the C level, C level friendly, executive friendly view of kind of cross-reference against the NIST framework.
00:14:20.399 --> 00:14:22.879
And by the way, no, I will be sharing.
00:14:23.279 --> 00:14:27.679
I just permissions don't let me share right now, but I will be sharing like both of them.
00:14:27.840 --> 00:14:33.200
Um and uh and yeah, like like what am I supposed to do with AI security posture management?
00:14:33.279 --> 00:14:41.440
Like whereas in the AI defend framework, it's a lot more robust, a lot more concrete and specific of a behavior that you can actually build for.
00:14:41.679 --> 00:14:43.600
Yep, yep, exactly, exactly.
00:14:43.759 --> 00:14:46.720
And as you uh as what Alex you described, right?
00:14:46.799 --> 00:14:50.320
Yeah, do you think it's not trying to replace any of these frameworks, right?
00:14:50.480 --> 00:14:57.279
It's just trying to tell you, uh fill in the how of what's missing in these uh in these frameworks.
00:14:57.440 --> 00:14:58.080
So yeah.
00:14:58.240 --> 00:14:58.559
Yeah.
00:14:59.200 --> 00:15:00.799
Well, I gotta ask.
00:15:00.879 --> 00:15:11.279
I mean, we obviously love all and any community open source work, but it's not always the easiest to be a maintainer of an open source initiative or repo.
00:15:11.519 --> 00:15:12.960
So how are you doing this?
00:15:13.120 --> 00:15:15.679
Is it a personal, unaffiliated initiative?
00:15:15.840 --> 00:15:18.080
What's it been like maintaining this solo?
00:15:18.639 --> 00:15:22.159
Are you getting any kind of help or support, maybe miter-backed efforts?
00:15:22.399 --> 00:15:26.000
Yeah, for now it's it's my personal effort mostly, right?
00:15:26.159 --> 00:15:32.399
I do get the uh pull requests from some of the uh community uh contributors, which I appreciate a lot.
00:15:32.559 --> 00:15:32.879
Yeah.
00:15:33.120 --> 00:15:39.919
And including those latest information on the defensive side into the AI different framework.
00:15:40.080 --> 00:15:42.320
That's something that I uh that I'm passionate about.
00:15:42.480 --> 00:15:55.759
That I'm doing almost every day, doing the research and track on the latest AI security trends and see if any of the new AI security defense techniques could be included into the AI defense framework.
00:15:55.840 --> 00:16:00.720
That's something that I am passionate about and happy to do when I wake up every morning.
00:16:00.879 --> 00:16:02.720
So that's fantastic.
00:16:02.960 --> 00:16:03.600
Same pattern.
00:16:03.759 --> 00:16:23.279
I mean, this is this is what the the community thrives off of is in MITRETAC did this also too in the early days where one person or maybe it was even a small group, but they just decided that the industry needed a shared dictionary or shared language, shared vocabulary, and you just build it and share it and just not you know not wait for permission.
00:16:23.679 --> 00:16:24.399
Actually, exactly.
00:16:24.639 --> 00:16:30.159
Actually, if you look at the uh the structure of AIDFEN framework, uh it contains tactics, right?
00:16:30.320 --> 00:16:30.879
Yep, yep.
00:16:31.039 --> 00:16:32.559
Model, harden, detect, yep.
00:16:32.720 --> 00:16:32.960
Yes.
00:16:33.120 --> 00:16:35.360
So the concept of those seven tactics.
00:16:35.519 --> 00:16:41.279
Wait, yes, as this is model, harden, detect, isolate, deceive, evict, restore, those tactics.
00:16:41.440 --> 00:16:43.039
I didn't reinvent the wheel.
00:16:43.200 --> 00:16:47.759
Those are inspired by another popular framework created by Mitre.
00:16:48.000 --> 00:16:49.679
It's called Defense Framework.
00:16:49.840 --> 00:16:53.360
And it's the defense, they spelled it as D and number three.
00:16:53.440 --> 00:16:54.480
D three.
00:16:54.559 --> 00:16:54.639
Yeah.
00:16:54.720 --> 00:17:01.200
Yeah, I and D, uh, which is a uh a defense framework that mapped to its own mitre attack framework.
00:17:01.279 --> 00:17:02.879
I I love both a lot, right?
00:17:02.960 --> 00:17:12.079
I mean, both of them have good, there are good indications of how secure in the security world, how attacks and how defense should be done uh from different perspectives.
00:17:12.319 --> 00:17:24.160
So, and one of the starting points of me creating AI Defense is that Mitre also created this amazing Atlas framework, which focuses on AI security, right?
00:17:24.319 --> 00:17:30.960
Attack is the uh attack uh techniques and uh mechanisms on the on the general uh world.
00:17:31.039 --> 00:17:36.799
And mitra Atlas is the the AI version of the attack framework, right?
00:17:36.960 --> 00:17:42.000
And so far, I haven't seen a defense version for the AI security.
00:17:42.559 --> 00:17:43.440
Atlas, yeah.
00:17:43.680 --> 00:17:44.720
Exactly, for Atlas.
00:17:45.039 --> 00:17:54.079
So uh that's where I, you know, that's when I thought about that, which is in about around June and last year, June 2025.
00:17:54.240 --> 00:17:56.400
And I think, hey, there's there's a gap here.
00:17:56.480 --> 00:18:04.799
Uh in my organization, uh, we really wanted to understand how defense technicism, how defense techniques and mechanisms should be done on our end.
00:18:04.960 --> 00:18:12.160
So, you know, we figure out, hey, why don't we make it a collection of a knowledge base or database so that we know what to do?
00:18:12.319 --> 00:18:14.480
And that's how I started this project.
00:18:14.720 --> 00:18:20.640
And then I feel that hey, this is something that I could contribute back to the uh security community.
00:18:20.799 --> 00:18:23.039
So that thing uh got started.
00:18:23.200 --> 00:18:23.759
Yes.
00:18:24.160 --> 00:18:25.119
Well, you're right.
00:18:25.200 --> 00:18:28.480
The uh miter attack has become such a household name now.
00:18:28.640 --> 00:18:33.759
We get less love for defend, let alone miter atlas.
00:18:33.920 --> 00:18:34.240
Right.
00:18:34.400 --> 00:18:35.200
Hi, yeah.
00:18:35.359 --> 00:18:35.680
Uh-huh.
00:18:36.000 --> 00:18:39.440
Yeah, I think there are a lot of reasons behind it, right?
00:18:39.599 --> 00:18:48.559
Uh, and I do think that as a security practitioner, right, I am happy to learn about the attacking mechanisms, how bad guys attack a system.
00:18:48.720 --> 00:18:49.200
That's great.
00:18:49.359 --> 00:18:53.519
Uh, but more importantly, at the same time, my management will ask about okay, great.
00:18:53.680 --> 00:18:57.680
Now you have all of these knowledge about how our systems could be attacked.
00:18:57.839 --> 00:19:00.400
Tell me how you can protect against these attacks.
00:19:00.559 --> 00:19:08.720
Then I'm like, oh yeah, we can use this tool, we can use this solution, I can create a script to detect this and that, things like that.
00:19:08.880 --> 00:19:28.160
They're all good, but I think it will be the best if we can collect all of them into one single place, and um you can keep track on all different kinds of defensive techniques, and then uh and and then you can decide um which one you should use to put into practice so that you can protect your organization better.
00:19:28.319 --> 00:19:33.279
No, no, that's the the thought of the uh that's the thought of uh AI defense framework behind the scenes.
00:19:33.359 --> 00:19:38.720
So yeah, no, it absolutely has a place, and you got on top of it pretty fast.
00:19:38.880 --> 00:19:45.440
Have you gotten into translating this defense uh detection work and operationalizing and implementing it at companies?
00:19:45.599 --> 00:19:52.799
I want to get into that because that's something that I think that is on a lot of our mind as an actual detection engineer.
00:19:53.200 --> 00:19:58.400
I think we would probably live, if I'm looking at your framework, on the detect side of the pillar.
00:19:58.720 --> 00:19:59.839
Sure, sure, sure thing.
00:19:59.920 --> 00:20:01.920
Now, if you look at uh the framework, right?
00:20:02.079 --> 00:20:09.119
Detection, yeah, detect is a tactic that consists of uh 17, if I'm not mistaken, 16 or 17 techniques.
00:20:09.279 --> 00:20:18.480
So there are definitely in AI security world, there are definitely uh detection engineering things that uh will be included, at least in the AI defense framework.
00:20:18.640 --> 00:20:25.920
I can share some of the things that I put into the framework, which I think that from detection engineering perspective is super important.
00:20:26.240 --> 00:20:28.079
Prompt level telemetry, right?
00:20:28.240 --> 00:20:38.000
You will need um structural features like injected content length related to system prompt, uh delimiter row confusion patterns, and sudden shift in prompt entropy.
00:20:38.240 --> 00:20:40.880
Those are the top things you want to look at at prompt level.
00:20:40.960 --> 00:20:45.920
And in agency system world, right, tool call graphs is super important.
00:20:46.079 --> 00:20:56.400
Uh, which tools got involved, uh, in what sequence, with what parameters, whether the per the sequence deviated from the from the agent's expected task.
00:20:56.480 --> 00:21:03.200
Those are the things that you need to focus you need to focus on when it comes to detection engineering in the agentic security world, right?
00:21:04.240 --> 00:21:05.359
And reg, right?
00:21:05.519 --> 00:21:06.079
And RAG.
00:21:06.319 --> 00:21:06.720
Exactly.
00:21:06.880 --> 00:21:10.480
Retrieval provenance is super important for RAG, right?
00:21:10.559 --> 00:21:15.440
Which document chunks actually got pulled into the context and from where, right?
00:21:15.519 --> 00:21:27.119
This is the uh these these are the things that are included in the techniques under the AI deep and detects tactics that are super important to look at in the from the context of AI AI security.
00:21:27.359 --> 00:21:34.880
Yeah, as well as the framework, the view by framework, it literally has every single defenses by every single technique.
00:21:35.039 --> 00:21:35.920
Yes, exactly.
00:21:36.160 --> 00:21:36.480
Exactly.
00:21:36.640 --> 00:21:37.279
Yes, yep.
00:21:37.519 --> 00:21:38.240
Oh, yeah.
00:21:38.400 --> 00:21:41.200
I I had not gone to that part to that tab.
00:21:41.839 --> 00:21:43.519
No worries, no worries, no worries.
00:21:43.599 --> 00:21:59.759
You will love the different things that I love about the about AI defense framework is that I try to make it as operationalized as possible so that in addition to tactics view, I also provide several views if you want to do good AI security.
00:22:00.480 --> 00:22:10.960
So, for example, in Pillar's view, you can see that the defensive mechanisms, the defensive techniques, they are split into four different pillars.
00:22:11.039 --> 00:22:18.319
They have security, infrastructure security, model, algorithm security, and application security.
00:22:18.480 --> 00:22:18.720
Right?
00:22:18.880 --> 00:22:24.160
So Pillar Vie is for is for people in different roles, right?
00:22:24.240 --> 00:22:27.680
Let's say I am in this company and I focus on infrastructure security.
00:22:27.839 --> 00:22:28.960
Okay, what do I need to do?
00:22:29.119 --> 00:22:36.960
Now, if I come to the pillars view, organize, yeah, that's that that's a list of things that I should be aware of or I should be doing.
00:22:37.440 --> 00:22:41.759
Now in addition, the the phases view, right?
00:22:42.000 --> 00:22:53.759
It is using the uh the concept of separating um separating security defense techniques into different phases when it comes to AI system deployment.
00:22:54.079 --> 00:23:14.079
So when we uh when we're deploying AI an AI system, uh we have different phases from system design scoping to model training, uh training and building, pre-deployment validation, pre-deployment validation, production operation, incident containment, and system restoration improvement.
00:23:14.160 --> 00:23:15.920
There are different phases, right?
00:23:16.160 --> 00:23:23.519
And in some companies, we have people that that are in charge of uh different phases of the AI system deployment.
00:23:23.680 --> 00:23:24.640
That's number one.
00:23:24.880 --> 00:23:36.000
And number two is that uh we can now clearly see that okay, for this system, now it's under the phase of model training and building and hardening.
00:23:36.079 --> 00:23:40.319
Okay, so here's the line that we of defense techniques that we should be looking at.
00:23:40.480 --> 00:23:46.400
And when we move to the next phase, then it's the next line of defensive techniques that we um that we should be looking at.
00:23:46.559 --> 00:23:47.920
So that's that that's a concept there.
00:23:48.160 --> 00:24:00.240
Yeah, I also I keep thinking about what you said about um you you're you're offering various options, like from free all the way to commercial, because that story will be different, right?
00:24:00.319 --> 00:24:02.559
Depending on depending on who you are.
00:24:02.720 --> 00:24:15.759
For example, if we take this like prompt injection, uh what a detection engineer needs to look for, sometimes it's gonna have to be, you know, versus uh signature versus behavioral baseline versus maybe an LLM judging another one.
00:24:15.839 --> 00:24:20.960
And that that that that means different the the cost of doing each one is gonna be very much different.
00:24:21.279 --> 00:24:21.680
Exactly.
00:24:21.759 --> 00:24:22.160
Yep.
00:24:22.400 --> 00:24:30.000
So so they can either go with more of an open source route of detection or they can fully now go buy a tool that helps with maybe something like this.
00:24:31.519 --> 00:24:31.839
Yeah.
00:24:32.240 --> 00:24:32.640
Exactly.
00:24:32.799 --> 00:24:35.359
That's that's the uh the concept behind it, right?
00:24:35.440 --> 00:24:44.480
The the goal of this framework is that organizations or teams in different size in with different budgets can do good AI security, right?
00:24:44.559 --> 00:24:51.519
So and it also depends on how uh critical that specific defense related to your environment.
00:24:51.680 --> 00:24:55.519
Maybe to your environment, a certain defense is super critical.
00:24:55.680 --> 00:25:07.279
Then my recommendation in that framework is go ahead and buy the best in class, super expensive solution if that works well for you, and I list out those uh commercial solutions for your reference.
00:25:07.440 --> 00:25:27.759
If it's not as critical as a functionality device to you, then I provide the example code as well as the open source resources that you can refer to as well as building it, building one for yourself to protect your system for to protect that uh that field of um security risk at certain basic level.
00:25:27.920 --> 00:25:33.200
So that's a thought behind uh the different information given in this AI defense framework.
00:25:33.440 --> 00:25:35.920
I want to touch back on the RAG part of it all.
00:25:36.079 --> 00:25:42.240
You said there's a lot of a potential opportunity detection engineering opportunities for the RAG pipeline specifically.
00:25:42.400 --> 00:25:46.319
Where would you tell a detection engineer to start instrumenting first?
00:25:46.480 --> 00:25:50.880
Is it the retrieval layer, the prompt lead construction, the model output?
00:25:51.200 --> 00:25:54.319
Do we really just have to wait on the until the downstream side of it?
00:25:54.480 --> 00:25:56.000
Like that's you're too late at that point.
00:25:56.240 --> 00:25:58.400
Yeah, that's that's uh that's a great question.
00:25:58.640 --> 00:26:01.279
I I personally have a strong opinion about it.
00:26:01.440 --> 00:26:04.559
So uh feel free to disagree with me if you would like to.
00:26:04.720 --> 00:26:08.880
And obviously, in security, we want to do layered security, right?
00:26:08.960 --> 00:26:13.200
There's no single silver bullet to do good security, right?
00:26:13.279 --> 00:26:14.640
We want to do defense in depth.
00:26:14.720 --> 00:26:20.799
And but I do have this strong opinion that if you can only pick one, the retrieval layer is super, super important.
00:26:21.200 --> 00:26:28.160
Almost every serious rec compromise incidents that I've seen, they enter at retrieval layer, right?
00:26:28.400 --> 00:26:40.640
So if you're not logging what retrieved and you know from from where, uh you have no way to reconstruct an incident after the fact, even though, even though you have contained the uh the impact that it is causing.
00:26:40.799 --> 00:26:49.759
But if you there if you have no way to trace back to how it happened, how it got happened, then uh it could still happen again next time, right?
00:26:50.400 --> 00:26:50.880
Yeah.
00:26:51.599 --> 00:26:58.720
Yeah, so uh, you know, definitely for rack and pipeline, you want to do good security at uh retrieval layer.
00:26:59.279 --> 00:27:00.240
At the retrieval layer.
00:27:00.480 --> 00:27:01.599
At retrieval layer, yes.
00:27:02.640 --> 00:27:07.519
Prompt construction is also is also very super important.
00:27:07.839 --> 00:27:20.160
You want to have the visibility into how retrieve content got merged with the system prompt, and because that's where the injected uh instructions actually take effect.
00:27:20.480 --> 00:27:24.319
So uh that's I would say that's the second thing to look at.
00:27:24.400 --> 00:27:35.839
But retrieval layer is super important that uh you need to know and you need to understand where uh things were retrieved from at a very at a very basic level.
00:27:35.920 --> 00:27:36.559
So yes.
00:27:37.359 --> 00:27:40.960
Yes, and of course, just by keyword searching retrieval rag.
00:27:41.599 --> 00:27:42.480
Yep, yep.
00:27:43.359 --> 00:27:46.799
We'll we'll get we'll we'll we'll get populate populated some things.
00:27:46.880 --> 00:27:55.519
I'm already seeing some harden and some detect things, permission aware retrieval, of course, always always at the identity part as well, like defense and depth.
00:27:56.319 --> 00:28:02.799
For for OWASP specifically, for so for the OWASP top 10, it seems like now Agent AI it's getting its own OWASP.
00:28:03.440 --> 00:28:15.119
Uh and so what do you think is fundamentally different about detecting an agent doing something malicious versus an LLM just generating bad text?
00:28:15.759 --> 00:28:16.480
Yeah, yeah.
00:28:16.720 --> 00:28:21.279
So uh I think it's uh it's much closer to insider threat detection than malware detection.
00:28:21.359 --> 00:28:22.240
That's my thought, right?
00:28:22.400 --> 00:28:30.240
An agent is it is closer to a legitimate credentialed user that can be socially engineered by prompt, right?
00:28:30.319 --> 00:28:31.599
By malicious prompt.
00:28:32.480 --> 00:28:44.480
And when they got socially engineered, they are led into the field of they can do malicious things within or outside of their original intention, uh, but with legitimate permission.
00:28:44.559 --> 00:28:49.680
That's the worst part of it, because they have in that in that scenario, they have the legitimate permission.
00:28:49.920 --> 00:28:50.240
Yeah.
00:28:50.400 --> 00:28:58.799
And personally, I think that actually reframes the detection engineering posture that you're looking for behavior that's technically authorized, right?
00:28:58.880 --> 00:29:02.559
It's authorized by the uh correct source.
00:29:03.200 --> 00:29:05.680
Yep, but they are contextually wrong, right?
00:29:05.839 --> 00:29:15.039
Yeah, the agent using a tool it has every right to use in a way that it doesn't match the task that it was given, right?
00:29:15.759 --> 00:29:34.000
So my working point of view is that you look for the goal drift, which does the agent's tool call sequence stay coherent with the um original stated task, or does it branch off in a way that only makes sense if instructions were injected mixed stream from a external source?
00:29:34.160 --> 00:29:36.799
So yeah, that's my thought on how this should be done.
00:29:36.880 --> 00:29:37.039
Yes.
00:29:37.279 --> 00:29:51.200
Yeah, I think this is that's the conversation I feel like that I keep hearing across podcasts is how do you distinguish a true, you were talking about insider, a true human prompted request versus a malicious one.
00:29:51.359 --> 00:29:54.240
It's the line is getting blurry between what is what.
00:29:54.319 --> 00:29:58.480
I I've been I've been watching my own Claude Code request.
00:29:58.640 --> 00:30:00.559
They touch a bunch of my credential stores.
00:30:00.720 --> 00:30:09.440
And sometimes looking back, like I can't tell if it's just the tool doing the job or if it's sometimes getting abused because they look almost the exact same.
00:30:09.759 --> 00:30:10.880
Exactly, exactly.
00:30:11.039 --> 00:30:13.440
And uh it's super hard to detect.
00:30:13.599 --> 00:30:13.759
Yeah.
00:30:13.920 --> 00:30:26.799
So it's a hard job for detection engineering people that uh to distinguish between between the a legitimate prompt versus uh what's being maliciously injected into the system.
00:30:26.960 --> 00:30:28.799
So yeah, at different level, right?
00:30:28.880 --> 00:30:37.920
Uh you want to be able to detect the malicious or the not so malicious, but the prompts that could lead to goal drifting later on.
00:30:38.000 --> 00:30:39.759
Yeah, you want to be able to detect that.
00:30:39.920 --> 00:30:53.759
And at goal level, you want to check the agents or the uh MCP service that you tell that you you authorize uh um doing your work are actually doing the things that you're expecting.
00:30:54.319 --> 00:30:54.559
Yeah.
00:30:54.799 --> 00:31:02.240
So these are things that uh uh you know, a huge amount but but valuable job for detection engineering people.
00:31:03.359 --> 00:31:20.160
Oh uh do you think they should have their own set of credentials or like its own entity, its own employee ID, so that maybe on the governance side you you can keep track of them as like users and which one's an agent, which yeah.
00:31:20.240 --> 00:31:21.680
I mean should they have their own?
00:31:22.079 --> 00:31:26.400
Very similar to 10 or 15 years ago when we introduced cloud, right?
00:31:26.480 --> 00:31:28.160
That the the the cloud environment.
00:31:28.319 --> 00:31:31.680
We used to have human being account, now we have the service account.
00:31:31.920 --> 00:31:32.480
The service account.
00:31:32.640 --> 00:31:37.200
Which is used for the uh the the the system services and APIs.
00:31:37.359 --> 00:31:51.839
So I agree with that approach that uh from tracking perspective, it will be much easier and clearer if we have different if we have categorized identity for the agents and for different AI services.
00:31:51.920 --> 00:31:53.359
I agree with that approach.
00:31:53.759 --> 00:31:54.160
Yeah.
00:31:54.400 --> 00:31:59.440
No, if we think about the data feed of it all, I guess that maybe now it's getting better.
00:31:59.519 --> 00:32:07.519
I haven't, to be honest, looked at it in a while now, and I keep hearing that it's much better, that the Claude, specifically the OTEL telemetry.
00:32:07.759 --> 00:32:13.279
But before it used to just be like token usage and session duration.
00:32:13.440 --> 00:32:20.079
It's not really that uh helpful information to build a detection per se, a rule.
00:32:20.880 --> 00:32:28.240
What would you say is probably the best visibility if someone had let's say zero visibility into their company's LLM usage?
00:32:28.400 --> 00:32:30.480
Where would you get started with logging?
00:32:31.039 --> 00:32:34.319
What hotel you do would you have them stand up?
00:32:34.720 --> 00:32:37.599
Uh what kind of variables would you be looking for?
00:32:37.920 --> 00:32:38.480
Sure thing.
00:32:38.559 --> 00:32:38.799
Yeah.
00:32:38.880 --> 00:32:40.240
Uh if I have to choose, right?
00:32:40.319 --> 00:32:47.519
Uh I would say start with the tool call logs as well as the uh uh the API call login, right?
00:32:48.000 --> 00:32:52.720
And uh this can be done at the uh at the proxy and gateway level, right?
00:32:52.880 --> 00:33:06.799
Uh whatever sits between you and your users and the model provider, you know, if you have the uh gateway and proxy setup, then you will have visibility into the tool call history, uh the tool call locks and the API call logs.
00:33:06.960 --> 00:33:16.960
Yeah, I mean uh from there it gives you the uh the request volume, which models are used, yeah, which tools got involved, you know, without touching the content at all.
00:33:17.119 --> 00:33:23.440
So I think that's a good thing that you could use that you track the tool call and API call at the proxy level.
00:33:23.599 --> 00:33:32.240
And I think that the layer in retrieval provenance login, if you're running around, that is also super important, right?
00:33:32.400 --> 00:33:37.279
And then probably tool call parameter login if you're running uh agents, right?
00:33:37.359 --> 00:33:40.559
Uh that's that's the next things that I I'll be looking at.
00:33:40.640 --> 00:33:44.960
Yeah, much practical than the token usage, token maxing of it all.
00:33:45.200 --> 00:33:48.400
That doesn't tell you anything meaningful from security perspective.
00:33:48.559 --> 00:33:51.119
So, well, it is a well, the indicator, right?
00:33:51.279 --> 00:33:57.359
You can you do see that uh if there's a search on the usage of token, that that means something.
00:33:57.680 --> 00:34:04.160
But you know, models uh behave in ways that people cannot fully uh expect, right?
00:34:04.240 --> 00:34:08.880
So there are some legit tasks that uh cause a lot of usage in tokens.
00:34:09.039 --> 00:34:13.840
You think that's weird, but if you look at it, you know, it's normal, it's just how that works.
00:34:14.000 --> 00:34:18.880
So it's an indicator, but it's not the uh most useful indicator in the security world.
00:34:18.960 --> 00:34:19.760
That's what that's with.
00:34:20.079 --> 00:34:23.599
Yeah, that I've seen some good research from Trail of Bits.
00:34:23.760 --> 00:34:33.360
Uh, this would only apply if you if you have an MCP, since MCPs has now become more of the default way that agents are or talking to each other, to tools.
00:34:33.519 --> 00:34:41.360
And a trail of bits has like a like a like an MCP wrapper that will track all of the input-output requests into the MCP.
00:34:41.599 --> 00:34:45.280
Yeah, which but again, not that's not the only way agents can be called.
00:34:45.679 --> 00:34:46.079
That's true.
00:34:46.239 --> 00:34:46.719
That's true.
00:34:46.880 --> 00:35:01.679
So I do think that there are several good contributors in the security world and in the security field that there are several tools that try to that try to collect as as many logs in a meaningful way in detail at different levels.
00:35:01.840 --> 00:35:07.039
So I think there are several very problem and promising open source projects for that.
00:35:07.280 --> 00:35:08.000
Okay, yeah.
00:35:08.320 --> 00:35:15.280
I love seeing that because you know detection engineering is so important that it's almost the very first step.
00:35:15.360 --> 00:35:18.480
Well, second step after uh after log collection, right?
00:35:18.639 --> 00:35:26.559
You have to collect all the signals, but everything that you need to operationalize that comes after the actual detection happens.
00:35:26.800 --> 00:35:30.719
So yeah, that's something that I myself, I'm, you know, I look forward to happening.
00:35:30.880 --> 00:35:39.679
I look forward to see more tools get into this field that people can leverage to do detection engineering and the follow-up operationalization.
00:35:39.840 --> 00:35:44.000
So yeah, no, totally like the same way, like what Sysmon was for Windows.
00:35:44.159 --> 00:35:45.599
That that's what I'm I'm looking for.
00:35:45.840 --> 00:35:47.760
Trell of Bits was like the closest thing.
00:35:47.840 --> 00:35:50.159
This is this is also a step in that direction.
00:35:50.320 --> 00:35:56.559
Uh, there doesn't seem to be like a an open source telemetry enrichment type of thing for this yet.
00:35:56.639 --> 00:36:03.280
Uh, I think everyone is still building kind of bespoke logging for vendor uh or the tool calls, like you said.
00:36:04.000 --> 00:36:07.039
Yeah, have have a lot more to explore there.
00:36:07.760 --> 00:36:09.760
Uh we're getting to the end of the episode.
00:36:10.000 --> 00:36:23.440
I want to kind of close with what how exactly our detection engineers, who's never touched security, maybe has a backlog, you know, a mile long before they even start to get into some of the some of the techniques.
00:36:23.519 --> 00:36:28.079
Uh, what would you tell them to implement first, would you say, to get their best return for their efforts?
00:36:28.320 --> 00:36:29.280
Sure thing, sure thing.
00:36:29.440 --> 00:36:40.000
So I would say that depending on what your role is, depending on which phase that you're at when it comes to the AI deployment at your in your organization, right?
00:36:40.159 --> 00:36:47.519
And depending on what your management cares about in terms of uh in terms of framework or in terms of best practices guide, right?
00:36:47.679 --> 00:37:01.360
You can always refer to the uh the detect tactics under under AI defense and cross-map that with your with your role responsibility as well as the phase that your organization or the application that you is at.
00:37:01.519 --> 00:37:38.880
And and you know, look in look at the the those techniques that you think that will to your work when it comes to implementation or operationalization, and then exploring the tools that I mentioned in those in those under those techniques, which I think that uh it will be most beneficial for you to understand the tool first, and then you can decide how important that is to our organization, and then you can decide, okay, I want to build it myself, uh, or I want to build it within my team, or oh, that's a lot of things to do, and we have the budget, so let's you know, buy a good solution for that.
00:37:39.039 --> 00:37:46.639
So, you know, just remember that the the goal of this framework is not selling you things, right?
00:37:46.719 --> 00:37:55.519
It's not it's not another framework that tells you that tells you, hey, you need to do this, but without telling you how to do that, right?
00:37:55.840 --> 00:38:21.440
The goal of this is depending on your the reality that you're at, yeah, uh it tells you the different options that you can do based on the resources, based on the budget, based on the uh the the criticality of that specific technique, uh the that specific AI security uh defense perspective in your organization.
00:38:21.599 --> 00:38:22.480
So that's a goal.
00:38:22.639 --> 00:38:37.119
So if this uh the if if the tool or if the framework could help you a little bit in doing better AI security, then I am so happy about that.
00:38:37.280 --> 00:38:38.320
That's the goal of this.
00:38:39.199 --> 00:38:49.840
Spoke as like a true engine uh detection engineer, is starting with what you have, what you prioritize, like the good key, what your business landscape looks like before trying anything out.
00:38:50.480 --> 00:38:55.039
See, I I am I'm immediately drawn to the to the rogue agent discovery.
00:38:55.360 --> 00:38:56.239
Yep, yep, yep.
00:38:56.480 --> 00:38:58.880
You should uh yeah, that that's a good part.
00:38:59.039 --> 00:39:06.320
I uh I think I spent almost two days researching on that topic and then and then worked on that that part.
00:39:06.480 --> 00:39:13.199
That's uh that's a good part that I have that I I have pretty strong memory around uh the creation of the part.
00:39:13.440 --> 00:39:13.840
So yeah.
00:39:14.320 --> 00:39:22.559
Do you do you think the role of an um an AI detection engineer will will c will start to pop up in like LinkedIn?
00:39:23.360 --> 00:39:23.760
A lot.
00:39:24.000 --> 00:39:24.880
A lot, yes.
00:39:25.039 --> 00:39:36.480
So we are now talking about uh you can see a lot of job jobs around uh AI governance or even AI security governance that are open up today, right?
00:39:37.119 --> 00:39:39.440
Yeah, which which is great, right?
00:39:39.599 --> 00:39:49.679
We uh as more and more companies and organizations are deploying AI systems and no governance is the next thing that people start looking at, right?
00:39:49.920 --> 00:39:52.719
Yeah, okay, what do we need to govern?
00:39:53.039 --> 00:39:53.199
Right?
00:39:53.360 --> 00:39:54.880
That's the next question, right?
00:39:55.039 --> 00:40:05.920
So follow up on that question, the next thing we would need to understand what to govern before we can do good governance.
00:40:06.400 --> 00:40:06.639
Yeah.
00:40:06.960 --> 00:40:13.039
Where do we get the data of understanding what to govern from detection engineering, right?
00:40:13.119 --> 00:40:29.039
So I think the role, I think the role of the the the detection engineering will be uh will be thriving a lot, simply because of the need of more clarity and more governance are needed in the AI and both both AI and the uh agentic AI world.
00:40:29.199 --> 00:40:44.400
So uh that's that's uh that's something that I I would put a lot of effort into my uh my framework as well to help the uh detection engineering people getting more resources and getting more better guidance on how to do good security.
00:40:44.639 --> 00:40:44.880
Yeah.
00:40:44.960 --> 00:40:55.840
For for my for my more senior detection engineers, do you think that their access to the way they consume this framework is it mcp friendly a ball?
00:40:56.000 --> 00:41:06.639
So like if they have a whole, let's say, you know, codified CI CD pipeline as a part of their coverage assessment and everything is API driven and a part of this like agentic framework.
00:41:06.800 --> 00:41:15.760
Is is this front is this MCP friendly where they can query back into their system and their setup where they don't have to, let's say, you know, not go to the UI.
00:41:15.840 --> 00:41:17.199
They could do it in a headless way.
00:41:17.440 --> 00:41:18.079
Yeah, yes.
00:41:18.320 --> 00:41:24.079
So this framework, AI Defense, it does have uh come with the uh the MCP service, which I built.
00:41:24.239 --> 00:41:43.280
So if you go to AIDFend.net, you will see that there's a uh on the top, there's a UMCP server and service of AIDFEN, which you can integrate with your CI CD or operation operations pipeline uh to make better use of the knowledge in the uh that that is built in in AIDFAN.
00:41:43.519 --> 00:42:06.800
Uh also on the AIDFEN.net, the the website, uh I integrate it with the the web mcp, which means that if you use your your browser's AI tool and ask your browser's AI tool on the AIDFAN.net saying, hey, I want to do this, this, and that watch in, what's in here in this in this in this AIDFEN.net website, it acts like an MCP service.
00:42:06.960 --> 00:42:17.599
So it will give you the answer back with a more structured and accurate way, uh, as how we build it in the AIDFEN.NET uh framework structure.
00:42:17.760 --> 00:42:28.960
So yeah, it has the AIDFEN has both uh the MCP service provided, which you can integrate directly, or on the website it has it is integrated with the web mcp.
00:42:29.280 --> 00:42:30.079
Fabulous, fabulous.
00:42:30.159 --> 00:42:32.079
Yeah, and it's all linked to your GitHub here, I see.
00:42:32.239 --> 00:42:32.800
Yeah, yes.
00:42:32.960 --> 00:42:34.639
Well, thank you so much, Edward.
00:42:34.800 --> 00:42:40.320
I think we've covered quite a bit in terms of your repo, your contribution, your initiative to the community.
00:42:40.480 --> 00:42:44.800
I'm so glad we were finally able to catch up on the same time zone.
00:42:45.039 --> 00:42:47.199
Actually, are you in town for Black Hat?
00:42:47.519 --> 00:42:49.519
Not this year, so uh this year.
00:42:49.840 --> 00:42:57.599
Yes, I will be traveling in the uh Asia Pacific region, speaking in different conferences in August and in September.
00:42:57.760 --> 00:43:07.679
But I surely will keep my eye on the latest AI defense security trend and uh keep working on refining my the the AI defense framework.
00:43:07.760 --> 00:43:08.719
So that that's for sure.
00:43:09.039 --> 00:43:11.440
What's your plans for V2 of this?
00:43:11.599 --> 00:43:12.800
Just continuing refinement?
00:43:12.960 --> 00:43:15.119
Are you gonna productize this maybe?
00:43:15.519 --> 00:43:19.599
That's in the plan, but I think either I productize it or not.
00:43:19.679 --> 00:43:22.719
Uh the goal is to make it more operationalized, right?
00:43:22.960 --> 00:43:23.280
Okay.
00:43:23.519 --> 00:43:26.480
So then I think that let me put it that that way.
00:43:26.639 --> 00:43:34.000
That if we if I if there's a way that I could make aitfant.net more operation operationalizable.
00:43:34.079 --> 00:43:35.119
Is that a is that a word?
00:43:35.280 --> 00:43:36.000
Operationalized.
00:43:36.079 --> 00:43:36.639
Yeah, yeah.
00:43:36.880 --> 00:43:38.079
Operationalizable.
00:43:38.159 --> 00:43:39.599
Yeah, then I'll do it.
00:43:39.760 --> 00:43:42.079
Either open source it or making it a product.
00:43:42.159 --> 00:43:42.960
I'm happy to do it.
00:43:43.039 --> 00:44:06.079
Uh, in fact, there are some projects underway that one of them is that I am trying to grouping, trying to group different use cases into um, you know, trying to put different uh AI different techniques into groups so that you could, based on your use case, you could understand faster and easier what to do if you want to implement security force in in specific use case.
00:44:06.239 --> 00:44:08.639
Let's say you're doing uh rack, right?
00:44:08.800 --> 00:44:17.760
Then one of the projects that I'm working on is okay, if you're doing rack, out of these 200 techniques, these 30 techniques are the top ones that you want to look at.
00:44:17.920 --> 00:44:19.280
So it's easier for you.
00:44:19.519 --> 00:44:30.400
It categorize things for for people in different roles or working on different projects so that you can utilize and leverage the knowledge in the uh AI Defense framework better.
00:44:30.719 --> 00:44:31.199
Fantastic.
00:44:31.280 --> 00:44:32.639
That's I'm so happy to hear that.
00:44:32.800 --> 00:44:35.199
Well, uh, that has been AI Defend.
00:44:35.360 --> 00:44:38.000
Remember not to be confused by AI defense.
00:44:38.400 --> 00:44:45.760
There are there are several names from different companies, from different people around that, which I like a lot, right?
00:44:45.840 --> 00:45:03.519
I mean, that the more that we have the awareness that AI security is important, the more people that we have people, the more people that pour in and uh counsel people into this field, uh the more the the better, the better future that we have for knowledge and the technology in this field, which is something that we would love to see.
00:45:03.760 --> 00:45:06.320
And that's the spirit of community, which is what this is about.
00:45:07.599 --> 00:45:23.519
So thank you so much for the reminder that the fastest growing attack surface in most companies right now literally has a place where it could it can all live and have a shared language that we can yeah, that we can enforce and harden and build our detections against.
00:45:23.599 --> 00:45:29.280
So thank you so much for building this out in the open and for walking us through what it actually takes to operationalize it.
00:45:29.360 --> 00:45:34.960
And I look forward to continuing to see how you continue to operationalize it or make it more operationalizable.
00:45:35.280 --> 00:45:36.159
Yeah, sounds good.
00:45:36.320 --> 00:45:36.719
Sounds good.
00:45:37.039 --> 00:45:39.199
I really appreciate your comment, Alex.
00:45:39.360 --> 00:45:43.679
And yeah, I had a great time talking with you and the uh the audience here today.
00:45:44.000 --> 00:45:50.559
Likewise, to our listeners, if you're a detection engineer and you haven't looked at your own AI or agentic tooling as an attack service yet.
00:45:50.719 --> 00:45:54.079
I think this is a really fantastic place to start and you're prompt to go do that.
00:45:54.159 --> 00:45:56.800
I'll link all of the resources in our show notes.
00:45:56.960 --> 00:46:03.039
So go actually check out what you're logging and you know, not just bookmark it on and just save it for later.
00:46:03.199 --> 00:46:04.480
Literally actually tune in.
00:46:04.639 --> 00:46:07.360
So thank you all for joining us on Dispatch today.
00:46:07.599 --> 00:46:08.719
We'll see you next time.
