WEBVTT
00:00:03.520 --> 00:00:13.279
From our headquarters in Oslo Norway, and on behalf of our host Robby Peralta, welcome to the mnemonic security podcast.
00:00:15.279 --> 00:00:18.879
An attacking agent that fails might get caught.
00:00:18.879 --> 00:00:23.359
A defending agent that fails might take down its own business.
00:00:23.359 --> 00:00:27.679
Attackers get retries, defenders get consequences.
00:00:27.679 --> 00:00:32.000
And the recent Hugging Face incident exposed another problem.
00:00:32.000 --> 00:00:40.320
OpenAI's agent took more than 17,000 autonomous actions over a weekend, escaping its sandbox and ultimately breaking into Hugging Face.
00:00:40.320 --> 00:00:54.159
And when Hugging Face tried to investigate what happened, they said Claude Opus and Fable refused much of the reverse engineering work because their guardrails struggled to distinguish analyzing an exploit from actually carrying one out.
00:00:54.159 --> 00:00:55.600
Lovely.
00:00:55.600 --> 00:00:57.840
So what do defenders do?
00:00:57.840 --> 00:01:02.960
Do we keep relying on frontier models that can refuse us when things get uncomfortable?
00:01:02.960 --> 00:01:09.200
Or do we bring the models in-house, run them locally, and take control of the guardrails ourselves?
00:01:09.200 --> 00:01:17.120
One thing is for sure, that notion of attackers moving at machine speed is finally coming true.
00:01:18.319 --> 00:01:21.280
Matteo Strada, welcome to the podcast.
00:01:21.760 --> 00:01:23.680
Thank you very much for inviting me.
00:01:23.680 --> 00:01:24.480
It's a pleasure.
00:01:25.040 --> 00:01:25.680
Ciao Bello.
00:01:25.680 --> 00:01:26.959
You're in Milan, right?
00:01:27.120 --> 00:01:28.879
Yeah, right now I'm in Milan.
00:01:29.200 --> 00:01:29.920
Lovely.
00:01:29.920 --> 00:01:31.120
Lucky man.
00:01:31.120 --> 00:01:35.439
So I came across to you in your blog, uh, which I'll put in the show notes.
00:01:35.439 --> 00:01:39.200
I think you called it the asymmetric feature of AI and cybersecurity.
00:01:39.200 --> 00:01:41.760
What was running through your head when you wrote that?
00:01:42.079 --> 00:01:50.799
So I think that there is an symmetry, mainly because attackers can use these tools to perform cybersecurity attacks.
00:01:50.799 --> 00:01:55.120
And the worst scenario that they get is that they get detected.
00:01:55.120 --> 00:01:59.280
The vulnerability that they're trying to exploit gets fixed.
00:01:59.280 --> 00:02:03.280
And so this is their worst scenario, detection.
00:02:03.280 --> 00:02:17.919
On the other side, if we think about defenders in companies try to automate with AI agents all the uh cybersecurity processes, we have uh a different scenario.
00:02:17.919 --> 00:02:21.120
And here um we have the asymmetry.
00:02:21.120 --> 00:02:34.560
Because if the AI agent makes some mistakes, then you have some production issues and some business uh issue, and you're going to lose some money.
00:02:34.560 --> 00:02:42.960
So the the freedom that you have on one side as an attacker is much broader than if you are a defender.
00:02:42.960 --> 00:02:50.560
So I think that this is the the biggest asymmetry that we're going to see uh even more in the future.
00:02:50.560 --> 00:02:56.159
And that we also started to see uh with the aging face cybersecurity incident.
00:02:56.159 --> 00:03:00.639
Um so if you want, I can try to make a summary of what happened.
00:03:00.639 --> 00:03:01.919
Yeah, that'd be great.
00:03:01.919 --> 00:03:11.840
Yeah, so from from what we know uh right now, OpenAI was testing different models on different uh cybersecurity benchmarks.
00:03:11.840 --> 00:03:20.479
And during one of these executions, one model had a goal of solving one of these tasks of the benchmark.
00:03:20.479 --> 00:03:33.599
And the model to solve this task was able to find an exploit to go outside, to break the sandbox, which is like an isolated environment where these tests are usually run.
00:03:33.599 --> 00:03:53.280
And uh so it was able to break this sandbox, to go outside the safe network, to search for a library that the model was thinking that would allow him to solve the test, uh, to find a vulnerability inside that library and break into uh Hagging Face.
00:03:53.280 --> 00:03:58.000
So an extremely sophisticated attack.
00:03:58.000 --> 00:04:03.199
And I think that here is where the asymmetry came in.
00:04:03.199 --> 00:04:27.279
Because Hugging Face detected this breach, and as they wrote inside their incident post, they were not able to use like public models, like closed source model to investigate this issue because GPT was blocking their requests, thinking that Hugging Face was trying to use this this uh information in a malicious way.
00:04:27.279 --> 00:04:36.560
So they they had to use open source model, they use a GLM uh model to help them investigate this issue.
00:04:36.560 --> 00:04:41.120
So it's definitely clear that we have an asymmetry.
00:04:41.120 --> 00:04:49.839
And this topic is also linked on how the current models are are measuring whether we can do some actions and whether we cannot.
00:04:49.839 --> 00:05:05.680
And there we have a threshold, and we are moving that threshold one side, we we allow the model to release information, and if we move it on the other side, we are going to say to the model that uh it should not release too many information.
00:05:05.680 --> 00:05:12.000
And that threshold is basically one of the most important things in this in this asymmetry discussion.
00:05:12.000 --> 00:05:22.240
Because if you move it too much on one side, you have too many refusals from the model, and if you move it too much on the other side, it's going to release too many information.
00:05:22.240 --> 00:05:29.759
So I think that this is all this would be one of the um central point of of uh future discussions.
00:05:30.079 --> 00:05:34.319
Do you think it's normal for companies to use models in the incident response situation?
00:05:34.319 --> 00:05:37.279
Do you think that's like the the business as usual moving forward?
00:05:37.279 --> 00:05:46.160
Because usually you would call like a company if you didn't have the resources, but maybe now people just throw everything at a model instead of calling calling incident response firms.
00:05:46.160 --> 00:05:47.759
Do you have any thoughts about that?
00:05:48.079 --> 00:06:01.040
Yeah, I think that LLM models can can help uh this type of processes because they may see some some connections, uh some relations between logs that a human uh didn't see.
00:06:01.040 --> 00:06:11.199
Um at the same time, I also think that the the the human part still plays a central role in all the cybersecurity processes.
00:06:11.600 --> 00:06:20.000
So yeah, when they when they did use the model for incident response purposes, it just said we can't help you with this because it looks like you're trying to do something that we're not gonna allow you to do.
00:06:20.000 --> 00:06:25.040
At the same time, uh this was going on maybe a couple days before Kidmy was released.
00:06:25.040 --> 00:06:28.000
Uh would you explain why that's relevant here?
00:06:29.120 --> 00:06:29.680
Yeah.
00:06:29.680 --> 00:06:51.680
So right now we have closed source model that usually have better capabilities, like Fable, Mythos, GPT 5.6 SOL, and then we have open source model, GPT OSS, which is a bit old, but Deep Seek, uh V4, GLM, and uh and much more.
00:06:51.680 --> 00:06:59.360
So as we saw with the Alligame Phase incident, uh local models can have an extremely important role.
00:06:59.360 --> 00:07:02.879
For I think for three different reasons.
00:07:02.879 --> 00:07:10.240
So first reason is that you have some um some expected costs, right?
00:07:10.240 --> 00:07:20.160
So when you're using a closed source model like GPT, OpenAI may just change pricing tomorrow and you have to accept it, right?
00:07:20.160 --> 00:07:33.759
Uh if you create a pipeline for local models, you know what you're going to spend as an upfront cost for the hardware and how much you're going to spend for the usage of the model.
00:07:33.759 --> 00:07:38.319
The second element I think is about availability.
00:07:38.319 --> 00:07:49.279
So a few weeks back, we saw how the US government blocked the release of uh of Fable and the GPT 5.6.
00:07:49.279 --> 00:07:55.199
So with uh if you have a local model, you don't have these problems.
00:07:55.199 --> 00:08:02.560
You are you're owning the model, you're owning the inference, so you will not have any availability problem.
00:08:02.560 --> 00:08:14.079
If tomorrow a jailbreak uh comes out for GPT 5.6 and the government is going to block it again, you you will not have access to the model, right?
00:08:14.079 --> 00:08:19.120
But if you if you have local inference, this is not an issue.
00:08:19.120 --> 00:08:22.079
And uh and finally you have more control.
00:08:22.079 --> 00:08:31.600
So as we saw uh with the aggregate incident, with local models, you usually can bypass the guard delays easier.
00:08:31.600 --> 00:08:39.840
Uh, you can perform uh steering with the model, so you have more control also on cybersecurity topics.
00:08:39.840 --> 00:08:44.639
And uh as a last point, you have a full data control, right?
00:08:44.639 --> 00:08:54.399
So I think that we should remember that when we are using um like a closed source AI model, we are sending information to a third party, right?
00:08:54.399 --> 00:09:11.120
So we we may, of course, accept this risk because there are contracts and we are trusting that party, but for some extremely confidential information, we may not want to send information data to a third party.
00:09:11.120 --> 00:09:16.000
And so I think that this is another important element to keep in mind.
00:09:16.320 --> 00:09:21.679
It just sounds like to me, there must be so few use cases where you actually need a frontier model.
00:09:21.679 --> 00:09:28.879
Like I get it if you're doing exploit development, because that's where you actually see the differences between the the frontier labs and the on-prem models.
00:09:28.879 --> 00:09:33.279
But it sounds like there are so many downsides to using one of these closed source models.
00:09:33.279 --> 00:09:34.320
Does everybody agree with that?
00:09:34.320 --> 00:09:36.399
Or is that sort of like a new thought?
00:09:36.879 --> 00:09:44.559
I think that the the most popular thought is that you want to use the best tool out there, right?
00:09:44.559 --> 00:09:48.000
So you're not trying other things, right?
00:09:48.000 --> 00:09:52.639
You're just using the best tool and that's uh that that works pretty good.
00:09:52.639 --> 00:09:53.919
Or do the job, yeah.
00:09:53.919 --> 00:09:54.480
Yeah.
00:09:54.480 --> 00:10:01.279
And if you're using it just for coding, for personal use, I think it's it's extremely fine.
00:10:01.279 --> 00:10:16.799
But if we talk about LLM as being an asset, like a strategic asset in an enterprise environment, I think that you should also think about the the long-term cost, the availability, and all the other uh points that we discussed before.
00:10:17.200 --> 00:10:22.559
How should one be architecting the usage of LLMs for normal employees?
00:10:22.559 --> 00:10:25.759
I'm thinking of uh the concept of like token maxing.
00:10:25.759 --> 00:10:32.480
You shouldn't be using the best models for what I use it for, like, hey, sum up this uh blog post that I read.
00:10:32.799 --> 00:10:36.639
Yeah, so the first option is uh awareness, right?
00:10:36.639 --> 00:10:48.879
If you explain to people that for some tasks, using some models may be better than using other models, they are going just to choose the right model for a specific task.
00:10:48.879 --> 00:10:50.080
In the browser, right?
00:10:50.159 --> 00:10:51.759
You just click the yeah.
00:10:52.159 --> 00:10:57.279
Um at the same time, I I understand that this may be a bit complicated.
00:10:57.279 --> 00:11:04.480
You just want to write something and uh uh you don't want to make the choice for every task.
00:11:04.480 --> 00:11:14.320
So um there are different orchestrators that based on the request, they're going to use like a router that is going to select the best model.
00:11:14.320 --> 00:11:19.440
So if for easy tasks, they're going to select a very cheap model.
00:11:19.440 --> 00:11:26.960
And if a task requires more reasoning, they're going to select a bigger and more advanced model.
00:11:26.960 --> 00:11:33.360
The model selector could also um could also see if you uploaded any sensitive information.
00:11:33.360 --> 00:11:36.799
And in that case, it's going to throw you to the local models.
00:11:36.799 --> 00:11:39.679
That that may be another uh interesting use case.
00:11:39.679 --> 00:11:46.240
But again, right now we don't uh we don't have the like the cost issue, right?
00:11:46.240 --> 00:11:53.120
Everybody just spending tokens and tokens, but I don't think that that's sustainable over time, right?
00:11:53.120 --> 00:11:57.200
We should start to think about these um these optimizations.
00:11:57.519 --> 00:12:02.480
One thing I want to touch on before we uh go away from um open source versus frontier lab stuff.
00:12:02.480 --> 00:12:05.039
Did you see the interview with Palantir CEO?
00:12:05.519 --> 00:12:06.320
Which one?
00:12:06.639 --> 00:12:24.320
Basically, he came out and said that he thinks it's crazy that enterprises are building their business, their processes inside of the frontier labs because they're basically giving all their intellectual property to the frontier labs and giving them all they need to be able to copy their business process, right?
00:12:24.320 --> 00:12:27.440
They can literally go in and see how exactly my business works.
00:12:27.440 --> 00:12:30.799
And if it was novel enough and they saw an opportunity, they could just steal it.
00:12:30.799 --> 00:12:31.759
Do you see his point?
00:12:31.759 --> 00:12:33.519
Or what do you what do you think about that?
00:12:33.919 --> 00:12:38.320
I don't know specifically the open AI uh policy.
00:12:38.320 --> 00:12:46.960
Uh I assume that they're not able to to create a a profile of of Robbie and make decisions based on that.
00:12:46.960 --> 00:13:05.840
Uh, but if you are a company and you're using these tools to reason over uh secret projects or IPs, then I think you should it should be like a center of discussion, the fact that you're you're sending data to a third party.
00:13:06.159 --> 00:13:09.360
What are the downsides of using open source models?
00:13:10.879 --> 00:13:14.559
So the first downside is upfront cost.
00:13:14.559 --> 00:13:15.519
Hardware.
00:13:15.519 --> 00:13:16.639
Yeah.
00:13:16.639 --> 00:13:20.320
And uh right now hardware costs um a lot.
00:13:20.320 --> 00:13:28.159
Nvidia is selling the GPU that we're using for inference, so that we're using when we use a model, right?
00:13:28.159 --> 00:13:39.919
For an enterprise use, of course, you are expecting to buy some some very powerful GPUs because then you have to distribute the model across different people.
00:13:39.919 --> 00:13:48.480
If you're talking about just personal use, they're starting to set some um some workstation, they're usually pretty small.
00:13:48.480 --> 00:13:56.240
For example, um NVIDIA, I think it's called NVIDIA Spark, uh, which is three or four thousand dollars.
00:13:56.240 --> 00:13:59.360
Of course, um Apple MacBook.
00:13:59.360 --> 00:14:05.600
Um, since they have a unified memory you can use to for the model inference.
00:14:05.600 --> 00:14:11.679
And um we are starting to see different uh local inference engine projects.
00:14:11.679 --> 00:14:24.480
A famous one is uh is Dwalf Stars from AntiReds, the creator of uh Redis, where with MacBook Pro, you can you can basically run DeepSec uh Flash with a quite high speed.
00:14:24.480 --> 00:14:36.240
And um and also they they're starting to implement also SSD streaming, where you can use the SSD to um as a as a storage for for the model.
00:14:36.240 --> 00:14:42.159
Uh so even if you don't have high uh memory, you can you can use it.
00:14:42.159 --> 00:14:47.919
Of course, uh uh it would be a bit slower, uh, but you can you can still use it.
00:14:48.320 --> 00:15:01.679
That means you can be running a model locally and asking it whatever you want without the oversight of anybody because it's running locally, but you're still being able to use the fancy model because it's some you're pulling it from somewhere else.
00:15:02.000 --> 00:15:03.200
Yeah, yeah, exactly.
00:15:03.200 --> 00:15:11.600
Of course, the model will be uh smaller, so you will not have the same reasoning capabilities of uh using closed source model.
00:15:11.600 --> 00:15:20.879
Um but as we discussed before, you may not need uh such a bigger model for for every task and for the most confidential ones.
00:15:20.879 --> 00:15:23.279
You that that that may be enough.
00:15:23.759 --> 00:15:29.519
When you have an on-prem model that's running locally, nobody can control what you're doing anymore or see what you're doing.
00:15:29.519 --> 00:15:31.440
Would you mind elaborating that a little more?
00:15:31.440 --> 00:15:33.200
Or did I understand that correctly?
00:15:33.519 --> 00:15:35.840
Yeah, no, that that's an another open point.
00:15:35.840 --> 00:15:42.879
Mainly because with local model, it's easier to bypass all the guard rates that the model the model have.
00:15:42.879 --> 00:15:55.440
So when we are using like GPT, you have different protection layers, different guard lays, of course, you have the alignment phase, but also other types of guard rates that they put before.
00:15:55.440 --> 00:15:59.679
If you use a local model, you have access to the model itself.
00:15:59.679 --> 00:16:03.200
So you can you can try to brute force the model.
00:16:03.200 --> 00:16:07.919
So you can use maybe hundreds of different prompt and try to jailbreak the model.
00:16:07.919 --> 00:16:14.799
While if you use GPT or anthropic models, they're going to detect it uh like pretty fast.
00:16:14.799 --> 00:16:44.559
So while local model on one side allows you the freedom on the defense side, so on all the benign usage, on the other side, it may be an interesting tool for attackers, where you if you can get a powerful local model and you're able to unlock it and through the usage of some jailbreak, you can use it to perform attacks.
00:16:44.559 --> 00:17:10.559
And while this is just like a forecasting game, I don't know uh what the future will be, but definitely one of the possible use cases where you have maybe tens or hundreds of these models that are going to analyze the system that you want to attack, they're going to coordinate one with the other, and they're going to perform the attack.
00:17:10.559 --> 00:17:29.519
So what one one note on this possible uh scenario is that right now we don't have any model except for like GPT 5.6 Sol and the Mythos that are able to make cybersecurity attacks in a consistent ways, right?
00:17:29.519 --> 00:17:53.759
So one interesting study from the AI security research team from the UK, I don't know if if the name is exactly that, but they tested with a benchmark where a model was asked to bridge into a system that had like 30 different steps to do to reach the um like the final uh step.
00:17:53.759 --> 00:18:11.200
And uh a lot of models were able to do the first like five or six steps as an average result, while uh bigger model like Mythos in 5.6 were able in the best run to achieve the latest uh step.
00:18:11.200 --> 00:18:15.599
And we're talking about like 30 different steps to to reach the final one.
00:18:15.599 --> 00:18:19.759
And this is the this was the best attempt.
00:18:19.759 --> 00:18:24.480
So it was not able to make it for every run.
00:18:24.480 --> 00:18:30.559
So right now we don't have like the consistency, but we don't need it probably.
00:18:30.559 --> 00:18:43.200
It is enough if you if you try and try it again, it is enough that one of these times you you're able to to breach because the model finds the right pattern, and and basically you're you're breaching a system.
00:18:43.200 --> 00:18:54.799
So when you're using tens of of these agents, you're you're increasing the likelihood of of finding like the right combination, the right path to to breach a system, basically.
00:18:56.640 --> 00:19:03.599
So alert triage, reverse engineering, vulnerability prioritization, threat intelligence summarization, detection engineering.
00:19:03.599 --> 00:19:06.000
You mentioned all these things in your your blog posts.
00:19:06.000 --> 00:19:08.480
How do you look at those aspects these days?
00:19:08.880 --> 00:19:16.720
Yeah, I think that all these areas of cybersecurity basically share two properties uh in common.
00:19:16.720 --> 00:19:22.799
So the first one is that there is a human who can cheaply check the output of a model.
00:19:22.799 --> 00:19:31.920
And uh the second one is that the the model uh if you get the answer wrong, it's not touching like any live system.
00:19:31.920 --> 00:19:37.599
And this is linked to the asymmetry of damage that a defender can do.
00:19:37.599 --> 00:19:47.359
In software development, we saw that these models are much faster than human developers when it comes to writing uh functional code.
00:19:47.359 --> 00:19:58.559
There is uh like a bottleneck where uh software developers who are reviewing the code basically are slowing down the entire process, right?
00:19:58.559 --> 00:20:08.640
Because if you're using the a state-of-the-art model to write code, you are uh writing code much faster than a human could do.
00:20:08.640 --> 00:20:14.160
But if you're using a human to review the entire code, basically the the speed is the same, right?
00:20:14.160 --> 00:20:29.839
Um so you you you can try to to fix this bottleneck by using very large test suite where you can test the software and uh you see the results and you can fix and if you find any bug.
00:20:29.839 --> 00:20:34.079
I think that in cybersecurity it's uh it's a bit different.
00:20:34.079 --> 00:20:41.279
So you you want that human bottleneck for for the decision, right?
00:20:41.279 --> 00:20:55.119
What you can use the LMs for is to reduce the information, to reduce the number of alerts, to reduce the number of tickets that you get inside of the bottleneck, right?
00:20:55.119 --> 00:21:08.480
So you're reducing the sources uh that comes inside the bottleneck, but you still want a human bottleneck that is going to perform the last decisions, right?
00:21:08.480 --> 00:21:36.960
So I think the LLMs can help to uh prioritize vulnerabilities, to prioritize alerts, but at the at the end of the pipeline, you still want a human that perform like the the highest values decision that an agent may miss due to the context, or may may miss because it doesn't have some some information that a human uh may have.
00:21:38.160 --> 00:21:52.880
The implementations you've been doing, is it has it been building new things using LLMs, or have you been using a lot of time to in like you just kind of pointed to increase the efficiency and accuracy and improve existing processes?
00:21:53.279 --> 00:21:59.759
I think it was an approach from from Google where you use uh like 70% of your time.
00:21:59.759 --> 00:22:10.400
To run daily activities, then you use 20% of the time to optimize things, and then you use 10% of the time to experiment, right?
00:22:10.400 --> 00:22:27.279
You want to to make high risk testing on uh idea that you have that may work or may not, but you want to also use more time to optimize process that you already have, right?
00:22:27.279 --> 00:22:31.119
And then of course you have to uh do your your day job.
00:22:31.119 --> 00:22:31.839
Your day job.
00:22:31.839 --> 00:22:32.640
Yeah.
00:22:32.640 --> 00:22:34.799
At the end of the day, you do you have to do it.
00:22:34.799 --> 00:22:42.720
And linked to that, I think that it should not be just something that an internal AI team uh can do, right?
00:22:42.720 --> 00:22:52.400
Each team is made of the best person that can find the right idea to optimize their workflow, right?
00:22:52.400 --> 00:23:00.559
If I'm part of the AI team or the security team, I have no idea what's the best tool for a marketing team, right?
00:23:00.559 --> 00:23:17.440
So I think that the marketing team or whatever team it is should have the ability of creating with uh with these new tools, of course, in protected environment without creating a risk for the company.
00:23:17.440 --> 00:23:29.440
But I think they should have these this opportunity to build tools, to build internal software, something that they think can bring additional value and can optimize their workflow.
00:23:29.440 --> 00:23:45.680
Then these these types of tools should be reviewed by some other teams that are going to say, I mean, it's something that we can keep or it's something that it's not possible to do from the security team who will have to check what kind of information are involved.
00:23:45.680 --> 00:23:54.799
So you you have something uh very useful for that team, and that you know that it exists because you have a process in place.
00:23:55.200 --> 00:24:04.000
Yeah, I do feel that a lot of people these days are doing the they're maybe doing 70, 30, and then 50% of their uh evening time on the innovation part.
00:24:04.000 --> 00:24:07.759
But uh that's the life that we live in these days, trying not to fall behind.
00:24:07.759 --> 00:24:17.119
So there's security companies out there that you know help you with pen testing, GRC, security operations.
00:24:17.119 --> 00:24:24.960
There there must be a shift in the economy these days for security partners because everybody was used to doing what they're doing before.
00:24:24.960 --> 00:24:32.400
Now here comes large language models, and they are kind of eating security companies lunch, right?
00:24:32.400 --> 00:24:35.920
Because you could just ask a model today to do all these things.
00:24:35.920 --> 00:24:39.759
Where do you see the future of security vendors?
00:24:39.759 --> 00:24:44.160
Like when I started with cybersecurity, you would you would call people that knew what they're talking about, and you'd say, Hey, I have this project.
00:24:44.160 --> 00:24:44.960
What do you recommend?
00:24:44.960 --> 00:24:50.000
And maybe there'd be some software from some popular vendors, or you need an architect.
00:24:50.000 --> 00:24:55.759
But now I could I could totally see those people just going to an LLM and asking them and then just going with whatever it says.
00:24:59.279 --> 00:25:02.480
I think that it is maybe a risk, right?
00:25:02.480 --> 00:25:06.400
Overconfidence towards the LLM.
00:25:06.400 --> 00:25:14.480
I think that in the next years we are going to to see, I would call bubbles, uh, like internet bubbles and so on.
00:25:14.480 --> 00:25:26.000
And as part of this process, I think we should use LLMs on on our own to try to understand which limits LLMs have, right?
00:25:26.000 --> 00:25:54.400
Because if you if you know the limits of LLM, you can understand whether someone is selling you something that doesn't have any additional value that you can do on your own, or if it's something that can can bring some interesting feature uh that can simplify your your your working day, or find new vulnerabilities or solve some security issues.
00:25:54.400 --> 00:26:16.640
So at the end of the day, I think that as every other tool, we should use it, we should study it, we should uh understand it, and and when we have made more practice with it, we can make a more aware decision uh on whether to choose or not choose, for example, a vendor.
00:26:17.680 --> 00:26:19.440
70 2010, Mateo.
00:26:19.440 --> 00:26:20.400
I'm gonna go for that.
00:26:20.400 --> 00:26:23.920
I'm gonna make some calls to make sure that that's how we're doing it mnemonic as well.
00:26:23.920 --> 00:26:27.200
Thank you so much for your time today.
00:26:27.200 --> 00:26:28.079
We'll talk to you soon.
00:26:28.079 --> 00:26:28.480
Take care.
00:26:28.720 --> 00:26:29.920
Thank you very much, Robby.
00:26:30.240 --> 00:26:30.400
Bye.
00:26:30.400 --> 00:26:31.279
It was a pleasure.
00:26:31.279 --> 00:26:31.599
Thank you.
00:26:31.599 --> 00:26:31.920
Bye.
00:26:31.920 --> 00:26:33.359
It's always ciao.
00:26:34.319 --> 00:26:36.079
Well, that's all for today, folks.
00:26:36.079 --> 00:26:38.799
Thank you for tuning in to the mnemonic security podcast.
00:26:38.799 --> 00:26:47.119
If you have any concepts or ideas that you'd like us to discuss on future episodes, please feel free to hit me up on LinkedIn or to send us a mail to podcast@mneonic.no.
00:26:48.319 --> 00:26:49.920
Thank you for listening.
00:26:49.920 --> 00:26:51.200
We will see you next time.
00:00:03.520 --> 00:00:13.279
From our headquarters in Oslo Norway, and on behalf of our host Robby Peralta, welcome to the mnemonic security podcast.
00:00:15.279 --> 00:00:18.879
An attacking agent that fails might get caught.
00:00:18.879 --> 00:00:23.359
A defending agent that fails might take down its own business.
00:00:23.359 --> 00:00:27.679
Attackers get retries, defenders get consequences.
00:00:27.679 --> 00:00:32.000
And the recent Hugging Face incident exposed another problem.
00:00:32.000 --> 00:00:40.320
OpenAI's agent took more than 17,000 autonomous actions over a weekend, escaping its sandbox and ultimately breaking into Hugging Face.
00:00:40.320 --> 00:00:54.159
And when Hugging Face tried to investigate what happened, they said Claude Opus and Fable refused much of the reverse engineering work because their guardrails struggled to distinguish analyzing an exploit from actually carrying one out.
00:00:54.159 --> 00:00:55.600
Lovely.
00:00:55.600 --> 00:00:57.840
So what do defenders do?
00:00:57.840 --> 00:01:02.960
Do we keep relying on frontier models that can refuse us when things get uncomfortable?
00:01:02.960 --> 00:01:09.200
Or do we bring the models in-house, run them locally, and take control of the guardrails ourselves?
00:01:09.200 --> 00:01:17.120
One thing is for sure, that notion of attackers moving at machine speed is finally coming true.
00:01:18.319 --> 00:01:21.280
Matteo Strada, welcome to the podcast.
00:01:21.760 --> 00:01:23.680
Thank you very much for inviting me.
00:01:23.680 --> 00:01:24.480
It's a pleasure.
00:01:25.040 --> 00:01:25.680
Ciao Bello.
00:01:25.680 --> 00:01:26.959
You're in Milan, right?
00:01:27.120 --> 00:01:28.879
Yeah, right now I'm in Milan.
00:01:29.200 --> 00:01:29.920
Lovely.
00:01:29.920 --> 00:01:31.120
Lucky man.
00:01:31.120 --> 00:01:35.439
So I came across to you in your blog, uh, which I'll put in the show notes.
00:01:35.439 --> 00:01:39.200
I think you called it the asymmetric feature of AI and cybersecurity.
00:01:39.200 --> 00:01:41.760
What was running through your head when you wrote that?
00:01:42.079 --> 00:01:50.799
So I think that there is an symmetry, mainly because attackers can use these tools to perform cybersecurity attacks.
00:01:50.799 --> 00:01:55.120
And the worst scenario that they get is that they get detected.
00:01:55.120 --> 00:01:59.280
The vulnerability that they're trying to exploit gets fixed.
00:01:59.280 --> 00:02:03.280
And so this is their worst scenario, detection.
00:02:03.280 --> 00:02:17.919
On the other side, if we think about defenders in companies try to automate with AI agents all the uh cybersecurity processes, we have uh a different scenario.
00:02:17.919 --> 00:02:21.120
And here um we have the asymmetry.
00:02:21.120 --> 00:02:34.560
Because if the AI agent makes some mistakes, then you have some production issues and some business uh issue, and you're going to lose some money.
00:02:34.560 --> 00:02:42.960
So the the freedom that you have on one side as an attacker is much broader than if you are a defender.
00:02:42.960 --> 00:02:50.560
So I think that this is the the biggest asymmetry that we're going to see uh even more in the future.
00:02:50.560 --> 00:02:56.159
And that we also started to see uh with the aging face cybersecurity incident.
00:02:56.159 --> 00:03:00.639
Um so if you want, I can try to make a summary of what happened.
00:03:00.639 --> 00:03:01.919
Yeah, that'd be great.
00:03:01.919 --> 00:03:11.840
Yeah, so from from what we know uh right now, OpenAI was testing different models on different uh cybersecurity benchmarks.
00:03:11.840 --> 00:03:20.479
And during one of these executions, one model had a goal of solving one of these tasks of the benchmark.
00:03:20.479 --> 00:03:33.599
And the model to solve this task was able to find an exploit to go outside, to break the sandbox, which is like an isolated environment where these tests are usually run.
00:03:33.599 --> 00:03:53.280
And uh so it was able to break this sandbox, to go outside the safe network, to search for a library that the model was thinking that would allow him to solve the test, uh, to find a vulnerability inside that library and break into uh Hagging Face.
00:03:53.280 --> 00:03:58.000
So an extremely sophisticated attack.
00:03:58.000 --> 00:04:03.199
And I think that here is where the asymmetry came in.
00:04:03.199 --> 00:04:27.279
Because Hugging Face detected this breach, and as they wrote inside their incident post, they were not able to use like public models, like closed source model to investigate this issue because GPT was blocking their requests, thinking that Hugging Face was trying to use this this uh information in a malicious way.
00:04:27.279 --> 00:04:36.560
So they they had to use open source model, they use a GLM uh model to help them investigate this issue.
00:04:36.560 --> 00:04:41.120
So it's definitely clear that we have an asymmetry.
00:04:41.120 --> 00:04:49.839
And this topic is also linked on how the current models are are measuring whether we can do some actions and whether we cannot.
00:04:49.839 --> 00:05:05.680
And there we have a threshold, and we are moving that threshold one side, we we allow the model to release information, and if we move it on the other side, we are going to say to the model that uh it should not release too many information.
00:05:05.680 --> 00:05:12.000
And that threshold is basically one of the most important things in this in this asymmetry discussion.
00:05:12.000 --> 00:05:22.240
Because if you move it too much on one side, you have too many refusals from the model, and if you move it too much on the other side, it's going to release too many information.
00:05:22.240 --> 00:05:29.759
So I think that this is all this would be one of the um central point of of uh future discussions.
00:05:30.079 --> 00:05:34.319
Do you think it's normal for companies to use models in the incident response situation?
00:05:34.319 --> 00:05:37.279
Do you think that's like the the business as usual moving forward?
00:05:37.279 --> 00:05:46.160
Because usually you would call like a company if you didn't have the resources, but maybe now people just throw everything at a model instead of calling calling incident response firms.
00:05:46.160 --> 00:05:47.759
Do you have any thoughts about that?
00:05:48.079 --> 00:06:01.040
Yeah, I think that LLM models can can help uh this type of processes because they may see some some connections, uh some relations between logs that a human uh didn't see.
00:06:01.040 --> 00:06:11.199
Um at the same time, I also think that the the the human part still plays a central role in all the cybersecurity processes.
00:06:11.600 --> 00:06:20.000
So yeah, when they when they did use the model for incident response purposes, it just said we can't help you with this because it looks like you're trying to do something that we're not gonna allow you to do.
00:06:20.000 --> 00:06:25.040
At the same time, uh this was going on maybe a couple days before Kidmy was released.
00:06:25.040 --> 00:06:28.000
Uh would you explain why that's relevant here?
00:06:29.120 --> 00:06:29.680
Yeah.
00:06:29.680 --> 00:06:51.680
So right now we have closed source model that usually have better capabilities, like Fable, Mythos, GPT 5.6 SOL, and then we have open source model, GPT OSS, which is a bit old, but Deep Seek, uh V4, GLM, and uh and much more.
00:06:51.680 --> 00:06:59.360
So as we saw with the Alligame Phase incident, uh local models can have an extremely important role.
00:06:59.360 --> 00:07:02.879
For I think for three different reasons.
00:07:02.879 --> 00:07:10.240
So first reason is that you have some um some expected costs, right?
00:07:10.240 --> 00:07:20.160
So when you're using a closed source model like GPT, OpenAI may just change pricing tomorrow and you have to accept it, right?
00:07:20.160 --> 00:07:33.759
Uh if you create a pipeline for local models, you know what you're going to spend as an upfront cost for the hardware and how much you're going to spend for the usage of the model.
00:07:33.759 --> 00:07:38.319
The second element I think is about availability.
00:07:38.319 --> 00:07:49.279
So a few weeks back, we saw how the US government blocked the release of uh of Fable and the GPT 5.6.
00:07:49.279 --> 00:07:55.199
So with uh if you have a local model, you don't have these problems.
00:07:55.199 --> 00:08:02.560
You are you're owning the model, you're owning the inference, so you will not have any availability problem.
00:08:02.560 --> 00:08:14.079
If tomorrow a jailbreak uh comes out for GPT 5.6 and the government is going to block it again, you you will not have access to the model, right?
00:08:14.079 --> 00:08:19.120
But if you if you have local inference, this is not an issue.
00:08:19.120 --> 00:08:22.079
And uh and finally you have more control.
00:08:22.079 --> 00:08:31.600
So as we saw uh with the aggregate incident, with local models, you usually can bypass the guard delays easier.
00:08:31.600 --> 00:08:39.840
Uh, you can perform uh steering with the model, so you have more control also on cybersecurity topics.
00:08:39.840 --> 00:08:44.639
And uh as a last point, you have a full data control, right?
00:08:44.639 --> 00:08:54.399
So I think that we should remember that when we are using um like a closed source AI model, we are sending information to a third party, right?
00:08:54.399 --> 00:09:11.120
So we we may, of course, accept this risk because there are contracts and we are trusting that party, but for some extremely confidential information, we may not want to send information data to a third party.
00:09:11.120 --> 00:09:16.000
And so I think that this is another important element to keep in mind.
00:09:16.320 --> 00:09:21.679
It just sounds like to me, there must be so few use cases where you actually need a frontier model.
00:09:21.679 --> 00:09:28.879
Like I get it if you're doing exploit development, because that's where you actually see the differences between the the frontier labs and the on-prem models.
00:09:28.879 --> 00:09:33.279
But it sounds like there are so many downsides to using one of these closed source models.
00:09:33.279 --> 00:09:34.320
Does everybody agree with that?
00:09:34.320 --> 00:09:36.399
Or is that sort of like a new thought?
00:09:36.879 --> 00:09:44.559
I think that the the most popular thought is that you want to use the best tool out there, right?
00:09:44.559 --> 00:09:48.000
So you're not trying other things, right?
00:09:48.000 --> 00:09:52.639
You're just using the best tool and that's uh that that works pretty good.
00:09:52.639 --> 00:09:53.919
Or do the job, yeah.
00:09:53.919 --> 00:09:54.480
Yeah.
00:09:54.480 --> 00:10:01.279
And if you're using it just for coding, for personal use, I think it's it's extremely fine.
00:10:01.279 --> 00:10:16.799
But if we talk about LLM as being an asset, like a strategic asset in an enterprise environment, I think that you should also think about the the long-term cost, the availability, and all the other uh points that we discussed before.
00:10:17.200 --> 00:10:22.559
How should one be architecting the usage of LLMs for normal employees?
00:10:22.559 --> 00:10:25.759
I'm thinking of uh the concept of like token maxing.
00:10:25.759 --> 00:10:32.480
You shouldn't be using the best models for what I use it for, like, hey, sum up this uh blog post that I read.
00:10:32.799 --> 00:10:36.639
Yeah, so the first option is uh awareness, right?
00:10:36.639 --> 00:10:48.879
If you explain to people that for some tasks, using some models may be better than using other models, they are going just to choose the right model for a specific task.
00:10:48.879 --> 00:10:50.080
In the browser, right?
00:10:50.159 --> 00:10:51.759
You just click the yeah.
00:10:52.159 --> 00:10:57.279
Um at the same time, I I understand that this may be a bit complicated.
00:10:57.279 --> 00:11:04.480
You just want to write something and uh uh you don't want to make the choice for every task.
00:11:04.480 --> 00:11:14.320
So um there are different orchestrators that based on the request, they're going to use like a router that is going to select the best model.
00:11:14.320 --> 00:11:19.440
So if for easy tasks, they're going to select a very cheap model.
00:11:19.440 --> 00:11:26.960
And if a task requires more reasoning, they're going to select a bigger and more advanced model.
00:11:26.960 --> 00:11:33.360
The model selector could also um could also see if you uploaded any sensitive information.
00:11:33.360 --> 00:11:36.799
And in that case, it's going to throw you to the local models.
00:11:36.799 --> 00:11:39.679
That that may be another uh interesting use case.
00:11:39.679 --> 00:11:46.240
But again, right now we don't uh we don't have the like the cost issue, right?
00:11:46.240 --> 00:11:53.120
Everybody just spending tokens and tokens, but I don't think that that's sustainable over time, right?
00:11:53.120 --> 00:11:57.200
We should start to think about these um these optimizations.
00:11:57.519 --> 00:12:02.480
One thing I want to touch on before we uh go away from um open source versus frontier lab stuff.
00:12:02.480 --> 00:12:05.039
Did you see the interview with Palantir CEO?
00:12:05.519 --> 00:12:06.320
Which one?
00:12:06.639 --> 00:12:24.320
Basically, he came out and said that he thinks it's crazy that enterprises are building their business, their processes inside of the frontier labs because they're basically giving all their intellectual property to the frontier labs and giving them all they need to be able to copy their business process, right?
00:12:24.320 --> 00:12:27.440
They can literally go in and see how exactly my business works.
00:12:27.440 --> 00:12:30.799
And if it was novel enough and they saw an opportunity, they could just steal it.
00:12:30.799 --> 00:12:31.759
Do you see his point?
00:12:31.759 --> 00:12:33.519
Or what do you what do you think about that?
00:12:33.919 --> 00:12:38.320
I don't know specifically the open AI uh policy.
00:12:38.320 --> 00:12:46.960
Uh I assume that they're not able to to create a a profile of of Robbie and make decisions based on that.
00:12:46.960 --> 00:13:05.840
Uh, but if you are a company and you're using these tools to reason over uh secret projects or IPs, then I think you should it should be like a center of discussion, the fact that you're you're sending data to a third party.
00:13:06.159 --> 00:13:09.360
What are the downsides of using open source models?
00:13:10.879 --> 00:13:14.559
So the first downside is upfront cost.
00:13:14.559 --> 00:13:15.519
Hardware.
00:13:15.519 --> 00:13:16.639
Yeah.
00:13:16.639 --> 00:13:20.320
And uh right now hardware costs um a lot.
00:13:20.320 --> 00:13:28.159
Nvidia is selling the GPU that we're using for inference, so that we're using when we use a model, right?
00:13:28.159 --> 00:13:39.919
For an enterprise use, of course, you are expecting to buy some some very powerful GPUs because then you have to distribute the model across different people.
00:13:39.919 --> 00:13:48.480
If you're talking about just personal use, they're starting to set some um some workstation, they're usually pretty small.
00:13:48.480 --> 00:13:56.240
For example, um NVIDIA, I think it's called NVIDIA Spark, uh, which is three or four thousand dollars.
00:13:56.240 --> 00:13:59.360
Of course, um Apple MacBook.
00:13:59.360 --> 00:14:05.600
Um, since they have a unified memory you can use to for the model inference.
00:14:05.600 --> 00:14:11.679
And um we are starting to see different uh local inference engine projects.
00:14:11.679 --> 00:14:24.480
A famous one is uh is Dwalf Stars from AntiReds, the creator of uh Redis, where with MacBook Pro, you can you can basically run DeepSec uh Flash with a quite high speed.
00:14:24.480 --> 00:14:36.240
And um and also they they're starting to implement also SSD streaming, where you can use the SSD to um as a as a storage for for the model.
00:14:36.240 --> 00:14:42.159
Uh so even if you don't have high uh memory, you can you can use it.
00:14:42.159 --> 00:14:47.919
Of course, uh uh it would be a bit slower, uh, but you can you can still use it.
00:14:48.320 --> 00:15:01.679
That means you can be running a model locally and asking it whatever you want without the oversight of anybody because it's running locally, but you're still being able to use the fancy model because it's some you're pulling it from somewhere else.
00:15:02.000 --> 00:15:03.200
Yeah, yeah, exactly.
00:15:03.200 --> 00:15:11.600
Of course, the model will be uh smaller, so you will not have the same reasoning capabilities of uh using closed source model.
00:15:11.600 --> 00:15:20.879
Um but as we discussed before, you may not need uh such a bigger model for for every task and for the most confidential ones.
00:15:20.879 --> 00:15:23.279
You that that that may be enough.
00:15:23.759 --> 00:15:29.519
When you have an on-prem model that's running locally, nobody can control what you're doing anymore or see what you're doing.
00:15:29.519 --> 00:15:31.440
Would you mind elaborating that a little more?
00:15:31.440 --> 00:15:33.200
Or did I understand that correctly?
00:15:33.519 --> 00:15:35.840
Yeah, no, that that's an another open point.
00:15:35.840 --> 00:15:42.879
Mainly because with local model, it's easier to bypass all the guard rates that the model the model have.
00:15:42.879 --> 00:15:55.440
So when we are using like GPT, you have different protection layers, different guard lays, of course, you have the alignment phase, but also other types of guard rates that they put before.
00:15:55.440 --> 00:15:59.679
If you use a local model, you have access to the model itself.
00:15:59.679 --> 00:16:03.200
So you can you can try to brute force the model.
00:16:03.200 --> 00:16:07.919
So you can use maybe hundreds of different prompt and try to jailbreak the model.
00:16:07.919 --> 00:16:14.799
While if you use GPT or anthropic models, they're going to detect it uh like pretty fast.
00:16:14.799 --> 00:16:44.559
So while local model on one side allows you the freedom on the defense side, so on all the benign usage, on the other side, it may be an interesting tool for attackers, where you if you can get a powerful local model and you're able to unlock it and through the usage of some jailbreak, you can use it to perform attacks.
00:16:44.559 --> 00:17:10.559
And while this is just like a forecasting game, I don't know uh what the future will be, but definitely one of the possible use cases where you have maybe tens or hundreds of these models that are going to analyze the system that you want to attack, they're going to coordinate one with the other, and they're going to perform the attack.
00:17:10.559 --> 00:17:29.519
So what one one note on this possible uh scenario is that right now we don't have any model except for like GPT 5.6 Sol and the Mythos that are able to make cybersecurity attacks in a consistent ways, right?
00:17:29.519 --> 00:17:53.759
So one interesting study from the AI security research team from the UK, I don't know if if the name is exactly that, but they tested with a benchmark where a model was asked to bridge into a system that had like 30 different steps to do to reach the um like the final uh step.
00:17:53.759 --> 00:18:11.200
And uh a lot of models were able to do the first like five or six steps as an average result, while uh bigger model like Mythos in 5.6 were able in the best run to achieve the latest uh step.
00:18:11.200 --> 00:18:15.599
And we're talking about like 30 different steps to to reach the final one.
00:18:15.599 --> 00:18:19.759
And this is the this was the best attempt.
00:18:19.759 --> 00:18:24.480
So it was not able to make it for every run.
00:18:24.480 --> 00:18:30.559
So right now we don't have like the consistency, but we don't need it probably.
00:18:30.559 --> 00:18:43.200
It is enough if you if you try and try it again, it is enough that one of these times you you're able to to breach because the model finds the right pattern, and and basically you're you're breaching a system.
00:18:43.200 --> 00:18:54.799
So when you're using tens of of these agents, you're you're increasing the likelihood of of finding like the right combination, the right path to to breach a system, basically.
00:18:56.640 --> 00:19:03.599
So alert triage, reverse engineering, vulnerability prioritization, threat intelligence summarization, detection engineering.
00:19:03.599 --> 00:19:06.000
You mentioned all these things in your your blog posts.
00:19:06.000 --> 00:19:08.480
How do you look at those aspects these days?
00:19:08.880 --> 00:19:16.720
Yeah, I think that all these areas of cybersecurity basically share two properties uh in common.
00:19:16.720 --> 00:19:22.799
So the first one is that there is a human who can cheaply check the output of a model.
00:19:22.799 --> 00:19:31.920
And uh the second one is that the the model uh if you get the answer wrong, it's not touching like any live system.
00:19:31.920 --> 00:19:37.599
And this is linked to the asymmetry of damage that a defender can do.
00:19:37.599 --> 00:19:47.359
In software development, we saw that these models are much faster than human developers when it comes to writing uh functional code.
00:19:47.359 --> 00:19:58.559
There is uh like a bottleneck where uh software developers who are reviewing the code basically are slowing down the entire process, right?
00:19:58.559 --> 00:20:08.640
Because if you're using the a state-of-the-art model to write code, you are uh writing code much faster than a human could do.
00:20:08.640 --> 00:20:14.160
But if you're using a human to review the entire code, basically the the speed is the same, right?
00:20:14.160 --> 00:20:29.839
Um so you you you can try to to fix this bottleneck by using very large test suite where you can test the software and uh you see the results and you can fix and if you find any bug.
00:20:29.839 --> 00:20:34.079
I think that in cybersecurity it's uh it's a bit different.
00:20:34.079 --> 00:20:41.279
So you you want that human bottleneck for for the decision, right?
00:20:41.279 --> 00:20:55.119
What you can use the LMs for is to reduce the information, to reduce the number of alerts, to reduce the number of tickets that you get inside of the bottleneck, right?
00:20:55.119 --> 00:21:08.480
So you're reducing the sources uh that comes inside the bottleneck, but you still want a human bottleneck that is going to perform the last decisions, right?
00:21:08.480 --> 00:21:36.960
So I think the LLMs can help to uh prioritize vulnerabilities, to prioritize alerts, but at the at the end of the pipeline, you still want a human that perform like the the highest values decision that an agent may miss due to the context, or may may miss because it doesn't have some some information that a human uh may have.
00:21:38.160 --> 00:21:52.880
The implementations you've been doing, is it has it been building new things using LLMs, or have you been using a lot of time to in like you just kind of pointed to increase the efficiency and accuracy and improve existing processes?
00:21:53.279 --> 00:21:59.759
I think it was an approach from from Google where you use uh like 70% of your time.
00:21:59.759 --> 00:22:10.400
To run daily activities, then you use 20% of the time to optimize things, and then you use 10% of the time to experiment, right?
00:22:10.400 --> 00:22:27.279
You want to to make high risk testing on uh idea that you have that may work or may not, but you want to also use more time to optimize process that you already have, right?
00:22:27.279 --> 00:22:31.119
And then of course you have to uh do your your day job.
00:22:31.119 --> 00:22:31.839
Your day job.
00:22:31.839 --> 00:22:32.640
Yeah.
00:22:32.640 --> 00:22:34.799
At the end of the day, you do you have to do it.
00:22:34.799 --> 00:22:42.720
And linked to that, I think that it should not be just something that an internal AI team uh can do, right?
00:22:42.720 --> 00:22:52.400
Each team is made of the best person that can find the right idea to optimize their workflow, right?
00:22:52.400 --> 00:23:00.559
If I'm part of the AI team or the security team, I have no idea what's the best tool for a marketing team, right?
00:23:00.559 --> 00:23:17.440
So I think that the marketing team or whatever team it is should have the ability of creating with uh with these new tools, of course, in protected environment without creating a risk for the company.
00:23:17.440 --> 00:23:29.440
But I think they should have these this opportunity to build tools, to build internal software, something that they think can bring additional value and can optimize their workflow.
00:23:29.440 --> 00:23:45.680
Then these these types of tools should be reviewed by some other teams that are going to say, I mean, it's something that we can keep or it's something that it's not possible to do from the security team who will have to check what kind of information are involved.
00:23:45.680 --> 00:23:54.799
So you you have something uh very useful for that team, and that you know that it exists because you have a process in place.
00:23:55.200 --> 00:24:04.000
Yeah, I do feel that a lot of people these days are doing the they're maybe doing 70, 30, and then 50% of their uh evening time on the innovation part.
00:24:04.000 --> 00:24:07.759
But uh that's the life that we live in these days, trying not to fall behind.
00:24:07.759 --> 00:24:17.119
So there's security companies out there that you know help you with pen testing, GRC, security operations.
00:24:17.119 --> 00:24:24.960
There there must be a shift in the economy these days for security partners because everybody was used to doing what they're doing before.
00:24:24.960 --> 00:24:32.400
Now here comes large language models, and they are kind of eating security companies lunch, right?
00:24:32.400 --> 00:24:35.920
Because you could just ask a model today to do all these things.
00:24:35.920 --> 00:24:39.759
Where do you see the future of security vendors?
00:24:39.759 --> 00:24:44.160
Like when I started with cybersecurity, you would you would call people that knew what they're talking about, and you'd say, Hey, I have this project.
00:24:44.160 --> 00:24:44.960
What do you recommend?
00:24:44.960 --> 00:24:50.000
And maybe there'd be some software from some popular vendors, or you need an architect.
00:24:50.000 --> 00:24:55.759
But now I could I could totally see those people just going to an LLM and asking them and then just going with whatever it says.
00:24:59.279 --> 00:25:02.480
I think that it is maybe a risk, right?
00:25:02.480 --> 00:25:06.400
Overconfidence towards the LLM.
00:25:06.400 --> 00:25:14.480
I think that in the next years we are going to to see, I would call bubbles, uh, like internet bubbles and so on.
00:25:14.480 --> 00:25:26.000
And as part of this process, I think we should use LLMs on on our own to try to understand which limits LLMs have, right?
00:25:26.000 --> 00:25:54.400
Because if you if you know the limits of LLM, you can understand whether someone is selling you something that doesn't have any additional value that you can do on your own, or if it's something that can can bring some interesting feature uh that can simplify your your your working day, or find new vulnerabilities or solve some security issues.
00:25:54.400 --> 00:26:16.640
So at the end of the day, I think that as every other tool, we should use it, we should study it, we should uh understand it, and and when we have made more practice with it, we can make a more aware decision uh on whether to choose or not choose, for example, a vendor.
00:26:17.680 --> 00:26:19.440
70 2010, Mateo.
00:26:19.440 --> 00:26:20.400
I'm gonna go for that.
00:26:20.400 --> 00:26:23.920
I'm gonna make some calls to make sure that that's how we're doing it mnemonic as well.
00:26:23.920 --> 00:26:27.200
Thank you so much for your time today.
00:26:27.200 --> 00:26:28.079
We'll talk to you soon.
00:26:28.079 --> 00:26:28.480
Take care.
00:26:28.720 --> 00:26:29.920
Thank you very much, Robby.
00:26:30.240 --> 00:26:30.400
Bye.
00:26:30.400 --> 00:26:31.279
It was a pleasure.
00:26:31.279 --> 00:26:31.599
Thank you.
00:26:31.599 --> 00:26:31.920
Bye.
00:26:31.920 --> 00:26:33.359
It's always ciao.
00:26:34.319 --> 00:26:36.079
Well, that's all for today, folks.
00:26:36.079 --> 00:26:38.799
Thank you for tuning in to the mnemonic security podcast.
00:26:38.799 --> 00:26:47.119
If you have any concepts or ideas that you'd like us to discuss on future episodes, please feel free to hit me up on LinkedIn or to send us a mail to podcast@mneonic.no.
00:26:48.319 --> 00:26:49.920
Thank you for listening.
00:26:49.920 --> 00:26:51.200
We will see you next time.
