S4E22 – HuggingFace compromised by agentic attack, Gold Eagle program

WEBVTT
00:00:00.000 --> 00:00:02.759
If the question of, are there any AI enabled
00:00:02.759 --> 00:00:05.400
attacks? I think the answer is absolutely. There
00:00:05.400 --> 00:00:09.000
are absolutely autonomous attack harnesses that
00:00:09.000 --> 00:00:10.519
are being put out there and people are using
00:00:10.519 --> 00:00:29.469
them. Hey, Sunil. Good to see you, as always,
00:00:29.550 --> 00:00:32.210
on the Adversarial Podcast. Welcome. Hey, Jerry.
00:00:32.630 --> 00:00:36.289
Yeah, thanks. Hey, we're lacking Mario this week,
00:00:36.429 --> 00:00:38.250
unfortunately. It's summer and there's a lot
00:00:38.250 --> 00:00:40.869
of travel, and we've done a pretty good job of
00:00:40.869 --> 00:00:42.450
aligning three different schedules for a long
00:00:42.450 --> 00:00:44.850
time, but it was just near impossible, and I'm
00:00:44.850 --> 00:00:47.149
the guilty party because I'm off on holiday starting
00:00:47.149 --> 00:00:51.649
tomorrow. So this is a special Only Sunil and
00:00:51.649 --> 00:00:55.340
Jerry short episode. And we will bring Mario
00:00:55.340 --> 00:00:56.899
back into the fold as soon as we're all back
00:00:56.899 --> 00:00:59.640
together again in a couple of weeks. Oh fudge.
00:01:00.820 --> 00:01:02.479
Yeah, I know we need some sound bites to just
00:01:02.479 --> 00:01:06.819
say what the fudge Mario we miss you. We'll be
00:01:06.819 --> 00:01:09.700
back soon. Well, one of the reasons we want to
00:01:09.700 --> 00:01:11.939
jump in and do something anyway is because there's
00:01:11.939 --> 00:01:15.579
no shortage of headlines and here we go. So let's
00:01:15.579 --> 00:01:18.299
launch right into it. One of the things that
00:01:18.299 --> 00:01:23.519
is like top of mind for us this week. is a I
00:01:23.519 --> 00:01:25.099
was about to say a really cool incident disclosure.
00:01:25.500 --> 00:01:28.099
I'll tell you like in general, I think some of
00:01:28.099 --> 00:01:32.620
my favorite reading is when there's a really
00:01:32.620 --> 00:01:36.519
founder led company, where the founders understand
00:01:36.519 --> 00:01:38.000
what's going on in the nuts and bolts pretty
00:01:38.000 --> 00:01:41.280
closely. And then they have some kind of security
00:01:41.280 --> 00:01:45.579
intrusion or incident. And because the same person
00:01:45.579 --> 00:01:48.060
has the ability to pick how forthright they're
00:01:48.060 --> 00:01:51.599
going to be the CEO in this case. who also understands
00:01:51.599 --> 00:01:54.060
all the details, then you get these amazing write
00:01:54.060 --> 00:01:56.019
-ups like Matthew Prince, CloudFlare, I think
00:01:56.019 --> 00:01:58.680
is really well known for this. And in this case,
00:01:58.799 --> 00:02:03.719
it's out of Hugging Face, where they had a AI
00:02:03.719 --> 00:02:08.979
-related intrusion. And I think this is a watershed
00:02:08.979 --> 00:02:14.180
moment for a few reasons. Yes, Sunil? Yeah, it's...
00:02:14.379 --> 00:02:17.180
One of the few, I mean, I wouldn't say it's the
00:02:17.180 --> 00:02:20.819
only, but one of the few direct cases where they're
00:02:20.819 --> 00:02:25.599
publicizing or sharing details of a AI -driven
00:02:25.599 --> 00:02:28.599
attack. But what also makes this one particularly
00:02:28.599 --> 00:02:31.259
special is that they've shared details on how
00:02:31.259 --> 00:02:34.099
they've used their own autonomous AI agent system
00:02:34.099 --> 00:02:37.620
to defend as well. So this is, I think, one of
00:02:37.620 --> 00:02:41.879
the first write -ups that I've seen of AI attacking
00:02:41.879 --> 00:02:45.840
and AI defending. Yeah, and, you know, the defending
00:02:45.840 --> 00:02:47.560
is really important. And before we get into that,
00:02:47.639 --> 00:02:49.099
and you something that comes to mind for me is,
00:02:49.460 --> 00:02:51.759
you know, like I mentioned, you have the confidence
00:02:51.759 --> 00:02:54.300
to Venn diagram overlap between being willing
00:02:54.300 --> 00:02:55.560
to talk about it and knowing what the hell you're
00:02:55.560 --> 00:02:58.580
talking about. And that's, that's uncommon, you
00:02:58.580 --> 00:03:00.219
know, in general with incident management and
00:03:00.219 --> 00:03:03.439
cyber. And what is a similar thing was like,
00:03:03.439 --> 00:03:05.900
if you wind back the clock 15 years was kind
00:03:05.900 --> 00:03:09.509
of the not AI involved in attack, but APTs. And
00:03:09.509 --> 00:03:11.870
if you remember, there's I'd say a wave where
00:03:11.870 --> 00:03:13.830
everything was blamed on APTs. And it was like,
00:03:13.909 --> 00:03:16.490
well, we had a breach, but it was an APT. So
00:03:16.490 --> 00:03:19.030
you can't blame us. And what can we do? I bet
00:03:19.030 --> 00:03:21.090
we're gonna see some of that too. And I know
00:03:21.090 --> 00:03:23.509
this isn't the first time I've heard about AI
00:03:23.509 --> 00:03:26.270
involved in attacks. But this is exciting to
00:03:26.270 --> 00:03:27.870
me because when I read it, I actually believed
00:03:27.870 --> 00:03:30.509
it. And I'm not sure when I see that all the
00:03:30.509 --> 00:03:34.759
time. I mean, If you just use Claude while you're
00:03:34.759 --> 00:03:36.560
attacking someone of like, Hey, can you spell
00:03:36.560 --> 00:03:38.840
check this email? Is that an AI powered attack
00:03:38.840 --> 00:03:41.460
the way a lay person would interpret it? So this
00:03:41.460 --> 00:03:43.780
one I think actually is, and we should talk a
00:03:43.780 --> 00:03:45.939
little bit about like, what is it? What would
00:03:45.939 --> 00:03:49.759
it take for you to read something an incident
00:03:49.759 --> 00:03:52.580
report and be like, Oh yeah, this, this was AI.
00:03:53.180 --> 00:03:55.939
Yeah. Yeah. I think so from the, from the attacker
00:03:55.939 --> 00:03:59.080
or what they observed from the, uh, as a defender.
00:03:59.780 --> 00:04:01.560
There are a couple of characteristics I think
00:04:01.560 --> 00:04:04.240
we would look at and say, oh yeah, this is definitely
00:04:04.240 --> 00:04:06.319
not a human doing this. This is a machine doing
00:04:06.319 --> 00:04:09.280
it. So there's one perspective, which is if you
00:04:09.280 --> 00:04:12.879
see a certain activity that is just scripted
00:04:12.879 --> 00:04:15.479
and you can tell that it's a script that's running.
00:04:16.060 --> 00:04:18.220
And if all of the certain fairly predictable
00:04:18.220 --> 00:04:21.279
pattern or there's very little variation, then
00:04:21.279 --> 00:04:25.300
you have what hugging face saw, which was the
00:04:25.300 --> 00:04:27.860
speed and scale of what appears to be a scripted
00:04:27.860 --> 00:04:31.540
attack. But with Changes that are happening dynamically
00:04:31.540 --> 00:04:34.920
based on what's coming back, right? So iteration
00:04:34.920 --> 00:04:38.199
iterative attacking iterative attacking adjustments
00:04:38.199 --> 00:04:41.379
on the on the fly and at machine speed scale,
00:04:41.420 --> 00:04:46.139
which You wouldn't expect a script will fail
00:04:46.139 --> 00:04:49.100
Unless it's a really really advanced script that
00:04:49.100 --> 00:04:51.899
yeah You look at any logs and for failed logins
00:04:51.899 --> 00:04:53.740
on an internet connected system You see an example
00:04:53.740 --> 00:04:54.839
of what you're talking about where it's just
00:04:54.839 --> 00:04:56.439
the same crap over and over and over and over
00:04:56.439 --> 00:04:58.500
again Right, yeah, and so I think this notion
00:04:58.500 --> 00:05:02.240
that they're observing this really rapid adjustment
00:05:02.240 --> 00:05:05.300
is what makes this more of an agentic attack
00:05:05.300 --> 00:05:10.199
rather than just simple automation, that machine
00:05:10.199 --> 00:05:12.860
automation that does it at the speed that we're
00:05:12.860 --> 00:05:15.199
already accustomed to, but not of the kind of
00:05:15.199 --> 00:05:18.079
variations that I think Huggy and Faye saw. Yeah,
00:05:18.079 --> 00:05:21.199
and what struck me was exactly examples of what
00:05:21.199 --> 00:05:23.279
you're talking about, which is iteration, pivoting,
00:05:23.379 --> 00:05:25.720
real -time pivoting. And they didn't give like
00:05:25.720 --> 00:05:29.480
a X per Y rate metric through this, but I got
00:05:29.480 --> 00:05:31.779
the feeling they're saying they saw in a volume
00:05:31.779 --> 00:05:34.920
that that also tipped off a little bit too, that
00:05:34.920 --> 00:05:37.339
there was something, if not just like sheer volume
00:05:37.339 --> 00:05:39.620
and brute force, more like the speed at which
00:05:39.620 --> 00:05:42.319
the pivots would happen. That's right. Yeah.
00:05:42.779 --> 00:05:45.860
I think it was something like almost like 20
00:05:45.860 --> 00:05:48.600
,000 some events that happened in a relatively
00:05:48.600 --> 00:05:51.459
short period of time that, and again, you know,
00:05:51.480 --> 00:05:54.180
that's not that. many events in the grand scheme
00:05:54.180 --> 00:05:56.300
of things, but if that happens in a short period
00:05:56.300 --> 00:05:58.740
of time and you see clear evidence of pivoting
00:05:58.740 --> 00:06:02.120
or adjustments, then that strongly suggests that
00:06:02.120 --> 00:06:05.240
there's some sort of agentic harness behind the
00:06:05.240 --> 00:06:08.000
scenes that's guiding the actions of the agent.
00:06:08.819 --> 00:06:11.660
Yeah, and as I read this, and by the way, my
00:06:11.660 --> 00:06:14.639
source on this was their blog, which was wonderful,
00:06:15.199 --> 00:06:19.220
and the press coverage I saw this basically just...
00:06:19.069 --> 00:06:21.449
play drives their blog at best. I didn't have
00:06:21.449 --> 00:06:24.529
much other to say on that. But when you but it
00:06:24.529 --> 00:06:27.769
did mention, which I thought is interesting,
00:06:28.050 --> 00:06:30.449
it mentioned exploits. And I say that's interesting,
00:06:30.509 --> 00:06:33.050
because I think the to the lay person or the
00:06:33.050 --> 00:06:36.230
common narrative is that exploits like CV popping
00:06:36.230 --> 00:06:39.829
has a key role in the majority of intrusions
00:06:39.829 --> 00:06:42.540
that the world cares about. And I usually will
00:06:42.540 --> 00:06:44.560
kind of take the contrarian viewpoint say, yeah,
00:06:44.560 --> 00:06:46.800
it's more likely account takeover and social
00:06:46.800 --> 00:06:49.980
engineering. And, and so therefore, like, literal
00:06:49.980 --> 00:06:52.160
mythos and more CVs aren't as big a deal, because
00:06:52.160 --> 00:06:54.360
that's, it has its place, and it happens sometimes,
00:06:54.379 --> 00:06:58.120
but it's not massive. And here, it's saying that
00:06:58.120 --> 00:07:00.420
there were a couple of exploits, it didn't use
00:07:00.420 --> 00:07:02.579
the term CV, but I think he's the term vulnerability,
00:07:02.800 --> 00:07:05.360
just interesting, and that they fix those. So
00:07:05.360 --> 00:07:07.829
that implies that it's fixable. But what they
00:07:07.829 --> 00:07:12.089
didn't say was if there was like a zero day devised
00:07:12.089 --> 00:07:15.889
through the iteration, or if it was more just
00:07:15.889 --> 00:07:18.529
hard to find, but yeah, this was a vulnerability.
00:07:18.550 --> 00:07:19.790
And if they had known about it, it would have
00:07:19.790 --> 00:07:22.389
been on their risk register and or fixed. So
00:07:22.389 --> 00:07:24.829
it's hard to know whether I don't think it's
00:07:24.829 --> 00:07:28.589
a zero day live crafting, or that probably they
00:07:28.589 --> 00:07:31.370
wouldn't be able to resist sharing that, I think.
00:07:31.990 --> 00:07:35.240
But it is pretty interesting. that it was able
00:07:35.240 --> 00:07:38.680
to find and exploit and then pivot in lateral
00:07:38.680 --> 00:07:40.800
movement in the newer sense, not like machine
00:07:40.800 --> 00:07:44.720
to machine. Yeah, but keep in mind, it could
00:07:44.720 --> 00:07:48.300
have been as simple as this. The agentic harness
00:07:48.300 --> 00:07:51.579
could have been, hey, I've been contracted out
00:07:51.579 --> 00:07:55.720
by hugging face to go attack the public facing
00:07:55.720 --> 00:07:59.160
website, find a novel vulnerability, exploit
00:07:59.160 --> 00:08:03.129
it and see how far you can get. Right. That's
00:08:03.129 --> 00:08:07.670
what it did, right? And so they, in many ways,
00:08:08.029 --> 00:08:11.829
that simple expression of go try to find an awful
00:08:11.829 --> 00:08:13.629
vulnerability, which means that there is no,
00:08:13.850 --> 00:08:16.730
well, more than likely, there was no existing
00:08:16.730 --> 00:08:19.709
CVE, presuming that Huggy Base has done its basic
00:08:19.709 --> 00:08:23.269
due diligence to look for the common sort of
00:08:23.269 --> 00:08:25.649
craft that you would typically look for. But
00:08:25.649 --> 00:08:31.850
now you point an LLM. at your public -facing,
00:08:33.509 --> 00:08:38.389
customer -facing website and then have an agent
00:08:38.389 --> 00:08:42.370
go find something. If you don't give it, hey,
00:08:42.529 --> 00:08:45.730
please stop here, don't go further, just report
00:08:45.730 --> 00:08:48.830
on this and come back to me versus go attack
00:08:48.830 --> 00:08:52.110
and find out what you can find. I think that's
00:08:52.110 --> 00:08:54.039
the difference that you'll see. I, you know,
00:08:54.039 --> 00:08:56.679
we've done, I've done a ton of, um, pen testing
00:08:56.679 --> 00:08:59.639
with these agents, but you usually have it stop
00:08:59.639 --> 00:09:01.620
and say, give me a report before you go and act
00:09:01.620 --> 00:09:04.299
on anything. What I think happened here, what,
00:09:04.299 --> 00:09:07.000
what appears to be the case here is it found
00:09:07.000 --> 00:09:09.720
it and then act them on it and, and kept going.
00:09:09.879 --> 00:09:12.940
And, uh, that's the nature of the agent activity
00:09:12.940 --> 00:09:15.960
that I was referring to. Yeah. You know, it's,
00:09:16.080 --> 00:09:18.139
it's pretty interesting. So like on the initial
00:09:18.139 --> 00:09:20.720
exploit or, um, when I was initial on the exploit
00:09:20.720 --> 00:09:23.700
part of it, like. To be specific on it, because
00:09:23.700 --> 00:09:25.019
I remember I caught my eye and I just looked
00:09:25.019 --> 00:09:27.879
it back up again. What they said was, it was
00:09:27.879 --> 00:09:31.039
a data set code execution path or two of them
00:09:31.039 --> 00:09:34.200
actually, right. And I'm interpreting that as
00:09:34.200 --> 00:09:37.679
the product is designed so that people can upload
00:09:37.679 --> 00:09:41.600
data, let's say, but code. And oh, lo and behold,
00:09:41.720 --> 00:09:45.360
that code gets included into a routine that will
00:09:45.360 --> 00:09:47.639
execute what's in there with more privilege than
00:09:47.639 --> 00:09:50.990
you would like for a user to have. So you upload
00:09:50.990 --> 00:09:54.169
some stuff and purportedly to have it analyzed,
00:09:54.190 --> 00:09:55.710
but you know, really, or at least you're testing
00:09:55.710 --> 00:09:57.289
to see if it actually get executed. And it did
00:09:57.289 --> 00:09:59.889
in this case. And that was the vulnerability.
00:10:00.570 --> 00:10:02.509
So it could be so bespoke that it's not like
00:10:02.509 --> 00:10:04.570
CVE type material anyway, it could just be something
00:10:04.570 --> 00:10:08.350
very bespoke to hugging face. And then from there,
00:10:08.409 --> 00:10:10.169
it moved pretty quickly in the lateral movement.
00:10:11.070 --> 00:10:13.409
You know, it's interesting with the prevalence
00:10:13.409 --> 00:10:16.850
of attack service management. And I'm using that
00:10:16.850 --> 00:10:19.070
term broadly and inclusive of you and red teaming,
00:10:19.070 --> 00:10:22.409
right, like just look for the problems with the
00:10:22.409 --> 00:10:24.250
prevalence of companies in that space that are
00:10:24.250 --> 00:10:27.009
pivoting to AI to automate all of this. Because,
00:10:27.210 --> 00:10:30.049
you know, if you employed, if you bought a attack
00:10:30.049 --> 00:10:32.809
to watch tower attack service management or bug
00:10:32.809 --> 00:10:36.539
crowd bug bounty or, or red team. in the past,
00:10:36.580 --> 00:10:38.419
it was exactly like you said, whether it was
00:10:38.419 --> 00:10:40.460
manual or not, they'd get to a certain point.
00:10:40.460 --> 00:10:42.240
And then you'd give them the green light. And
00:10:42.240 --> 00:10:43.759
then they would say, Oh, actually, it turns out
00:10:43.759 --> 00:10:45.340
it was a nothing burger. And you kind of iterate.
00:10:46.419 --> 00:10:49.820
And so the Holy Grail is to have things automatically
00:10:49.820 --> 00:10:52.860
pivot and go on and ultimately get results. And
00:10:52.860 --> 00:10:55.399
in the process of creating those benevolent products,
00:10:55.440 --> 00:10:59.299
you're going to probably have things like this
00:10:59.299 --> 00:11:01.700
happen by accident. So I don't think it's out
00:11:01.700 --> 00:11:04.620
of the realm of possibility that at some point
00:11:04.620 --> 00:11:09.240
they say, My bad, sorry and embarrassing but
00:11:09.240 --> 00:11:13.139
uh One of one of your customers put you in as
00:11:13.139 --> 00:11:15.840
a vendor and we automatically crawled the vendors
00:11:15.840 --> 00:11:19.059
and kind of Didn't guard rail ourselves and went
00:11:19.059 --> 00:11:21.399
off to the races right that that could be something
00:11:21.399 --> 00:11:25.000
that comes out of this. We'll see Yeah, I think
00:11:25.000 --> 00:11:29.320
if Hopefully those who are a bit more cautious
00:11:29.320 --> 00:11:33.899
And aren't just going operating in Yolo mode
00:11:34.250 --> 00:11:38.610
will be a bit more conscientious about checking
00:11:38.610 --> 00:11:41.889
with a human before proceeding with actually
00:11:41.889 --> 00:11:45.570
attacking a system. And I think again, this is
00:11:45.570 --> 00:11:48.269
the distinction I would make. I wouldn't be too
00:11:48.269 --> 00:11:50.649
surprised if an attacker actually scouted it
00:11:50.649 --> 00:11:53.049
out first and then said, okay, now I've found
00:11:53.049 --> 00:11:58.529
the potential weaknesses, go take it to its max.
00:11:59.129 --> 00:12:00.990
The other general point I would also make here
00:12:00.990 --> 00:12:05.100
is that there is a asymmetric advantage for defenders,
00:12:05.379 --> 00:12:08.879
for attackers here, which is one's ability to
00:12:08.879 --> 00:12:14.379
determine that a particular vulnerability and
00:12:14.379 --> 00:12:16.519
subsequent exploit does achieve the objective
00:12:16.519 --> 00:12:20.379
that you have or not, is something that is easily
00:12:20.379 --> 00:12:23.820
verifiable by a machine. So the machine can continue
00:12:23.820 --> 00:12:26.419
to iterate over and over again until it gets
00:12:26.419 --> 00:12:29.820
to the point that it has verified that yes, this
00:12:29.820 --> 00:12:38.480
was exploitable. For defenders, the asymmetry
00:12:38.480 --> 00:12:41.840
exists for defenders on the configuration end,
00:12:42.120 --> 00:12:44.419
which is, do I know that my environment is configured
00:12:44.419 --> 00:12:49.720
properly? That's a very verifiable thing. So
00:12:49.720 --> 00:12:54.480
I think there's an asymmetry on both ends that
00:12:54.480 --> 00:12:56.659
we need to take advantage of. Rather, the attackers
00:12:56.659 --> 00:12:59.360
are taking advantage of the asymmetry and easily
00:12:59.360 --> 00:13:02.419
verifying exploits. We need to somehow take advantage
00:13:02.419 --> 00:13:05.159
of easily verifying configuration and ensuring
00:13:05.159 --> 00:13:07.620
that configurations are solid as well. Yeah.
00:13:07.620 --> 00:13:09.340
I mean, it's almost proving a negative versus
00:13:09.340 --> 00:13:11.960
proving a positive and naturally proving negatives
00:13:11.960 --> 00:13:15.279
is a lot harder. And, you know, I could look
00:13:15.279 --> 00:13:17.419
at it as like, restating that as like the success
00:13:17.419 --> 00:13:21.740
criteria user to define for the attacker, which
00:13:21.740 --> 00:13:23.899
is stole all the data, shut down the system,
00:13:24.019 --> 00:13:26.519
whatever, boom. I used to call it ringing the
00:13:26.519 --> 00:13:30.659
bell. When we would do, what was that? Those
00:13:30.659 --> 00:13:33.139
damn partner categories, breach and attack simulation,
00:13:33.279 --> 00:13:34.659
Baz, right? Maybe who knows what it's called
00:13:34.659 --> 00:13:37.259
these days. But that was this 10 years ago, right?
00:13:37.299 --> 00:13:41.019
It was, let's automate going in. And what we
00:13:41.019 --> 00:13:43.419
would say is like, we didn't just want to have
00:13:43.419 --> 00:13:45.340
like a test go in and then say without those
00:13:45.340 --> 00:13:47.480
vulnerabilities. We also want to test response.
00:13:47.840 --> 00:13:49.659
So we didn't want to tell the defenders that
00:13:49.659 --> 00:13:51.679
we're automating testing, and we wanted them
00:13:51.679 --> 00:13:55.110
to find it. and not feel like, oh, it was just
00:13:55.110 --> 00:13:57.470
a stupid test, you're wasting my time. So we
00:13:57.470 --> 00:13:59.210
wanted them to have to exercise the real run
00:13:59.210 --> 00:14:01.210
book and all of that. And at some point, say,
00:14:01.850 --> 00:14:04.830
I found you. And that was us trying to like devise
00:14:04.830 --> 00:14:07.610
how do you on the attacker, you ring the bell
00:14:07.610 --> 00:14:09.590
when you see the data come out or whatever. On
00:14:09.590 --> 00:14:11.769
the defender, you mentioned restoring the system
00:14:11.769 --> 00:14:15.169
to a known good state, which is never really
00:14:15.169 --> 00:14:17.009
perfect, because you never really know that was
00:14:17.009 --> 00:14:20.070
known good. And so we would say like, we would
00:14:20.070 --> 00:14:22.210
have some kind of token embedded in the attack
00:14:22.210 --> 00:14:24.309
string, whether it was in the data set or in
00:14:24.309 --> 00:14:28.250
a command or in a header. And we would have the
00:14:28.250 --> 00:14:30.250
good and benevolent processes of the defenders
00:14:30.250 --> 00:14:32.950
end up finding that if it was going to be there,
00:14:32.950 --> 00:14:34.809
and then that's their aha, it was the red team
00:14:34.809 --> 00:14:36.509
and they put it in a little thing and it validates.
00:14:36.649 --> 00:14:38.870
Yes, that's the token you found us. And that
00:14:38.870 --> 00:14:42.710
sort of thing. Really hard requires a lot of
00:14:42.710 --> 00:14:45.529
and that's just for testing. In the wild, you're
00:14:45.529 --> 00:14:48.190
right, it's very hard. I should also clarify
00:14:48.190 --> 00:14:50.509
something. It's not necessarily, I'm not suggesting
00:14:50.509 --> 00:14:54.529
a known secure state, but just a pretty unknown
00:14:54.529 --> 00:14:57.169
good state. So when I was at Bank of America,
00:14:57.610 --> 00:15:01.610
I ran an experiment to see if this was both feasible
00:15:01.610 --> 00:15:10.789
and effective, which was if I, so I had a container
00:15:10.789 --> 00:15:15.340
that I had a process running on top of a container
00:15:15.340 --> 00:15:17.659
that basically wiped the container as soon as
00:15:17.659 --> 00:15:21.440
I saw anything that was outside of the known
00:15:21.440 --> 00:15:24.899
state. Okay, known good state. So if someone
00:15:24.899 --> 00:15:29.100
logged in, okay, and ran who am I, or even tried
00:15:29.100 --> 00:15:31.059
to run, yeah, just ran the command who am I,
00:15:31.360 --> 00:15:35.100
it would blow away the container. Okay. So the
00:15:35.100 --> 00:15:37.340
attacker was able to still get into the container,
00:15:37.639 --> 00:15:39.799
but they could practically do nothing with it
00:15:39.799 --> 00:15:41.779
because as soon as they tried to do anything
00:15:41.779 --> 00:15:46.470
that was not what was defined appropriate or
00:15:46.470 --> 00:15:49.769
useful or allowed, it basically would say, oh,
00:15:50.029 --> 00:15:54.769
that's a like LS, okay? Like you never, no one,
00:15:55.529 --> 00:15:57.870
a system should not ever have to LS anything
00:15:57.870 --> 00:15:59.809
inside of a container because you know exactly
00:15:59.809 --> 00:16:01.529
what's inside the container. Only a user would
00:16:01.529 --> 00:16:04.830
do that. So as soon as I saw anything like that,
00:16:04.909 --> 00:16:07.149
we would just blow away the container. So you
00:16:07.149 --> 00:16:09.669
log in, it's an insecure system. You're able
00:16:09.669 --> 00:16:12.919
to break in. And as soon as you break in, you
00:16:12.919 --> 00:16:16.919
do anything that's not what the system is designed
00:16:16.919 --> 00:16:19.940
to do, and it wipes the system starts over. So
00:16:19.940 --> 00:16:21.659
it was maddening for the anyone who was trying
00:16:21.659 --> 00:16:24.639
to do anything, because it would get wiped within
00:16:24.639 --> 00:16:27.019
seconds of you getting onto the system and trying
00:16:27.019 --> 00:16:29.799
anything. It's like playing Operation. Zap, yeah,
00:16:29.919 --> 00:16:33.659
right, exactly. So anyway, to your point earlier,
00:16:34.159 --> 00:16:36.740
I don't know what... There's no guarantee of
00:16:36.740 --> 00:16:39.240
a secure state, anything, right? But I can at
00:16:39.240 --> 00:16:42.139
least revert to a known good state and constantly
00:16:42.139 --> 00:16:45.600
blow away the system if it diverges from that.
00:16:46.820 --> 00:16:48.860
Yeah. Well, so like talking about the defending
00:16:48.860 --> 00:16:50.879
side on this, so that I think you alluded to
00:16:50.879 --> 00:16:52.460
earlier, one of the most interesting components
00:16:52.460 --> 00:16:58.019
of this story. So the team said, for one, just
00:16:58.019 --> 00:17:02.220
to be clear, they did say they had some AI assisted
00:17:02.220 --> 00:17:04.119
detection mechanisms that turned them on to this
00:17:04.119 --> 00:17:07.559
activity. So behavioral. But then they went on
00:17:07.559 --> 00:17:10.740
to say that when they were going through logs
00:17:10.740 --> 00:17:13.900
to figure out what happened, that they kept getting
00:17:13.900 --> 00:17:17.740
their hand slapped by guard rails on frontier
00:17:17.740 --> 00:17:20.940
models or a frontier model, because they were
00:17:20.940 --> 00:17:23.099
basically loading in all these exploit strings
00:17:23.099 --> 00:17:27.819
and all of this exploitation and command and
00:17:27.819 --> 00:17:31.640
control type activity and they were, you know,
00:17:31.819 --> 00:17:33.259
constantly getting hit. No, you can't do that.
00:17:33.319 --> 00:17:36.059
No, you're going against this. And what's funny,
00:17:36.119 --> 00:17:38.180
and this is just like hyper sophisticated. And
00:17:38.180 --> 00:17:41.319
I think this is where most security teams would
00:17:41.319 --> 00:17:44.519
kind of be shamed and want to pretend that that
00:17:44.519 --> 00:17:46.079
they do this to and just leave the room before
00:17:46.079 --> 00:17:48.500
they get found out. But it was basically like
00:17:49.039 --> 00:17:52.440
Hey, since our automated process of pulling all
00:17:52.440 --> 00:17:54.640
of the forensic data and feeding it to a frontier
00:17:54.640 --> 00:17:57.819
AI model ran into guardrails, we quickly pivoted
00:17:57.819 --> 00:18:01.839
to our on prem GLM model that we use on our own
00:18:01.839 --> 00:18:04.160
hardware, you guys should make sure you all have
00:18:04.160 --> 00:18:06.099
one of those two. And that's where people are
00:18:06.099 --> 00:18:09.019
Oh, yeah, I'll do that. And oh my god, I'm such
00:18:09.019 --> 00:18:12.460
a imposter. So first of all, don't feel bad out
00:18:12.460 --> 00:18:15.859
there if you're not at that level of sophistication.
00:18:15.950 --> 00:18:18.549
Well, okay, first of all, this is hugging face.
00:18:18.609 --> 00:18:21.809
So I expect them to have, I mean, I shall say
00:18:21.809 --> 00:18:25.150
- Yeah, but it's the security team, not the ML
00:18:25.150 --> 00:18:28.930
team. Well, you'd be surprised. I mean, when
00:18:28.930 --> 00:18:31.650
I was talking to folks at OpenAI and Anthropic,
00:18:31.789 --> 00:18:33.769
they're like, the security team, it's a whole
00:18:33.769 --> 00:18:36.609
different mindset because you have, like for
00:18:36.609 --> 00:18:41.829
them, throwing large volumes of logs. It's like,
00:18:41.869 --> 00:18:43.970
there is no third party problem for Anthropic.
00:18:45.039 --> 00:18:49.180
It's like for Amazon, AWS is first party, right?
00:18:50.119 --> 00:18:54.440
So in many cases with hugging face, I mean, they
00:18:54.440 --> 00:18:57.619
run, they're having, they operate some of these
00:18:57.619 --> 00:19:00.599
things as a service for others too. So I'm not
00:19:00.599 --> 00:19:04.759
surprised that they have GLM 5 .2 fully unquantized
00:19:04.759 --> 00:19:09.339
or whatever to run locally. So I was somewhat
00:19:09.339 --> 00:19:11.279
surprised that they didn't turn to that first.
00:19:12.839 --> 00:19:15.440
The fact that they went with frontier models
00:19:15.440 --> 00:19:19.259
was a little bit of a head -scratcher. Well,
00:19:19.400 --> 00:19:22.000
I mean, the adversary went there first. It seems
00:19:22.000 --> 00:19:24.859
that they didn't use frontier models. Yeah, I
00:19:24.859 --> 00:19:30.559
mean, well, if they did, then there's more cartwheels
00:19:30.559 --> 00:19:32.839
that the frontier models need to create. But
00:19:32.839 --> 00:19:38.440
yes, I think the perspective of using your open
00:19:38.440 --> 00:19:41.400
weight models. Unless you're a front -end model
00:19:41.400 --> 00:19:43.700
provider and you consider the front -end models
00:19:43.700 --> 00:19:47.039
to be a first party, I think there is going to
00:19:47.039 --> 00:19:49.740
be a demand for folks to be able to say, hey,
00:19:49.940 --> 00:19:52.559
I need to be able to send really sensitive content
00:19:52.559 --> 00:19:56.079
or some kind of content like this at large volumes
00:19:56.079 --> 00:19:59.559
into an LLM. Let's have a resource like this
00:19:59.559 --> 00:20:02.480
internally so we can use that. But there's another
00:20:02.480 --> 00:20:04.160
part of what you were pointing out that's worth
00:20:04.160 --> 00:20:08.809
noting, which is that defenders got refusals,
00:20:08.809 --> 00:20:11.890
right? And we're trying to defend, and we get
00:20:11.890 --> 00:20:15.029
these refusals, which really cripples or undermines,
00:20:15.049 --> 00:20:18.930
obviously, the ability for defenders to do this
00:20:18.930 --> 00:20:21.930
properly. So one question that I have for the
00:20:21.930 --> 00:20:24.349
Hugging Face team is, did you not sign up for
00:20:24.349 --> 00:20:27.650
the trusted access program or the cyber verification
00:20:27.650 --> 00:20:29.730
program? There's a couple different variants
00:20:29.730 --> 00:20:32.690
of that. Yeah, good question. So as a defender,
00:20:32.849 --> 00:20:35.900
if you haven't already done that, Uh, then you
00:20:35.900 --> 00:20:37.779
should do that. You should go ahead and do that
00:20:37.779 --> 00:20:40.519
anyway. Uh, that gives you the ability to use
00:20:40.519 --> 00:20:43.099
these commercial, uh, frontier models. If you
00:20:43.099 --> 00:20:45.799
choose to do so and not get the refusals that,
00:20:45.799 --> 00:20:48.599
that hugging face ran into. Yeah. Right. I mean,
00:20:48.720 --> 00:20:51.720
the open question, like you said, did they have
00:20:51.720 --> 00:20:55.220
that and still got slapped or not? And I think
00:20:55.220 --> 00:20:56.559
to your earlier point about sophistication in
00:20:56.559 --> 00:20:59.519
that program, you think they did. So I don't
00:20:59.519 --> 00:21:02.430
know. We'll find out I mean there has to be some
00:21:02.430 --> 00:21:04.910
point where the guardrails are it can't just
00:21:04.910 --> 00:21:07.569
be all or nothing So I and if what they're running
00:21:07.569 --> 00:21:10.670
was, you know, really hot there I could see Yeah,
00:21:10.670 --> 00:21:12.430
they could still run into it. So maybe we'll
00:21:12.430 --> 00:21:16.230
learn more about that So I looked to bring this
00:21:16.230 --> 00:21:20.410
this topic home for the audience Is this watershed
00:21:20.410 --> 00:21:22.170
moment that people need to be worried about AI
00:21:22.170 --> 00:21:26.549
enabled text now when people says help Move to
00:21:26.549 --> 00:21:29.839
an open source on -prem model, I don't even have
00:21:29.839 --> 00:21:31.920
AI. Forget about it. I don't have AI assisted
00:21:31.920 --> 00:21:33.259
sock. I don't even have a sock. I don't even
00:21:33.259 --> 00:21:36.119
have good logs. I don't have reliable telemetry
00:21:36.119 --> 00:21:37.980
when I do have a problem that takes me six weeks
00:21:37.980 --> 00:21:39.839
to even reconstruct it. And then I find out I
00:21:39.839 --> 00:21:43.140
don't have all the data. Does this just say you
00:21:43.140 --> 00:21:45.720
better fix all those problems now because you
00:21:45.720 --> 00:21:49.099
can't keep them for long? So here's the thing
00:21:49.099 --> 00:21:52.880
that concerns me to a large degree. This was
00:21:52.880 --> 00:21:56.039
what appears to be an AI enabled attack. Was
00:21:56.039 --> 00:21:58.589
it only discovered because it had because Hugging
00:21:58.589 --> 00:22:02.589
Face had AI enabled defenses. If the question
00:22:02.589 --> 00:22:05.029
of are there any AI enabled attacks, I think
00:22:05.029 --> 00:22:08.009
the answer is absolutely. There are absolutely
00:22:08.009 --> 00:22:12.390
autonomous attack harnesses that are being put
00:22:12.390 --> 00:22:14.789
out there and people are using them. Are we not
00:22:14.789 --> 00:22:16.789
hearing about or seeing them or being able to
00:22:16.789 --> 00:22:19.049
even distinguish that they are such because we
00:22:19.049 --> 00:22:23.019
just don't have. because we lack either know
00:22:23.019 --> 00:22:26.940
-how or these AI enabled defending tools, defense
00:22:26.940 --> 00:22:32.420
tools to help us discover these attacks. I'm
00:22:32.420 --> 00:22:37.140
hoping, well, I can be hopeful and say, oh no,
00:22:37.279 --> 00:22:40.660
there aren't too many agentic AI attacks happening
00:22:40.660 --> 00:22:44.299
right now. Um, but I, I just can't believe that
00:22:44.299 --> 00:22:46.960
that's true because it's so easy. It's so easy
00:22:46.960 --> 00:22:49.279
to just throw like an autonomous agent framework
00:22:49.279 --> 00:22:53.359
at a framework attack framework at an agent at
00:22:53.359 --> 00:22:57.240
a large language model and say, go. Yeah. Yeah.
00:22:57.299 --> 00:22:58.980
I mean, and, and yeah, I'm tying that back to
00:22:58.980 --> 00:23:00.480
what you said earlier about like the ability
00:23:00.480 --> 00:23:02.579
to have success criteria and know when you win.
00:23:03.099 --> 00:23:05.980
Um, that begets an agentic framework as well.
00:23:06.279 --> 00:23:08.019
You know, an actual goal, which really helps
00:23:08.019 --> 00:23:10.480
with that. And if we're all able to use cloud
00:23:10.480 --> 00:23:14.000
code every day to steer us towards some product
00:23:14.000 --> 00:23:17.099
vision already, then something this specific,
00:23:17.279 --> 00:23:20.759
you'd be a no -brainer. And one really important
00:23:20.759 --> 00:23:23.019
thing to mention, you talked about breach and
00:23:23.019 --> 00:23:25.660
attack simulation. The way I looked at it when
00:23:25.660 --> 00:23:29.900
I was at Bank of America was the primary goal
00:23:29.900 --> 00:23:32.380
that I had for breach and attack simulation was
00:23:32.380 --> 00:23:35.480
to evaluate not my response capability, but my
00:23:35.480 --> 00:23:38.400
detective capability. Did my detective controls
00:23:38.400 --> 00:23:41.400
even notice the breach or the attacks that were
00:23:41.400 --> 00:23:45.319
being simulated? So I have a very specific progression,
00:23:45.339 --> 00:23:48.380
which may not align with how everyone thinks
00:23:48.380 --> 00:23:50.500
about this, but I look at, for example, vulnerability
00:23:50.500 --> 00:23:53.940
scanning as a test against my identified controls.
00:23:54.099 --> 00:23:56.980
Can I proactively identify my vulnerabilities
00:23:56.980 --> 00:23:59.640
and address them before they become discoverable?
00:24:00.019 --> 00:24:04.000
Then I look at pen testing as a test against
00:24:04.000 --> 00:24:07.769
my protect. capabilities and my protective controls.
00:24:08.230 --> 00:24:09.750
Then I looked at breach and attack simulation
00:24:09.750 --> 00:24:13.650
as my mechanism to test my detective controls.
00:24:14.549 --> 00:24:19.430
I look at Red Team as my test against my response
00:24:19.430 --> 00:24:22.569
controls. The response being, once I know an
00:24:22.569 --> 00:24:25.029
attacker is in there, can I actually kick them
00:24:25.029 --> 00:24:29.269
out thoroughly, completely? So my Red Team, in
00:24:29.269 --> 00:24:31.230
many cases, we just said, here are our credentials,
00:24:31.509 --> 00:24:34.920
go and trench yourself. And Blue Team. try to
00:24:34.920 --> 00:24:36.680
kick them out. We'll even tell you where they
00:24:36.680 --> 00:24:38.960
might be, right? Do what you can to kick them
00:24:38.960 --> 00:24:40.440
out. And Red Team, of course... You have to go
00:24:40.440 --> 00:24:42.579
through that. Yeah. Red Team, of course, is like
00:24:42.579 --> 00:24:45.700
just persistence everywhere, entrench yourself
00:24:45.700 --> 00:24:49.359
everywhere, do things that were really, really
00:24:49.359 --> 00:24:52.960
hard to get rid of. And the goal was to see what
00:24:52.960 --> 00:24:55.539
our response was. But anyway, going back to the
00:24:55.539 --> 00:24:56.880
sequence there, the reason why I mentioned this
00:24:56.880 --> 00:25:00.410
is... Your detective controls are really, are
00:25:00.410 --> 00:25:02.589
what we need to evaluate. And what Hug and Face
00:25:02.589 --> 00:25:05.369
I think really showed us is they had, they clearly
00:25:05.369 --> 00:25:08.430
had good detective controls, maybe because of
00:25:08.430 --> 00:25:14.250
the AI defense, AI enabled defense. But to understand
00:25:14.250 --> 00:25:16.170
whether or not you actually can defend against
00:25:16.170 --> 00:25:21.069
that or that you can see this, see the, the detective
00:25:21.069 --> 00:25:23.789
controls light up is to take your own attack
00:25:23.789 --> 00:25:27.640
harness and go say, Hey, here's my Public -facing
00:25:27.640 --> 00:25:31.579
website go attack it and don't stop and very
00:25:31.579 --> 00:25:33.960
few of us are willing to let that run on its
00:25:33.960 --> 00:25:35.880
own like that but that's essentially what the
00:25:35.880 --> 00:25:39.579
attacker I think has done here and Again, fortunately
00:25:39.579 --> 00:25:42.180
hugging face fought found it, but I don't I wouldn't
00:25:42.180 --> 00:25:44.440
bet that others would have had the same success
00:25:44.440 --> 00:25:47.299
Yeah, yeah, I think that's right. And I mean,
00:25:47.900 --> 00:25:50.240
I think there's so many said for you know, when
00:25:50.240 --> 00:25:53.039
you said early on the prominence of hugging face
00:25:53.039 --> 00:25:58.470
and I'd like to believe that they're on rarefied
00:25:58.470 --> 00:26:00.410
earth when it comes to the targeting as well,
00:26:00.430 --> 00:26:03.930
which would be good news for PetSmart or, you
00:26:03.930 --> 00:26:06.670
know, the average company, regardless of revenue.
00:26:07.250 --> 00:26:10.329
But to your other point, I think this is kind
00:26:10.329 --> 00:26:13.789
of a bellwether of the tradecraft and it's going
00:26:13.789 --> 00:26:15.269
to become part and it's just a matter of whether
00:26:15.269 --> 00:26:18.349
people really want to attack you or not. And
00:26:18.349 --> 00:26:21.430
if that's the case and the whole world, no less
00:26:21.430 --> 00:26:25.200
country, will ultimately, you know, anybody who
00:26:25.200 --> 00:26:26.779
used to be under the ransomware target, which
00:26:26.779 --> 00:26:28.940
means everyone will now be under this kind of
00:26:28.940 --> 00:26:32.220
trade craft. Then we can just like fast forward
00:26:32.220 --> 00:26:33.839
and say people are ultimately going to say the
00:26:33.839 --> 00:26:35.880
government has to get involved. We can't afford
00:26:35.880 --> 00:26:37.619
or expect to have the kind of sophistication
00:26:37.619 --> 00:26:41.900
to defend against us. So that brings us to an
00:26:41.900 --> 00:26:45.440
executive order on the, I don't even want to
00:26:45.440 --> 00:26:47.299
yell it because I'm in my office and people might
00:26:47.299 --> 00:26:50.200
think I'm under assault over here, but it's gold
00:26:50.200 --> 00:26:53.089
eagle. Screamed really loudly in all capital
00:26:53.089 --> 00:26:58.650
letters So What do you know about this gold eagle
00:26:58.650 --> 00:27:00.089
initiative? Are you up to speed on this thing?
00:27:00.109 --> 00:27:02.470
And do you read about this? Yeah, I'm pretty
00:27:02.470 --> 00:27:05.089
Anyway, yeah, I'm familiar enough with it The
00:27:05.089 --> 00:27:07.329
funny thing is when you have a program in all
00:27:07.329 --> 00:27:08.809
caps, you know It came from the intelligence
00:27:08.809 --> 00:27:11.269
community or it started somewhere in the intelligence
00:27:11.269 --> 00:27:14.009
community. That's typically the yeah the common
00:27:14.009 --> 00:27:16.930
signal anyway Yeah, so there are a couple things
00:27:16.930 --> 00:27:19.230
about gold eagle that I thought was interesting
00:27:20.059 --> 00:27:23.819
So for those who are not familiar with it, it's
00:27:23.819 --> 00:27:28.519
in theory an effort to help coordinate the discovery
00:27:28.519 --> 00:27:32.740
of vulnerabilities and then subsequently the
00:27:32.740 --> 00:27:36.960
fixing and maybe even the patching, who knows.
00:27:39.049 --> 00:27:40.450
But there were a couple of things that I thought
00:27:40.450 --> 00:27:42.109
was interesting. One is that there's nothing
00:27:42.109 --> 00:27:44.990
about that coordinating and the scanning. So
00:27:44.990 --> 00:27:47.250
it talks about at the very beginning, but this
00:27:47.250 --> 00:27:48.930
notion that we have all these people scanning
00:27:48.930 --> 00:27:51.230
the same repository, same open source software,
00:27:51.609 --> 00:27:54.589
and no coordination happening, and certainly
00:27:54.589 --> 00:27:57.829
very little coordination of the reporting process
00:27:57.829 --> 00:28:01.390
to the maintainers. I would argue that the Accredits
00:28:01.390 --> 00:28:05.349
project that was launched a couple of weeks ago,
00:28:06.109 --> 00:28:10.130
I guess, we've already kind of started that process.
00:28:10.970 --> 00:28:13.170
So the question is, where can government actually
00:28:13.170 --> 00:28:15.410
help here? If the government's here to help,
00:28:15.950 --> 00:28:19.309
okay, what can you actually do to help? And ultimately,
00:28:19.349 --> 00:28:22.130
I think we all know this, but discovery is not
00:28:22.130 --> 00:28:24.829
the constraint. It's never been a constraint.
00:28:25.250 --> 00:28:28.849
I would argue even finding patches isn't the
00:28:28.849 --> 00:28:32.730
constraint. It's always been about fixing, actually
00:28:32.730 --> 00:28:34.450
implementing the patch. And so we have to go
00:28:34.450 --> 00:28:37.130
and understand, why are people hesitant to patch?
00:28:37.309 --> 00:28:39.829
Um, and is there anything that the government
00:28:39.829 --> 00:28:45.109
has imposed that makes that harder? Okay. That's
00:28:45.109 --> 00:28:47.089
the focus area that I would have taken, which
00:28:47.089 --> 00:28:50.410
is what is it? Where, uh, where are the inflection
00:28:50.410 --> 00:28:52.190
points or the pressure points that government
00:28:52.190 --> 00:28:57.930
can release? Uh, or help, uh, create proper backstops
00:28:57.930 --> 00:29:02.630
for it to say, look, if you patch something that
00:29:02.630 --> 00:29:06.289
comes out of this program and your product. fails,
00:29:06.430 --> 00:29:08.470
breaks, blah, blah, blah, blah, then you are
00:29:08.470 --> 00:29:12.089
released from liability, okay? That would be
00:29:12.089 --> 00:29:15.970
wonderful. Some people may die, things will break,
00:29:17.230 --> 00:29:19.269
things will crash. Yes, yes, we all know that.
00:29:19.549 --> 00:29:23.990
That's the result of patches. But to be released
00:29:23.990 --> 00:29:28.109
from liability definitely creates a, releases
00:29:28.109 --> 00:29:31.109
some pressure here where right now I think government
00:29:31.109 --> 00:29:35.399
forces many organizations to have everything
00:29:35.399 --> 00:29:39.799
recertified or properly validated after any change.
00:29:40.220 --> 00:29:42.500
Okay. Not just a patch, but any change at all.
00:29:43.000 --> 00:29:45.460
And that makes sense as to why we'd want to do
00:29:45.460 --> 00:29:48.660
that. So if the government really wants us to
00:29:48.660 --> 00:29:51.119
patch things as well with a higher degree of
00:29:51.119 --> 00:29:54.880
urgency, then it has to correspond with being
00:29:54.880 --> 00:29:56.880
released from some of the certification that
00:29:56.880 --> 00:30:01.140
happens after the fact. Yeah. I'll play that
00:30:01.140 --> 00:30:02.799
back and then take it forward from that. So like
00:30:02.799 --> 00:30:06.799
to me, So with the EO and Project Golden Eagle,
00:30:07.019 --> 00:30:10.000
the ideas there is there's going to be some central
00:30:10.000 --> 00:30:14.660
kind of coordination of, Hey, we found this,
00:30:14.660 --> 00:30:16.799
this critical issue with this code base. Oh,
00:30:16.819 --> 00:30:18.400
I was getting to a cal stop. Now you already
00:30:18.400 --> 00:30:19.799
found it. Okay, great. Let's all work together
00:30:19.799 --> 00:30:22.240
on making sure the vendors aware of the maintainers
00:30:22.240 --> 00:30:25.880
and we get a solution for this. And then the
00:30:25.880 --> 00:30:28.579
90 % that's left is actually getting that thing
00:30:28.579 --> 00:30:31.220
out into the wild and into infrastructure, which
00:30:31.220 --> 00:30:34.210
is a change control problem and not a exploit
00:30:34.210 --> 00:30:37.930
problem. And I totally agree with you. It's so
00:30:37.930 --> 00:30:40.369
funny, even when we talk about zero days. In
00:30:40.369 --> 00:30:42.869
reality, it's always a one day, right? Because
00:30:42.869 --> 00:30:45.509
it's there is a patch, right? And part of the
00:30:45.509 --> 00:30:47.009
story is always why didn't they patch or didn't
00:30:47.009 --> 00:30:49.450
patch up? It's very rare that it's you know,
00:30:49.450 --> 00:30:51.849
there was no patch at the time it happened. It
00:30:51.849 --> 00:30:53.769
doesn't exist. But it's very rare, not enough
00:30:53.769 --> 00:30:56.069
to drive in this podcast, we wouldn't exist if
00:30:56.069 --> 00:30:57.390
that was the only problem we were worried about.
00:30:58.390 --> 00:31:02.849
So I'll rephrase and say like, a better almost
00:31:02.849 --> 00:31:06.349
like moonshot would be how do we when there's
00:31:06.349 --> 00:31:08.589
issues make sure that the collective ecosystem
00:31:08.589 --> 00:31:12.029
can get up to up to speed with the protections
00:31:12.029 --> 00:31:15.029
that are available. And you mentioned what all
00:31:15.029 --> 00:31:17.269
rephrases regulatory relief to some standpoint,
00:31:17.849 --> 00:31:21.210
like instead of when people rush out and they
00:31:21.210 --> 00:31:24.509
kind of YOLO with change control to some degree.
00:31:25.430 --> 00:31:27.589
When things break, hey, get cut them some slack.
00:31:28.150 --> 00:31:32.579
I think You know of the entities that get penalized
00:31:32.579 --> 00:31:36.440
for breaking things I think the place where government
00:31:36.440 --> 00:31:38.859
has a role in that and where they're currently
00:31:38.859 --> 00:31:41.920
disincentivizing people from innovating is pretty
00:31:41.920 --> 00:31:46.220
minimal in that When I worked in a regulatory
00:31:46.220 --> 00:31:50.119
environment If we had an outage nothing all do
00:31:50.119 --> 00:31:52.720
it cyber We'd be penalized for that because we
00:31:52.720 --> 00:31:54.859
had a guarantee of fair and orderly market on
00:31:54.859 --> 00:31:58.819
a stock exchange. And so if we broke that Um,
00:31:59.039 --> 00:32:01.200
we would get our hand slap for sure. And sometimes
00:32:01.200 --> 00:32:03.319
that would be as a result of, um, what you can
00:32:03.319 --> 00:32:05.240
generally call patching. It's, you know, innovation
00:32:05.240 --> 00:32:08.160
upgrades, code releases, whatever it may be.
00:32:09.200 --> 00:32:12.319
Had we the ability to say, well, well, well,
00:32:12.559 --> 00:32:16.740
I want to cite clause 55B. This was related to
00:32:16.740 --> 00:32:19.660
security. So back off. And we can't have a cause
00:32:19.660 --> 00:32:22.519
of action here, SEC or CFTC, because this was
00:32:22.519 --> 00:32:26.150
related to security. One. I think that would
00:32:26.150 --> 00:32:28.349
give some perverse incentives to for people to
00:32:28.349 --> 00:32:30.089
automatically every time there's an issue, start
00:32:30.089 --> 00:32:33.109
blaming security and just always a way to do
00:32:33.109 --> 00:32:35.970
that, you know. But perhaps more importantly,
00:32:36.650 --> 00:32:39.809
I think the overwhelming majority of hand slapping
00:32:39.809 --> 00:32:42.049
around change control is not government or regulation
00:32:42.049 --> 00:32:44.990
born. I think it's, it's either commercial, hey,
00:32:45.089 --> 00:32:46.990
we're down during Black Friday and we're losing
00:32:46.990 --> 00:32:51.049
the business, or contractual, right, which could
00:32:51.049 --> 00:32:54.559
be This big customer of mine is going to come
00:32:54.559 --> 00:32:58.500
after me. So you'd almost have to have this like
00:32:58.500 --> 00:33:00.779
tort injection where you're able in court to
00:33:00.779 --> 00:33:04.799
say, well, according to this national law, because
00:33:04.799 --> 00:33:08.079
what I did was related to security, you can't
00:33:08.079 --> 00:33:10.240
sue me for violating my uptime requirements on
00:33:10.240 --> 00:33:14.599
my SLAs. That seems really ripe for abuse and
00:33:14.599 --> 00:33:18.960
manipulation. Yeah, I don't, I would definitely
00:33:18.960 --> 00:33:22.769
not suggest that. It can't be abused, right?
00:33:22.910 --> 00:33:25.069
I think that's really the... What I'm trying
00:33:25.069 --> 00:33:27.150
to do is characterize a trade -off and I think
00:33:27.150 --> 00:33:32.009
the trade -off is ultimately more evident. It's
00:33:32.009 --> 00:33:34.809
not just that, hey, why can't you guys apply
00:33:34.809 --> 00:33:37.410
these patches? There's a whole bunch of other
00:33:37.410 --> 00:33:39.430
considerations that do need to be accounted for.
00:33:40.089 --> 00:33:43.250
And I have jokingly said, oh, you know, people
00:33:43.250 --> 00:33:45.619
are going to die. Well, guess what? That... the
00:33:45.619 --> 00:33:47.859
people who die are gonna care a lot, right? They're
00:33:47.859 --> 00:33:51.240
gonna be like, no, don't patch. Thank you very
00:33:51.240 --> 00:33:54.299
much. I need to get my MRI or I need to get this
00:33:54.299 --> 00:33:57.259
medical procedure done. How dare you patch in
00:33:57.259 --> 00:34:01.299
the midst of that, right? But we can understand
00:34:01.299 --> 00:34:04.140
now what are the real trade -offs that government
00:34:04.140 --> 00:34:08.559
can weigh in on. And again, if there are things
00:34:08.559 --> 00:34:11.380
that the government can truly do, that industry
00:34:11.380 --> 00:34:14.389
can't do, it would be something like this. Well,
00:34:14.389 --> 00:34:19.409
I anecdote here I remember circa 1995 maybe a
00:34:19.409 --> 00:34:23.429
buddy and I in college Performing LLC weren't
00:34:23.429 --> 00:34:25.570
doing all kinds of work to pay tuition and so
00:34:25.570 --> 00:34:29.989
on related IT and we went to a school and part
00:34:29.989 --> 00:34:32.909
of the statement of work was so that they it
00:34:32.909 --> 00:34:35.610
was a public school system and They said hey,
00:34:35.650 --> 00:34:37.710
can you come in? It has to be after hours So
00:34:37.710 --> 00:34:39.929
come in on the weekend the janitor will let you
00:34:39.929 --> 00:34:42.820
in not gonna see anybody related to us We have
00:34:42.820 --> 00:34:45.739
an old Novell system, and it hasn't been backed
00:34:45.739 --> 00:34:49.940
up in decades, or at least a decade. And because
00:34:49.940 --> 00:34:52.079
it doesn't have a backup, it doesn't have a tape
00:34:52.079 --> 00:34:53.860
backup drives as before doing over the network
00:34:53.860 --> 00:34:57.260
was really plausible. So what we want for you
00:34:57.260 --> 00:34:59.539
guys to do, because this system is so at risk,
00:34:59.539 --> 00:35:01.519
and we're worried about losing all the data is
00:35:01.519 --> 00:35:04.099
to install a backup drive on it. So we go in
00:35:04.099 --> 00:35:06.059
there and I remember quite vividly, and it was
00:35:06.059 --> 00:35:08.260
a desktop format system sitting on top of a desk.
00:35:09.039 --> 00:35:12.449
And it was kind of like anchored, like built,
00:35:12.869 --> 00:35:16.590
constructed, formed by dust. Like the whole thing
00:35:16.590 --> 00:35:20.369
was encased in dust. And so we said, well, let's
00:35:20.369 --> 00:35:22.710
put in the backup drive. So we, first thing we
00:35:22.710 --> 00:35:25.210
did was, you know, power it down. And that was
00:35:25.210 --> 00:35:27.610
the end of it. We powered it. I mean, we didn't
00:35:27.610 --> 00:35:28.889
figure it out until we opened the case, put in
00:35:28.889 --> 00:35:30.170
the backup drive and I went to power back up.
00:35:30.269 --> 00:35:32.929
But once we powered it down, it was done. It's
00:35:32.929 --> 00:35:35.650
completely done there. That's change control.
00:35:36.150 --> 00:35:38.329
That's what companies are thinking about when
00:35:38.329 --> 00:35:41.030
it comes to patching a lot is not like. Hey,
00:35:41.150 --> 00:35:42.969
can you give me a get a jail free card? It's
00:35:42.969 --> 00:35:46.230
more like, oh, this legacy code base. Oh, if
00:35:46.230 --> 00:35:48.590
I do not touch that thing, we fired all those
00:35:48.590 --> 00:35:50.630
developers six years ago. Nobody knows anything
00:35:50.630 --> 00:35:53.050
about it is still generating revenue. I don't
00:35:53.050 --> 00:35:55.690
care who says it's okay. You will not. That's
00:35:55.690 --> 00:35:57.989
what patch management is like a change control,
00:35:58.170 --> 00:36:01.090
I think in a lot of organizations. So I'm not
00:36:01.090 --> 00:36:04.309
sure if, if the government could do something
00:36:04.309 --> 00:36:06.510
to help with like the metaphorical projection
00:36:06.510 --> 00:36:08.829
of that kind of situation writ large across the
00:36:08.829 --> 00:36:13.079
world. Yeah, I agree. There's a limit to what
00:36:13.079 --> 00:36:15.039
government can actually do in some of these cases
00:36:15.039 --> 00:36:17.360
where it's really an internal corporate decision.
00:36:18.920 --> 00:36:21.019
But at the end of the day, that is where the
00:36:21.019 --> 00:36:23.019
constraint exists, right? The change control,
00:36:24.619 --> 00:36:27.800
the actual change itself, and people's unwillingness
00:36:27.800 --> 00:36:31.579
to break other things, potentially break other
00:36:31.579 --> 00:36:35.730
things for the sake of security. So anyway, all
00:36:35.730 --> 00:36:37.849
the efforts around vulnerability disclosure and
00:36:37.849 --> 00:36:40.650
coordination, that's nice. But if you think about,
00:36:41.849 --> 00:36:44.829
it's called Goldratz's theory of constraints.
00:36:45.969 --> 00:36:50.349
So Goldratz indicated, he wrote a book, I forget
00:36:50.349 --> 00:36:53.349
the name of the book, but it's basically the
00:36:53.349 --> 00:36:55.289
premise that if you try to fix anything besides
00:36:55.289 --> 00:36:57.789
what's at the constraint, you only exacerbate
00:36:57.789 --> 00:37:01.010
that constraint. So we're scaling discovery,
00:37:01.369 --> 00:37:04.119
but remediation isn't scaling. And so. increased
00:37:04.119 --> 00:37:07.019
discovery. Well, it's just gonna make the remediation
00:37:07.019 --> 00:37:09.960
even worse. Yeah, that seems out and applicable
00:37:09.960 --> 00:37:12.320
here. Well, not to let the government off the
00:37:12.320 --> 00:37:18.820
hook. Next topic on here. So, you know, I'm more
00:37:18.820 --> 00:37:21.619
ignorant to the CMMC process, not completely.
00:37:22.340 --> 00:37:25.079
But I forget what it even stands for, but Capability
00:37:25.079 --> 00:37:28.139
Maturity Model certification, certification,
00:37:28.159 --> 00:37:30.489
okay. Yeah, I want you to give us a recap of
00:37:30.489 --> 00:37:34.110
what it was and what changed this year before
00:37:34.110 --> 00:37:37.050
July 13th and what the state of play was with
00:37:37.050 --> 00:37:40.289
CMMC. Okay, so full disclosure as well, I sit
00:37:40.289 --> 00:37:44.130
on the board of the Cyber AB, which oversees
00:37:44.130 --> 00:37:47.809
the CMMC certification, the program itself in
00:37:47.809 --> 00:37:51.590
terms of certifying third -party assessors. So
00:37:51.590 --> 00:37:55.340
I do have a... Even though I try to remain as
00:37:55.340 --> 00:37:58.179
independent, as unbiased as possible, there's
00:37:58.179 --> 00:38:00.059
at least a perception of bias here because it
00:38:00.059 --> 00:38:03.260
means that I would naturally have a favor towards
00:38:03.260 --> 00:38:06.460
this program. Because of how much they pay you.
00:38:07.909 --> 00:38:10.570
Yeah, it's a voluntary role. I pushed the rest
00:38:10.570 --> 00:38:12.409
of the role to realize how these voluntary roles
00:38:12.409 --> 00:38:14.929
work for Indian cybersecurity, but yeah. It is
00:38:14.929 --> 00:38:19.730
a non -profit, but it is a notable board position,
00:38:19.929 --> 00:38:22.130
which I think is kind of interesting. But that's
00:38:22.130 --> 00:38:25.750
it, yes. There's a natural bent or bias towards
00:38:25.750 --> 00:38:28.550
it because of my position on here. But I'll try
00:38:28.550 --> 00:38:31.849
to give as unbiased a view as possible, because
00:38:31.849 --> 00:38:35.010
I'm also good friends with the person that suspended
00:38:35.010 --> 00:38:39.300
the requirements. So anyway, the basic premise
00:38:39.300 --> 00:38:42.099
here is that if you're doing business with the
00:38:42.099 --> 00:38:45.500
government or with DoD, then you're supposed
00:38:45.500 --> 00:38:48.739
to meet certain compliance requirements as it
00:38:48.739 --> 00:38:52.360
relates to security, specifically NIST 800 .171.
00:38:52.719 --> 00:39:00.079
And this has been the acquisition language. for
00:39:00.079 --> 00:39:02.260
a long time. It's been there for a long time,
00:39:02.559 --> 00:39:04.739
but basically companies would say, yep, we're
00:39:04.739 --> 00:39:08.980
compliant with 800 .171. But everyone pretty
00:39:08.980 --> 00:39:11.440
much realized that they're just lying, right?
00:39:11.500 --> 00:39:14.699
They're not actually compliant, but they're stating
00:39:14.699 --> 00:39:19.739
that they're doing so. And so because we all
00:39:19.739 --> 00:39:22.340
recognize that they're just lying to their teeth,
00:39:23.119 --> 00:39:24.440
there were a couple of things that needed to
00:39:24.440 --> 00:39:26.760
happen or that people wanted to have happen.
00:39:26.820 --> 00:39:30.000
One is to certify, to go through a third party
00:39:30.000 --> 00:39:33.460
certification and validate that, yes, this organization
00:39:33.880 --> 00:39:37.280
is actually meeting those requirements. And as
00:39:37.280 --> 00:39:40.900
a result, they are eligible for DOD contracts
00:39:40.900 --> 00:39:44.500
or Department of War contracts. The other sort
00:39:44.500 --> 00:39:49.320
of perspective is that making such claims makes
00:39:49.320 --> 00:39:51.380
you susceptible to what's called the False Claims
00:39:51.380 --> 00:39:54.639
Act. So there's been recent enforcement of the
00:39:54.639 --> 00:39:57.039
False Claims Act with fines in the millions and
00:39:57.039 --> 00:39:59.619
millions of dollars against organizations that
00:39:59.619 --> 00:40:04.880
have claimed that they're compliant to to 8171,
00:40:05.039 --> 00:40:08.320
but they clearly aren't. And so it's starting
00:40:08.320 --> 00:40:11.019
to get more teeth into the enforcement side of
00:40:11.019 --> 00:40:13.400
it. Not just saying, hey, you can't do business,
00:40:13.460 --> 00:40:16.559
but you will also get fined as a part of that.
00:40:18.139 --> 00:40:20.360
So the false claims that, or the requirement
00:40:20.360 --> 00:40:25.239
to meet this levels of compliance has not been
00:40:25.239 --> 00:40:27.699
taken off the books. That's still the case. That's
00:40:27.699 --> 00:40:32.070
still in force. and you could still be taken
00:40:32.070 --> 00:40:36.349
to court through the False Claims Act. What has
00:40:36.349 --> 00:40:41.969
changed here is the actual certification process
00:40:41.969 --> 00:40:47.349
itself. Should you be pre -certified to say that
00:40:47.349 --> 00:40:51.469
you adhere to this because what's the biggest
00:40:51.469 --> 00:40:54.760
concern from the White House and from... uh,
00:40:54.760 --> 00:40:57.280
the current leadership in the Pentagon is, uh,
00:40:57.280 --> 00:41:00.119
this whole certification process, uh, is potentially,
00:41:00.119 --> 00:41:04.960
uh, too expensive for a smaller, um, organization,
00:41:05.179 --> 00:41:08.199
smaller manufacturers and such. And there's a
00:41:08.199 --> 00:41:13.059
huge, huge, huge push to increase our manufacturing
00:41:13.059 --> 00:41:17.260
capability, especially for building more, um,
00:41:17.260 --> 00:41:20.840
uh, arms, uh, arsenal, uh, more to expand our
00:41:20.840 --> 00:41:23.079
arsenal. Right. So it's called the arsenal of
00:41:23.079 --> 00:41:27.480
freedom. And I'm sure no one's surprised by the
00:41:27.480 --> 00:41:29.679
need for this, right? We're expending lots of
00:41:29.679 --> 00:41:33.179
armaments in Iran. We've sent a lot to Ukraine.
00:41:33.460 --> 00:41:37.239
And in the next two years, might we have a conflict
00:41:37.239 --> 00:41:40.840
with China? And so this sort of perspective is
00:41:40.840 --> 00:41:44.260
driving a perspective of how do we increase our
00:41:44.260 --> 00:41:46.539
manufacturing capability as quickly as possible?
00:41:47.460 --> 00:41:49.659
Still try to remain as secure as possible, but
00:41:49.659 --> 00:41:52.000
not over -rotate on compliance because we know
00:41:52.000 --> 00:41:57.019
that If I had to spend money on compliance, I'm
00:41:57.019 --> 00:41:58.960
not spending money on something else, right?
00:41:59.420 --> 00:42:01.440
And it's not spending money on compliance, it's
00:42:01.440 --> 00:42:03.780
spending money on the certification that you're
00:42:03.780 --> 00:42:06.219
compliant versus actually spending, like I don't
00:42:06.219 --> 00:42:08.820
think anyone disagrees that you should try to
00:42:08.820 --> 00:42:11.639
secure your environment. It's should you spend
00:42:11.639 --> 00:42:13.519
additional money on getting certified on top
00:42:13.519 --> 00:42:17.699
of that? That last part is what was suspended
00:42:17.699 --> 00:42:21.849
from what I can gather. Yeah, and being much
00:42:21.849 --> 00:42:23.590
more ignorant of the process than you, I don't
00:42:23.590 --> 00:42:25.010
have to worry about tripping over anything. But
00:42:25.010 --> 00:42:27.329
so when I read it with absolute zero insight
00:42:27.329 --> 00:42:31.750
knowledge, I detected some color there, like
00:42:31.750 --> 00:42:36.250
he was saying, this is rescinded, because it's
00:42:36.250 --> 00:42:37.929
not an effective use of people's time, which
00:42:37.929 --> 00:42:39.349
I mean, you said that in a very diplomatic way
00:42:39.349 --> 00:42:43.769
as well. But my question for you is, do you feel
00:42:43.769 --> 00:42:46.829
like the next step out of this is more in the
00:42:46.829 --> 00:42:50.690
direction of there's natural remedies for this,
00:42:50.750 --> 00:42:52.650
we didn't need this process, we're in a good
00:42:52.650 --> 00:42:56.190
direction. Or, yeah, we do need attestation and
00:42:56.190 --> 00:42:58.250
validation around security of vendors, but this
00:42:58.250 --> 00:42:59.730
is not a good process. So we're going to create
00:42:59.730 --> 00:43:03.449
a new one. And just to spell it out, it did say
00:43:03.449 --> 00:43:04.769
in the release around this, that there's gonna
00:43:04.769 --> 00:43:09.329
be a task force for reforming this is is that
00:43:09.329 --> 00:43:10.949
form going to be reform going to be encouraged
00:43:10.949 --> 00:43:14.710
to come up with a new version, or to back out
00:43:14.710 --> 00:43:16.250
of the whole initiative more? What do you think?
00:43:17.750 --> 00:43:20.070
I'm not sure if it's... I think it's ultimately
00:43:20.070 --> 00:43:23.269
to find more cost -effective ways to expedite
00:43:23.269 --> 00:43:24.730
the whole process while still... Yeah, but do
00:43:24.730 --> 00:43:26.949
you think like cost aside that if people were
00:43:26.949 --> 00:43:28.630
willing to shoulder all of the costs, that the
00:43:28.630 --> 00:43:30.710
outcome is benevolent and good, or do you feel
00:43:30.710 --> 00:43:32.349
like it's regardless of cost, it's just not a
00:43:32.349 --> 00:43:34.409
good outcome? Yeah, let's just, for argument's
00:43:34.409 --> 00:43:36.730
sake, say, you know what? DoD will reimburse
00:43:36.730 --> 00:43:40.849
your cost, okay? Yeah, right. They would have
00:43:40.849 --> 00:43:43.250
to ask for more budget, but I can also imagine
00:43:43.250 --> 00:43:46.489
the percentage of spend for that would be...
00:43:46.250 --> 00:43:51.050
you know, a small fraction of our total spend
00:43:51.050 --> 00:43:53.690
for that department. But let's suppose they could
00:43:53.690 --> 00:43:57.530
do that, right? Does it remove the, does it just
00:43:57.530 --> 00:43:59.289
take it off the table? Yeah, to some degree,
00:43:59.489 --> 00:44:02.190
it's not really about the cost as much as then
00:44:02.190 --> 00:44:05.530
it's about, is this actually effective? I think,
00:44:05.929 --> 00:44:10.889
let me give an analogy for those who know, who
00:44:10.889 --> 00:44:12.969
complain about this as well. It's like SOC 2,
00:44:13.190 --> 00:44:16.679
right? Let's make SOC 2 free. You can get a sock
00:44:16.679 --> 00:44:19.440
too for free. You get a sock too, you get a toot
00:44:19.440 --> 00:44:23.639
too. Everyone gets a sock too. Does it really
00:44:23.639 --> 00:44:28.179
move the needle in any meaningful way? No. I
00:44:28.179 --> 00:44:30.639
don't know if that was rhetorical or not. I mean,
00:44:30.639 --> 00:44:32.659
I'll argue that they are free now. Like the prices
00:44:32.659 --> 00:44:35.420
come down so much. And we're going through ours
00:44:35.420 --> 00:44:38.880
right now. And I'm just disgusted with it. I
00:44:38.880 --> 00:44:42.820
can't wait till the next cycle to just be the
00:44:42.820 --> 00:44:44.360
company that says, we're not doing that because
00:44:44.360 --> 00:44:47.000
it's so dumb. but here's a viable alternative.
00:44:47.199 --> 00:44:50.539
To me, it's free and still bad. Yes. So I agree
00:44:50.539 --> 00:44:52.820
that one, the price point for a SOC 2 is to drop,
00:44:53.019 --> 00:44:54.719
you know, drop to the, it's a race to the bottom.
00:44:55.079 --> 00:44:57.579
It's a race to the bottom and you can get one
00:44:57.579 --> 00:45:00.119
fairly cheaply, but that's really, is it worth
00:45:00.119 --> 00:45:01.960
more than what's on the paper that it's written
00:45:01.960 --> 00:45:04.460
on? And the answer generally speaking is, you
00:45:04.460 --> 00:45:08.699
do it because there's a general industry expectation
00:45:08.699 --> 00:45:10.920
that you have one, but beyond the fact that you
00:45:10.920 --> 00:45:13.659
have one, there's not much beyond that. And I
00:45:13.659 --> 00:45:16.800
would say for a SOC 2, Uh, for most of what we
00:45:16.800 --> 00:45:19.219
do, vast majority of what, you know, you might
00:45:19.219 --> 00:45:21.420
think what you do is important and really, uh,
00:45:21.559 --> 00:45:23.599
life -changing, whatever. But at the end of the
00:45:23.599 --> 00:45:27.460
day, it doesn't matter as much as, uh, weapons
00:45:27.460 --> 00:45:31.610
of war and things that actually can. change in
00:45:31.610 --> 00:45:35.190
a lot of geopolitical situations. So now that
00:45:35.190 --> 00:45:39.349
I've upped the game in some way, right? We now,
00:45:39.809 --> 00:45:42.510
is the notion of a SOC 2, not the actual SOC
00:45:42.510 --> 00:45:45.309
2 itself, but something like a SOC 2 where someone
00:45:45.309 --> 00:45:48.309
comes in and audits you, everyone gets audited
00:45:48.309 --> 00:45:52.469
in some meaningful way. Is that a worthwhile
00:45:52.469 --> 00:45:58.710
exercise when there is more at stake? And so
00:45:58.710 --> 00:46:01.670
an example would be like medical equipment, right?
00:46:02.630 --> 00:46:06.869
I would generally want to have medical equipment
00:46:06.869 --> 00:46:11.570
be assessed, not even sampled. I would want all
00:46:11.570 --> 00:46:14.030
of them to be tested, right, in some way or another.
00:46:15.530 --> 00:46:19.489
And so I think that's really kind of the... I'm
00:46:19.489 --> 00:46:21.449
just giving the extreme here just to say there
00:46:21.449 --> 00:46:24.980
are things that I think we would say... A self
00:46:24.980 --> 00:46:27.000
-certification is not gonna be sufficient. You
00:46:27.000 --> 00:46:29.719
need somebody else to come in and certify that.
00:46:30.280 --> 00:46:32.480
So then this gets to the core of what CMMC is
00:46:32.480 --> 00:46:35.340
all about, which is we're doing this for what's
00:46:35.340 --> 00:46:39.199
called confidential, but unclassified information.
00:46:40.239 --> 00:46:44.820
Yeah, so CUI, C -U -I. So it's not secret information
00:46:44.820 --> 00:46:50.440
or top secret information. It's just, it's specifications
00:46:50.440 --> 00:46:56.579
on bullets, okay? which in some aggregate form
00:46:56.579 --> 00:47:00.199
probably reaches a little level of secret, maybe
00:47:00.199 --> 00:47:03.559
even top secret in some cases. But that's how
00:47:03.559 --> 00:47:07.820
this type of data is how China ended up building
00:47:07.820 --> 00:47:09.800
their version of the Joint Strike Fighter that
00:47:09.800 --> 00:47:14.039
looks identical to ours, right? And so this perspective
00:47:14.039 --> 00:47:17.360
of what's the right threshold of... of importance
00:47:17.360 --> 00:47:20.280
to this, are we putting that causes us to treat
00:47:20.280 --> 00:47:22.219
this like more medical equipment rather than
00:47:22.219 --> 00:47:26.360
a SOC 2? Yeah. Okay. So, I mean, I'd say you
00:47:26.360 --> 00:47:28.340
definitely answered the question and it was more
00:47:28.340 --> 00:47:33.920
about the threshold under which we have to require
00:47:33.920 --> 00:47:36.780
some kind of attestation. And I'm more focused
00:47:36.780 --> 00:47:39.880
on, even when you do say this is really important
00:47:39.880 --> 00:47:42.920
than a material vendor and it matters, I don't
00:47:42.920 --> 00:47:45.579
know. I mean, certainly SOC 2, but... All of
00:47:45.579 --> 00:47:48.340
the, and I think, sorry, but I think a lot of
00:47:48.340 --> 00:47:51.440
it comes from NIST too, that there's these, like
00:47:51.440 --> 00:47:52.940
we're going through SOC two and it's, hey, where's
00:47:52.940 --> 00:47:55.340
the evidence of the fire suppression? And I'm
00:47:55.340 --> 00:47:58.460
rolling my eyes like, like that's such a distraction
00:47:58.460 --> 00:48:02.179
for someone. And I think what would be great
00:48:02.179 --> 00:48:04.440
thresholds aside and whether or not this vendor
00:48:04.440 --> 00:48:08.579
rises to the occasion. Once one does, I think
00:48:08.579 --> 00:48:11.039
the next answer isn't to go to NIST and say,
00:48:11.159 --> 00:48:13.780
what are the hallmarks of maturity? But rather
00:48:13.780 --> 00:48:17.059
to red team it. And say, oh, okay. We think if
00:48:17.059 --> 00:48:19.219
this thing, if this person has, you know, this
00:48:19.219 --> 00:48:21.579
entity has weak security, then we can lose some
00:48:21.579 --> 00:48:22.980
data here. And it's going to be really important
00:48:22.980 --> 00:48:26.300
to us. Red team it and do it over and over again.
00:48:26.300 --> 00:48:29.000
And if you find something who would argue with
00:48:29.000 --> 00:48:31.099
fixing it, and if you don't, then that's a pass.
00:48:31.460 --> 00:48:33.699
Like, and maybe there's something where if, I
00:48:33.699 --> 00:48:35.340
mean, you rise to the occasion where we're definitely
00:48:35.340 --> 00:48:37.699
going to buy if you pass that. So now if you
00:48:37.699 --> 00:48:39.000
want to go forward, you're going to have to submit
00:48:39.000 --> 00:48:41.199
to red teaming. And maybe if it's just a total
00:48:41.199 --> 00:48:43.039
disaster in Swiss cheese and you're wasting a
00:48:43.039 --> 00:48:45.110
lot of time, then you. You have to foot the bill
00:48:45.110 --> 00:48:48.250
or something like that. But as we threat based
00:48:48.250 --> 00:48:51.110
and exploit based and aligned with the outcome
00:48:51.110 --> 00:48:54.090
that you're trying to avoid and not you test
00:48:54.090 --> 00:48:56.269
my fire suppression system with a questionnaire.
00:48:56.289 --> 00:48:58.090
So you can see whether or not I'm going to get
00:48:58.090 --> 00:49:00.469
hacked and lose all your data. Which is what's
00:49:00.469 --> 00:49:04.769
going on right now in TPRM. Yeah, I have, um,
00:49:04.769 --> 00:49:06.769
I have a bit of struggle on what you're suggesting
00:49:06.769 --> 00:49:08.690
because in some level, even going back to the
00:49:08.690 --> 00:49:12.329
earlier conversation with the, uh, with the,
00:49:12.429 --> 00:49:16.789
uh, gold eagle. discovery of, no, it's just,
00:49:16.829 --> 00:49:19.190
actually, it's just called Gold Eagle. You would
00:49:19.190 --> 00:49:21.010
think it's called Golden Eagle, but no, it's
00:49:21.010 --> 00:49:26.289
just Gold Eagle. Weird. Anyway, you would think,
00:49:26.349 --> 00:49:30.090
anyway, the perspective of Gold Eagle is simply
00:49:30.090 --> 00:49:33.670
that finding vulnerabilities and breaking into
00:49:33.670 --> 00:49:36.789
your environment is now potentially truly easy,
00:49:37.170 --> 00:49:41.340
right? And if you've tried to patch everything,
00:49:42.300 --> 00:49:45.619
will you patch everything? Are there novel things
00:49:45.619 --> 00:49:49.019
that one will discover? Oh, and by the way, part
00:49:49.019 --> 00:49:52.179
of the Gold Eagle premise was, hey, let's not
00:49:52.179 --> 00:49:55.059
rescan the same thing over and over again. I
00:49:55.059 --> 00:49:57.579
would actually disagree with that. I would say
00:49:57.579 --> 00:50:00.159
scan the same thing, try different harnesses,
00:50:00.159 --> 00:50:02.360
and you may find different things. That's exactly
00:50:02.360 --> 00:50:04.320
what we're finding. The probabilism AI kind of
00:50:04.320 --> 00:50:06.489
makes everyone a unique scan. Yeah, even if you
00:50:06.489 --> 00:50:08.269
did it twice with the exact same harness, you're
00:50:08.269 --> 00:50:10.769
going to definitely get two results. But what
00:50:10.769 --> 00:50:13.650
we're also discovering is that different harnesses
00:50:13.650 --> 00:50:19.710
have different levels of... You'll end up with
00:50:19.710 --> 00:50:21.510
potentially higher quality outcomes depending
00:50:21.510 --> 00:50:25.070
on the harness you use. So I would say no, it's
00:50:25.070 --> 00:50:27.670
kind of the same code base multiple times as
00:50:27.670 --> 00:50:29.730
long as you know that you're coordinating the
00:50:29.730 --> 00:50:31.610
different harnesses that are being used and that
00:50:31.610 --> 00:50:33.889
becomes more interesting. Anyway, my point though
00:50:33.889 --> 00:50:39.420
is that... With any sort of Does the compliance
00:50:39.420 --> 00:50:43.840
regime only? says that you're this tall but we
00:50:43.840 --> 00:50:47.019
all recognize that that a determined attacker
00:50:47.019 --> 00:50:50.619
or just even a basic AI system now can easily
00:50:50.619 --> 00:50:54.219
leap over that the basic you're this tall threshold
00:50:54.219 --> 00:50:57.019
III don't know I think that's kind of defeatist
00:50:57.019 --> 00:50:59.320
because I mean to say like we will don't don't
00:50:59.320 --> 00:51:00.940
test me on red teaming because everybody knows
00:51:00.940 --> 00:51:03.280
you could break in then don't use me because
00:51:04.279 --> 00:51:08.340
I think maybe given enough time and resource,
00:51:08.739 --> 00:51:11.119
but there's a spectrum of results from that,
00:51:11.179 --> 00:51:12.500
too. It doesn't have to be binary, and it could
00:51:12.500 --> 00:51:14.519
be you're really hard to break into, and that's
00:51:14.519 --> 00:51:17.079
good enough. Right, and I agree. So I think the
00:51:17.079 --> 00:51:21.039
basic threshold is with X amount of tokens and
00:51:21.039 --> 00:51:25.019
with commodity models, can you break in? And
00:51:25.019 --> 00:51:29.989
the answer is no? Good. Will that hold up for
00:51:29.989 --> 00:51:33.289
the next generation frontier model or even the
00:51:33.289 --> 00:51:36.829
next generation OSS open weight model? Yeah.
00:51:37.389 --> 00:51:39.150
The answer is going to be no, it's not going
00:51:39.150 --> 00:51:40.769
to be good enough. You're going to have to rise
00:51:40.769 --> 00:51:43.250
to the occasion. So maybe we finally have a meaningful
00:51:43.250 --> 00:51:45.869
metric, which is the number of tokens it takes
00:51:45.869 --> 00:51:49.030
to compromise. And so that will be the new, and
00:51:49.030 --> 00:51:50.550
let's say like for this particular contract,
00:51:50.630 --> 00:51:54.289
we're expecting a 65 trillion token security
00:51:54.289 --> 00:51:58.239
posture. We've solved security. Way to go, Sunil.
00:51:58.559 --> 00:52:00.159
Where's Mario? Or at least we just forget how
00:52:00.159 --> 00:52:05.960
to measure it. Yeah. Well, great. Yeah, there's
00:52:05.960 --> 00:52:07.619
no shortage of topics. It's always fun talking.
00:52:08.579 --> 00:52:11.539
I certainly miss Mario here. Hopefully, Tilson
00:52:11.539 --> 00:52:14.539
can edit in some What the Fudge is periodically
00:52:14.539 --> 00:52:17.840
through here. But otherwise, I hope your summer's
00:52:17.840 --> 00:52:20.400
going well. And man, I don't know about you,
00:52:20.400 --> 00:52:22.940
but the next time we talk, my kids might be back
00:52:22.940 --> 00:52:25.079
in school already. So things are moving fast.
00:52:25.789 --> 00:52:28.849
Well, have a good summer, otherwise. Thanks,
00:52:28.929 --> 00:52:29.769
Anil. Speak soon.