00:00:02.574 --> 00:00:03.091
[MUSIC]
00:00:03.092 --> 00:00:06.800
KATHLEEN SULLIVAN: Welcome
to AI Testing and Evaluation:
00:00:06.800 --> 00:00:11.600
Learnings from Science and Industry.
I'm your host, Kathleen Sullivan.
00:00:11.600 --> 00:00:16.640
As generative AI continues to advance, Microsoft
has gathered a range of experts—from genome
00:00:16.640 --> 00:00:21.040
editing to cybersecurity—to share how
their fields approach evaluation and
00:00:21.040 --> 00:00:26.240
risk assessment. Our goal is to learn from
their successes and their stumbles to move
00:00:26.240 --> 00:00:31.360
the science and practice of AI testing
forward. In this series, we'll explore
00:00:31.360 --> 00:00:39.268
how these insights might help guide the future of
AI development, deployment, and responsible use.
00:00:39.268 --> 00:00:39.280
[MUSIC ENDS]
00:00:39.280 --> 00:00:44.240
Today, I'm excited to welcome Ciaran Martin to the
podcast to explore testing and risk assessment in
00:00:44.240 --> 00:00:50.000
cybersecurity. Ciaran is a professor of practice
in the management of public organizations at the
00:00:50.000 --> 00:00:54.960
University of Oxford. He had previously
founded and served as chief executive of
00:00:54.960 --> 00:01:00.720
the National Cyber Security Centre within the
UK's intelligence, security, and cyber agency.
00:01:00.720 --> 00:01:05.520
And after our conversation, we'll talk to
Microsoft's Tori Westerhoff, a principal director
00:01:05.520 --> 00:01:12.080
on Microsoft’s AI Red Team, about how we should
think about these insights in the context of AI.
00:01:12.080 --> 00:01:14.785
Hi, Ciaran. Thank you so
much for being here today.
00:01:14.785 --> 00:01:17.520
CIARAN MARTIN: Well, thanks so much
for inviting me. It’s great to be here.
00:01:17.520 --> 00:01:21.440
SULLIVAN: Ciaran, before we get into some
regulatory specifics, it'd be great to hear
00:01:21.440 --> 00:01:27.200
a little bit more about your origin story, and
just take us to that day—who tapped you on the
00:01:27.200 --> 00:01:31.760
shoulder and said, “Ciaran, we need you to run a
national cyber center! Do you fancy building one?”
00:01:31.760 --> 00:01:38.160
MARTIN: You could argue that I owe my job to
Edward Snowden. Not an obvious thing to say.
00:01:38.160 --> 00:01:43.440
So the National Cyber Security Centre, which
didn't exist at the time—I was invited to join
00:01:43.440 --> 00:01:50.080
the British government's cybersecurity effort in
a leadership role—is now a subset of GCHQ. That's
00:01:50.080 --> 00:01:54.951
the digital intelligence agency. The equivalent
in the US obviously is the NSA [National Security
00:01:54.951 --> 00:02:00.560
Agency]. It had been convulsed by the Snowden
disclosures. It was an unprecedented challenge.
00:02:00.560 --> 00:02:08.240
I was a 17-year career government fixer with some
national security experience. So I was asked to go
00:02:08.240 --> 00:02:13.760
out and help with the policy response, the media
response, the legal response. But I said, look,
00:02:13.760 --> 00:02:19.040
any crisis, even one as big as this, is over
one way or the other in six months. What should
00:02:19.040 --> 00:02:24.960
I do long term? And they said, well, we were
thinking of asking you to try to help transform
00:02:24.960 --> 00:02:30.080
our cybersecurity mission. So the National Cyber
Security Centre was born, and I was very proud to
00:02:30.080 --> 00:02:37.040
lead it, and all in all, I did it for seven years
from startup to handing it on to somebody else.
00:02:37.040 --> 00:02:40.080
SULLIVAN: I mean, it's incredible.
And just building on that,
00:02:40.080 --> 00:02:44.960
people spend a significant portion of their
lives online now with a variety of devices,
00:02:44.960 --> 00:02:48.800
and maybe for listeners who are newer
to cybersecurity, could you give us the
00:02:48.800 --> 00:02:54.000
90-second lightning talk? Kind of, what does risk
assessment and testing look like in this space?
00:02:54.000 --> 00:02:58.240
MARTIN: Well, risk assessment and testing,
I think, are two different things. You can't
00:02:58.240 --> 00:03:03.440
defend everything. If you defend everything,
you're defending nothing. So broadly speaking,
00:03:03.440 --> 00:03:09.200
organizations face three threats. One is
complete disruption of their systems. So
00:03:09.200 --> 00:03:15.440
just imagine not being able to access your
system. The second is data protection, and
00:03:15.440 --> 00:03:20.400
that could be sensitive customer information. It
could be intellectual property. And the third is,
00:03:20.400 --> 00:03:25.280
of course, you could be at risk of just
straightforward being stolen from. I mean,
00:03:25.280 --> 00:03:28.320
you don't want any of them to happen, but
you have to have a hierarchy of harm.
00:03:28.320 --> 00:03:30.560
SULLIVAN: Yes.
MARTIN: So that's your risk assessment.
00:03:30.560 --> 00:03:35.200
The testing side, I think, is slightly different.
One of the paradoxes, I think, of cybersecurity is
00:03:35.200 --> 00:03:42.960
for such a scientific, data-rich subject, the sort
of metrics about what works are very, very hard
00:03:42.960 --> 00:03:51.440
to come by. So you've got boards and corporate
leadership and senior governmental structures, and
00:03:51.440 --> 00:03:56.320
they say, “Look, how do I run this organization
safely and securely?” And a cybersecurity chief
00:03:56.320 --> 00:03:59.840
within the organization will say, “Well,
we could get this capability in.” Well,
00:03:59.840 --> 00:04:04.400
the classic question for a leadership team to ask
is, well, what risk and harm will this reduce,
00:04:04.400 --> 00:04:08.880
by how much, and what's the cost-benefit
analysis? And we find that really hard.
00:04:08.880 --> 00:04:14.080
So that's really where testing and assurance
comes in. And also as technology changes so fast,
00:04:14.080 --> 00:04:18.800
we have to figure out, well, if we're worried
about post-quantum cryptography, for example,
00:04:18.800 --> 00:04:22.320
what standards does it have to meet? How do you
assess whether it's meeting those standards? So
00:04:22.320 --> 00:04:27.600
it's a huge issue in cybersecurity and one that
we're always very conscious of. It’s really hard.
00:04:27.600 --> 00:04:32.560
SULLIVAN: Given the scope of cybersecurity, are
there any differences in testing, let's say,
00:04:32.560 --> 00:04:37.280
for maybe a small business versus a critical
infrastructure operator? Are there any,
00:04:37.280 --> 00:04:42.640
sort of, metrics we can look at in terms
of distinguishing risk or assessment?
00:04:42.640 --> 00:04:48.880
MARTIN: There have to be. One of the reasons I
think why we have to be is that no small business
00:04:48.880 --> 00:04:54.320
can be expected to take on a hostile nation-state
that's well equipped. You have to be realistic.
00:04:54.320 --> 00:04:58.640
If you look at government guidance, certainly
in the UK 15 years ago on cybersecurity,
00:04:58.640 --> 00:05:02.160
you were telling small businesses that
are living hand to mouth, week by week,
00:05:02.160 --> 00:05:06.400
trying to make payments at the end of each
month, we were telling them they needed sort of
00:05:06.400 --> 00:05:10.240
nation-state-level cyber defenses. That was never
going to happen, even if they could afford it,
00:05:10.240 --> 00:05:15.120
which they couldn't. So you have to
have some differentiation. So again,
00:05:15.120 --> 00:05:19.040
you've got assessment frameworks and so forth
where you have to meet higher standards. So
00:05:19.040 --> 00:05:24.080
there absolutely has to be that distinction.
Otherwise, you end up in a crazy world of
00:05:24.080 --> 00:05:30.400
crippling small businesses with just unmanageable
requirements which they're never going to meet.
00:05:30.400 --> 00:05:33.760
SULLIVAN: It's such a great point. You
touched on this a little bit earlier, as well,
00:05:33.760 --> 00:05:39.440
but just cybersecurity governance operates in a
fast-moving technology and threat environment.
00:05:39.440 --> 00:05:44.880
How have testing standards evolved, and where
do new technical standards usually originate?
00:05:44.880 --> 00:05:47.600
MARTIN: I keep saying this is very difficult,
00:05:47.600 --> 00:05:56.320
and it is. [LAUGHTER] So I think there are two
challenges. One is actually about the balance,
00:05:56.320 --> 00:06:02.160
and this applies to the technology of today as
well as the technology of tomorrow. This is about,
00:06:02.160 --> 00:06:07.040
how do you make sure things are good enough
without crowding out new entrants? You want
00:06:07.040 --> 00:06:11.520
people to be innovative and dynamic.
You want disruptors in this business.
00:06:11.520 --> 00:06:17.600
But if you say to them, “Look, well, you have to
meet these 14 impossibly high technical standards
00:06:17.600 --> 00:06:21.600
before you can even sell to anybody or sell to
the government,” whatever, then you've got a
00:06:21.600 --> 00:06:24.320
problem. And I think we've wrestled with that,
and there's no perfect answer. You just have
00:06:24.320 --> 00:06:32.160
to try and go to … find the sweet spot between two
ends of a spectrum. And that's going to evolve.
00:06:32.160 --> 00:06:36.880
The second point, which in some respects if you've
got the right capabilities is slightly easier but
00:06:36.880 --> 00:06:44.320
still a big call, is around, you know, those newer
and evolving technologies. And here, having, you
00:06:44.320 --> 00:06:48.320
know, been a bit sort of gloomy and pessimistic,
here I think is actually an opportunity. So one
00:06:48.320 --> 00:06:53.440
of the things we always say in cybersecurity is
that the internet was built and developed without
00:06:53.440 --> 00:06:57.760
security in mind. And that was kind of true in the
’90s and the noughties, as we call them over here.
00:06:59.520 --> 00:07:03.760
But I think as you move into things like
post-quantum computing, applied use of AI,
00:07:03.760 --> 00:07:08.400
and so on, you can actually set the standards
at the beginning. And that's really good
00:07:08.400 --> 00:07:12.080
because it's saying to people that these
are the things that are going to matter in
00:07:12.080 --> 00:07:15.520
the post-quantum age. Here's the outline of
the standards you're going to have to meet;
00:07:15.520 --> 00:07:19.440
start looking at them. So there's an opportunity
actually to make technology safer by design,
00:07:19.440 --> 00:07:22.360
by getting ahead of it. And I
think that's the era we're in now.
00:07:22.360 --> 00:07:25.040
SULLIVAN: That makes a lot of
sense. Just building on that,
00:07:25.040 --> 00:07:29.440
do businesses and the public trust these
standards? And I guess, which standard do
00:07:29.440 --> 00:07:34.000
you wish the world would just adopt already,
and what's the real reason they haven't?
00:07:34.000 --> 00:07:38.560
MARTIN: Well, again, where do you start? I
mean, most members of the public quite rightly
00:07:38.560 --> 00:07:44.960
haven't heard of any of these standards.
I think public trust and public capital
00:07:44.960 --> 00:07:50.480
in any society matters. But I think it is
important that these things are credible.
00:07:50.480 --> 00:07:56.480
And there's quite a lot of convergence between,
you know, the top-level frameworks. And obviously
00:07:56.480 --> 00:07:58.800
in the US, you know, the NIST [National Institute
of Standards and Technology] framework is the one
00:07:58.800 --> 00:08:02.000
that's most popular for cybersecurity,
but it bears quite a strong resemblance
00:08:02.000 --> 00:08:07.920
to the international one, ISO[/IEC] 27001, and
there are others, as well. But fundamentally,
00:08:07.920 --> 00:08:12.320
they boil down to kind of five things.
Do a risk assessment; work out what your
00:08:12.320 --> 00:08:18.640
crown jewels are. Protect your perimeter as
best you can. Those are the first two.
00:08:18.640 --> 00:08:23.120
The third one then is when your perimeter's
breached, be able to detect it more times
00:08:23.120 --> 00:08:29.840
than not. And when you can't do that,
you go to the fourth one, which is,
00:08:29.840 --> 00:08:34.400
can you mitigate it? And when all else fails,
how quickly can you recover and manage it?
00:08:34.400 --> 00:08:38.960
I mean, all the standards are expressed
in way more technical language than that,
00:08:38.960 --> 00:08:44.640
but fundamentally, if everybody adopted those
five things and operated them in a simple way,
00:08:44.640 --> 00:08:48.720
you wouldn't eliminate the harm, but
you would reduce it quite substantially.
00:08:48.720 --> 00:08:50.560
SULLIVAN: Which policy initiatives are
00:08:50.560 --> 00:08:54.560
most promising for incentivizing
companies to undertake, you know,
00:08:54.560 --> 00:09:00.320
these cybersecurity testing parameters that you’ve
just outlined? Governments, including the UK,
00:09:00.320 --> 00:09:06.880
have used carrots and sticks, but what do
you think will actually move the needle?
00:09:06.880 --> 00:09:13.360
MARTIN: I think there are two answers to
that, and it comes back to your split between
00:09:13.360 --> 00:09:19.360
smaller businesses and critically important
businesses. In the critically important services,
00:09:19.360 --> 00:09:26.240
I think it's easier because most industries are
looking for a level playing field. In other words,
00:09:26.240 --> 00:09:30.240
they realize there have to be rules and
they want to apply them to everyone.
00:09:30.240 --> 00:09:35.360
We had a fascinating experience when I was
in government back in around 2018 where the
00:09:35.360 --> 00:09:39.040
telecom sector, they came to us and they
said, we've got a very good cooperative
00:09:39.040 --> 00:09:43.520
relationship with the British government,
but it needs to be put on a proper legal
00:09:43.520 --> 00:09:49.280
footing because you're just asking us nicely to
do expensive things. And in a regulated sector,
00:09:49.280 --> 00:09:53.440
if you actually put in some rules—and
please develop them jointly with us;
00:09:53.440 --> 00:09:57.840
that's the crucial part—then that will help
because it means that we're not going to our
00:09:57.840 --> 00:10:01.680
boards and saying, or our shareholders, and saying
that we should do this, and they're saying, “Well,
00:10:01.680 --> 00:10:05.440
do you have to do it? Are our competitors
doing it?” And if the answer to that is,
00:10:05.440 --> 00:10:09.360
yes, we have to, and, yes, our competitors
are doing it, then it tends to be OK.
00:10:09.360 --> 00:10:15.440
The harder nut to crack is the smaller business.
And I think there's a real mystery here:
00:10:15.440 --> 00:10:20.320
why has nobody cracked a really good and easy
solution for small business? We need to be
00:10:20.320 --> 00:10:25.600
careful about this because, you know, you can't
throttle small businesses with onerous regulation.
00:10:25.600 --> 00:10:31.760
At the same time, we're not brilliant, I
think, in any part of the world at using
00:10:31.760 --> 00:10:37.120
the normal corporate governance rules to try and
get people to figure out how to do cybersecurity.
00:10:37.120 --> 00:10:41.360
There are initiatives there that are not
the sort of pretty heavy stick that you
00:10:41.360 --> 00:10:45.520
might have to take to a critical function,
but they could help. But that is a hard nut
00:10:45.520 --> 00:10:49.520
to crack. And I look around the world,
and, you know, I think if this was easy,
00:10:49.520 --> 00:10:52.960
somebody would have figured it out by now.
I think most of the developed economies
00:10:52.960 --> 00:10:56.720
around the world really struggle with
cybersecurity for smaller businesses.
00:10:56.720 --> 00:11:00.800
SULLIVAN: Yeah, it's a great point. Actually
building on one of the comments you made on
00:11:00.800 --> 00:11:06.240
the role of, kind of, government, how
do you see the role of private-public
00:11:06.240 --> 00:11:10.720
partnerships scaling and strengthening,
you know, robust cybersecurity testing?
00:11:10.720 --> 00:11:14.640
MARTIN: I think they're crucial, but they
have to be practical. I've got a slight,
00:11:14.640 --> 00:11:18.320
sort of, high horse on this, if you don't
mind, Kathleen. It's sort of … [LAUGHS]
00:11:18.320 --> 00:11:18.680
SULLIVAN: Of course.
00:11:18.680 --> 00:11:22.800
MARTIN: I think that there are two
types of public-private partnership.
00:11:22.800 --> 00:11:27.360
One involves committees saying that we should
strengthen partnerships and we should all work
00:11:27.360 --> 00:11:31.360
together and collaborate and share stuff.
And we tried that for a very long time,
00:11:31.360 --> 00:11:34.240
and it didn't get us very
far. There are other types.
00:11:34.240 --> 00:11:38.720
We had some at the National Cyber Security Centre
where we paid companies to do spectacularly
00:11:38.720 --> 00:11:43.840
good technical work that the market wouldn't
provide. So I think it's sort of partnership
00:11:43.840 --> 00:11:47.840
with a purpose. I think sometimes, and I
understand the human instinct to do this,
00:11:47.840 --> 00:11:51.280
particularly in governments and big business,
they think you need to get around a table
00:11:51.280 --> 00:11:55.520
and work out some grand strategy to fix
everything, and the scale of the … not just
00:11:55.520 --> 00:11:59.200
the problem but the scale of the whole
technology is just too big to do that.
00:11:59.200 --> 00:12:04.640
So pick a bit of the problem. Find some
ways of doing it. Don't over-lawyer it.
00:12:04.640 --> 00:12:08.400
[LAUGHTER] I think sometimes people get very
nervous. Oh, well, is this our role? You know,
00:12:08.400 --> 00:12:11.360
should we be doing this, that, and the other?
Well, you know, sometimes certainly in this
00:12:11.360 --> 00:12:15.920
country, you think, well, who's actually
going to sue you over this, you know? So
00:12:16.480 --> 00:12:22.160
I wouldn't over-programmatize it. Just get
stuck practically into solving some problems.
00:12:22.160 --> 00:12:24.240
SULLIVAN: I love that.
Actually, [it] made me think,
00:12:24.240 --> 00:12:26.720
are there any surprising allies
that you've gained—you know,
00:12:26.720 --> 00:12:31.987
maybe someone who you never expected to be
a cybersecurity champion—through your work?
00:12:31.987 --> 00:12:41.440
MARTIN: Ooh! That's a … that's a… what a
question! To give you a slightly disappointing
00:12:41.440 --> 00:12:48.240
answer, but it relates to your previous
question. In the early part of my career,
00:12:48.240 --> 00:12:54.400
I was working in institutions like the UK
Treasury long before I was in cybersecurity,
00:12:54.400 --> 00:12:58.080
and the treasury and the British civil
service in general, but the treasury in
00:12:58.080 --> 00:13:04.240
particular sort of trained you to believe that
the private sector was amoral, not immoral,
00:13:04.240 --> 00:13:09.360
amoral. It just didn't have values. It just had
bottom line, and, you know, its job essentially
00:13:09.360 --> 00:13:14.720
was to provide employment and revenue then for
the government to spend on good things that people
00:13:14.720 --> 00:13:20.640
cared about. And when I got into cybersecurity and
people said, look, you need to develop relations
00:13:20.640 --> 00:13:25.280
with this cybersecurity company, often in the US,
actually. I thought, well, what's in it for them?
00:13:25.280 --> 00:13:30.160
And, sure, sometimes you were paying them for
specific services, but other times, there was a
00:13:30.160 --> 00:13:35.280
real public spiritedness about this. There was
a realization that if you tried to delineate
00:13:35.280 --> 00:13:40.160
public-private boundaries, that it wouldn't really
work. It was a shared risk. And you could analyze
00:13:40.160 --> 00:13:44.080
where the boundaries fell or you could actually
go on and do something about it together. So
00:13:44.080 --> 00:13:53.280
I was genuinely surprised at the allyship from
the cybersecurity sector. Absolutely, I really,
00:13:53.280 --> 00:13:58.520
really was. And I think it's a really positive
part of certainly the UK cybersecurity ecosystem.
00:13:58.520 --> 00:14:02.240
SULLIVAN: Wonderful. Well, we're
coming to the end of our time here,
00:14:02.240 --> 00:14:08.320
but is there any maybe last thoughts or perhaps
requests you have for our listeners today?
00:14:08.320 --> 00:14:14.640
MARTIN: I think that standards,
assurance, and testing really matter,
00:14:14.640 --> 00:14:21.360
but it's a bit like the discussion we're having
over AI. Get all these things to take you 80,
00:14:21.360 --> 00:14:28.720
90% of the way and then really apply
your judgment. There's been some bad
00:14:28.720 --> 00:14:33.680
regulation under the auspices of standards and
assurance. First of all, it’s, have you done this
00:14:33.680 --> 00:14:39.920
assessment? Have you done that? Have you looked
at this? Well, fine. And you can tick that box,
00:14:39.920 --> 00:14:45.040
but what does it actually mean when you do it?
What bits that you know in your heart of hearts
00:14:45.040 --> 00:14:50.240
are really important to the defense of your
organization that may not be covered by this
00:14:50.240 --> 00:14:56.120
and just go and do those anyway. Because
sure it helps, but it's not everything.
00:14:56.120 --> 00:14:59.680
SULLIVAN: No. Great, great
closing sentiment. Well, Ciaran,
00:14:59.680 --> 00:15:03.200
thank you for joining us today. This
has been just a super fun conversation
00:15:03.200 --> 00:15:07.520
and really insightful. Just really
enjoyed the conversation. Thank you.
00:15:07.520 --> 00:15:14.800
MARTIN: My pleasure, Kathleen, thank you.
00:15:14.800 --> 00:15:18.417
[TRANSITION MUSIC]
00:15:18.417 --> 00:15:20.320
SULLIVAN:
00:15:20.320 --> 00:15:26.080
Now, I'm happy to introduce Tori Westerhoff. As a
principal director on the Microsoft AI Red Team,
00:15:26.080 --> 00:15:30.160
Tori leads all AI security and
safety red team operations,
00:15:30.160 --> 00:15:34.880
as well as dangerous capability testing,
to directly inform C-suite decision-makers.
00:15:34.880 --> 00:15:36.863
So, Tori, welcome!
00:15:36.863 --> 00:15:39.360
TORI WESTERHOFF: Thanks. I
am so excited to be here.
00:15:39.360 --> 00:15:43.680
SULLIVAN: I'd love to just start a little bit
more learning about your background. You've
00:15:43.680 --> 00:15:48.800
worn some very intriguing hats. I mean,
cognitive neuroscience grad from Yale,
00:15:48.800 --> 00:15:54.800
national security consultant, strategist
in augmented and virtual reality … how do
00:15:54.800 --> 00:15:59.040
those experiences help shape the way
you lead the Microsoft AI Red Team?
00:15:59.040 --> 00:16:01.760
WESTERHOFF: I always joke this is the only role I
00:16:01.760 --> 00:16:08.880
think will always combine the entire
patchwork LinkedIn résumé. [LAUGHS]
00:16:08.880 --> 00:16:18.640
I think I use those experiences to help me
understand the really broad approach that
00:16:18.640 --> 00:16:28.720
AI Red Team—artist also known as AIRT; I'm sure
I'll slip into our acronym—how we frame up the
00:16:28.720 --> 00:16:35.200
broad security implications of AI. So I think the
cognitive neuroscience element really helped me
00:16:35.200 --> 00:16:43.440
initially approach AI hacking, right. There's
a lot of social engineering and manipulation
00:16:43.440 --> 00:16:51.040
within chat interfaces that are enabled by
AI. And also, kind of, this, like, metaphor
00:16:51.040 --> 00:16:59.920
for understanding how to find soft spots in the
way that you see human heuristics show up, too.
00:16:59.920 --> 00:17:08.000
And so I think that was actually my personal “in”
to getting hooked into AI red teaming generally.
00:17:08.960 --> 00:17:15.920
But my experience in national security and I'd
also say working through the AR/VR/metaverse
00:17:15.920 --> 00:17:25.200
space at the time where I was in it helped
me balance both how our impact is framed,
00:17:25.200 --> 00:17:30.720
how we're thinking about critical industries, how
we're really trying to push our understanding of
00:17:30.720 --> 00:17:38.720
where security of AI can help people the most.
And also do it in a really breakneck speed in
00:17:38.720 --> 00:17:42.800
an industry that's evolving all of the time,
that's really pushing you to always be at the
00:17:42.800 --> 00:17:50.080
bleeding edge of your understanding. So I draw
a lot of the energy and the mission criticality
00:17:50.080 --> 00:17:54.480
and the speed from those experiences
as we're shaping up how we approach it.
00:17:54.480 --> 00:17:58.960
SULLIVAN: Can you just give us a quick rundown?
What does the Red Team do? What actually,
00:17:58.960 --> 00:18:02.720
kind of, is involved on a day-to-day
basis? And then as we think about,
00:18:02.720 --> 00:18:05.840
you know, our engagements with
large enterprises and companies,
00:18:05.840 --> 00:18:10.480
how do we work alongside some of
those companies in terms of testing?
00:18:10.480 --> 00:18:19.200
WESTERHOFF: The way I see our team is almost
like an indicator light that works really part
00:18:19.200 --> 00:18:25.200
and parcel with product development. So the
way we've organized our expert red teaming
00:18:25.200 --> 00:18:30.400
efforts is that we work with product
development before anything ships out
00:18:30.400 --> 00:18:36.240
to anyone who can use it. And our job
is to act as expert AI manipulators,
00:18:36.240 --> 00:18:43.520
AI hackers. And we are supposed to take the
theories and methods and new research and
00:18:43.520 --> 00:18:50.400
harness it to find examples of vulnerabilities
or soft spots in products to enable product
00:18:50.400 --> 00:18:55.760
teams to harden those soft spots before anything
actually reaches someone who wants to use it.
00:18:55.760 --> 00:19:00.320
So if we're the indicator light,
we are also not the full workup,
00:19:00.320 --> 00:19:04.800
right. I see that as measurement and
evals. And we also are not the mechanic,
00:19:04.800 --> 00:19:10.080
which is that product development team that's
creating mitigations. It's platform-security
00:19:10.080 --> 00:19:16.400
folks who are creating mitigations at scale. And
there's a really great throughput of insights from
00:19:16.400 --> 00:19:24.720
those groups back into our area where we love to
inform about them, but we also love to add on to,
00:19:24.720 --> 00:19:28.560
how do we break the next thing,
right? So it's a continuous cycle.
00:19:30.160 --> 00:19:37.600
And part of that is just being really creative and
thinking outside of a traditional cybersecurity
00:19:37.600 --> 00:19:44.000
box. And part of that is also really thinking
about how we pull in research—we have a research
00:19:44.000 --> 00:19:49.040
function within our AI Red Team—and
how we automate and scale. This year,
00:19:49.040 --> 00:19:54.160
we've pulled a lot of those assets and
insights into the Azure [AI] Foundry AI
00:19:54.160 --> 00:19:59.120
Red Teaming Agent. And so folks can
now access a lot of our mechanisms
00:19:59.680 --> 00:20:05.560
through that. So you can get a little taste of
what we do day to day in the AI Red Teaming Agent.
00:20:05.560 --> 00:20:10.400
SULLIVAN: You recently—actually, with your
team—published a report that outlined lessons
00:20:10.400 --> 00:20:15.920
from testing over a hundred generative
AI products. But could you share a bit
00:20:15.920 --> 00:20:19.360
about what you learned? What were some of
the important lessons? Where do you see
00:20:19.360 --> 00:20:25.320
opportunities to improve the state of red
teaming as a method for probing AI safety?
00:20:25.320 --> 00:20:33.360
WESTERHOFF: I think the most important
takeaway from those lessons is that AI
00:20:33.360 --> 00:20:40.960
security is truly a team sport. You'll
hear cybersecurity folks say that a lot.
00:20:40.960 --> 00:20:50.480
And part of the rationale there is that the
defense in depth and integrating and a view
00:20:50.480 --> 00:20:57.040
towards AI security through the entire
development of AI systems is really the
00:20:57.040 --> 00:21:03.360
way that we're going to approach this
with intentionality and responsibility.
00:21:03.360 --> 00:21:10.160
So in our space, we really focus on novel
harm categories. We are pushing bleeding edge,
00:21:10.160 --> 00:21:17.840
and we also are pushing iterative and, like,
contextually based red teaming in product
00:21:17.840 --> 00:21:26.080
dev. So outside of those hundred that we've done,
there's a community [LAUGHS] through the entire,
00:21:26.080 --> 00:21:32.640
again, multistage life cycle of a product that
is really trying to push the cost of attacking
00:21:32.640 --> 00:21:38.960
those AI systems higher and higher with all
of the expertise they bring. So we may be,
00:21:38.960 --> 00:21:44.720
like, the experts in AI hacking in that
line, but there are also so many partners
00:21:44.720 --> 00:21:50.240
in the Microsoft ecosystem who are thinking
about their market context or they really,
00:21:50.240 --> 00:21:53.920
really know the people who love their
products. How are they using it?
00:21:53.920 --> 00:21:57.760
And then when you bubble out, you also
have industry and government who are
00:21:57.760 --> 00:22:06.000
working together to push towards the most
secure AI implementation for people, right?
00:22:06.000 --> 00:22:12.480
And I think our team in particular, we feel
really grateful to be part of the big AI safety
00:22:12.480 --> 00:22:17.880
and security ecosystem at Microsoft and also to
be able to contribute to the industry writ large.
00:22:17.880 --> 00:22:21.440
SULLIVAN: As you know, we had a chance
to speak with Professor Ciaran Martin
00:22:21.440 --> 00:22:25.200
from the University of Oxford about the
cybersecurity industry and governance
00:22:25.200 --> 00:22:29.440
there. What are some of the ideas and
tools from that space that are surfacing
00:22:29.440 --> 00:22:33.360
in how we think about approaching red
teaming and AI governance broadly?
00:22:33.360 --> 00:22:41.200
WESTERHOFF: Yeah, I think it's such a
broad set of perspectives to bring in,
00:22:41.200 --> 00:22:50.160
in the AI instance. Something that I've noticed
interjecting into security at the AI junction,
00:22:50.160 --> 00:22:56.240
right, is that cybersecurity has so many
decades of experience of working through
00:22:56.240 --> 00:23:03.520
how to build trustworthy computing, for example,
or bring an entire industry to bear in that way.
00:23:03.520 --> 00:23:11.360
And I think that AI security and safety can learn
a lot of lessons of how to bring clarity and
00:23:11.360 --> 00:23:17.920
transparency across the industry to push universal
understanding of where the threats really are.
00:23:19.040 --> 00:23:26.320
So frameworks coming out of NIST, coming out of
MITRE that help us have a universal language that
00:23:26.320 --> 00:23:32.720
inform governance, I think, are really important
because it brings clarity irrespective of where
00:23:32.720 --> 00:23:39.520
you are looking into AI security, irrespective
of your company size, what you're working on. It
00:23:39.520 --> 00:23:45.440
means you all understand, “Hey, we are really
worried about this fundamental impact.” And
00:23:45.440 --> 00:23:52.160
I think cybersecurity has done a really good job
of driving towards impact as their organizational
00:23:52.160 --> 00:23:58.960
vector. And I am starting to see that in the AI
space, too, where we're trying to really clarify
00:23:58.960 --> 00:24:03.360
terms and threats. And you see it in updates of
those frameworks, as well, that I really love.
00:24:03.360 --> 00:24:08.560
So I think that the innovation is in
transparency to folks who are really
00:24:08.560 --> 00:24:12.720
innovating and doing the work so
we all have a shared language,
00:24:12.720 --> 00:24:20.480
and from that, it really creates communal
goals across security instead of a lot of
00:24:20.480 --> 00:24:23.680
people being worried about the same thing
and talking about it in a different way.
00:24:23.680 --> 00:24:28.800
SULLIVAN: Mm-hmm. In the cybersecurity context,
Ciaran really stressed matching risk frameworks
00:24:28.800 --> 00:24:34.720
to an organization's role and scale. Microsoft
plays many roles, including building models
00:24:34.720 --> 00:24:40.600
and shipping applications. How does your red
teaming approach shift across those layers?
00:24:40.600 --> 00:24:46.320
WESTERHOFF: I love this question also
because I love it as part of our work.
00:24:46.320 --> 00:24:51.120
So one of the most fascinating things
about working on this team has been the
00:24:51.120 --> 00:24:56.560
diversity of the technology that we end
up red teaming and testing. And it feels
00:24:56.560 --> 00:25:01.440
like we're in the crucible in that
way. Because we see AI applied to so
00:25:01.440 --> 00:25:10.640
many different architectures, tech stacks,
individual features, models, you name it.
00:25:10.640 --> 00:25:16.320
Part of my answer is that we still care
about the highest-impact things. And so
00:25:16.320 --> 00:25:21.440
irrespective of the iteration, which
is really fascinating and I love,
00:25:23.200 --> 00:25:29.120
I still think that our team drives to say, “OK,
what is that critical vulnerability that is
00:25:29.120 --> 00:25:36.000
going to affect people in the largest ways, and
can we battle test to see if that can occur?”
00:25:36.000 --> 00:25:44.720
So in some ways, the task is always the same.
I think in the ways that we change our testing,
00:25:44.720 --> 00:25:55.440
we customize a lot to the access
to systems and data and also
00:25:57.280 --> 00:26:05.760
people's trust almost as different variables
that could affect the impact, right.
00:26:05.760 --> 00:26:10.640
So a good example is if we're thinking
through agentic frameworks that have
00:26:10.640 --> 00:26:17.200
access to functions and tools and
preferential ability to act on data,
00:26:17.200 --> 00:26:22.880
it's really different to spaces where
that action may not be feasible,
00:26:22.880 --> 00:26:31.040
right. And so I think the tailoring of the way
to get to that impact is hyper-custom every time
00:26:31.040 --> 00:26:39.360
we start an engagement. And part of it is very
thesis driven and almost mechanizing empathy.
00:26:39.360 --> 00:26:46.000
You almost need to really focus on how people
could use, or misuse, in such a way that you
00:26:46.000 --> 00:26:53.520
can emulate it before to a really great signal
to product development, to say this is truly
00:26:53.520 --> 00:26:59.760
what people could do and we want to deliver
the highest-impact scenarios so you can solve
00:26:59.760 --> 00:27:06.000
for those and also solve the underlying patterns,
actually, that could contribute to maybe that one
00:27:06.000 --> 00:27:12.080
piece of evidence but also all the related pieces
of evidence. So singular drive but like hyper-,
00:27:12.080 --> 00:27:17.480
hyper-customization to what that piece
of tech could do and has access to.
00:27:17.480 --> 00:27:21.040
SULLIVAN: What are some of the unexplored
testing approaches or considerations from
00:27:21.040 --> 00:27:24.960
cybersecurity that you think we
should encourage AI technologists,
00:27:24.960 --> 00:27:28.000
policymakers, and other stakeholders to focus on?
00:27:28.000 --> 00:27:34.400
WESTERHOFF: I do love that AI humbles us
each and every day with new capabilities
00:27:34.400 --> 00:27:38.640
and the potential for new capabilities. It's
not just saying, “Hey, there's one test that
00:27:38.640 --> 00:27:43.520
we want to try,” but more, “Hey, can we
create a methodology that we feel really,
00:27:43.520 --> 00:27:48.640
really solid about so that when we are
asked a question we haven't even thought of,
00:27:48.640 --> 00:27:53.440
we feel confident that we have
the resources and the system?”
00:27:53.440 --> 00:28:01.360
So part of me is really intrigued by the
process that we're asked to make without
00:28:01.360 --> 00:28:07.440
knowing what those capabilities are really
going to bring. And then I think tactically,
00:28:07.440 --> 00:28:14.560
AIRT is really pushing on how we create new
research methodologies. How are we investing in,
00:28:14.560 --> 00:28:20.080
kind of, these longer-term iterations
of red teaming? So we're really excited
00:28:20.080 --> 00:28:27.840
about pushing out those insights in
an experimental and longer-term way.
00:28:28.480 --> 00:28:34.320
I think another element is a little bit of
that evolution of how industry standards
00:28:34.320 --> 00:28:43.760
and frameworks are updating to the AI moment and
really articulating where AI is either furthering
00:28:43.760 --> 00:28:55.600
adversarial ability to create those harms or
threats or identifying where AI has a net new
00:28:55.600 --> 00:29:01.040
harm. And I think that demystifies a little
bit about what we talked about in terms of the
00:29:01.040 --> 00:29:07.200
lessons learned, that fundamentally, a lot of the
things that we talk about are traditional security
00:29:07.200 --> 00:29:16.320
vulnerabilities, and we are standing on kind of
that cybersecurity shoulder. And I'm starting
00:29:16.320 --> 00:29:25.840
to see those updates translate in spaces that
are already considered trustworthy and kind of
00:29:27.440 --> 00:29:32.720
the basis on which not only
cybersecurity folks build their
00:29:32.720 --> 00:29:39.920
work but also business decision-makers
make decisions on those frameworks.
00:29:39.920 --> 00:29:44.640
So to me, integration of AI into
those frameworks by those same
00:29:44.640 --> 00:29:50.800
standards means that we're evolving
security to include AI. We aren't
00:29:50.800 --> 00:29:55.440
creating an entirely new industry
of AI security and that, I think,
00:29:55.440 --> 00:30:02.000
really helps anchor people in the really solid
foundation that we have in cybersecurity anyways.
00:30:02.000 --> 00:30:14.560
I think there's also some work around how
the cyber, like, defenses will actually
00:30:14.560 --> 00:30:21.040
benefit from AI. So we think a lot about
threats because that's our job. But the
00:30:21.040 --> 00:30:27.760
other side of cybersecurity is offense.
And I'm seeing a ton of people come out
00:30:27.760 --> 00:30:31.680
with frameworks and methodologies,
especially in the research space,
00:30:31.680 --> 00:30:37.840
on how defensive networks are going to be
benefited from things like agentic systems.
00:30:38.400 --> 00:30:45.520
Generally speaking, I think the best practice
is to realize that we're fundamentally still
00:30:45.520 --> 00:30:52.400
talking about the same impacts, and we can use
the same avenues, conversations, and frameworks.
00:30:52.400 --> 00:31:00.280
We just really want them to be crisply updated
with that understanding of AI applications.
00:31:00.280 --> 00:31:04.240
SULLIVAN: How do you think about
bringing others into the fold there?
00:31:04.240 --> 00:31:09.600
I think those standards and frameworks are often
informed by technologists. But I'd love for you
00:31:09.600 --> 00:31:15.200
to expand [that to] policymakers or other
kind of stakeholders in our ecosystem, even,
00:31:15.200 --> 00:31:19.680
you know, end consumers of these products.
Like, how do we communicate some of this
00:31:19.680 --> 00:31:24.520
to them in a way that resonates
and it has an impactful meaning?
00:31:24.520 --> 00:31:32.240
WESTERHOFF: I've found the AI security-safety
space to be one of the more collaborative. I
00:31:32.240 --> 00:31:36.560
actually think the fact that I'm talking to
you today is probably evidence that a ton
00:31:36.560 --> 00:31:42.000
of people are bringing in perspectives
that don't only come from a long-term
00:31:42.000 --> 00:31:50.400
cybersecurity view. And I see that as a trend
in how AI is being approached opposed to how
00:31:50.400 --> 00:31:57.120
those areas were moving earlier. So I think
that speed and the idea of conversations and
00:31:57.120 --> 00:32:01.760
not always having the perfect answer but
really trying to be transparent with what
00:32:01.760 --> 00:32:09.520
everyone does know is kind of a communal
energy in the communities, at least,
00:32:09.520 --> 00:32:14.200
where we're playing. [LAUGHS] So I am pretty
biased but at least the spaces where we are.
00:32:14.200 --> 00:32:16.160
SULLIVAN: No, I think we're seeing
that across the board. I mean,
00:32:16.160 --> 00:32:20.720
I'd echo [that] sitting in research,
as well, like, that ability to have
00:32:20.720 --> 00:32:26.320
impact now and at speed to getting the
amazing technology and models that we're
00:32:26.320 --> 00:32:31.360
creating into the hands of our customers and
partners and ecosystem is just underscored.
00:32:32.240 --> 00:32:37.520
So on the note of speed, let's shift gears a
little bit to just a quick lightning round. I'd
00:32:37.520 --> 00:32:43.120
love to get maybe some quick thoughts from you,
just 30-second answers here. I'll start with one.
00:32:43.120 --> 00:32:48.040
Which headline-grabbing AI threat
do you think is mostly hot air?
00:32:48.040 --> 00:32:56.400
WESTERHOFF: I think we should pay attention
to it all. I'm a red team lead. I love a good
00:32:56.400 --> 00:33:05.040
question to see if we can find an answer in
real life. So no hot air, just questions.
00:33:05.040 --> 00:33:09.280
SULLIVAN: Is there some sort of maybe new tool
00:33:09.280 --> 00:33:13.560
that you can't wait to sneak
into the red team arsenal?
00:33:13.560 --> 00:33:20.240
WESTERHOFF: I think there are really
interesting methodologies that break
00:33:20.240 --> 00:33:26.000
our understanding of cybersecurity by
looking at the intersection between
00:33:26.000 --> 00:33:33.920
different layers of AI and how you
can manipulate AI-to-AI interaction,
00:33:33.920 --> 00:33:41.480
especially now when we're looking at agentic
systems. So I would say a method, not a tool.
00:33:41.480 --> 00:33:44.080
SULLIVAN: So maybe ending on a
little bit of a lighter note,
00:33:44.080 --> 00:33:47.280
do you have a go-to snack during
an all-night red teaming session?
00:33:47.280 --> 00:33:52.160
WESTERHOFF: Always coffee. I would
love it to be a protein smoothie,
00:33:52.160 --> 00:33:57.280
but honestly, it is probably
Trader Joe's elote chips. Like
00:33:57.280 --> 00:34:03.983
the whole bag. [LAUGHTER] It’s going to get me
through. I'm going to not love that I did it.
00:34:03.983 --> 00:34:03.990
[MUSIC]
00:34:03.990 --> 00:34:07.040
SULLIVAN: Amazing. Well, Tori,
thanks so much for joining us today,
00:34:07.040 --> 00:34:10.800
and just a huge thanks also to
Ciaran for his insights, as well.
00:34:10.800 --> 00:34:14.040
WESTERHOFF: Thank you so much
for having me. This was a joy.
00:34:14.040 --> 00:34:18.800
SULLIVAN: And to our listeners, thanks
for tuning in. You can find resources
00:34:18.800 --> 00:34:22.240
related to this podcast in the
show notes. And if you want to
00:34:22.240 --> 00:34:29.680
learn more about how Microsoft approaches AI
governance, you can visit microsoft.com/RAI.
00:34:29.680 --> 00:34:35.360
See you next time!
00:34:35.360 --> 00:34:36.201
[MUSIC FADES]
[MUSIC]
00:00:03.092 --> 00:00:06.800
KATHLEEN SULLIVAN: Welcome
to AI Testing and Evaluation:
00:00:06.800 --> 00:00:11.600
Learnings from Science and Industry.
I'm your host, Kathleen Sullivan.
00:00:11.600 --> 00:00:16.640
As generative AI continues to advance, Microsoft
has gathered a range of experts—from genome
00:00:16.640 --> 00:00:21.040
editing to cybersecurity—to share how
their fields approach evaluation and
00:00:21.040 --> 00:00:26.240
risk assessment. Our goal is to learn from
their successes and their stumbles to move
00:00:26.240 --> 00:00:31.360
the science and practice of AI testing
forward. In this series, we'll explore
00:00:31.360 --> 00:00:39.268
how these insights might help guide the future of
AI development, deployment, and responsible use.
00:00:39.268 --> 00:00:39.280
[MUSIC ENDS]
00:00:39.280 --> 00:00:44.240
Today, I'm excited to welcome Ciaran Martin to the
podcast to explore testing and risk assessment in
00:00:44.240 --> 00:00:50.000
cybersecurity. Ciaran is a professor of practice
in the management of public organizations at the
00:00:50.000 --> 00:00:54.960
University of Oxford. He had previously
founded and served as chief executive of
00:00:54.960 --> 00:01:00.720
the National Cyber Security Centre within the
UK's intelligence, security, and cyber agency.
00:01:00.720 --> 00:01:05.520
And after our conversation, we'll talk to
Microsoft's Tori Westerhoff, a principal director
00:01:05.520 --> 00:01:12.080
on Microsoft’s AI Red Team, about how we should
think about these insights in the context of AI.
00:01:12.080 --> 00:01:14.785
Hi, Ciaran. Thank you so
much for being here today.
00:01:14.785 --> 00:01:17.520
CIARAN MARTIN: Well, thanks so much
for inviting me. It’s great to be here.
00:01:17.520 --> 00:01:21.440
SULLIVAN: Ciaran, before we get into some
regulatory specifics, it'd be great to hear
00:01:21.440 --> 00:01:27.200
a little bit more about your origin story, and
just take us to that day—who tapped you on the
00:01:27.200 --> 00:01:31.760
shoulder and said, “Ciaran, we need you to run a
national cyber center! Do you fancy building one?”
00:01:31.760 --> 00:01:38.160
MARTIN: You could argue that I owe my job to
Edward Snowden. Not an obvious thing to say.
00:01:38.160 --> 00:01:43.440
So the National Cyber Security Centre, which
didn't exist at the time—I was invited to join
00:01:43.440 --> 00:01:50.080
the British government's cybersecurity effort in
a leadership role—is now a subset of GCHQ. That's
00:01:50.080 --> 00:01:54.951
the digital intelligence agency. The equivalent
in the US obviously is the NSA [National Security
00:01:54.951 --> 00:02:00.560
Agency]. It had been convulsed by the Snowden
disclosures. It was an unprecedented challenge.
00:02:00.560 --> 00:02:08.240
I was a 17-year career government fixer with some
national security experience. So I was asked to go
00:02:08.240 --> 00:02:13.760
out and help with the policy response, the media
response, the legal response. But I said, look,
00:02:13.760 --> 00:02:19.040
any crisis, even one as big as this, is over
one way or the other in six months. What should
00:02:19.040 --> 00:02:24.960
I do long term? And they said, well, we were
thinking of asking you to try to help transform
00:02:24.960 --> 00:02:30.080
our cybersecurity mission. So the National Cyber
Security Centre was born, and I was very proud to
00:02:30.080 --> 00:02:37.040
lead it, and all in all, I did it for seven years
from startup to handing it on to somebody else.
00:02:37.040 --> 00:02:40.080
SULLIVAN: I mean, it's incredible.
And just building on that,
00:02:40.080 --> 00:02:44.960
people spend a significant portion of their
lives online now with a variety of devices,
00:02:44.960 --> 00:02:48.800
and maybe for listeners who are newer
to cybersecurity, could you give us the
00:02:48.800 --> 00:02:54.000
90-second lightning talk? Kind of, what does risk
assessment and testing look like in this space?
00:02:54.000 --> 00:02:58.240
MARTIN: Well, risk assessment and testing,
I think, are two different things. You can't
00:02:58.240 --> 00:03:03.440
defend everything. If you defend everything,
you're defending nothing. So broadly speaking,
00:03:03.440 --> 00:03:09.200
organizations face three threats. One is
complete disruption of their systems. So
00:03:09.200 --> 00:03:15.440
just imagine not being able to access your
system. The second is data protection, and
00:03:15.440 --> 00:03:20.400
that could be sensitive customer information. It
could be intellectual property. And the third is,
00:03:20.400 --> 00:03:25.280
of course, you could be at risk of just
straightforward being stolen from. I mean,
00:03:25.280 --> 00:03:28.320
you don't want any of them to happen, but
you have to have a hierarchy of harm.
00:03:28.320 --> 00:03:30.560
SULLIVAN: Yes.
MARTIN: So that's your risk assessment.
00:03:30.560 --> 00:03:35.200
The testing side, I think, is slightly different.
One of the paradoxes, I think, of cybersecurity is
00:03:35.200 --> 00:03:42.960
for such a scientific, data-rich subject, the sort
of metrics about what works are very, very hard
00:03:42.960 --> 00:03:51.440
to come by. So you've got boards and corporate
leadership and senior governmental structures, and
00:03:51.440 --> 00:03:56.320
they say, “Look, how do I run this organization
safely and securely?” And a cybersecurity chief
00:03:56.320 --> 00:03:59.840
within the organization will say, “Well,
we could get this capability in.” Well,
00:03:59.840 --> 00:04:04.400
the classic question for a leadership team to ask
is, well, what risk and harm will this reduce,
00:04:04.400 --> 00:04:08.880
by how much, and what's the cost-benefit
analysis? And we find that really hard.
00:04:08.880 --> 00:04:14.080
So that's really where testing and assurance
comes in. And also as technology changes so fast,
00:04:14.080 --> 00:04:18.800
we have to figure out, well, if we're worried
about post-quantum cryptography, for example,
00:04:18.800 --> 00:04:22.320
what standards does it have to meet? How do you
assess whether it's meeting those standards? So
00:04:22.320 --> 00:04:27.600
it's a huge issue in cybersecurity and one that
we're always very conscious of. It’s really hard.
00:04:27.600 --> 00:04:32.560
SULLIVAN: Given the scope of cybersecurity, are
there any differences in testing, let's say,
00:04:32.560 --> 00:04:37.280
for maybe a small business versus a critical
infrastructure operator? Are there any,
00:04:37.280 --> 00:04:42.640
sort of, metrics we can look at in terms
of distinguishing risk or assessment?
00:04:42.640 --> 00:04:48.880
MARTIN: There have to be. One of the reasons I
think why we have to be is that no small business
00:04:48.880 --> 00:04:54.320
can be expected to take on a hostile nation-state
that's well equipped. You have to be realistic.
00:04:54.320 --> 00:04:58.640
If you look at government guidance, certainly
in the UK 15 years ago on cybersecurity,
00:04:58.640 --> 00:05:02.160
you were telling small businesses that
are living hand to mouth, week by week,
00:05:02.160 --> 00:05:06.400
trying to make payments at the end of each
month, we were telling them they needed sort of
00:05:06.400 --> 00:05:10.240
nation-state-level cyber defenses. That was never
going to happen, even if they could afford it,
00:05:10.240 --> 00:05:15.120
which they couldn't. So you have to
have some differentiation. So again,
00:05:15.120 --> 00:05:19.040
you've got assessment frameworks and so forth
where you have to meet higher standards. So
00:05:19.040 --> 00:05:24.080
there absolutely has to be that distinction.
Otherwise, you end up in a crazy world of
00:05:24.080 --> 00:05:30.400
crippling small businesses with just unmanageable
requirements which they're never going to meet.
00:05:30.400 --> 00:05:33.760
SULLIVAN: It's such a great point. You
touched on this a little bit earlier, as well,
00:05:33.760 --> 00:05:39.440
but just cybersecurity governance operates in a
fast-moving technology and threat environment.
00:05:39.440 --> 00:05:44.880
How have testing standards evolved, and where
do new technical standards usually originate?
00:05:44.880 --> 00:05:47.600
MARTIN: I keep saying this is very difficult,
00:05:47.600 --> 00:05:56.320
and it is. [LAUGHTER] So I think there are two
challenges. One is actually about the balance,
00:05:56.320 --> 00:06:02.160
and this applies to the technology of today as
well as the technology of tomorrow. This is about,
00:06:02.160 --> 00:06:07.040
how do you make sure things are good enough
without crowding out new entrants? You want
00:06:07.040 --> 00:06:11.520
people to be innovative and dynamic.
You want disruptors in this business.
00:06:11.520 --> 00:06:17.600
But if you say to them, “Look, well, you have to
meet these 14 impossibly high technical standards
00:06:17.600 --> 00:06:21.600
before you can even sell to anybody or sell to
the government,” whatever, then you've got a
00:06:21.600 --> 00:06:24.320
problem. And I think we've wrestled with that,
and there's no perfect answer. You just have
00:06:24.320 --> 00:06:32.160
to try and go to … find the sweet spot between two
ends of a spectrum. And that's going to evolve.
00:06:32.160 --> 00:06:36.880
The second point, which in some respects if you've
got the right capabilities is slightly easier but
00:06:36.880 --> 00:06:44.320
still a big call, is around, you know, those newer
and evolving technologies. And here, having, you
00:06:44.320 --> 00:06:48.320
know, been a bit sort of gloomy and pessimistic,
here I think is actually an opportunity. So one
00:06:48.320 --> 00:06:53.440
of the things we always say in cybersecurity is
that the internet was built and developed without
00:06:53.440 --> 00:06:57.760
security in mind. And that was kind of true in the
’90s and the noughties, as we call them over here.
00:06:59.520 --> 00:07:03.760
But I think as you move into things like
post-quantum computing, applied use of AI,
00:07:03.760 --> 00:07:08.400
and so on, you can actually set the standards
at the beginning. And that's really good
00:07:08.400 --> 00:07:12.080
because it's saying to people that these
are the things that are going to matter in
00:07:12.080 --> 00:07:15.520
the post-quantum age. Here's the outline of
the standards you're going to have to meet;
00:07:15.520 --> 00:07:19.440
start looking at them. So there's an opportunity
actually to make technology safer by design,
00:07:19.440 --> 00:07:22.360
by getting ahead of it. And I
think that's the era we're in now.
00:07:22.360 --> 00:07:25.040
SULLIVAN: That makes a lot of
sense. Just building on that,
00:07:25.040 --> 00:07:29.440
do businesses and the public trust these
standards? And I guess, which standard do
00:07:29.440 --> 00:07:34.000
you wish the world would just adopt already,
and what's the real reason they haven't?
00:07:34.000 --> 00:07:38.560
MARTIN: Well, again, where do you start? I
mean, most members of the public quite rightly
00:07:38.560 --> 00:07:44.960
haven't heard of any of these standards.
I think public trust and public capital
00:07:44.960 --> 00:07:50.480
in any society matters. But I think it is
important that these things are credible.
00:07:50.480 --> 00:07:56.480
And there's quite a lot of convergence between,
you know, the top-level frameworks. And obviously
00:07:56.480 --> 00:07:58.800
in the US, you know, the NIST [National Institute
of Standards and Technology] framework is the one
00:07:58.800 --> 00:08:02.000
that's most popular for cybersecurity,
but it bears quite a strong resemblance
00:08:02.000 --> 00:08:07.920
to the international one, ISO[/IEC] 27001, and
there are others, as well. But fundamentally,
00:08:07.920 --> 00:08:12.320
they boil down to kind of five things.
Do a risk assessment; work out what your
00:08:12.320 --> 00:08:18.640
crown jewels are. Protect your perimeter as
best you can. Those are the first two.
00:08:18.640 --> 00:08:23.120
The third one then is when your perimeter's
breached, be able to detect it more times
00:08:23.120 --> 00:08:29.840
than not. And when you can't do that,
you go to the fourth one, which is,
00:08:29.840 --> 00:08:34.400
can you mitigate it? And when all else fails,
how quickly can you recover and manage it?
00:08:34.400 --> 00:08:38.960
I mean, all the standards are expressed
in way more technical language than that,
00:08:38.960 --> 00:08:44.640
but fundamentally, if everybody adopted those
five things and operated them in a simple way,
00:08:44.640 --> 00:08:48.720
you wouldn't eliminate the harm, but
you would reduce it quite substantially.
00:08:48.720 --> 00:08:50.560
SULLIVAN: Which policy initiatives are
00:08:50.560 --> 00:08:54.560
most promising for incentivizing
companies to undertake, you know,
00:08:54.560 --> 00:09:00.320
these cybersecurity testing parameters that you’ve
just outlined? Governments, including the UK,
00:09:00.320 --> 00:09:06.880
have used carrots and sticks, but what do
you think will actually move the needle?
00:09:06.880 --> 00:09:13.360
MARTIN: I think there are two answers to
that, and it comes back to your split between
00:09:13.360 --> 00:09:19.360
smaller businesses and critically important
businesses. In the critically important services,
00:09:19.360 --> 00:09:26.240
I think it's easier because most industries are
looking for a level playing field. In other words,
00:09:26.240 --> 00:09:30.240
they realize there have to be rules and
they want to apply them to everyone.
00:09:30.240 --> 00:09:35.360
We had a fascinating experience when I was
in government back in around 2018 where the
00:09:35.360 --> 00:09:39.040
telecom sector, they came to us and they
said, we've got a very good cooperative
00:09:39.040 --> 00:09:43.520
relationship with the British government,
but it needs to be put on a proper legal
00:09:43.520 --> 00:09:49.280
footing because you're just asking us nicely to
do expensive things. And in a regulated sector,
00:09:49.280 --> 00:09:53.440
if you actually put in some rules—and
please develop them jointly with us;
00:09:53.440 --> 00:09:57.840
that's the crucial part—then that will help
because it means that we're not going to our
00:09:57.840 --> 00:10:01.680
boards and saying, or our shareholders, and saying
that we should do this, and they're saying, “Well,
00:10:01.680 --> 00:10:05.440
do you have to do it? Are our competitors
doing it?” And if the answer to that is,
00:10:05.440 --> 00:10:09.360
yes, we have to, and, yes, our competitors
are doing it, then it tends to be OK.
00:10:09.360 --> 00:10:15.440
The harder nut to crack is the smaller business.
And I think there's a real mystery here:
00:10:15.440 --> 00:10:20.320
why has nobody cracked a really good and easy
solution for small business? We need to be
00:10:20.320 --> 00:10:25.600
careful about this because, you know, you can't
throttle small businesses with onerous regulation.
00:10:25.600 --> 00:10:31.760
At the same time, we're not brilliant, I
think, in any part of the world at using
00:10:31.760 --> 00:10:37.120
the normal corporate governance rules to try and
get people to figure out how to do cybersecurity.
00:10:37.120 --> 00:10:41.360
There are initiatives there that are not
the sort of pretty heavy stick that you
00:10:41.360 --> 00:10:45.520
might have to take to a critical function,
but they could help. But that is a hard nut
00:10:45.520 --> 00:10:49.520
to crack. And I look around the world,
and, you know, I think if this was easy,
00:10:49.520 --> 00:10:52.960
somebody would have figured it out by now.
I think most of the developed economies
00:10:52.960 --> 00:10:56.720
around the world really struggle with
cybersecurity for smaller businesses.
00:10:56.720 --> 00:11:00.800
SULLIVAN: Yeah, it's a great point. Actually
building on one of the comments you made on
00:11:00.800 --> 00:11:06.240
the role of, kind of, government, how
do you see the role of private-public
00:11:06.240 --> 00:11:10.720
partnerships scaling and strengthening,
you know, robust cybersecurity testing?
00:11:10.720 --> 00:11:14.640
MARTIN: I think they're crucial, but they
have to be practical. I've got a slight,
00:11:14.640 --> 00:11:18.320
sort of, high horse on this, if you don't
mind, Kathleen. It's sort of … [LAUGHS]
00:11:18.320 --> 00:11:18.680
SULLIVAN: Of course.
00:11:18.680 --> 00:11:22.800
MARTIN: I think that there are two
types of public-private partnership.
00:11:22.800 --> 00:11:27.360
One involves committees saying that we should
strengthen partnerships and we should all work
00:11:27.360 --> 00:11:31.360
together and collaborate and share stuff.
And we tried that for a very long time,
00:11:31.360 --> 00:11:34.240
and it didn't get us very
far. There are other types.
00:11:34.240 --> 00:11:38.720
We had some at the National Cyber Security Centre
where we paid companies to do spectacularly
00:11:38.720 --> 00:11:43.840
good technical work that the market wouldn't
provide. So I think it's sort of partnership
00:11:43.840 --> 00:11:47.840
with a purpose. I think sometimes, and I
understand the human instinct to do this,
00:11:47.840 --> 00:11:51.280
particularly in governments and big business,
they think you need to get around a table
00:11:51.280 --> 00:11:55.520
and work out some grand strategy to fix
everything, and the scale of the … not just
00:11:55.520 --> 00:11:59.200
the problem but the scale of the whole
technology is just too big to do that.
00:11:59.200 --> 00:12:04.640
So pick a bit of the problem. Find some
ways of doing it. Don't over-lawyer it.
00:12:04.640 --> 00:12:08.400
[LAUGHTER] I think sometimes people get very
nervous. Oh, well, is this our role? You know,
00:12:08.400 --> 00:12:11.360
should we be doing this, that, and the other?
Well, you know, sometimes certainly in this
00:12:11.360 --> 00:12:15.920
country, you think, well, who's actually
going to sue you over this, you know? So
00:12:16.480 --> 00:12:22.160
I wouldn't over-programmatize it. Just get
stuck practically into solving some problems.
00:12:22.160 --> 00:12:24.240
SULLIVAN: I love that.
Actually, [it] made me think,
00:12:24.240 --> 00:12:26.720
are there any surprising allies
that you've gained—you know,
00:12:26.720 --> 00:12:31.987
maybe someone who you never expected to be
a cybersecurity champion—through your work?
00:12:31.987 --> 00:12:41.440
MARTIN: Ooh! That's a … that's a… what a
question! To give you a slightly disappointing
00:12:41.440 --> 00:12:48.240
answer, but it relates to your previous
question. In the early part of my career,
00:12:48.240 --> 00:12:54.400
I was working in institutions like the UK
Treasury long before I was in cybersecurity,
00:12:54.400 --> 00:12:58.080
and the treasury and the British civil
service in general, but the treasury in
00:12:58.080 --> 00:13:04.240
particular sort of trained you to believe that
the private sector was amoral, not immoral,
00:13:04.240 --> 00:13:09.360
amoral. It just didn't have values. It just had
bottom line, and, you know, its job essentially
00:13:09.360 --> 00:13:14.720
was to provide employment and revenue then for
the government to spend on good things that people
00:13:14.720 --> 00:13:20.640
cared about. And when I got into cybersecurity and
people said, look, you need to develop relations
00:13:20.640 --> 00:13:25.280
with this cybersecurity company, often in the US,
actually. I thought, well, what's in it for them?
00:13:25.280 --> 00:13:30.160
And, sure, sometimes you were paying them for
specific services, but other times, there was a
00:13:30.160 --> 00:13:35.280
real public spiritedness about this. There was
a realization that if you tried to delineate
00:13:35.280 --> 00:13:40.160
public-private boundaries, that it wouldn't really
work. It was a shared risk. And you could analyze
00:13:40.160 --> 00:13:44.080
where the boundaries fell or you could actually
go on and do something about it together. So
00:13:44.080 --> 00:13:53.280
I was genuinely surprised at the allyship from
the cybersecurity sector. Absolutely, I really,
00:13:53.280 --> 00:13:58.520
really was. And I think it's a really positive
part of certainly the UK cybersecurity ecosystem.
00:13:58.520 --> 00:14:02.240
SULLIVAN: Wonderful. Well, we're
coming to the end of our time here,
00:14:02.240 --> 00:14:08.320
but is there any maybe last thoughts or perhaps
requests you have for our listeners today?
00:14:08.320 --> 00:14:14.640
MARTIN: I think that standards,
assurance, and testing really matter,
00:14:14.640 --> 00:14:21.360
but it's a bit like the discussion we're having
over AI. Get all these things to take you 80,
00:14:21.360 --> 00:14:28.720
90% of the way and then really apply
your judgment. There's been some bad
00:14:28.720 --> 00:14:33.680
regulation under the auspices of standards and
assurance. First of all, it’s, have you done this
00:14:33.680 --> 00:14:39.920
assessment? Have you done that? Have you looked
at this? Well, fine. And you can tick that box,
00:14:39.920 --> 00:14:45.040
but what does it actually mean when you do it?
What bits that you know in your heart of hearts
00:14:45.040 --> 00:14:50.240
are really important to the defense of your
organization that may not be covered by this
00:14:50.240 --> 00:14:56.120
and just go and do those anyway. Because
sure it helps, but it's not everything.
00:14:56.120 --> 00:14:59.680
SULLIVAN: No. Great, great
closing sentiment. Well, Ciaran,
00:14:59.680 --> 00:15:03.200
thank you for joining us today. This
has been just a super fun conversation
00:15:03.200 --> 00:15:07.520
and really insightful. Just really
enjoyed the conversation. Thank you.
00:15:07.520 --> 00:15:14.800
MARTIN: My pleasure, Kathleen, thank you.
00:15:14.800 --> 00:15:18.417
[TRANSITION MUSIC]
00:15:18.417 --> 00:15:20.320
SULLIVAN:
00:15:20.320 --> 00:15:26.080
Now, I'm happy to introduce Tori Westerhoff. As a
principal director on the Microsoft AI Red Team,
00:15:26.080 --> 00:15:30.160
Tori leads all AI security and
safety red team operations,
00:15:30.160 --> 00:15:34.880
as well as dangerous capability testing,
to directly inform C-suite decision-makers.
00:15:34.880 --> 00:15:36.863
So, Tori, welcome!
00:15:36.863 --> 00:15:39.360
TORI WESTERHOFF: Thanks. I
am so excited to be here.
00:15:39.360 --> 00:15:43.680
SULLIVAN: I'd love to just start a little bit
more learning about your background. You've
00:15:43.680 --> 00:15:48.800
worn some very intriguing hats. I mean,
cognitive neuroscience grad from Yale,
00:15:48.800 --> 00:15:54.800
national security consultant, strategist
in augmented and virtual reality … how do
00:15:54.800 --> 00:15:59.040
those experiences help shape the way
you lead the Microsoft AI Red Team?
00:15:59.040 --> 00:16:01.760
WESTERHOFF: I always joke this is the only role I
00:16:01.760 --> 00:16:08.880
think will always combine the entire
patchwork LinkedIn résumé. [LAUGHS]
00:16:08.880 --> 00:16:18.640
I think I use those experiences to help me
understand the really broad approach that
00:16:18.640 --> 00:16:28.720
AI Red Team—artist also known as AIRT; I'm sure
I'll slip into our acronym—how we frame up the
00:16:28.720 --> 00:16:35.200
broad security implications of AI. So I think the
cognitive neuroscience element really helped me
00:16:35.200 --> 00:16:43.440
initially approach AI hacking, right. There's
a lot of social engineering and manipulation
00:16:43.440 --> 00:16:51.040
within chat interfaces that are enabled by
AI. And also, kind of, this, like, metaphor
00:16:51.040 --> 00:16:59.920
for understanding how to find soft spots in the
way that you see human heuristics show up, too.
00:16:59.920 --> 00:17:08.000
And so I think that was actually my personal “in”
to getting hooked into AI red teaming generally.
00:17:08.960 --> 00:17:15.920
But my experience in national security and I'd
also say working through the AR/VR/metaverse
00:17:15.920 --> 00:17:25.200
space at the time where I was in it helped
me balance both how our impact is framed,
00:17:25.200 --> 00:17:30.720
how we're thinking about critical industries, how
we're really trying to push our understanding of
00:17:30.720 --> 00:17:38.720
where security of AI can help people the most.
And also do it in a really breakneck speed in
00:17:38.720 --> 00:17:42.800
an industry that's evolving all of the time,
that's really pushing you to always be at the
00:17:42.800 --> 00:17:50.080
bleeding edge of your understanding. So I draw
a lot of the energy and the mission criticality
00:17:50.080 --> 00:17:54.480
and the speed from those experiences
as we're shaping up how we approach it.
00:17:54.480 --> 00:17:58.960
SULLIVAN: Can you just give us a quick rundown?
What does the Red Team do? What actually,
00:17:58.960 --> 00:18:02.720
kind of, is involved on a day-to-day
basis? And then as we think about,
00:18:02.720 --> 00:18:05.840
you know, our engagements with
large enterprises and companies,
00:18:05.840 --> 00:18:10.480
how do we work alongside some of
those companies in terms of testing?
00:18:10.480 --> 00:18:19.200
WESTERHOFF: The way I see our team is almost
like an indicator light that works really part
00:18:19.200 --> 00:18:25.200
and parcel with product development. So the
way we've organized our expert red teaming
00:18:25.200 --> 00:18:30.400
efforts is that we work with product
development before anything ships out
00:18:30.400 --> 00:18:36.240
to anyone who can use it. And our job
is to act as expert AI manipulators,
00:18:36.240 --> 00:18:43.520
AI hackers. And we are supposed to take the
theories and methods and new research and
00:18:43.520 --> 00:18:50.400
harness it to find examples of vulnerabilities
or soft spots in products to enable product
00:18:50.400 --> 00:18:55.760
teams to harden those soft spots before anything
actually reaches someone who wants to use it.
00:18:55.760 --> 00:19:00.320
So if we're the indicator light,
we are also not the full workup,
00:19:00.320 --> 00:19:04.800
right. I see that as measurement and
evals. And we also are not the mechanic,
00:19:04.800 --> 00:19:10.080
which is that product development team that's
creating mitigations. It's platform-security
00:19:10.080 --> 00:19:16.400
folks who are creating mitigations at scale. And
there's a really great throughput of insights from
00:19:16.400 --> 00:19:24.720
those groups back into our area where we love to
inform about them, but we also love to add on to,
00:19:24.720 --> 00:19:28.560
how do we break the next thing,
right? So it's a continuous cycle.
00:19:30.160 --> 00:19:37.600
And part of that is just being really creative and
thinking outside of a traditional cybersecurity
00:19:37.600 --> 00:19:44.000
box. And part of that is also really thinking
about how we pull in research—we have a research
00:19:44.000 --> 00:19:49.040
function within our AI Red Team—and
how we automate and scale. This year,
00:19:49.040 --> 00:19:54.160
we've pulled a lot of those assets and
insights into the Azure [AI] Foundry AI
00:19:54.160 --> 00:19:59.120
Red Teaming Agent. And so folks can
now access a lot of our mechanisms
00:19:59.680 --> 00:20:05.560
through that. So you can get a little taste of
what we do day to day in the AI Red Teaming Agent.
00:20:05.560 --> 00:20:10.400
SULLIVAN: You recently—actually, with your
team—published a report that outlined lessons
00:20:10.400 --> 00:20:15.920
from testing over a hundred generative
AI products. But could you share a bit
00:20:15.920 --> 00:20:19.360
about what you learned? What were some of
the important lessons? Where do you see
00:20:19.360 --> 00:20:25.320
opportunities to improve the state of red
teaming as a method for probing AI safety?
00:20:25.320 --> 00:20:33.360
WESTERHOFF: I think the most important
takeaway from those lessons is that AI
00:20:33.360 --> 00:20:40.960
security is truly a team sport. You'll
hear cybersecurity folks say that a lot.
00:20:40.960 --> 00:20:50.480
And part of the rationale there is that the
defense in depth and integrating and a view
00:20:50.480 --> 00:20:57.040
towards AI security through the entire
development of AI systems is really the
00:20:57.040 --> 00:21:03.360
way that we're going to approach this
with intentionality and responsibility.
00:21:03.360 --> 00:21:10.160
So in our space, we really focus on novel
harm categories. We are pushing bleeding edge,
00:21:10.160 --> 00:21:17.840
and we also are pushing iterative and, like,
contextually based red teaming in product
00:21:17.840 --> 00:21:26.080
dev. So outside of those hundred that we've done,
there's a community [LAUGHS] through the entire,
00:21:26.080 --> 00:21:32.640
again, multistage life cycle of a product that
is really trying to push the cost of attacking
00:21:32.640 --> 00:21:38.960
those AI systems higher and higher with all
of the expertise they bring. So we may be,
00:21:38.960 --> 00:21:44.720
like, the experts in AI hacking in that
line, but there are also so many partners
00:21:44.720 --> 00:21:50.240
in the Microsoft ecosystem who are thinking
about their market context or they really,
00:21:50.240 --> 00:21:53.920
really know the people who love their
products. How are they using it?
00:21:53.920 --> 00:21:57.760
And then when you bubble out, you also
have industry and government who are
00:21:57.760 --> 00:22:06.000
working together to push towards the most
secure AI implementation for people, right?
00:22:06.000 --> 00:22:12.480
And I think our team in particular, we feel
really grateful to be part of the big AI safety
00:22:12.480 --> 00:22:17.880
and security ecosystem at Microsoft and also to
be able to contribute to the industry writ large.
00:22:17.880 --> 00:22:21.440
SULLIVAN: As you know, we had a chance
to speak with Professor Ciaran Martin
00:22:21.440 --> 00:22:25.200
from the University of Oxford about the
cybersecurity industry and governance
00:22:25.200 --> 00:22:29.440
there. What are some of the ideas and
tools from that space that are surfacing
00:22:29.440 --> 00:22:33.360
in how we think about approaching red
teaming and AI governance broadly?
00:22:33.360 --> 00:22:41.200
WESTERHOFF: Yeah, I think it's such a
broad set of perspectives to bring in,
00:22:41.200 --> 00:22:50.160
in the AI instance. Something that I've noticed
interjecting into security at the AI junction,
00:22:50.160 --> 00:22:56.240
right, is that cybersecurity has so many
decades of experience of working through
00:22:56.240 --> 00:23:03.520
how to build trustworthy computing, for example,
or bring an entire industry to bear in that way.
00:23:03.520 --> 00:23:11.360
And I think that AI security and safety can learn
a lot of lessons of how to bring clarity and
00:23:11.360 --> 00:23:17.920
transparency across the industry to push universal
understanding of where the threats really are.
00:23:19.040 --> 00:23:26.320
So frameworks coming out of NIST, coming out of
MITRE that help us have a universal language that
00:23:26.320 --> 00:23:32.720
inform governance, I think, are really important
because it brings clarity irrespective of where
00:23:32.720 --> 00:23:39.520
you are looking into AI security, irrespective
of your company size, what you're working on. It
00:23:39.520 --> 00:23:45.440
means you all understand, “Hey, we are really
worried about this fundamental impact.” And
00:23:45.440 --> 00:23:52.160
I think cybersecurity has done a really good job
of driving towards impact as their organizational
00:23:52.160 --> 00:23:58.960
vector. And I am starting to see that in the AI
space, too, where we're trying to really clarify
00:23:58.960 --> 00:24:03.360
terms and threats. And you see it in updates of
those frameworks, as well, that I really love.
00:24:03.360 --> 00:24:08.560
So I think that the innovation is in
transparency to folks who are really
00:24:08.560 --> 00:24:12.720
innovating and doing the work so
we all have a shared language,
00:24:12.720 --> 00:24:20.480
and from that, it really creates communal
goals across security instead of a lot of
00:24:20.480 --> 00:24:23.680
people being worried about the same thing
and talking about it in a different way.
00:24:23.680 --> 00:24:28.800
SULLIVAN: Mm-hmm. In the cybersecurity context,
Ciaran really stressed matching risk frameworks
00:24:28.800 --> 00:24:34.720
to an organization's role and scale. Microsoft
plays many roles, including building models
00:24:34.720 --> 00:24:40.600
and shipping applications. How does your red
teaming approach shift across those layers?
00:24:40.600 --> 00:24:46.320
WESTERHOFF: I love this question also
because I love it as part of our work.
00:24:46.320 --> 00:24:51.120
So one of the most fascinating things
about working on this team has been the
00:24:51.120 --> 00:24:56.560
diversity of the technology that we end
up red teaming and testing. And it feels
00:24:56.560 --> 00:25:01.440
like we're in the crucible in that
way. Because we see AI applied to so
00:25:01.440 --> 00:25:10.640
many different architectures, tech stacks,
individual features, models, you name it.
00:25:10.640 --> 00:25:16.320
Part of my answer is that we still care
about the highest-impact things. And so
00:25:16.320 --> 00:25:21.440
irrespective of the iteration, which
is really fascinating and I love,
00:25:23.200 --> 00:25:29.120
I still think that our team drives to say, “OK,
what is that critical vulnerability that is
00:25:29.120 --> 00:25:36.000
going to affect people in the largest ways, and
can we battle test to see if that can occur?”
00:25:36.000 --> 00:25:44.720
So in some ways, the task is always the same.
I think in the ways that we change our testing,
00:25:44.720 --> 00:25:55.440
we customize a lot to the access
to systems and data and also
00:25:57.280 --> 00:26:05.760
people's trust almost as different variables
that could affect the impact, right.
00:26:05.760 --> 00:26:10.640
So a good example is if we're thinking
through agentic frameworks that have
00:26:10.640 --> 00:26:17.200
access to functions and tools and
preferential ability to act on data,
00:26:17.200 --> 00:26:22.880
it's really different to spaces where
that action may not be feasible,
00:26:22.880 --> 00:26:31.040
right. And so I think the tailoring of the way
to get to that impact is hyper-custom every time
00:26:31.040 --> 00:26:39.360
we start an engagement. And part of it is very
thesis driven and almost mechanizing empathy.
00:26:39.360 --> 00:26:46.000
You almost need to really focus on how people
could use, or misuse, in such a way that you
00:26:46.000 --> 00:26:53.520
can emulate it before to a really great signal
to product development, to say this is truly
00:26:53.520 --> 00:26:59.760
what people could do and we want to deliver
the highest-impact scenarios so you can solve
00:26:59.760 --> 00:27:06.000
for those and also solve the underlying patterns,
actually, that could contribute to maybe that one
00:27:06.000 --> 00:27:12.080
piece of evidence but also all the related pieces
of evidence. So singular drive but like hyper-,
00:27:12.080 --> 00:27:17.480
hyper-customization to what that piece
of tech could do and has access to.
00:27:17.480 --> 00:27:21.040
SULLIVAN: What are some of the unexplored
testing approaches or considerations from
00:27:21.040 --> 00:27:24.960
cybersecurity that you think we
should encourage AI technologists,
00:27:24.960 --> 00:27:28.000
policymakers, and other stakeholders to focus on?
00:27:28.000 --> 00:27:34.400
WESTERHOFF: I do love that AI humbles us
each and every day with new capabilities
00:27:34.400 --> 00:27:38.640
and the potential for new capabilities. It's
not just saying, “Hey, there's one test that
00:27:38.640 --> 00:27:43.520
we want to try,” but more, “Hey, can we
create a methodology that we feel really,
00:27:43.520 --> 00:27:48.640
really solid about so that when we are
asked a question we haven't even thought of,
00:27:48.640 --> 00:27:53.440
we feel confident that we have
the resources and the system?”
00:27:53.440 --> 00:28:01.360
So part of me is really intrigued by the
process that we're asked to make without
00:28:01.360 --> 00:28:07.440
knowing what those capabilities are really
going to bring. And then I think tactically,
00:28:07.440 --> 00:28:14.560
AIRT is really pushing on how we create new
research methodologies. How are we investing in,
00:28:14.560 --> 00:28:20.080
kind of, these longer-term iterations
of red teaming? So we're really excited
00:28:20.080 --> 00:28:27.840
about pushing out those insights in
an experimental and longer-term way.
00:28:28.480 --> 00:28:34.320
I think another element is a little bit of
that evolution of how industry standards
00:28:34.320 --> 00:28:43.760
and frameworks are updating to the AI moment and
really articulating where AI is either furthering
00:28:43.760 --> 00:28:55.600
adversarial ability to create those harms or
threats or identifying where AI has a net new
00:28:55.600 --> 00:29:01.040
harm. And I think that demystifies a little
bit about what we talked about in terms of the
00:29:01.040 --> 00:29:07.200
lessons learned, that fundamentally, a lot of the
things that we talk about are traditional security
00:29:07.200 --> 00:29:16.320
vulnerabilities, and we are standing on kind of
that cybersecurity shoulder. And I'm starting
00:29:16.320 --> 00:29:25.840
to see those updates translate in spaces that
are already considered trustworthy and kind of
00:29:27.440 --> 00:29:32.720
the basis on which not only
cybersecurity folks build their
00:29:32.720 --> 00:29:39.920
work but also business decision-makers
make decisions on those frameworks.
00:29:39.920 --> 00:29:44.640
So to me, integration of AI into
those frameworks by those same
00:29:44.640 --> 00:29:50.800
standards means that we're evolving
security to include AI. We aren't
00:29:50.800 --> 00:29:55.440
creating an entirely new industry
of AI security and that, I think,
00:29:55.440 --> 00:30:02.000
really helps anchor people in the really solid
foundation that we have in cybersecurity anyways.
00:30:02.000 --> 00:30:14.560
I think there's also some work around how
the cyber, like, defenses will actually
00:30:14.560 --> 00:30:21.040
benefit from AI. So we think a lot about
threats because that's our job. But the
00:30:21.040 --> 00:30:27.760
other side of cybersecurity is offense.
And I'm seeing a ton of people come out
00:30:27.760 --> 00:30:31.680
with frameworks and methodologies,
especially in the research space,
00:30:31.680 --> 00:30:37.840
on how defensive networks are going to be
benefited from things like agentic systems.
00:30:38.400 --> 00:30:45.520
Generally speaking, I think the best practice
is to realize that we're fundamentally still
00:30:45.520 --> 00:30:52.400
talking about the same impacts, and we can use
the same avenues, conversations, and frameworks.
00:30:52.400 --> 00:31:00.280
We just really want them to be crisply updated
with that understanding of AI applications.
00:31:00.280 --> 00:31:04.240
SULLIVAN: How do you think about
bringing others into the fold there?
00:31:04.240 --> 00:31:09.600
I think those standards and frameworks are often
informed by technologists. But I'd love for you
00:31:09.600 --> 00:31:15.200
to expand [that to] policymakers or other
kind of stakeholders in our ecosystem, even,
00:31:15.200 --> 00:31:19.680
you know, end consumers of these products.
Like, how do we communicate some of this
00:31:19.680 --> 00:31:24.520
to them in a way that resonates
and it has an impactful meaning?
00:31:24.520 --> 00:31:32.240
WESTERHOFF: I've found the AI security-safety
space to be one of the more collaborative. I
00:31:32.240 --> 00:31:36.560
actually think the fact that I'm talking to
you today is probably evidence that a ton
00:31:36.560 --> 00:31:42.000
of people are bringing in perspectives
that don't only come from a long-term
00:31:42.000 --> 00:31:50.400
cybersecurity view. And I see that as a trend
in how AI is being approached opposed to how
00:31:50.400 --> 00:31:57.120
those areas were moving earlier. So I think
that speed and the idea of conversations and
00:31:57.120 --> 00:32:01.760
not always having the perfect answer but
really trying to be transparent with what
00:32:01.760 --> 00:32:09.520
everyone does know is kind of a communal
energy in the communities, at least,
00:32:09.520 --> 00:32:14.200
where we're playing. [LAUGHS] So I am pretty
biased but at least the spaces where we are.
00:32:14.200 --> 00:32:16.160
SULLIVAN: No, I think we're seeing
that across the board. I mean,
00:32:16.160 --> 00:32:20.720
I'd echo [that] sitting in research,
as well, like, that ability to have
00:32:20.720 --> 00:32:26.320
impact now and at speed to getting the
amazing technology and models that we're
00:32:26.320 --> 00:32:31.360
creating into the hands of our customers and
partners and ecosystem is just underscored.
00:32:32.240 --> 00:32:37.520
So on the note of speed, let's shift gears a
little bit to just a quick lightning round. I'd
00:32:37.520 --> 00:32:43.120
love to get maybe some quick thoughts from you,
just 30-second answers here. I'll start with one.
00:32:43.120 --> 00:32:48.040
Which headline-grabbing AI threat
do you think is mostly hot air?
00:32:48.040 --> 00:32:56.400
WESTERHOFF: I think we should pay attention
to it all. I'm a red team lead. I love a good
00:32:56.400 --> 00:33:05.040
question to see if we can find an answer in
real life. So no hot air, just questions.
00:33:05.040 --> 00:33:09.280
SULLIVAN: Is there some sort of maybe new tool
00:33:09.280 --> 00:33:13.560
that you can't wait to sneak
into the red team arsenal?
00:33:13.560 --> 00:33:20.240
WESTERHOFF: I think there are really
interesting methodologies that break
00:33:20.240 --> 00:33:26.000
our understanding of cybersecurity by
looking at the intersection between
00:33:26.000 --> 00:33:33.920
different layers of AI and how you
can manipulate AI-to-AI interaction,
00:33:33.920 --> 00:33:41.480
especially now when we're looking at agentic
systems. So I would say a method, not a tool.
00:33:41.480 --> 00:33:44.080
SULLIVAN: So maybe ending on a
little bit of a lighter note,
00:33:44.080 --> 00:33:47.280
do you have a go-to snack during
an all-night red teaming session?
00:33:47.280 --> 00:33:52.160
WESTERHOFF: Always coffee. I would
love it to be a protein smoothie,
00:33:52.160 --> 00:33:57.280
but honestly, it is probably
Trader Joe's elote chips. Like
00:33:57.280 --> 00:34:03.983
the whole bag. [LAUGHTER] It’s going to get me
through. I'm going to not love that I did it.
00:34:03.983 --> 00:34:03.990
[MUSIC]
00:34:03.990 --> 00:34:07.040
SULLIVAN: Amazing. Well, Tori,
thanks so much for joining us today,
00:34:07.040 --> 00:34:10.800
and just a huge thanks also to
Ciaran for his insights, as well.
00:34:10.800 --> 00:34:14.040
WESTERHOFF: Thank you so much
for having me. This was a joy.
00:34:14.040 --> 00:34:18.800
SULLIVAN: And to our listeners, thanks
for tuning in. You can find resources
00:34:18.800 --> 00:34:22.240
related to this podcast in the
show notes. And if you want to
00:34:22.240 --> 00:34:29.680
learn more about how Microsoft approaches AI
governance, you can visit microsoft.com/RAI.
00:34:29.680 --> 00:34:35.360
See you next time!
00:34:35.360 --> 00:34:36.201
[MUSIC FADES]
