WEBVTT
Kind: captions
Language: en-US
00:00:02.404 --> 00:00:03.106
[MUSIC]
00:00:03.106 --> 00:00:06.800
KATHLEEN SULLIVAN: Welcome
to AI Testing and Evaluation:
00:00:06.800 --> 00:00:11.520
Learnings from Science and Industry.
I’m your host, Kathleen Sullivan.
00:00:11.520 --> 00:00:16.640
As generative AI continues to advance, Microsoft
has gathered a range of experts—from genome
00:00:16.640 --> 00:00:21.360
editing to cybersecurity—to share how
their fields approach evaluation and risk
00:00:21.360 --> 00:00:26.000
assessment. Our goal is to learn from
their successes and their stumbles to
00:00:26.000 --> 00:00:31.520
move the science and practice of AI testing
forward. In this series, we’ll explore how
00:00:31.520 --> 00:00:40.304
these insights might help guide the future of AI
development, deployment, and responsible use.
00:00:40.293 --> 00:00:40.892
[MUSIC ENDS]
00:00:40.892 --> 00:00:45.760
For our final episode of the series, I’m thrilled
to once again be joined by Amanda Craig Deckard,
00:00:45.760 --> 00:00:50.240
senior director of public policy in
Microsoft’s Office of Responsible AI.
00:00:50.240 --> 00:00:52.538
Amanda, welcome back to the podcast!
00:00:52.538 --> 00:00:53.360
AMANDA CRAIG DECKARD: Thank you so much.
00:00:53.360 --> 00:00:57.840
SULLIVAN: In our intro episode, you really
helped set the stage for this series. And
00:00:57.840 --> 00:01:03.280
it’s been great, because since then, we’ve had
the pleasure of speaking with governance experts
00:01:03.280 --> 00:01:10.400
about genome editing, pharma, medical devices,
cybersecurity, and we’ve also gotten to spend
00:01:10.400 --> 00:01:15.840
some time with our own Microsoft responsible
AI leaders and hear reflections from them.
00:01:15.840 --> 00:01:20.000
And here’s what stuck with me, and I’d
love to hear from you on this, as well:
00:01:20.000 --> 00:01:26.800
testing builds trust; context is shaping risk;
and every field is really thinking about striking
00:01:26.800 --> 00:01:32.080
its own balance between pre-deployment
testing and post-deployment monitoring.
00:01:32.080 --> 00:01:37.280
So drawing on what you’ve learned from
the workshop and the case studies,
00:01:37.280 --> 00:01:42.625
what headline insights do you think
matter the most for AI governance?
00:01:42.625 --> 00:01:46.240
CRAIG DECKARD: It's been really interesting
to learn from all of these different domains,
00:01:46.240 --> 00:01:50.560
and there are, you know, lots of
really interesting takeaways.
00:01:50.560 --> 00:01:56.800
I think a starting point for me is actually pretty
similar to where you landed, which is just that
00:01:56.800 --> 00:02:05.040
testing is really important for trust, and it's
also really hard [LAUGHS] to figure out exactly,
00:02:05.040 --> 00:02:09.280
you know, how to get it right, how to
make sure that you're addressing risks,
00:02:09.280 --> 00:02:16.560
that you're not constraining innovation, that you
are recognizing that a lot of the industry that's
00:02:16.560 --> 00:02:20.960
impacted is really different. You have small
organizations, you have large organizations,
00:02:20.960 --> 00:02:25.440
and you want to enable that opportunity that is
enabled by the technology across the board.
00:02:25.440 --> 00:02:30.320
And so it's just difficult to, kind
of, get all of these dynamics right,
00:02:30.320 --> 00:02:34.960
especially when, you know, I think we heard
from other domains, testing is not some,
00:02:34.960 --> 00:02:38.960
sort of, like, oh, simple thing, right.
There's not this linear path from, like,
00:02:38.960 --> 00:02:41.360
A to B where you just test the
one thing and you're done.
00:02:41.360 --> 00:02:41.985
SULLIVAN: Right.
00:02:41.985 --> 00:02:47.280
CRAIG DECKARD: It's complex, right. Testing
is multistage. There's a lot of testing by
00:02:47.280 --> 00:02:53.680
different actors. There are a lot of different
purposes for which you might test. As I think
00:02:53.680 --> 00:02:58.640
it was Dan Carpenter who talked about it's not
just about testing for safety. It's also about
00:02:58.640 --> 00:03:04.640
testing for efficacy and building confidence
in the right dosage for pharmaceuticals,
00:03:04.640 --> 00:03:07.920
for example. And that's across the
board for all of these domains,
00:03:07.920 --> 00:03:12.080
right. That you're really thinking about
the performance of the technology. You're
00:03:12.080 --> 00:03:16.960
thinking about safety. You're trying
to also calibrate for efficiency.
00:03:16.960 --> 00:03:23.120
And so those tradeoffs, every expert
shared that navigating those is really
00:03:23.120 --> 00:03:29.920
challenging. And also that there were real
impacts to early choices in the, sort of,
00:03:29.920 --> 00:03:34.880
governance of risk in these different domains
and the development of the testing, sort of,
00:03:34.880 --> 00:03:38.160
expectations, and that in some cases,
this had been difficult to reverse,
00:03:38.160 --> 00:03:42.480
which also just layers on that complexity
and that difficulty in a different way.
00:03:42.480 --> 00:03:47.760
So that’s the super high-level takeaway. But
maybe if I could just quickly distill, like,
00:03:47.760 --> 00:03:54.080
three takeaways that I think really are applicable
to AI in a bit more of a granular way.
00:03:54.080 --> 00:04:02.080
You know, one is about, how is the testing
exactly used? For what purpose? And the
00:04:02.080 --> 00:04:07.520
second is what emphasis there is on this
pre- versus post-deployment testing and
00:04:07.520 --> 00:04:13.600
monitoring. And then the third is how
rigid versus adaptive the, sort of,
00:04:13.600 --> 00:04:17.520
testing regimes or frameworks are
in these different domains.
00:04:17.520 --> 00:04:24.720
So on the first—how is testing used?—so is
testing something that impacts market entry,
00:04:24.720 --> 00:04:32.160
for example? Or is it something that might be used
more for informing how risk is evolving in the
00:04:32.160 --> 00:04:38.720
domain and how broader risk management strategies
might need to be applied? We have examples,
00:04:38.720 --> 00:04:43.920
like the pharmaceutical or medical device industry
experts with whom you spoke, that's really,
00:04:43.920 --> 00:04:49.280
you know, testing … there is a pre-deployment
requirement. So that's one question.
00:04:49.280 --> 00:04:56.560
The second is this emphasis on pre- versus
post-deployment testing and monitoring,
00:04:56.560 --> 00:05:02.400
and we really did see across domains
that in many cases, there is a desire
00:05:02.400 --> 00:05:06.160
for both pre- and post-deployment,
sort of, testing and monitoring,
00:05:06.160 --> 00:05:12.560
but also that, sort of, naturally in these
different domains, a degree of emphasis on
00:05:12.560 --> 00:05:19.040
one or the other had evolved and that had a
real impact on governance and tradeoffs.
00:05:19.040 --> 00:05:24.240
And the third is just how rigid versus
adaptive these testing and evaluation
00:05:24.240 --> 00:05:29.200
regimes or frameworks are in these different
domains. We saw, you know, in some domains,
00:05:29.200 --> 00:05:37.200
the testing requirements were more rigid as
you might expect in more of the pharmaceutical
00:05:37.200 --> 00:05:43.440
or medical devices industries, for example. And
in other domains, there was this more, sort of,
00:05:43.440 --> 00:05:50.480
adaptive approach to how testing might get
used. So, for example, in the case of our
00:05:50.480 --> 00:05:56.240
other general-purpose technologies, you know,
you spoke with Alta Charo on genome editing,
00:05:56.240 --> 00:06:01.920
and in our case studies, we also explored
this in the context of nanotechnology. In
00:06:01.920 --> 00:06:09.840
those general-purpose technology domains, there is
more emphasis on downstream or application-context
00:06:09.840 --> 00:06:17.680
testing that is more, sort of, adaptive to the use
scenario of the technology and, you know, having
00:06:17.680 --> 00:06:23.200
that work in conjunction with testing more at
the, kind of, level of the technology itself.
00:06:23.200 --> 00:06:26.560
SULLIVAN: I want to double-click on
a number of the things we just talked
00:06:26.560 --> 00:06:29.840
about. But actually, before we go too much deeper,
00:06:29.840 --> 00:06:34.000
a question on if there's anything that really
surprised you or challenged maybe some of your
00:06:34.000 --> 00:06:39.665
own assumptions in this space from some of the
discussions that we had over the series.
00:06:39.665 --> 00:06:43.920
CRAIG DECKARD: Yeah. You know, I know I've already
just mentioned this pre- versus post-deployment
00:06:43.920 --> 00:06:50.160
testing and monitoring issue, but it was
something that was very interesting to me and
00:06:50.160 --> 00:06:56.960
in some ways surprised me or made me just realize
something that I hadn't fully connected before,
00:06:56.960 --> 00:07:03.440
about how these, sort of, regimes might evolve
in different contexts and why. And in part,
00:07:03.440 --> 00:07:10.240
I couldn't help but bring the context I have
from cybersecurity policy into this, kind of,
00:07:10.240 --> 00:07:14.560
processing of what we learned and
reflection because there was a real
00:07:14.560 --> 00:07:20.080
contrast for me between the pharmaceutical
industry and the cybersecurity domain when
00:07:20.080 --> 00:07:24.240
I think about the emphasis on pre-
versus post-deployment monitoring.
00:07:24.240 --> 00:07:31.680
And on the one hand, we have in the pharmaceutical
domain a real emphasis that has developed around
00:07:31.680 --> 00:07:38.160
pre-market testing. And there is also an
expectation in some circumstances in the
00:07:38.160 --> 00:07:43.280
pharmaceutical domain for post-deployment
testing, as well. But as we learned from
00:07:43.280 --> 00:07:51.280
our experts in that domain, there has naturally
been a real, kind of, emphasis on the pre-market
00:07:51.280 --> 00:07:58.240
portion of that testing. And in reality, even
where post-market monitoring is required and
00:07:58.240 --> 00:08:04.400
post-market testing is required, it does not
always actually happen. And the experts really
00:08:04.400 --> 00:08:09.440
explained that, you know, part of it is just the
incentive structure around the emphasis around,
00:08:09.440 --> 00:08:16.160
you know, the testing as a pre-market, sort of,
entry requirement. And also just the resources
00:08:16.160 --> 00:08:22.400
that exist among regulators, right. There's
limited resources, right. And so there are just
00:08:22.400 --> 00:08:26.720
choices and tradeoffs that they need to make
in their own, sort of, enforcement work.
00:08:26.720 --> 00:08:34.320
And then on the other hand, you know, in
cybersecurity, I never thought about the, kind of,
00:08:34.320 --> 00:08:40.560
emphasis on things like coordinated vulnerability
disclosure and bug bounties that have really
00:08:40.560 --> 00:08:47.280
developed in the cybersecurity domain. But it's a
really important part of how we secure technology
00:08:47.280 --> 00:08:53.600
and enhance cybersecurity over time, where we have
these norms that have developed where, you know,
00:08:53.600 --> 00:08:58.960
security researchers are doing really important
research. They're finding vulnerabilities in
00:08:58.960 --> 00:09:03.920
products. And we have norms developed where
they report those to the companies that are in a
00:09:03.920 --> 00:09:09.280
position to address those vulnerabilities. And in
some cases, those companies actually pay, through
00:09:09.280 --> 00:09:15.520
bug bounties, the researchers. And perhaps in
some ways, the role of coordinated vulnerability
00:09:15.520 --> 00:09:20.400
disclosure and bug bounties has evolved the way
that it has because there hasn't been as much
00:09:20.400 --> 00:09:26.720
emphasis on the pre-market testing across the
board at least in the context of software.
00:09:26.720 --> 00:09:32.080
And so you look at those two industries and it
was interesting to me to study them to some extent
00:09:32.080 --> 00:09:39.200
in contrast with each other as this way that
the incentives and the resources that need to
00:09:39.200 --> 00:09:45.680
be applied to testing, sort of, evolve to address
where there's, kind of, more or less emphasis.
00:09:45.680 --> 00:09:49.120
SULLIVAN: It's a great point. I mean, I
think what we're hearing—and what you're
00:09:49.120 --> 00:09:52.400
saying—is just exactly this choice …
like, is there a binary choice between
00:09:52.400 --> 00:09:57.120
focusing on pre-deployment testing or
post-deployment monitoring? And, you know,
00:09:57.120 --> 00:10:01.520
I think our assumption is that we need to do
both. But I'd love to hear from you on that.
00:10:01.520 --> 00:10:08.320
CRAIG DECKARD: Absolutely. I think we need to
do both. I'm very persuaded by this inclination
00:10:08.320 --> 00:10:14.560
always that there's value in trying to really
do it all in a risk management context.
00:10:14.560 --> 00:10:20.640
And also, we know one of the principles of risk
management is you have to prioritize because there
00:10:20.640 --> 00:10:27.120
are finite resources. And I think that's where we
get to this challenge in really thinking deeply,
00:10:27.120 --> 00:10:32.240
especially as we're in the early days of AI
governance, and we need to be very thoughtful
00:10:32.240 --> 00:10:37.680
about, you know, tradeoffs that we may not
want to be making but we are because, again,
00:10:37.680 --> 00:10:42.480
these are finite choices and we, kind of,
can't help but put our finger on the dial in
00:10:42.480 --> 00:10:47.280
different directions with our choices that, you
know, it's going to be very difficult to have,
00:10:47.280 --> 00:10:52.000
sort of, equal emphasis on both. And we
need to invest in both, but we need to be
00:10:52.000 --> 00:10:59.840
very deliberate about the roles of each and how
they complement each other and who does which
00:10:59.840 --> 00:11:05.280
and how we use what we learn from pre- versus
post-deployment testing and monitoring.
00:11:05.280 --> 00:11:09.280
SULLIVAN: Maybe just spending a little bit more
time here … you know, a lot of attention goes
00:11:09.280 --> 00:11:14.400
into testing models upstream, but risk often
shows up once they're wired into real products
00:11:14.400 --> 00:11:20.400
and workflows. How much does deployment context
change the risk picture from your perspective?
00:11:20.400 --> 00:11:24.640
CRAIG DECKARD: Yeah, I … such an
important question. I really agree that
00:11:24.640 --> 00:11:29.920
there has been a lot of emphasis to date
on, sort of, testing models upstream,
00:11:29.920 --> 00:11:39.280
the AI model evaluation. And it's also really
important that we bring more attention into
00:11:39.280 --> 00:11:46.960
evaluation at the system or application level. And
I actually see that in governance conversations,
00:11:46.960 --> 00:11:54.640
this is actually increasingly raised, this need
to have system-level evaluation. We see this
00:11:54.640 --> 00:12:00.480
across regulation. We also see it in the context
of just organizations trying to put in governance
00:12:00.480 --> 00:12:04.640
requirements for how their organization is going
to operate in deploying this technology.
00:12:04.640 --> 00:12:11.040
And there's a gap today in terms of best
practices around system-level testing,
00:12:11.040 --> 00:12:19.280
perhaps even more than model-level evaluation. And
it's really important because in a lot of cases,
00:12:19.280 --> 00:12:25.680
the deployment context really does impact
the risk picture, especially with AI,
00:12:25.680 --> 00:12:30.560
which is a general-purpose technology,
and we really saw this in our study of
00:12:30.560 --> 00:12:34.320
other domains that represented
general-purpose technology.
00:12:34.320 --> 00:12:40.640
So in the case study that you can find online
on nanotechnology, you know, there's a real
00:12:40.640 --> 00:12:48.640
distinction between the risk evaluation and
the governance of nanotechnology in different
00:12:48.640 --> 00:12:55.840
deployment contexts. So the chapter that our
expert on nanotechnology wrote really goes into
00:12:55.840 --> 00:13:01.200
incredibly interesting detail around, you know,
deployment of nanotechnology in the context of,
00:13:01.200 --> 00:13:07.440
like, chemical applications versus consumer
electronics versus pharmaceuticals versus
00:13:07.440 --> 00:13:13.280
construction and how the way that nanoparticles
are basically delivered in all those different
00:13:13.280 --> 00:13:19.520
deployment contexts, as well as, like, what
the risk of the actual use scenario is just
00:13:19.520 --> 00:13:25.760
varies so much. And so there's a real need to
do that kind of risk evaluation and testing in
00:13:25.760 --> 00:13:30.160
the deployment context, and this difference
in terms of risks and what we learned in
00:13:30.160 --> 00:13:36.720
these other domains where, you know, there are
these different approaches to trying to really
00:13:36.720 --> 00:13:41.600
think about and gain efficiencies and address
risks at a horizontal level versus, you know,
00:13:41.600 --> 00:13:47.760
taking a real sector-by-sector approach. And
to some extent, it seems like it's more time
00:13:47.760 --> 00:13:54.080
intensive to do that sectoral deployment-specific
work. And at the same time, perhaps there are
00:13:54.080 --> 00:14:00.320
efficiencies to be gained by actually doing
the work in the context in which, you know, you
00:14:00.320 --> 00:14:07.040
have a better understanding of the risk that can
result from really deploying this technology.
00:14:07.040 --> 00:14:12.560
And ultimately, [LAUGHS] really what we also
need to think about here is probably, in the end,
00:14:12.560 --> 00:14:17.200
just like pre- and post-deployment testing,
you need both. Not probably; certainly!
00:14:17.200 --> 00:14:24.080
So effectively we need to think about evaluation
at the model level and the system level as being
00:14:24.080 --> 00:14:29.360
really important. And it's really important
to get system evaluation right so that we can
00:14:29.360 --> 00:14:36.160
actually get trust in this technology in
deployment context so we enable adoption
00:14:37.040 --> 00:14:43.840
in low- and in high-risk deployments in a way that
means that we've done risk evaluation in each of
00:14:43.840 --> 00:14:49.520
those contexts in a way that really makes sense in
terms of the resources that we need to apply and
00:14:49.520 --> 00:14:56.760
ultimately we are able to unlock more applications
of this technology in a risk-informed way.
00:14:56.760 --> 00:15:01.840
SULLIVAN: That's great. I mean, I couldn't agree
more. I think these contexts, the approaches are
00:15:01.840 --> 00:15:07.600
so important for trust and adoption, and I'd love
to hear from you, what do we need to advance AI
00:15:07.600 --> 00:15:12.800
evaluation and testing in our ecosystem? What
are some of the big gaps that you're seeing,
00:15:12.800 --> 00:15:18.480
and what role can different stakeholders play
in filling them? And maybe an add-on, actually:
00:15:18.480 --> 00:15:23.505
is there some sort of network effect
that could 10x our testing capacity?
00:15:23.505 --> 00:15:29.280
CRAIG DECKARD: Absolutely. So there's
a lot of work that needs to be done,
00:15:29.280 --> 00:15:35.280
and there's a lot of work in process to
really level up our whole evaluation and
00:15:35.280 --> 00:15:41.760
testing ecosystem. We learned, across
domains, that there’s really a need to
00:15:41.760 --> 00:15:48.720
advance our thinking and our practice
in three areas: rigor of testing;
00:15:48.720 --> 00:15:55.200
standardization of methodologies and processes;
and interpretability of test results.
00:15:55.200 --> 00:16:01.280
So what we mean by rigor is that
we are ensuring that what we are
00:16:01.280 --> 00:16:05.920
ultimately evaluating in terms of risks
is defined in a scientifically valid way
00:16:05.920 --> 00:16:12.320
and we are able to measure against that
risk in a scientifically valid way.
00:16:12.320 --> 00:16:19.200
By standardization, what we mean is that there's
really an accepted and well-understood and,
00:16:19.200 --> 00:16:25.280
again, a scientifically valid
methodology for doing that testing
00:16:25.280 --> 00:16:33.200
and for actually producing artifacts out
of that testing that are meeting those
00:16:33.200 --> 00:16:39.760
standards. And that sets us up for the final
portion on interpretability, which is, like,
00:16:39.760 --> 00:16:45.520
really the process by which you can trust that
the testing has been done in this rigorous and
00:16:45.520 --> 00:16:52.560
standardized way and that then you have artifacts
that result from the testing process that can
00:16:52.560 --> 00:16:58.160
really be used in the risk management context
because they can be interpreted, right.
00:16:58.160 --> 00:17:05.760
We understand how to, like, apply weight to them
for our risk-management decisions. We actually are
00:17:05.760 --> 00:17:11.680
able to interpret them in a way that perhaps
they inform other downstream risk mitigations
00:17:11.680 --> 00:17:16.320
that address the risks that we see through the
testing results and that we actually understand
00:17:16.320 --> 00:17:23.040
what limitations apply to the test results and
why they may or may not be valid in certain,
00:17:23.040 --> 00:17:27.680
sort of, deployment contexts, for example,
and especially in the context of other risk
00:17:27.680 --> 00:17:33.760
mitigations that we need to apply. So there's a
need to advance all three of those things—rigor,
00:17:33.760 --> 00:17:40.640
standardization, and interpretability—to level up
the whole testing and evaluation ecosystem.
00:17:40.640 --> 00:17:48.480
And when we think about what actors should
be involved in that work … really everybody,
00:17:48.480 --> 00:17:54.400
which is both complex to orchestrate but
also really important. And so, you know,
00:17:54.400 --> 00:18:01.040
you need to have the entire value chain
involved in really advancing this work.
00:18:01.040 --> 00:18:05.680
You need the model developers, but you also
need the system developers and deployers that
00:18:05.680 --> 00:18:12.480
are really engaged in advancing the science
of evaluation and advancing how we are using
00:18:12.480 --> 00:18:16.880
these testing artifacts in the
risk management process.
00:18:16.880 --> 00:18:24.080
When we think about what could actually 10x our
testing capacity—that's the dream, right? We all
00:18:24.080 --> 00:18:29.440
want to accelerate our progress in this space.
You know, I think we need work across all three
00:18:29.440 --> 00:18:34.560
of those areas of rigor, standardization, and
interpretability, but I think one that will really
00:18:34.560 --> 00:18:43.120
help accelerate our progress across the board is
that standardization work, because ultimately,
00:18:43.120 --> 00:18:50.240
you're going to need to have these tests be done
and applied across so many different contexts,
00:18:50.240 --> 00:18:54.880
and ultimately, while we want the whole
value chain engaged in the development
00:18:54.880 --> 00:18:59.280
of the thinking and the science and the
standards in this space, we also need
00:18:59.280 --> 00:19:05.360
to realize that not every organization is
necessarily going to have the capacity to,
00:19:05.360 --> 00:19:10.880
kind of, contribute to developing the ways
that we create and use these tests. And
00:19:10.880 --> 00:19:14.080
there are going to be many organizations
that are going to benefit from there being
00:19:14.080 --> 00:19:20.640
standardization of the methodologies and the
artifacts that they can pick up and use.
00:19:20.640 --> 00:19:26.320
One thing that I know we've heard throughout this
podcast series from our experts in other domains,
00:19:26.320 --> 00:19:31.360
including Timo [Minssen] in the medical
devices context and Ciaran [Martin] in
00:19:31.360 --> 00:19:37.040
the cybersecurity context, is that there's been
a recognition, as those domains have evolved,
00:19:37.040 --> 00:19:42.320
that there's a need to calibrate our, sort of,
expectations for different actors in the ecosystem
00:19:42.320 --> 00:19:46.320
and really understand that small businesses,
for example, just cannot apply the same degree
00:19:46.320 --> 00:19:52.720
of resources that others may be able to, to do
testing and evaluation and risk management. And
00:19:52.720 --> 00:19:59.760
so the benefit of having standardized approaches
is that those organizations are able to, kind of,
00:19:59.760 --> 00:20:03.600
integrate into the broader supply chain
ecosystem and apply their own, kind of,
00:20:03.600 --> 00:20:09.520
risk management practices in their own
context in a way that is more efficient.
00:20:09.520 --> 00:20:14.160
And finally, the last stakeholder that I
think is really important to think about
00:20:14.160 --> 00:20:18.400
in terms of partnership across the
ecosystem to really advance the whole
00:20:18.400 --> 00:20:23.680
testing and evaluation work that needs
to happen is government partners, right,
00:20:23.680 --> 00:20:28.800
and thinking beyond the value chain, the AI supply
chain, and really thinking about public-private
00:20:28.800 --> 00:20:35.360
partnership. That's going to be incredibly
important to advancing this ecosystem.
00:20:35.360 --> 00:20:42.800
You know, I think there's been real progress
already in the AI evaluation and testing
00:20:42.800 --> 00:20:49.600
ecosystem in the public-private partnership
context. We have been really supportive of
00:20:49.600 --> 00:20:55.440
the work of the International Network of AI
Safety and Security Institutes and the Center
00:20:55.440 --> 00:21:02.320
for AI Standards and Innovation that all allow
for that kind of public-private partnership on
00:21:02.320 --> 00:21:09.520
actually testing and advancing the science
and best practices around standards.
00:21:09.520 --> 00:21:14.800
And there are other innovative, kind of,
partnerships, as well, in the ecosystem. You know,
00:21:14.800 --> 00:21:20.880
Singapore has recently launched their Global AI
Assurance Pilot findings. And that effort really
00:21:20.880 --> 00:21:27.280
paired application deployers and testers so that
consequential impacts at deployment could really
00:21:27.280 --> 00:21:34.480
be tested. And that's a really fruitful, sort
of, effort that complements the work of these
00:21:34.480 --> 00:21:38.960
institutes and centers that are more focused on
evaluation at the model level, for example.
00:21:39.600 --> 00:21:44.480
And in general, you know, I think that there's
just really a lot of benefits for us thinking
00:21:44.480 --> 00:21:49.600
expansively about what we can accomplish through
deep, meaningful public-private partnership in
00:21:49.600 --> 00:21:55.120
this space. I'm really excited to see where we
can go from here with building on, you know,
00:21:55.120 --> 00:22:01.280
partnerships across AI supply chains and with
governments and public-private partnerships.
00:22:01.280 --> 00:22:05.280
SULLIVAN: I couldn't agree more. I mean, this
notion of more engagement across the ecosystem and
00:22:05.280 --> 00:22:11.280
value chain is super important for us and informs
how we think about the space completely.
00:22:11.280 --> 00:22:15.680
If you could invite any other
industry to the next workshop,
00:22:15.680 --> 00:22:18.640
maybe quantum safety, space tech, even gaming,
00:22:18.640 --> 00:22:22.865
who's on your wish list? And maybe what are some
of the things you'd want to go deeper on?
00:22:22.865 --> 00:22:29.680
CRAIG DECKARD: This is something that we really
welcome feedback on if anyone listening has ideas
00:22:29.680 --> 00:22:33.440
about other domains that would be interesting
to study. I will say, I think I shared at the
00:22:33.440 --> 00:22:40.880
outset of this podcast series, the domains that
we added in this round of our efforts in studying
00:22:40.880 --> 00:22:46.320
other domains actually all came from feedback
that we received from, you know, folks we’d
00:22:46.320 --> 00:22:51.920
engaged with our first study of other domains and
multilateral, sort of, governance institutions.
00:22:51.920 --> 00:22:57.440
And so we're really keen to think about what
other domains could be interesting to study. And
00:22:57.440 --> 00:23:05.040
we are also keen to go deeper, building on what we
learned in this round of effort going forward.
00:23:05.040 --> 00:23:10.560
One of the areas that I am particularly
really interested in is going deeper on,
00:23:10.560 --> 00:23:17.280
what, sort of, transparency and information
sharing about risk evaluation and testing
00:23:17.280 --> 00:23:24.000
will be really useful to share in different
contexts? So across the AI supply chain,
00:23:24.000 --> 00:23:30.640
what is the information that's going to be
really meaningful to share between developers
00:23:30.640 --> 00:23:35.440
and deployers of models and systems and those
that are ultimately using this technology in
00:23:35.440 --> 00:23:42.880
particular deployment contexts? And, you know,
I think that we could have much to learn from
00:23:42.880 --> 00:23:50.800
other general-purpose technologies like genome
editing and nanotechnology and cybersecurity,
00:23:50.800 --> 00:23:57.120
where we could learn a bit more about the kinds
of information that they have shared across the
00:23:57.120 --> 00:24:02.960
development and deployment life cycle and
how that has strengthened risk management
00:24:02.960 --> 00:24:07.680
in general as well as provided a really strong
feedback loop around testing and evaluation.
00:24:08.560 --> 00:24:13.200
What kind of testing is most useful
to do at what point in the life cycle,
00:24:13.200 --> 00:24:19.360
and what artifacts are most useful to share as a
result of that testing and evaluation work?
00:24:19.360 --> 00:24:26.240
I'll say, as Microsoft, we have been really
investing in how we are sharing information
00:24:26.240 --> 00:24:32.400
with our various stakeholders. We also have been
engaged with others in industry in reporting what
00:24:32.400 --> 00:24:38.800
we've done in the context of the Hiroshima AI
Process, or HAIP, Reporting Framework. This is an
00:24:38.800 --> 00:24:48.320
effort that is really just in its first round of
really exploring how this kind of reporting can be
00:24:48.320 --> 00:24:54.640
really additive to risk management understanding.
And again, I think there's real opportunity here
00:24:54.640 --> 00:25:01.760
to look at this kind of reporting and understand,
you know, what's valuable for stakeholders and
00:25:01.760 --> 00:25:07.840
where is there opportunity to go further in really
informing value chains and policymakers and the
00:25:07.840 --> 00:25:14.960
public about AI risk and opportunity and what
can we learn again from other domains that have
00:25:14.960 --> 00:25:20.960
done this kind of work over decades to really
refine that kind of information sharing.
00:25:20.960 --> 00:25:25.120
SULLIVAN: It's really great to hear about all the
advances that we're making on these reports. I'm
00:25:25.120 --> 00:25:30.640
guessing a lot of the metrics in there are
technical, but sociotechnical impacts—jobs,
00:25:30.640 --> 00:25:36.000
maybe misinformation, well-being—are
harder to score. What new measurement
00:25:36.000 --> 00:25:40.720
ideas are you excited about, and do you have any
thoughts on, like, who needs to pilot those?
00:25:40.720 --> 00:25:46.560
CRAIG DECKARD: Yeah, it's an incredibly
interesting question that I think also
00:25:46.560 --> 00:25:50.640
just speaks to, you know, the breadth
of, sort of, testing and evaluation
00:25:50.640 --> 00:25:57.360
that's needed at different points along
that AI life cycle and really not getting
00:25:57.360 --> 00:26:01.840
lost in one particular kind of testing
or another pre- or post-deployment and
00:26:01.840 --> 00:26:05.520
thinking expansively about the risks that we're
trying to address through this testing.
00:26:05.520 --> 00:26:09.600
You know, for example, even with the
UK's AI Security Institute that has
00:26:09.600 --> 00:26:12.880
just recently launched a new program, a new team,
00:26:12.880 --> 00:26:18.400
that's focused on societal resilience research.
I think it's going to be a really important area
00:26:18.400 --> 00:26:24.720
from a sociotechnical impact perspective to
bring some focus into as this technology is
00:26:24.720 --> 00:26:30.400
more widely deployed. Are we understanding
the impacts over time as different people
00:26:30.400 --> 00:26:35.040
and different cultures adopt and use this
technology for different purposes?
00:26:35.040 --> 00:26:39.920
And I think that's an area where there really
is opportunity for greater public-private
00:26:39.920 --> 00:26:45.840
partnership in this research. Because we all
share this long-term interest in ensuring that
00:26:45.840 --> 00:26:53.200
this technology is really serving people and
we have to understand the impacts so that we
00:26:53.200 --> 00:26:58.880
understand, you know, what adjustments we can
actually pursue sooner upstream to address
00:26:58.880 --> 00:27:03.520
those impacts and make sure that this
technology is really going to work for
00:27:03.520 --> 00:27:07.480
all of us and in a way that is consistent
with the societal values that we want.
00:27:07.480 --> 00:27:11.440
SULLIVAN: So, Amanda, looking ahead,
I would love to hear just what's going
00:27:11.440 --> 00:27:15.025
to be on your radar? What's top of
mind for you in the coming weeks?
00:27:15.025 --> 00:27:19.920
CRAIG DECKARD: Well, we are certainly continuing
to process all the learnings that we've had from
00:27:19.920 --> 00:27:26.080
studying these domains. It’s really been a rich
set of insights that we want to make sure we,
00:27:26.080 --> 00:27:33.680
kind of, fully take advantage of. And, you
know, I think these hard questions and,
00:27:33.680 --> 00:27:40.480
you know, real opportunities to be thoughtful
in these early days of AI governance are not,
00:27:40.480 --> 00:27:45.920
sort of, going away or being easily resolved
soon. And so I think we continue to see value
00:27:45.920 --> 00:27:51.360
in really learning from others, thinking
about what's distinct in the AI context,
00:27:51.360 --> 00:27:56.000
but also what we can apply in terms of
what other domains have learned.
00:27:56.000 --> 00:28:00.080
SULLIVAN: Well, Amanda, it has been such a
special experience for me to help illuminate
00:28:00.080 --> 00:28:05.120
the work of the Office of Responsible
AI and our team in Microsoft Research,
00:28:05.120 --> 00:28:08.000
and [MUSIC] it's just really special to
see all of the work that we're doing to
00:28:08.000 --> 00:28:12.000
help set the standard for responsible
development and deployment of AI. So
00:28:12.000 --> 00:28:18.560
thank you for joining us today, and thanks
for your reflections and discussion.
00:28:18.560 --> 00:28:22.240
And to our listeners, thank you so much for
joining us for the series. We really hope you
00:28:22.240 --> 00:28:29.760
enjoyed it! To check out all of our episodes,
visit aka.ms/AITestingandEvaluation, and if you
00:28:29.760 --> 00:28:36.240
want to learn more about how Microsoft approaches
AI governance, you can visit microsoft.com/RAI.
00:28:36.240 --> 00:28:49.840
See you next time!
00:28:49.840 --> 00:28:50.670
[MUSIC FADES]
Kind: captions
Language: en-US
00:00:02.404 --> 00:00:03.106
[MUSIC]
00:00:03.106 --> 00:00:06.800
KATHLEEN SULLIVAN: Welcome
to AI Testing and Evaluation:
00:00:06.800 --> 00:00:11.520
Learnings from Science and Industry.
I’m your host, Kathleen Sullivan.
00:00:11.520 --> 00:00:16.640
As generative AI continues to advance, Microsoft
has gathered a range of experts—from genome
00:00:16.640 --> 00:00:21.360
editing to cybersecurity—to share how
their fields approach evaluation and risk
00:00:21.360 --> 00:00:26.000
assessment. Our goal is to learn from
their successes and their stumbles to
00:00:26.000 --> 00:00:31.520
move the science and practice of AI testing
forward. In this series, we’ll explore how
00:00:31.520 --> 00:00:40.304
these insights might help guide the future of AI
development, deployment, and responsible use.
00:00:40.293 --> 00:00:40.892
[MUSIC ENDS]
00:00:40.892 --> 00:00:45.760
For our final episode of the series, I’m thrilled
to once again be joined by Amanda Craig Deckard,
00:00:45.760 --> 00:00:50.240
senior director of public policy in
Microsoft’s Office of Responsible AI.
00:00:50.240 --> 00:00:52.538
Amanda, welcome back to the podcast!
00:00:52.538 --> 00:00:53.360
AMANDA CRAIG DECKARD: Thank you so much.
00:00:53.360 --> 00:00:57.840
SULLIVAN: In our intro episode, you really
helped set the stage for this series. And
00:00:57.840 --> 00:01:03.280
it’s been great, because since then, we’ve had
the pleasure of speaking with governance experts
00:01:03.280 --> 00:01:10.400
about genome editing, pharma, medical devices,
cybersecurity, and we’ve also gotten to spend
00:01:10.400 --> 00:01:15.840
some time with our own Microsoft responsible
AI leaders and hear reflections from them.
00:01:15.840 --> 00:01:20.000
And here’s what stuck with me, and I’d
love to hear from you on this, as well:
00:01:20.000 --> 00:01:26.800
testing builds trust; context is shaping risk;
and every field is really thinking about striking
00:01:26.800 --> 00:01:32.080
its own balance between pre-deployment
testing and post-deployment monitoring.
00:01:32.080 --> 00:01:37.280
So drawing on what you’ve learned from
the workshop and the case studies,
00:01:37.280 --> 00:01:42.625
what headline insights do you think
matter the most for AI governance?
00:01:42.625 --> 00:01:46.240
CRAIG DECKARD: It's been really interesting
to learn from all of these different domains,
00:01:46.240 --> 00:01:50.560
and there are, you know, lots of
really interesting takeaways.
00:01:50.560 --> 00:01:56.800
I think a starting point for me is actually pretty
similar to where you landed, which is just that
00:01:56.800 --> 00:02:05.040
testing is really important for trust, and it's
also really hard [LAUGHS] to figure out exactly,
00:02:05.040 --> 00:02:09.280
you know, how to get it right, how to
make sure that you're addressing risks,
00:02:09.280 --> 00:02:16.560
that you're not constraining innovation, that you
are recognizing that a lot of the industry that's
00:02:16.560 --> 00:02:20.960
impacted is really different. You have small
organizations, you have large organizations,
00:02:20.960 --> 00:02:25.440
and you want to enable that opportunity that is
enabled by the technology across the board.
00:02:25.440 --> 00:02:30.320
And so it's just difficult to, kind
of, get all of these dynamics right,
00:02:30.320 --> 00:02:34.960
especially when, you know, I think we heard
from other domains, testing is not some,
00:02:34.960 --> 00:02:38.960
sort of, like, oh, simple thing, right.
There's not this linear path from, like,
00:02:38.960 --> 00:02:41.360
A to B where you just test the
one thing and you're done.
00:02:41.360 --> 00:02:41.985
SULLIVAN: Right.
00:02:41.985 --> 00:02:47.280
CRAIG DECKARD: It's complex, right. Testing
is multistage. There's a lot of testing by
00:02:47.280 --> 00:02:53.680
different actors. There are a lot of different
purposes for which you might test. As I think
00:02:53.680 --> 00:02:58.640
it was Dan Carpenter who talked about it's not
just about testing for safety. It's also about
00:02:58.640 --> 00:03:04.640
testing for efficacy and building confidence
in the right dosage for pharmaceuticals,
00:03:04.640 --> 00:03:07.920
for example. And that's across the
board for all of these domains,
00:03:07.920 --> 00:03:12.080
right. That you're really thinking about
the performance of the technology. You're
00:03:12.080 --> 00:03:16.960
thinking about safety. You're trying
to also calibrate for efficiency.
00:03:16.960 --> 00:03:23.120
And so those tradeoffs, every expert
shared that navigating those is really
00:03:23.120 --> 00:03:29.920
challenging. And also that there were real
impacts to early choices in the, sort of,
00:03:29.920 --> 00:03:34.880
governance of risk in these different domains
and the development of the testing, sort of,
00:03:34.880 --> 00:03:38.160
expectations, and that in some cases,
this had been difficult to reverse,
00:03:38.160 --> 00:03:42.480
which also just layers on that complexity
and that difficulty in a different way.
00:03:42.480 --> 00:03:47.760
So that’s the super high-level takeaway. But
maybe if I could just quickly distill, like,
00:03:47.760 --> 00:03:54.080
three takeaways that I think really are applicable
to AI in a bit more of a granular way.
00:03:54.080 --> 00:04:02.080
You know, one is about, how is the testing
exactly used? For what purpose? And the
00:04:02.080 --> 00:04:07.520
second is what emphasis there is on this
pre- versus post-deployment testing and
00:04:07.520 --> 00:04:13.600
monitoring. And then the third is how
rigid versus adaptive the, sort of,
00:04:13.600 --> 00:04:17.520
testing regimes or frameworks are
in these different domains.
00:04:17.520 --> 00:04:24.720
So on the first—how is testing used?—so is
testing something that impacts market entry,
00:04:24.720 --> 00:04:32.160
for example? Or is it something that might be used
more for informing how risk is evolving in the
00:04:32.160 --> 00:04:38.720
domain and how broader risk management strategies
might need to be applied? We have examples,
00:04:38.720 --> 00:04:43.920
like the pharmaceutical or medical device industry
experts with whom you spoke, that's really,
00:04:43.920 --> 00:04:49.280
you know, testing … there is a pre-deployment
requirement. So that's one question.
00:04:49.280 --> 00:04:56.560
The second is this emphasis on pre- versus
post-deployment testing and monitoring,
00:04:56.560 --> 00:05:02.400
and we really did see across domains
that in many cases, there is a desire
00:05:02.400 --> 00:05:06.160
for both pre- and post-deployment,
sort of, testing and monitoring,
00:05:06.160 --> 00:05:12.560
but also that, sort of, naturally in these
different domains, a degree of emphasis on
00:05:12.560 --> 00:05:19.040
one or the other had evolved and that had a
real impact on governance and tradeoffs.
00:05:19.040 --> 00:05:24.240
And the third is just how rigid versus
adaptive these testing and evaluation
00:05:24.240 --> 00:05:29.200
regimes or frameworks are in these different
domains. We saw, you know, in some domains,
00:05:29.200 --> 00:05:37.200
the testing requirements were more rigid as
you might expect in more of the pharmaceutical
00:05:37.200 --> 00:05:43.440
or medical devices industries, for example. And
in other domains, there was this more, sort of,
00:05:43.440 --> 00:05:50.480
adaptive approach to how testing might get
used. So, for example, in the case of our
00:05:50.480 --> 00:05:56.240
other general-purpose technologies, you know,
you spoke with Alta Charo on genome editing,
00:05:56.240 --> 00:06:01.920
and in our case studies, we also explored
this in the context of nanotechnology. In
00:06:01.920 --> 00:06:09.840
those general-purpose technology domains, there is
more emphasis on downstream or application-context
00:06:09.840 --> 00:06:17.680
testing that is more, sort of, adaptive to the use
scenario of the technology and, you know, having
00:06:17.680 --> 00:06:23.200
that work in conjunction with testing more at
the, kind of, level of the technology itself.
00:06:23.200 --> 00:06:26.560
SULLIVAN: I want to double-click on
a number of the things we just talked
00:06:26.560 --> 00:06:29.840
about. But actually, before we go too much deeper,
00:06:29.840 --> 00:06:34.000
a question on if there's anything that really
surprised you or challenged maybe some of your
00:06:34.000 --> 00:06:39.665
own assumptions in this space from some of the
discussions that we had over the series.
00:06:39.665 --> 00:06:43.920
CRAIG DECKARD: Yeah. You know, I know I've already
just mentioned this pre- versus post-deployment
00:06:43.920 --> 00:06:50.160
testing and monitoring issue, but it was
something that was very interesting to me and
00:06:50.160 --> 00:06:56.960
in some ways surprised me or made me just realize
something that I hadn't fully connected before,
00:06:56.960 --> 00:07:03.440
about how these, sort of, regimes might evolve
in different contexts and why. And in part,
00:07:03.440 --> 00:07:10.240
I couldn't help but bring the context I have
from cybersecurity policy into this, kind of,
00:07:10.240 --> 00:07:14.560
processing of what we learned and
reflection because there was a real
00:07:14.560 --> 00:07:20.080
contrast for me between the pharmaceutical
industry and the cybersecurity domain when
00:07:20.080 --> 00:07:24.240
I think about the emphasis on pre-
versus post-deployment monitoring.
00:07:24.240 --> 00:07:31.680
And on the one hand, we have in the pharmaceutical
domain a real emphasis that has developed around
00:07:31.680 --> 00:07:38.160
pre-market testing. And there is also an
expectation in some circumstances in the
00:07:38.160 --> 00:07:43.280
pharmaceutical domain for post-deployment
testing, as well. But as we learned from
00:07:43.280 --> 00:07:51.280
our experts in that domain, there has naturally
been a real, kind of, emphasis on the pre-market
00:07:51.280 --> 00:07:58.240
portion of that testing. And in reality, even
where post-market monitoring is required and
00:07:58.240 --> 00:08:04.400
post-market testing is required, it does not
always actually happen. And the experts really
00:08:04.400 --> 00:08:09.440
explained that, you know, part of it is just the
incentive structure around the emphasis around,
00:08:09.440 --> 00:08:16.160
you know, the testing as a pre-market, sort of,
entry requirement. And also just the resources
00:08:16.160 --> 00:08:22.400
that exist among regulators, right. There's
limited resources, right. And so there are just
00:08:22.400 --> 00:08:26.720
choices and tradeoffs that they need to make
in their own, sort of, enforcement work.
00:08:26.720 --> 00:08:34.320
And then on the other hand, you know, in
cybersecurity, I never thought about the, kind of,
00:08:34.320 --> 00:08:40.560
emphasis on things like coordinated vulnerability
disclosure and bug bounties that have really
00:08:40.560 --> 00:08:47.280
developed in the cybersecurity domain. But it's a
really important part of how we secure technology
00:08:47.280 --> 00:08:53.600
and enhance cybersecurity over time, where we have
these norms that have developed where, you know,
00:08:53.600 --> 00:08:58.960
security researchers are doing really important
research. They're finding vulnerabilities in
00:08:58.960 --> 00:09:03.920
products. And we have norms developed where
they report those to the companies that are in a
00:09:03.920 --> 00:09:09.280
position to address those vulnerabilities. And in
some cases, those companies actually pay, through
00:09:09.280 --> 00:09:15.520
bug bounties, the researchers. And perhaps in
some ways, the role of coordinated vulnerability
00:09:15.520 --> 00:09:20.400
disclosure and bug bounties has evolved the way
that it has because there hasn't been as much
00:09:20.400 --> 00:09:26.720
emphasis on the pre-market testing across the
board at least in the context of software.
00:09:26.720 --> 00:09:32.080
And so you look at those two industries and it
was interesting to me to study them to some extent
00:09:32.080 --> 00:09:39.200
in contrast with each other as this way that
the incentives and the resources that need to
00:09:39.200 --> 00:09:45.680
be applied to testing, sort of, evolve to address
where there's, kind of, more or less emphasis.
00:09:45.680 --> 00:09:49.120
SULLIVAN: It's a great point. I mean, I
think what we're hearing—and what you're
00:09:49.120 --> 00:09:52.400
saying—is just exactly this choice …
like, is there a binary choice between
00:09:52.400 --> 00:09:57.120
focusing on pre-deployment testing or
post-deployment monitoring? And, you know,
00:09:57.120 --> 00:10:01.520
I think our assumption is that we need to do
both. But I'd love to hear from you on that.
00:10:01.520 --> 00:10:08.320
CRAIG DECKARD: Absolutely. I think we need to
do both. I'm very persuaded by this inclination
00:10:08.320 --> 00:10:14.560
always that there's value in trying to really
do it all in a risk management context.
00:10:14.560 --> 00:10:20.640
And also, we know one of the principles of risk
management is you have to prioritize because there
00:10:20.640 --> 00:10:27.120
are finite resources. And I think that's where we
get to this challenge in really thinking deeply,
00:10:27.120 --> 00:10:32.240
especially as we're in the early days of AI
governance, and we need to be very thoughtful
00:10:32.240 --> 00:10:37.680
about, you know, tradeoffs that we may not
want to be making but we are because, again,
00:10:37.680 --> 00:10:42.480
these are finite choices and we, kind of,
can't help but put our finger on the dial in
00:10:42.480 --> 00:10:47.280
different directions with our choices that, you
know, it's going to be very difficult to have,
00:10:47.280 --> 00:10:52.000
sort of, equal emphasis on both. And we
need to invest in both, but we need to be
00:10:52.000 --> 00:10:59.840
very deliberate about the roles of each and how
they complement each other and who does which
00:10:59.840 --> 00:11:05.280
and how we use what we learn from pre- versus
post-deployment testing and monitoring.
00:11:05.280 --> 00:11:09.280
SULLIVAN: Maybe just spending a little bit more
time here … you know, a lot of attention goes
00:11:09.280 --> 00:11:14.400
into testing models upstream, but risk often
shows up once they're wired into real products
00:11:14.400 --> 00:11:20.400
and workflows. How much does deployment context
change the risk picture from your perspective?
00:11:20.400 --> 00:11:24.640
CRAIG DECKARD: Yeah, I … such an
important question. I really agree that
00:11:24.640 --> 00:11:29.920
there has been a lot of emphasis to date
on, sort of, testing models upstream,
00:11:29.920 --> 00:11:39.280
the AI model evaluation. And it's also really
important that we bring more attention into
00:11:39.280 --> 00:11:46.960
evaluation at the system or application level. And
I actually see that in governance conversations,
00:11:46.960 --> 00:11:54.640
this is actually increasingly raised, this need
to have system-level evaluation. We see this
00:11:54.640 --> 00:12:00.480
across regulation. We also see it in the context
of just organizations trying to put in governance
00:12:00.480 --> 00:12:04.640
requirements for how their organization is going
to operate in deploying this technology.
00:12:04.640 --> 00:12:11.040
And there's a gap today in terms of best
practices around system-level testing,
00:12:11.040 --> 00:12:19.280
perhaps even more than model-level evaluation. And
it's really important because in a lot of cases,
00:12:19.280 --> 00:12:25.680
the deployment context really does impact
the risk picture, especially with AI,
00:12:25.680 --> 00:12:30.560
which is a general-purpose technology,
and we really saw this in our study of
00:12:30.560 --> 00:12:34.320
other domains that represented
general-purpose technology.
00:12:34.320 --> 00:12:40.640
So in the case study that you can find online
on nanotechnology, you know, there's a real
00:12:40.640 --> 00:12:48.640
distinction between the risk evaluation and
the governance of nanotechnology in different
00:12:48.640 --> 00:12:55.840
deployment contexts. So the chapter that our
expert on nanotechnology wrote really goes into
00:12:55.840 --> 00:13:01.200
incredibly interesting detail around, you know,
deployment of nanotechnology in the context of,
00:13:01.200 --> 00:13:07.440
like, chemical applications versus consumer
electronics versus pharmaceuticals versus
00:13:07.440 --> 00:13:13.280
construction and how the way that nanoparticles
are basically delivered in all those different
00:13:13.280 --> 00:13:19.520
deployment contexts, as well as, like, what
the risk of the actual use scenario is just
00:13:19.520 --> 00:13:25.760
varies so much. And so there's a real need to
do that kind of risk evaluation and testing in
00:13:25.760 --> 00:13:30.160
the deployment context, and this difference
in terms of risks and what we learned in
00:13:30.160 --> 00:13:36.720
these other domains where, you know, there are
these different approaches to trying to really
00:13:36.720 --> 00:13:41.600
think about and gain efficiencies and address
risks at a horizontal level versus, you know,
00:13:41.600 --> 00:13:47.760
taking a real sector-by-sector approach. And
to some extent, it seems like it's more time
00:13:47.760 --> 00:13:54.080
intensive to do that sectoral deployment-specific
work. And at the same time, perhaps there are
00:13:54.080 --> 00:14:00.320
efficiencies to be gained by actually doing
the work in the context in which, you know, you
00:14:00.320 --> 00:14:07.040
have a better understanding of the risk that can
result from really deploying this technology.
00:14:07.040 --> 00:14:12.560
And ultimately, [LAUGHS] really what we also
need to think about here is probably, in the end,
00:14:12.560 --> 00:14:17.200
just like pre- and post-deployment testing,
you need both. Not probably; certainly!
00:14:17.200 --> 00:14:24.080
So effectively we need to think about evaluation
at the model level and the system level as being
00:14:24.080 --> 00:14:29.360
really important. And it's really important
to get system evaluation right so that we can
00:14:29.360 --> 00:14:36.160
actually get trust in this technology in
deployment context so we enable adoption
00:14:37.040 --> 00:14:43.840
in low- and in high-risk deployments in a way that
means that we've done risk evaluation in each of
00:14:43.840 --> 00:14:49.520
those contexts in a way that really makes sense in
terms of the resources that we need to apply and
00:14:49.520 --> 00:14:56.760
ultimately we are able to unlock more applications
of this technology in a risk-informed way.
00:14:56.760 --> 00:15:01.840
SULLIVAN: That's great. I mean, I couldn't agree
more. I think these contexts, the approaches are
00:15:01.840 --> 00:15:07.600
so important for trust and adoption, and I'd love
to hear from you, what do we need to advance AI
00:15:07.600 --> 00:15:12.800
evaluation and testing in our ecosystem? What
are some of the big gaps that you're seeing,
00:15:12.800 --> 00:15:18.480
and what role can different stakeholders play
in filling them? And maybe an add-on, actually:
00:15:18.480 --> 00:15:23.505
is there some sort of network effect
that could 10x our testing capacity?
00:15:23.505 --> 00:15:29.280
CRAIG DECKARD: Absolutely. So there's
a lot of work that needs to be done,
00:15:29.280 --> 00:15:35.280
and there's a lot of work in process to
really level up our whole evaluation and
00:15:35.280 --> 00:15:41.760
testing ecosystem. We learned, across
domains, that there’s really a need to
00:15:41.760 --> 00:15:48.720
advance our thinking and our practice
in three areas: rigor of testing;
00:15:48.720 --> 00:15:55.200
standardization of methodologies and processes;
and interpretability of test results.
00:15:55.200 --> 00:16:01.280
So what we mean by rigor is that
we are ensuring that what we are
00:16:01.280 --> 00:16:05.920
ultimately evaluating in terms of risks
is defined in a scientifically valid way
00:16:05.920 --> 00:16:12.320
and we are able to measure against that
risk in a scientifically valid way.
00:16:12.320 --> 00:16:19.200
By standardization, what we mean is that there's
really an accepted and well-understood and,
00:16:19.200 --> 00:16:25.280
again, a scientifically valid
methodology for doing that testing
00:16:25.280 --> 00:16:33.200
and for actually producing artifacts out
of that testing that are meeting those
00:16:33.200 --> 00:16:39.760
standards. And that sets us up for the final
portion on interpretability, which is, like,
00:16:39.760 --> 00:16:45.520
really the process by which you can trust that
the testing has been done in this rigorous and
00:16:45.520 --> 00:16:52.560
standardized way and that then you have artifacts
that result from the testing process that can
00:16:52.560 --> 00:16:58.160
really be used in the risk management context
because they can be interpreted, right.
00:16:58.160 --> 00:17:05.760
We understand how to, like, apply weight to them
for our risk-management decisions. We actually are
00:17:05.760 --> 00:17:11.680
able to interpret them in a way that perhaps
they inform other downstream risk mitigations
00:17:11.680 --> 00:17:16.320
that address the risks that we see through the
testing results and that we actually understand
00:17:16.320 --> 00:17:23.040
what limitations apply to the test results and
why they may or may not be valid in certain,
00:17:23.040 --> 00:17:27.680
sort of, deployment contexts, for example,
and especially in the context of other risk
00:17:27.680 --> 00:17:33.760
mitigations that we need to apply. So there's a
need to advance all three of those things—rigor,
00:17:33.760 --> 00:17:40.640
standardization, and interpretability—to level up
the whole testing and evaluation ecosystem.
00:17:40.640 --> 00:17:48.480
And when we think about what actors should
be involved in that work … really everybody,
00:17:48.480 --> 00:17:54.400
which is both complex to orchestrate but
also really important. And so, you know,
00:17:54.400 --> 00:18:01.040
you need to have the entire value chain
involved in really advancing this work.
00:18:01.040 --> 00:18:05.680
You need the model developers, but you also
need the system developers and deployers that
00:18:05.680 --> 00:18:12.480
are really engaged in advancing the science
of evaluation and advancing how we are using
00:18:12.480 --> 00:18:16.880
these testing artifacts in the
risk management process.
00:18:16.880 --> 00:18:24.080
When we think about what could actually 10x our
testing capacity—that's the dream, right? We all
00:18:24.080 --> 00:18:29.440
want to accelerate our progress in this space.
You know, I think we need work across all three
00:18:29.440 --> 00:18:34.560
of those areas of rigor, standardization, and
interpretability, but I think one that will really
00:18:34.560 --> 00:18:43.120
help accelerate our progress across the board is
that standardization work, because ultimately,
00:18:43.120 --> 00:18:50.240
you're going to need to have these tests be done
and applied across so many different contexts,
00:18:50.240 --> 00:18:54.880
and ultimately, while we want the whole
value chain engaged in the development
00:18:54.880 --> 00:18:59.280
of the thinking and the science and the
standards in this space, we also need
00:18:59.280 --> 00:19:05.360
to realize that not every organization is
necessarily going to have the capacity to,
00:19:05.360 --> 00:19:10.880
kind of, contribute to developing the ways
that we create and use these tests. And
00:19:10.880 --> 00:19:14.080
there are going to be many organizations
that are going to benefit from there being
00:19:14.080 --> 00:19:20.640
standardization of the methodologies and the
artifacts that they can pick up and use.
00:19:20.640 --> 00:19:26.320
One thing that I know we've heard throughout this
podcast series from our experts in other domains,
00:19:26.320 --> 00:19:31.360
including Timo [Minssen] in the medical
devices context and Ciaran [Martin] in
00:19:31.360 --> 00:19:37.040
the cybersecurity context, is that there's been
a recognition, as those domains have evolved,
00:19:37.040 --> 00:19:42.320
that there's a need to calibrate our, sort of,
expectations for different actors in the ecosystem
00:19:42.320 --> 00:19:46.320
and really understand that small businesses,
for example, just cannot apply the same degree
00:19:46.320 --> 00:19:52.720
of resources that others may be able to, to do
testing and evaluation and risk management. And
00:19:52.720 --> 00:19:59.760
so the benefit of having standardized approaches
is that those organizations are able to, kind of,
00:19:59.760 --> 00:20:03.600
integrate into the broader supply chain
ecosystem and apply their own, kind of,
00:20:03.600 --> 00:20:09.520
risk management practices in their own
context in a way that is more efficient.
00:20:09.520 --> 00:20:14.160
And finally, the last stakeholder that I
think is really important to think about
00:20:14.160 --> 00:20:18.400
in terms of partnership across the
ecosystem to really advance the whole
00:20:18.400 --> 00:20:23.680
testing and evaluation work that needs
to happen is government partners, right,
00:20:23.680 --> 00:20:28.800
and thinking beyond the value chain, the AI supply
chain, and really thinking about public-private
00:20:28.800 --> 00:20:35.360
partnership. That's going to be incredibly
important to advancing this ecosystem.
00:20:35.360 --> 00:20:42.800
You know, I think there's been real progress
already in the AI evaluation and testing
00:20:42.800 --> 00:20:49.600
ecosystem in the public-private partnership
context. We have been really supportive of
00:20:49.600 --> 00:20:55.440
the work of the International Network of AI
Safety and Security Institutes and the Center
00:20:55.440 --> 00:21:02.320
for AI Standards and Innovation that all allow
for that kind of public-private partnership on
00:21:02.320 --> 00:21:09.520
actually testing and advancing the science
and best practices around standards.
00:21:09.520 --> 00:21:14.800
And there are other innovative, kind of,
partnerships, as well, in the ecosystem. You know,
00:21:14.800 --> 00:21:20.880
Singapore has recently launched their Global AI
Assurance Pilot findings. And that effort really
00:21:20.880 --> 00:21:27.280
paired application deployers and testers so that
consequential impacts at deployment could really
00:21:27.280 --> 00:21:34.480
be tested. And that's a really fruitful, sort
of, effort that complements the work of these
00:21:34.480 --> 00:21:38.960
institutes and centers that are more focused on
evaluation at the model level, for example.
00:21:39.600 --> 00:21:44.480
And in general, you know, I think that there's
just really a lot of benefits for us thinking
00:21:44.480 --> 00:21:49.600
expansively about what we can accomplish through
deep, meaningful public-private partnership in
00:21:49.600 --> 00:21:55.120
this space. I'm really excited to see where we
can go from here with building on, you know,
00:21:55.120 --> 00:22:01.280
partnerships across AI supply chains and with
governments and public-private partnerships.
00:22:01.280 --> 00:22:05.280
SULLIVAN: I couldn't agree more. I mean, this
notion of more engagement across the ecosystem and
00:22:05.280 --> 00:22:11.280
value chain is super important for us and informs
how we think about the space completely.
00:22:11.280 --> 00:22:15.680
If you could invite any other
industry to the next workshop,
00:22:15.680 --> 00:22:18.640
maybe quantum safety, space tech, even gaming,
00:22:18.640 --> 00:22:22.865
who's on your wish list? And maybe what are some
of the things you'd want to go deeper on?
00:22:22.865 --> 00:22:29.680
CRAIG DECKARD: This is something that we really
welcome feedback on if anyone listening has ideas
00:22:29.680 --> 00:22:33.440
about other domains that would be interesting
to study. I will say, I think I shared at the
00:22:33.440 --> 00:22:40.880
outset of this podcast series, the domains that
we added in this round of our efforts in studying
00:22:40.880 --> 00:22:46.320
other domains actually all came from feedback
that we received from, you know, folks we’d
00:22:46.320 --> 00:22:51.920
engaged with our first study of other domains and
multilateral, sort of, governance institutions.
00:22:51.920 --> 00:22:57.440
And so we're really keen to think about what
other domains could be interesting to study. And
00:22:57.440 --> 00:23:05.040
we are also keen to go deeper, building on what we
learned in this round of effort going forward.
00:23:05.040 --> 00:23:10.560
One of the areas that I am particularly
really interested in is going deeper on,
00:23:10.560 --> 00:23:17.280
what, sort of, transparency and information
sharing about risk evaluation and testing
00:23:17.280 --> 00:23:24.000
will be really useful to share in different
contexts? So across the AI supply chain,
00:23:24.000 --> 00:23:30.640
what is the information that's going to be
really meaningful to share between developers
00:23:30.640 --> 00:23:35.440
and deployers of models and systems and those
that are ultimately using this technology in
00:23:35.440 --> 00:23:42.880
particular deployment contexts? And, you know,
I think that we could have much to learn from
00:23:42.880 --> 00:23:50.800
other general-purpose technologies like genome
editing and nanotechnology and cybersecurity,
00:23:50.800 --> 00:23:57.120
where we could learn a bit more about the kinds
of information that they have shared across the
00:23:57.120 --> 00:24:02.960
development and deployment life cycle and
how that has strengthened risk management
00:24:02.960 --> 00:24:07.680
in general as well as provided a really strong
feedback loop around testing and evaluation.
00:24:08.560 --> 00:24:13.200
What kind of testing is most useful
to do at what point in the life cycle,
00:24:13.200 --> 00:24:19.360
and what artifacts are most useful to share as a
result of that testing and evaluation work?
00:24:19.360 --> 00:24:26.240
I'll say, as Microsoft, we have been really
investing in how we are sharing information
00:24:26.240 --> 00:24:32.400
with our various stakeholders. We also have been
engaged with others in industry in reporting what
00:24:32.400 --> 00:24:38.800
we've done in the context of the Hiroshima AI
Process, or HAIP, Reporting Framework. This is an
00:24:38.800 --> 00:24:48.320
effort that is really just in its first round of
really exploring how this kind of reporting can be
00:24:48.320 --> 00:24:54.640
really additive to risk management understanding.
And again, I think there's real opportunity here
00:24:54.640 --> 00:25:01.760
to look at this kind of reporting and understand,
you know, what's valuable for stakeholders and
00:25:01.760 --> 00:25:07.840
where is there opportunity to go further in really
informing value chains and policymakers and the
00:25:07.840 --> 00:25:14.960
public about AI risk and opportunity and what
can we learn again from other domains that have
00:25:14.960 --> 00:25:20.960
done this kind of work over decades to really
refine that kind of information sharing.
00:25:20.960 --> 00:25:25.120
SULLIVAN: It's really great to hear about all the
advances that we're making on these reports. I'm
00:25:25.120 --> 00:25:30.640
guessing a lot of the metrics in there are
technical, but sociotechnical impacts—jobs,
00:25:30.640 --> 00:25:36.000
maybe misinformation, well-being—are
harder to score. What new measurement
00:25:36.000 --> 00:25:40.720
ideas are you excited about, and do you have any
thoughts on, like, who needs to pilot those?
00:25:40.720 --> 00:25:46.560
CRAIG DECKARD: Yeah, it's an incredibly
interesting question that I think also
00:25:46.560 --> 00:25:50.640
just speaks to, you know, the breadth
of, sort of, testing and evaluation
00:25:50.640 --> 00:25:57.360
that's needed at different points along
that AI life cycle and really not getting
00:25:57.360 --> 00:26:01.840
lost in one particular kind of testing
or another pre- or post-deployment and
00:26:01.840 --> 00:26:05.520
thinking expansively about the risks that we're
trying to address through this testing.
00:26:05.520 --> 00:26:09.600
You know, for example, even with the
UK's AI Security Institute that has
00:26:09.600 --> 00:26:12.880
just recently launched a new program, a new team,
00:26:12.880 --> 00:26:18.400
that's focused on societal resilience research.
I think it's going to be a really important area
00:26:18.400 --> 00:26:24.720
from a sociotechnical impact perspective to
bring some focus into as this technology is
00:26:24.720 --> 00:26:30.400
more widely deployed. Are we understanding
the impacts over time as different people
00:26:30.400 --> 00:26:35.040
and different cultures adopt and use this
technology for different purposes?
00:26:35.040 --> 00:26:39.920
And I think that's an area where there really
is opportunity for greater public-private
00:26:39.920 --> 00:26:45.840
partnership in this research. Because we all
share this long-term interest in ensuring that
00:26:45.840 --> 00:26:53.200
this technology is really serving people and
we have to understand the impacts so that we
00:26:53.200 --> 00:26:58.880
understand, you know, what adjustments we can
actually pursue sooner upstream to address
00:26:58.880 --> 00:27:03.520
those impacts and make sure that this
technology is really going to work for
00:27:03.520 --> 00:27:07.480
all of us and in a way that is consistent
with the societal values that we want.
00:27:07.480 --> 00:27:11.440
SULLIVAN: So, Amanda, looking ahead,
I would love to hear just what's going
00:27:11.440 --> 00:27:15.025
to be on your radar? What's top of
mind for you in the coming weeks?
00:27:15.025 --> 00:27:19.920
CRAIG DECKARD: Well, we are certainly continuing
to process all the learnings that we've had from
00:27:19.920 --> 00:27:26.080
studying these domains. It’s really been a rich
set of insights that we want to make sure we,
00:27:26.080 --> 00:27:33.680
kind of, fully take advantage of. And, you
know, I think these hard questions and,
00:27:33.680 --> 00:27:40.480
you know, real opportunities to be thoughtful
in these early days of AI governance are not,
00:27:40.480 --> 00:27:45.920
sort of, going away or being easily resolved
soon. And so I think we continue to see value
00:27:45.920 --> 00:27:51.360
in really learning from others, thinking
about what's distinct in the AI context,
00:27:51.360 --> 00:27:56.000
but also what we can apply in terms of
what other domains have learned.
00:27:56.000 --> 00:28:00.080
SULLIVAN: Well, Amanda, it has been such a
special experience for me to help illuminate
00:28:00.080 --> 00:28:05.120
the work of the Office of Responsible
AI and our team in Microsoft Research,
00:28:05.120 --> 00:28:08.000
and [MUSIC] it's just really special to
see all of the work that we're doing to
00:28:08.000 --> 00:28:12.000
help set the standard for responsible
development and deployment of AI. So
00:28:12.000 --> 00:28:18.560
thank you for joining us today, and thanks
for your reflections and discussion.
00:28:18.560 --> 00:28:22.240
And to our listeners, thank you so much for
joining us for the series. We really hope you
00:28:22.240 --> 00:28:29.760
enjoyed it! To check out all of our episodes,
visit aka.ms/AITestingandEvaluation, and if you
00:28:29.760 --> 00:28:36.240
want to learn more about how Microsoft approaches
AI governance, you can visit microsoft.com/RAI.
00:28:36.240 --> 00:28:49.840
See you next time!
00:28:49.840 --> 00:28:50.670
[MUSIC FADES]
