TRANSKRIPT 🔗
00:00:02.679 --> 00:00:04.200
[MUSIC]
00:00:04.200 --> 00:00:06.800
KATHLEEN SULLIVAN: Welcome
to AI Testing and Evaluation:
00:00:06.800 --> 00:00:11.520
Learnings from Science and Industry.
I'm your host, Kathleen Sullivan.
00:00:11.520 --> 00:00:16.640
As generative AI continues to advance, Microsoft
has gathered a range of experts—from genome
00:00:16.640 --> 00:00:21.360
editing to cybersecurity—to share how
their fields approach evaluation and risk
00:00:21.360 --> 00:00:26.000
assessment. Our goal is to learn from
their successes and their stumbles to
00:00:26.000 --> 00:00:31.520
move the science and practice of AI testing
forward. In this series, we'll explore how
00:00:31.520 --> 00:00:40.547
these insights might help guide the future of AI
development, deployment, and responsible use.
00:00:40.547 --> 00:00:41.092
[MUSIC ENDS]
00:00:41.092 --> 00:00:45.120
Today I'm excited to welcome R.
Alta Charo, the Warren P. Knowles
00:00:45.120 --> 00:00:49.040
Professor Emerita of Law and Bioethics
at the University of Wisconsin–Madison,
00:00:49.840 --> 00:00:53.920
to explore testing and risk
assessment in genome editing.
00:00:53.920 --> 00:00:58.720
Professor Charo has been at the forefront of
biotechnology policy and governance for decades,
00:00:58.720 --> 00:01:04.240
advising former President Obama's transition team
on issues of medical research and public health,
00:01:04.240 --> 00:01:09.600
as well as serving as a senior policy advisor at
the Food and Drug Administration. She consults
00:01:09.600 --> 00:01:14.960
on gene therapy and genome editing for various
companies and organizations and has held positions
00:01:14.960 --> 00:01:21.360
on a number of advisory committees, including for
the National Academy of Sciences. Her committee
00:01:21.360 --> 00:01:27.360
work has spanned women's health, stem cell
research, genome editing, biosecurity, and more.
00:01:27.360 --> 00:01:31.360
After our conversation with Professor
Charo, we'll hear from Daniel Kluttz,
00:01:31.360 --> 00:01:36.160
a partner general manager in Microsoft's
Office of Responsible AI, about what these
00:01:36.160 --> 00:01:40.960
insights from biotech regulation could mean
for AI governance and risk assessment and
00:01:40.960 --> 00:01:45.680
his team's work governing sensitive
AI uses and emerging technologies.
00:01:45.680 --> 00:01:48.080
Alta, thank you so much for being here today.
00:01:48.080 --> 00:01:52.360
I'm a follower of your work and have really
been looking forward to our conversation.
00:01:52.360 --> 00:01:55.040
ALTA CHARO: It’s my pleasure.
Thanks for having me.
00:01:55.040 --> 00:01:59.360
SULLIVAN: Alta, I'd love to begin by stepping
back in time a bit before you became a leading
00:01:59.360 --> 00:02:05.120
figure in bioethics and legal policy. You've
shared that your interest in science was really
00:02:05.120 --> 00:02:09.520
inspired by your brothers’ interest in the topic
and that your upbringing really helped shape your
00:02:09.520 --> 00:02:15.520
perseverance and resilience. Can you talk to us
about what put you on the path to law and policy?
00:02:15.520 --> 00:02:20.960
CHARO: Well, I think it's true that many of
us are strongly influenced by our families and
00:02:20.960 --> 00:02:28.080
certainly my family had, kind of, a science-y,
techy orientation. My father was a refugee,
00:02:28.880 --> 00:02:34.560
you know, escaping the Nazis, and when he finally
was able to start working in the United States,
00:02:34.560 --> 00:02:40.480
he took advantage of the G.I. Bill to
learn how to repair televisions and radios,
00:02:40.480 --> 00:02:46.960
which were really just coming in in the 1950s.
So he was, kind of, technically oriented.
00:02:46.960 --> 00:02:53.680
My mother retrained from being a talented
amateur artist to becoming a math teacher,
00:02:53.680 --> 00:02:58.800
and not surprisingly, both my brothers began to
aim toward things like engineering and chemistry
00:02:58.800 --> 00:03:08.000
and physics. And our form of entertainment
was to watch PBS or Star Trek. [LAUGHTER]
00:03:08.000 --> 00:03:13.520
And so the interest comes from that
background coupled with, in the 1960s,
00:03:13.520 --> 00:03:19.280
this enormous surge of interest in the so-called
nature-versus-nurture debate about the degree to
00:03:19.280 --> 00:03:25.120
which we are destined by our biology or shaped
by our environments. It was a heady debate,
00:03:25.120 --> 00:03:28.960
and one that perfectly combined the
two interests in politics and science.
00:03:28.960 --> 00:03:34.880
SULLIVAN: For listeners who are brand
new to your field in genomic editing,
00:03:34.880 --> 00:03:40.800
can you give us what I'll call a “90-second
survey” of the space in perhaps plain language
00:03:40.800 --> 00:03:45.000
and why it's important to have a framework
for ensuring its responsible use.
00:03:45.000 --> 00:03:52.400
CHARO: Well, you know, genome editing
is both very old and very new. At base,
00:03:52.400 --> 00:04:01.360
what we're talking about is a way to either delete
sections of the genome, our collection of genes,
00:04:01.360 --> 00:04:09.520
or to add things or to alter what's there.
The goal is simply to be able to take what
00:04:09.520 --> 00:04:14.400
might not be healthy and make it healthy,
whether it's a plant, an animal, or a human.
00:04:15.280 --> 00:04:21.040
Many people have compared it to a word
processor, where you can edit text by
00:04:21.040 --> 00:04:27.360
swapping things in and out. You could change
the letter g to the letter h in every word,
00:04:27.360 --> 00:04:31.760
and in our genomes, you can
do similar kinds of things.
00:04:31.760 --> 00:04:40.160
But because of this, we have a responsibility
to make sure that whatever we change doesn't
00:04:40.160 --> 00:04:45.280
become
Load full transcript
dangerous and that it doesn't become
socially disruptive. Now the earliest forms
00:04:45.280 --> 00:04:51.520
of genome editing were very inefficient, and
so we didn't worry that much. But with the
00:04:51.520 --> 00:04:58.480
advances that were spearheaded by people like
Jennifer Doudna and Emmanuelle Charpentier,
00:04:58.480 --> 00:05:06.400
who won the Nobel Prize for their work in this
area, genome editing has become much easier to do.
00:05:06.400 --> 00:05:11.760
It's become more efficient. It doesn't require
as much sophisticated laboratory equipment.
00:05:12.640 --> 00:05:16.080
It's moved from being something that
only a few people can do to something
00:05:16.080 --> 00:05:22.080
that we're going to be seeing in our junior
high school biology labs. And that means you
00:05:22.080 --> 00:05:27.200
have to pay attention to who's doing it, why
are they doing it, what are they releasing,
00:05:27.200 --> 00:05:32.760
if anything, into the environment, what are they
trying to sell, and is it honest and is it safe?
00:05:32.760 --> 00:05:36.960
SULLIVAN: How would you describe the
risks, and are there, you know, sort of,
00:05:36.960 --> 00:05:43.520
specifically inherent risks in the technology
itself, or do those risks really emerge only when
00:05:43.520 --> 00:05:49.240
it's applied in certain contexts, like CRISPR
in agriculture or CRISPR for human therapies?
00:05:49.240 --> 00:05:54.000
CHARO: Well, to answer that, I'm going to
do something that may seem a little picky,
00:05:54.000 --> 00:06:00.000
even pedantic. [LAUGHTER] But I'm going
to distinguish between hazards and risks.
00:06:00.000 --> 00:06:05.680
So there are certain intrinsic hazards.
That is, there are things that can go wrong.
00:06:05.680 --> 00:06:11.520
You want to change one particular gene
or one particular portion of a gene,
00:06:11.520 --> 00:06:18.480
and you might accidentally change something else,
a so-called off-target effect. Or you might change
00:06:18.480 --> 00:06:24.400
something in a gene expecting a certain effect but
not necessarily anticipating that there's going
00:06:24.400 --> 00:06:29.360
to be an interaction between what you changed
and what was there, a gene-gene interaction,
00:06:29.360 --> 00:06:34.800
that might have an unanticipated kind
of result, a side effect essentially.
00:06:34.800 --> 00:06:41.680
So there are some intrinsic hazards, but risk is
a hazard coupled with the probability that it's
00:06:41.680 --> 00:06:47.920
going to actually create something harmful.
And that really depends upon the application.
00:06:47.920 --> 00:06:55.200
If you are doing something that is making
a change in a human being that is going
00:06:55.200 --> 00:07:03.840
to be a lifelong change, that enhances the
significance of that hazard. It amplifies what
00:07:03.840 --> 00:07:10.800
I call the risk because if something goes
wrong, then its consequences are greater.
00:07:11.760 --> 00:07:18.880
It may also be that in other settings, what you're
doing is going to have a much lower risk because
00:07:18.880 --> 00:07:24.640
you're working with a more familiar substance,
your predictive power is much greater, and it's
00:07:24.640 --> 00:07:30.720
not going into a human or an animal or into the
environment. So I think that you have to say that
00:07:30.720 --> 00:07:36.720
the risk and the benefits, by the way, all are
going to depend upon the particular application.
00:07:36.720 --> 00:07:40.560
SULLIVAN: Yeah, I think on
this point of application,
00:07:41.280 --> 00:07:44.720
there's many players involved in that,
right. Like, we often hear about this
00:07:44.720 --> 00:07:50.080
puzzle of who's actually responsible for
ensuring safety and a reasonable balance
00:07:50.080 --> 00:07:55.680
between risks and benefits or hazards and
benefits, to quote you. Is it the scientists,
00:07:55.680 --> 00:08:01.760
the biotech companies, government agencies?
And then if you could touch upon, as well,
00:08:01.760 --> 00:08:07.800
maybe how does the nature of genome editing risks
… how do those responsibilities get divvied up?
00:08:07.800 --> 00:08:15.040
CHARO: Well, in the 1980s, we had a very
significant policy discussion about whether
00:08:15.040 --> 00:08:20.800
we should regulate the technology—no matter
how it's used or for whatever purpose—or if
00:08:20.800 --> 00:08:24.480
we should simply fold the technology
in with all the other technologies
00:08:24.480 --> 00:08:29.600
that we currently have and regulate
its applications the way we regulate
00:08:29.600 --> 00:08:34.160
applications generally. And we went for the
second, the so-called coordinated framework.
00:08:34.160 --> 00:08:40.160
So what we have in the United States is a
system in which if you use genome editing
00:08:40.160 --> 00:08:46.560
in purely laboratory-based work, then you will
be regulated the way we regulate laboratories.
00:08:46.560 --> 00:08:52.640
There's also, at most universities because
of the way the government works with this,
00:08:52.640 --> 00:08:57.680
something called Institutional Biosafety
Committees, IBCs. You want to do research that
00:08:57.680 --> 00:09:03.200
involves recombinant DNA and modern biotechnology,
including genome editing but not limited to it,
00:09:03.200 --> 00:09:08.640
you have to go first to your IBC, and they
look and see what you're doing to decide
00:09:08.640 --> 00:09:13.200
if there's a danger there that you have not
anticipated that requires special attention.
00:09:13.200 --> 00:09:16.880
If what you're doing is going to get
released into the environment or it's
00:09:16.880 --> 00:09:20.640
going to be used to change an animal
that's going to be in the environment,
00:09:20.640 --> 00:09:25.280
then there are agencies that oversee
the safety of our environment,
00:09:25.280 -->
00:09:29.680
predominantly the Environmental Protection
Agency and the U.S. Department of Agriculture.
00:09:29.680 --> 00:09:33.520
If you're working with humans and
you're doing medical therapies,
00:09:33.520 --> 00:09:38.400
like you're doing the gene therapies that just
have been developed for things like sickle cell
00:09:38.400 --> 00:09:43.680
anemia, then you have to go through a very
elaborate regulatory process that's overseen
00:09:43.680 --> 00:09:50.320
by the Food and Drug Administration and also
seen locally at the research stages overseen
00:09:50.320 --> 00:09:55.280
by institutional review boards that make sure
the people who are being recruited into research
00:09:55.280 --> 00:09:58.880
understand what they're getting into, that
they're the right people to be recruited, etc.
00:09:59.760 --> 00:10:04.055
So we do have this kind of Jenga game …
00:10:04.055 --> 00:10:04.789
SULLIVAN: [LAUGHS] Yeah, sounds like it.
00:10:04.789 --> 00:10:10.560
CHARO: … of regulatory agencies. And on
top of all that, most of this involves
00:10:10.560 --> 00:10:15.680
professionals who've had to be licensed
in some way. There may be state laws
00:10:15.680 --> 00:10:21.280
specifically on licensing. If you are dealing
with things that might cross national borders,
00:10:21.280 --> 00:10:24.720
there may be international treaties
and agreements that cover this.
00:10:26.080 --> 00:10:29.920
And, of course, the insurance industry plays
a big part because they decide whether or
00:10:29.920 --> 00:10:35.920
not what you're doing is safe enough
to be insured. So all of these things
00:10:35.920 --> 00:10:40.320
come together in a way that is not at all
easy to understand if you're not, kind of,
00:10:40.320 --> 00:10:47.200
working in the field. But the bottom-line thing
to remember, the way to really think about it is,
00:10:47.200 --> 00:10:52.800
we don't regulate genome editing; we
regulate the things that use genome editing.
00:10:52.800 --> 00:10:54.640
SULLIVAN: Yeah, that makes
a lot of sense. Actually,
00:10:54.640 --> 00:10:59.440
maybe just following up a little bit on this
notion of a variety of different, particularly
00:10:59.440 --> 00:11:04.640
like government agencies being involved.
You know, in this multi-stakeholder model,
00:11:04.640 --> 00:11:09.120
where do you see gaps today that need to be
filled, some of the pros and cons to keep in
00:11:09.120 --> 00:11:14.000
mind, and, you know, just as we think about
distributing these systems at a global level,
00:11:14.000 --> 00:11:18.000
like, what are some of the considerations
you are keeping in mind on that front?
00:11:18.000 --> 00:11:22.560
CHARO: Well, certainly there are times
where the way the statutes were written
00:11:22.560 --> 00:11:28.400
that govern the regulation of drugs or the
regulation of foods did not anticipate this
00:11:28.400 --> 00:11:32.880
tremendous capacity we now have in the area of
biotechnology generally or genome editing in
00:11:32.880 --> 00:11:39.680
particular. And so you can find that there are
times where it feels a little bit ambiguous,
00:11:39.680 --> 00:11:45.040
and the agencies have to figure out
how to apply their existing rules.
00:11:45.040 --> 00:11:54.800
So an example. If you're going to make
alterations in an animal, right, we have a
00:11:54.800 --> 00:12:01.360
system for regulating drugs, including veterinary
drugs. But we didn't have something that regulated
00:12:01.360 --> 00:12:06.800
genome editing of animals. But in a sense, genome
editing of an animal is the same thing as using
00:12:06.800 --> 00:12:12.960
a veterinary drug. You're trying to affect the
animal's physical constitution in some fashion.
00:12:12.960 --> 00:12:18.720
And it took a long time within the FDA to,
sort of, work out how the regulation of
00:12:18.720 --> 00:12:23.440
veterinary drugs would apply if you think about
the genetic construct that's being used to alter
00:12:23.440 --> 00:12:28.720
the animal as the same thing as injecting
a chemically based drug. And on that basis,
00:12:28.720 --> 00:12:32.880
they now know here's the regulatory path—here are
the tests you have to do; here are the permissions
00:12:32.880 --> 00:12:36.960
you have to do; here's the surveillance
you have to do after it goes on the market.
00:12:37.760 --> 00:12:44.960
Even there, sometimes, it was confusing.
What happens when it's not the kind of
00:12:44.960 --> 00:12:47.920
animal you're thinking about when
you think about animal drugs? Like,
00:12:47.920 --> 00:12:51.440
we think about pigs and dogs,
but what about mosquitoes?
00:12:51.440 --> 00:12:59.040
Because there, you're really thinking more about
pests, and if you're editing the mosquito so that
00:12:59.040 --> 00:13:06.560
it can't, for example, transmit dengue fever,
right, it feels more like a public health thing
00:13:06.560 --> 00:13:12.960
than it is a drug for the mosquito itself, and
it, kind of, fell in between the agencies that
00:13:12.960 --> 00:13:17.600
possibly had jurisdiction. And it took a while
for the USDA, the Department of Agriculture,
00:13:17.600 --> 00:13:21.840
and the Food and Drug Administration to work
out an agreement about how they would share
00:13:21.840 --> 00:13:29.680
this responsibility. So you do get those kinds
of areas in which you have at least ambiguity.
00:13:30.720 --> 00:13:36.320
We also have situations where frankly the fact
that some things can move across national borders
00:13:36.320 --> 00:13:43.360
means you have to have a system for harmonizing
or coordinating national rules. If you want to,
00:13:43.360 --> 00:13:47.600
for example, genetically engineer
mosquitoes that can't transmit dengue,
00:13:47.600 --> 00:13:51.120
mosquitoes have a tendency to
fly. [LAUGHTER] And so ... they
00:13:51.120 --> 00:13:54.880
can't fly very far. That's good. That
actually makes it easier to control.
00:13:54.880 --> 00:13:59.040
But if you're doing work that's right near
a border, then you have to be sure that the
00:13:59.040 --> 00:14:04.640
country next to you has the same rules for
whether it's permitted to do this and how
00:14:04.640 --> 00:14:09.360
to surveil what you've done in order to be
sure that you got the results you wanted to
00:14:09.360 --> 00:14:13.440
get and no other results. And that also
is an area where we have a lot of work
00:14:13.440 --> 00:14:18.800
to be done in terms of coordinating across
government borders and harmonizing our rules.
00:14:18.800 --> 00:14:24.480
SULLIVAN: Yeah, I mean, you've touched on this
a little bit, but there is such this striking
00:14:24.480 --> 00:14:29.920
balance between advancing technology, ensuring
public safety, and sometimes, I think it feels
00:14:29.920 --> 00:14:34.880
just like you're walking a tightrope where, you
know, if we clamp down too hard, we'll stifle
00:14:34.880 --> 00:14:40.320
innovation, and if we're too lax, we risk some
of these unintended consequences. And on a global
00:14:40.320 --> 00:14:46.160
scale like you just mentioned, as well. How has
the field of genome editing found its balance?
00:14:46.160 --> 00:14:48.720
CHARO: It's still being worked out, frankly,
00:14:48.720 --> 00:14:57.040
but it's finding its balance application by
application. So in the United States, we have
00:14:57.040 --> 00:15:02.640
two very different approaches on regulation of
things that are going to go into the market.
00:15:02.640 --> 00:15:08.960
Some things can't be marketed until they've
gotten an approval from the government. So
00:15:08.960 --> 00:15:14.480
you come up with a new drug, you can't sell
that until it's gone through FDA approval.
00:15:14.480 --> 00:15:20.080
On the other hand, for most foods that
are made up of familiar kinds of things,
00:15:20.080 --> 00:15:26.240
you can go on the market, and it's only after
they're on the market that the FDA can act to
00:15:26.240 --> 00:15:32.080
withdraw it if a problem arises. So basically, we
have either pre-market controls: you can't go on
00:15:32.080 --> 00:15:37.120
without permission. Or post-market controls: we
can take you off the market if a problem occurs.
00:15:37.120 --> 00:15:41.600
How do we decide which one is appropriate
for a particular application? It's based
00:15:41.600 --> 00:15:48.560
on our experience. New drugs typically
are both less familiar than existing
00:15:48.560 --> 00:15:54.800
things on the market and also have a
higher potential for injury if they,
00:15:54.800 --> 00:15:59.200
in fact, are not effective or they
are, in fact, dangerous and toxic.
00:16:00.000 --> 00:16:04.720
If you have foods, even bioengineered foods,
that are basically the same as foods that are
00:16:04.720 --> 00:16:09.920
already here, it can go on
the market with
notice but without a prior approval. But
00:16:09.920 --> 00:16:15.200
if you create something truly novel, then
it has to go through a whole long process.
00:16:16.240 --> 00:16:23.520
And so that is the way that we make this balance.
We look at the application area. And we're just
00:16:23.520 --> 00:16:29.680
now seeing in the Department of Agriculture a new
approach on some of the animal editing, again,
00:16:29.680 --> 00:16:35.280
to try and distinguish between things that are
simply a more efficient way to make a familiar
00:16:35.280 --> 00:16:41.920
kind of animal variant and those things that are
genuinely novel and to have a regulatory process
00:16:41.920 --> 00:16:49.440
that is more rigid the more unfamiliar it is and
the more that we see a risk associated with it.
00:16:49.440 --> 00:16:54.560
SULLIVAN: I know we're at the end of our time here
and maybe just a quick kind of lightning-round
00:16:54.560 --> 00:17:00.800
of a question. For students, young scientists,
lawyers, or maybe even entrepreneurs listening
00:17:00.800 --> 00:17:05.040
who are inspired by your work, what's the
single piece of advice you give them if they're
00:17:05.040 --> 00:17:11.680
interested in policy, regulation, the ethical
side of things in genomics or other fields?
00:17:11.680 --> 00:17:18.880
CHARO: I'd say be a bio-optimist
and read a lot of science fiction.
00:17:19.600 --> 00:17:26.720
Because it expands your imagination about what the
world could be like. Is it going to be a world in
00:17:26.720 --> 00:17:32.160
which we're now going to be growing our buildings
instead of building them out of concrete?
00:17:32.160 --> 00:17:35.920
Is it going to be a world in which
our plants will glow in the evening
00:17:35.920 --> 00:17:40.160
so we don't need to be using batteries
or electrical power from other sources
00:17:40.160 --> 00:17:44.160
but instead our environment
is adapting to our needs?
00:17:44.800 --> 00:17:49.840
You know, expand your imagination
with a sense of optimism about what
00:17:49.840 --> 00:17:55.920
could be and see ethics and regulation
not as an obstacle but as a partner to
00:17:55.920 --> 00:18:01.491
bringing these things to fruition in a way
that's responsible and helpful to everyone.
00:18:01.491 --> 00:18:02.019
[TRANSITION MUSIC]
00:18:02.019 --> 00:18:06.000
SULLIVAN: Wonderful. Well, Alta, this has
been just an absolute pleasure. So thank you.
00:18:06.000 --> 00:18:10.880
CHARO: It was my pleasure.
Thank you for having me.
00:18:15.400 --> 00:18:16.400
SULLIVAN:
00:18:16.400 --> 00:18:21.680
Now, I'm happy to bring in Daniel Kluttz. As a
partner general manager in Microsoft's Office of
00:18:21.680 --> 00:18:27.280
Responsible AI, Daniel leads the group’s Sensitive
Uses and Emerging Technologies program.
00:18:27.280 --> 00:18:29.745
Daniel, it's great to have you
here. Thanks for coming in.
00:18:29.745
--> 00:18:30.920
DANIEL KLUTTZ: It's great to be here, Kathleen.
00:18:30.920 --> 00:18:36.000
SULLIVAN: Yeah. So maybe before
we unpack Alta Charo’s insights,
00:18:36.000 --> 00:18:39.840
I'd love to just understand the elevator
pitch here. What exactly is [the] Sensitive
00:18:39.840 --> 00:18:44.120
Uses and Emerging Tech program, and what
was the impetus for establishing it?
00:18:44.120 --> 00:18:49.360
KLUTTZ: Yeah. So the Sensitive Uses and Emerging
Technologies program sits within our Office of
00:18:49.360 --> 00:18:54.960
Responsible AI at Microsoft. And inherent in
the name, there are two real core functions.
00:18:54.960 --> 00:18:58.720
There's the sensitive uses and emerging
technologies. What does that mean?
00:18:58.720 --> 00:19:04.320
Sensitive uses, think of that as Microsoft's
internal consulting and oversight function for
00:19:04.320 --> 00:19:11.600
our higher-risk, most impactful AI system
deployments. And so my team is a team of
00:19:11.600 --> 00:19:18.320
multidisciplinary experts who engages in sort of
a white-glove-treatment sort of way with product
00:19:18.320 --> 00:19:24.240
teams at Microsoft that are designing, building,
and deploying these higher-risk AI systems,
00:19:24.240 --> 00:19:28.320
and where that sort of consulting
journey culminates is in a set of
00:19:28.320 --> 00:19:34.080
bespoke requirements tailored to the use case
of that given system that really implement
00:19:34.080 --> 00:19:40.400
and apply our more standardized, generalized
requirements that apply across the board.
00:19:40.400 --> 00:19:45.440
Then the emerging technologies function
of my team faces a little bit further out,
00:19:45.440 --> 00:19:51.280
trying to look around corners to see what new
and novel and emerging risks are coming out of
00:19:51.280 --> 00:19:57.040
new AI technologies with the idea that we work
with our researchers, our engineering partners,
00:19:57.040 --> 00:20:02.160
and, of course, product leaders across the
company to understand where Microsoft is going
00:20:02.160 --> 00:20:05.920
with those emerging technologies,
and we're developing sort of rapid,
00:20:05.920 --> 00:20:11.600
quick-fire-early steer guidance that
implements our policies ahead of that
00:20:11.600 --> 00:20:16.160
formal internal policymaking process, which can
take a bit of time. So it's designed to, sort of,
00:20:16.160 --> 00:20:23.280
both afford that innovation speed that we like
to optimize for at Microsoft but also integrate
00:20:23.280 --> 00:20:29.440
our responsible AI commitments and our AI
principles into emerging product development.
00:20:29.440 --> 00:20:34.480
SULLIVAN: That segues really nicely, actually, as
we met with Professor Charo and she was, you know,
00:20:34.480 --> 00:20:39.760
talking about the field of genome editing
and the governing at the application level.
00:20:39.760 --> 00:20:45.360
I'd love to just understand how similar or not is
that to managing the risks of AI in our world?
00:20:45.360 --> 00:20:49.440
KLUTTZ: Yeah. I mean, Professor Charo’s
comments were music to my ears because,
00:20:49.440 --> 00:20:53.200
you know, where we make our
bread and butter, so to speak,
00:20:53.200 --> 00:21:01.280
in our team is in applying to use cases. AI
systems, especially in this era of generative AI,
00:21:01.280 --> 00:21:08.000
are almost inherently multi-use, dual use. And so
what really matters is how you're going to apply
00:21:08.000 --> 00:21:13.040
that more general-purpose technology. Who's
going to use it? In what domain is it going
00:21:13.040 --> 00:21:19.920
to be deployed? And then tailor that oversight to
those use cases. Try to be risk proportionate.
00:21:19.920 --> 00:21:23.440
Professor Charo talked a little bit about
this, but if it's something that's been done
00:21:23.440 --> 00:21:28.400
before and it's just a new spin on an old
thing, maybe we're not so concerned about
00:21:28.400 --> 00:21:34.400
how closely we need to oversee and gate that
application of that technology, whereas if it's
00:21:34.400 --> 00:21:39.840
something new and novel or some new risk that
might be posed by that technology, we take a
00:21:39.840 --> 00:21:45.320
little bit closer look and we are overseeing
that in a more sort of high-touch way.
00:21:45.320 --> 00:21:47.280
SULLIVAN: Maybe following up on that, I mean,
00:21:47.280 --> 00:21:52.880
how do you define sensitive use or
maybe like high-impact application,
00:21:52.880 --> 00:21:56.560
and once that's labeled, what happens? Like,
what kind of steps kick in from there?
00:21:56.560 --> 00:22:02.000
KLUTTZ: Yeah. So we have this Sensitive Uses
program that's been at Microsoft since 2019.
00:22:02.000 --> 00:22:05.760
I came to Microsoft in 2019 when we were
starting this program in the Office of
00:22:05.760 --> 00:22:11.920
Responsible AI, and it had actually been incubated
in Microsoft Research with our Aether community
00:22:11.920 --> 00:22:18.320
of colleagues who are experts in sociotechnical
approaches to responsible AI, as well. Once we put
00:22:18.320 --> 00:22:23.030
it in the Office of Responsible AI, I came over. I
came from academia. I was a researcher myself …
00:22:23.030 --> 00:22:23.560
SULLIVAN: At Berkeley, right?
00:22:23.560 --> 00:22:28.400
KLUTTZ: At Berkeley. That's right. Yep.
Sociologist by training and a lawyer in a
00:22:28.400 --> 00:22:32.720
past life. [LAUGHTER] But that has helped
sort of bridge those fields for me.
00:22:32.720 --> 00:22:38.080
But Sensitive Uses, we force all of our
teams when they're envisioning their system
00:22:38.080 --> 00:22:45.280
design to think about, could the reasonably
foreseeable use or misuse of the system that
00:22:45.280 --> 00:22:51.600
they're developing in practice
result in
three really major, sort of, risk types.
00:22:51.600 --> 00:22:58.320
One is, could that deployment result in a
consequential impact on someone's legal position
00:22:58.320 --> 00:23:04.080
or life opportunity? Another category we have
is, could that foreseeable use or misuse result
00:23:04.080 --> 00:23:12.000
in significant psychological or physical injury
or harm? And then the third really ties in with
00:23:12.000 --> 00:23:16.560
a longstanding commitment we've had to human
rights at Microsoft. And so could that system
00:23:16.560 --> 00:23:24.960
in it's reasonably foreseeable use or misuse
result in human rights impacts and injurious
00:23:25.840 --> 00:23:29.360
consequences to folks along different
dimensions of human rights?
00:23:29.360 --> 00:23:34.640
Once you decide, we have a process to
reporting that project into my office,
00:23:34.640 --> 00:23:39.120
and we will triage that project, working
with the product team, for example,
00:23:39.120 --> 00:23:43.280
and our Responsible AI Champs community,
which are folks who are dispersed throughout
00:23:43.280 --> 00:23:49.360
the ecosystem at Microsoft and educated in our
responsible AI program, and then determine, OK,
00:23:49.360 --> 00:23:55.440
is it in scope for our program? If it is, say, OK,
we're going to go along for that ride with you,
00:23:55.440 --> 00:23:59.440
and then we get into that whole sort of
consulting arrangement that then culminates
00:23:59.440 --> 00:24:06.240
in this set of bespoke use-case-based
requirements applying our AI principles.
00:24:06.240 --> 00:24:10.240
SULLIVAN: That's super fascinating. What
are some of the approaches in the governance
00:24:10.240 --> 00:24:16.720
of genome editing are you maybe seeing
happening in AI governance or maybe just,
00:24:16.720 --> 00:24:18.640
like, bubbling up in conversations around it?
00:24:18.640 --> 00:24:23.440
KLUTTZ: Yeah, I mean, I think we've learned a lot
from fields like genome editing that Professor
00:24:23.440 --> 00:24:27.840
Charo talked about and others. And again, it gets
back to this, sort of, risk-proportionate-based
00:24:27.840 --> 00:24:32.800
approach. It's a balancing test. It's a
tradeoff of trying to, sort of, foster
00:24:32.800 --> 00:24:38.720
innovation and really look for the beneficial
uses of these technologies. I appreciated her
00:24:38.720 --> 00:24:45.280
speaking about that. What are the intended uses
of the system, right? And then getting to, OK,
00:24:45.280 --> 00:24:51.280
how do we balance trying to, again, foster
that innovation in a very fast-moving space,
00:24:51.280 --> 00:24:58.240
a pretty complex space, and a very unsettled space
contrasting to other, sort of, professional fields
00:24:58.240 --> 00:25:02.720
or technological fields that have a long history
and are relatively settled from an oversight and
00:25:02.720 --> 00:25:08.400
regulatory standpoint? This one is not, and
for good reason. It is still developing.
00:25:08.400 --> 00:25:13.520
And I think, you know, there are certain
oversight and policy regimes that exist
00:25:13.520 --> 00:25:19.280
today that can be applied. Professor Charo
talked about this, as well, where, you know,
00:25:19.280 --> 00:25:24.400
maybe you have certain policy and oversight
regimes that, depending on how the application
00:25:24.400 --> 00:25:30.720
of that technology is applied, applies there
versus some horizontal, overarching regulatory
00:25:30.720 --> 00:25:34.360
sort of framework. And I think that applies from
an internal governance standpoint, as well.
00:25:34.360 --> 00:25:40.640
SULLIVAN: Yeah. It's a great point. So what isn't
being explored from genome editing that, you know,
00:25:40.640 --> 00:25:47.569
maybe we think could be useful to AI governance,
or as we think about the evolving frameworks …
00:25:47.569 --> 00:25:49.040
KLUTTZ: Yeah.
SULLIVAN: … what maybe we should be taking into
00:25:49.040 --> 00:25:52.160
account from what Professor
Charo shared with us?
00:25:52.160 --> 00:25:56.000
KLUTTZ: So one of the things I've thought
about and took from Professor Charo’s
00:25:56.000 --> 00:26:03.200
discussion was she had just this amazing way
of framing up how genome editing regulation
00:26:03.200 --> 00:26:06.320
is done. And she said, you know,
we don't regulate genome editing;
00:26:06.320 --> 00:26:11.520
we regulate the things that use genome editing.
And while it's not a one-to-one analogy with
00:26:11.520 --> 00:26:18.400
the AI space because we do have this sort of very
general model level distinction versus application
00:26:18.400 --> 00:26:23.040
layer and even platform layer distinctions,
I think it's fair to say, you know, we don't
00:26:23.040 --> 00:26:31.200
regulate AI applications writ large. We regulate
the things that use AI in a very similar way. And
00:26:31.200 --> 00:26:35.600
that's how we think of our internal policy and
oversight process at Microsoft, as well.
00:26:35.600 --> 00:26:42.640
And maybe there are things that we regulated
and oversaw internally at the first instance
00:26:42.640 --> 00:26:46.640
and the first time we saw it come through,
and it graduates into more of a programmatic
00:26:46.640 --> 00:26:53.040
framework for how we manage that. So one good
example of that is some of our higher-risk AI
00:26:53.040 --> 00:26:56.640
systems that we offer out of Azure at the
platform level. When I say that, I mean
00:26:56.640 --> 00:27:03.120
APIs that you call that developers can then build
their own applications on top of. We were really
00:27:03.120 --> 00:27:09.520
deep in evaluating and assessing mitigations on
those platform systems in the first instance,
00:27:09.520 --> 00:27:14.800
but we also graduated them into what we call
our Limited Access AI services
program.
00:27:14.800 --> 00:27:18.800
And some of the things that Professor Charo
discussed really resonated with me. You know,
00:27:18.800 --> 00:27:24.000
she had this moment where she was mentioning how,
you know, you want to know who's using your tools
00:27:24.000 --> 00:27:29.120
and how they're being used. And it's the same
concepts. We want to have trust in our customers,
00:27:29.120 --> 00:27:33.840
we want to understand their use cases, and we want
to apply technical controls that, sort of, force
00:27:33.840 --> 00:27:39.920
those use cases or give us signal post-deployment
that use cases are being done in a way that may
00:27:39.920 --> 00:27:45.120
give us some level of concern, to reach out
and understand what those use cases are.
00:27:45.120 --> 00:27:48.560
SULLIVAN: Yeah, you're hitting on a
great point. And I love this kind of
00:27:48.560 --> 00:27:53.760
layered approach that we're taking and
that Alta highlighted, as well. Maybe
00:27:53.760 --> 00:27:58.640
to double-click a little bit just on that
post-market control and what we're tracking,
00:27:59.200 --> 00:28:04.080
kind of, once things are out and being used
by our customers. How do we take some of
00:28:04.080 --> 00:28:09.440
that deployment data and bring it back in to
maybe even better inform upfront governance
00:28:09.440 --> 00:28:12.320
or just how we think about some of the
frameworks that we're operating in?
00:28:12.320 --> 00:28:17.040
KLUTTZ: It's a great question. The number one
thing is for us at Microsoft, we want to know
00:28:17.040 --> 00:28:21.280
the voice of our customer. We want our customers
to talk to us. We don't want to just understand
00:28:21.280 --> 00:28:26.400
telemetry and data. But it's really getting out
there and understanding from our customers and not
00:28:26.400 --> 00:28:31.120
just our customers. I would say our stakeholders
is maybe a better term because that includes
00:28:31.120 --> 00:28:35.520
civil society organizations. It includes
governments. It includes all of these non,
00:28:35.520 --> 00:28:41.360
sort of, customer actors that we care about
and that we're trying to sort of optimize for,
00:28:41.360 --> 00:28:47.600
as well. It includes end users of our enterprise
customers. If we can gather data about how our
00:28:47.600 --> 00:28:53.360
products are being used and trying to understand
maybe areas that we didn't foresee how customers
00:28:53.360 --> 00:28:57.760
or users might be using those things, and then
we can tune those systems to better align with
00:28:57.760 --> 00:29:04.520
what both customers and users want but also our
own AI principles and policies and programs.
00:29:04.520 --> 00:29:09.280
SULLIVAN: Daniel, before coming to
Microsoft, you led social science research
00:29:09.280 --> 00:29:14.160
and sociotechnical applications
of AI-driven tech at Berkeley.
00:29:14.160 --> 00:29:18.800
What do you think some of the biggest challenges
are in defining and maybe even just, kind of,
00:29:18.800 --> 00:29:23.680
measuring at, like, a societal level some
of the impacts of AI more broadly?
00:29:23.680 --> 00:29:29.520
KLUTTZ: Measuring social phenomenon is a
difficult thing. And one of the things that,
00:29:29.520 --> 00:29:35.840
as social scientists, you're very interested
in is scientifically observing and measuring
00:29:35.840 --> 00:29:40.480
social phenomena. Well, that sounds great.
It sounds also very high level and jargony.
00:29:40.480 --> 00:29:46.560
What do we mean by that? You know, it's
very easy to say that you're collecting
00:29:46.560 --> 00:29:52.800
data and you're measuring, I don't know, trust
in AI, right? That's a very fuzzy concept.
00:29:52.800 --> 00:29:54.000
SULLIVAN: Right. Definitely.
00:29:54.000 --> 00:29:57.840
KLUTTZ: It is a concept that we want to get
to, but we have to unpack that, and we have
00:29:57.840 --> 00:30:06.240
to develop what we call measurable constructs.
What are the things that we might observe that
00:30:06.240 --> 00:30:12.560
could give us an indication toward what is a very
fuzzy and general concept. And there's challenges
00:30:12.560 --> 00:30:17.120
with that everywhere. And I'm extremely fortunate
to work at Microsoft with some of the world's
00:30:17.120 --> 00:30:24.480
leading sociotechnical researchers and some of
these folks who are thinking about—you know,
00:30:24.480 --> 00:30:29.040
very steeped in measurement theory,
literally PhDs in these fields—how
00:30:29.680 --> 00:30:39.840
to both measure and allow for a scalable way to do
that at a place the size of Microsoft. And that is
00:30:39.840 --> 00:30:46.240
trying to develop frameworks that are scalable
and repeatable and put into our platform that
00:30:46.240 --> 00:30:51.920
then serves our product teams. Are we providing,
as a platform, a service to those product teams
00:30:51.920 --> 00:30:57.360
that they can plug in and do their automated
evaluations at scale as much as possible and then
00:30:57.360 --> 00:31:02.040
go back in over the top and do some of your more
qualitative targeted testing and evaluations.
00:31:02.040 --> 00:31:06.080
SULLIVAN: Yeah, makes a lot
of sense. Before we close out,
00:31:06.080 --> 00:31:09.600
if you're game for it, maybe we do a
quick lightning round. Just 30-second
00:31:09.600 --> 00:31:14.320
answers here. Favorite real-world
sensitive use case you've ever
00:31:14.320 --> 00:31:15.112
reviewed.-second answers here. Favorite real-world
sensitive use case you've ever reviewed.
00:31:15.112 --> 00:31:20.080
KLUTTZ: Oh gosh. Wow, this is where
I get to be the social scientist.
00:31:20.080 --> 00:31:20.761
SULLIVAN: [LAUGHS] Yes.
00:31:20.761 --> 00:31:25.320
KLUTTZ: It’s like, define favorite, Kathleen.
[LAUGHS] Most memorable, most
painful.
00:31:25.320 --> 00:31:26.760
SULLIVAN: Let's do most memorable.
00:31:26.760 --> 00:31:27.840
KLUTTZ: We’ll do most memorable.
00:31:27.840 --> 00:31:28.626
SULLIVAN: Yeah.
00:31:28.626 --> 00:31:33.920
KLUTTZ: You know, I would say the most memorable
project I worked on was when we rolled out the
00:31:33.920 --> 00:31:41.760
new Bing Chat, which is no longer called Bing
Chat, because that was the first really big
00:31:41.760 --> 00:31:50.560
cross-company effort to deploy GPT-4, which was,
you know, the next step up in AI innovation from
00:31:50.560 --> 00:31:55.920
our partners at OpenAI. And I really value working
hand in hand with engineering teams and with
00:31:55.920 --> 00:32:02.000
researchers and that was us at our best and really
sort of turbocharged the model that we have.
00:32:02.000 --> 00:32:04.880
SULLIVAN: Wonderful. What's one of the most
00:32:04.880 --> 00:32:09.977
overused phrases that you have in
your AI governance meetings?
00:32:09.977 --> 00:32:13.600
KLUTTZ: Gosh. [LAUGHS] If I hear “We need to get
aligned; we need to align on this more” …
00:32:13.600 --> 00:32:14.320
SULLIVAN: [LAUGHS] Right.
00:32:14.320 --> 00:32:16.800
KLUTTZ: But, you know, it's said
for a reason. And I think it sort
00:32:16.800 --> 00:32:19.760
of speaks to that clever nature.
That's one that comes to mind.
00:32:19.760 --> 00:32:23.920
SULLIVAN: That's great. And then maybe, maybe
last one. What are you most excited about in
00:32:23.920 --> 00:32:27.560
the next, I don't know, let's say three
months? This world is moving so fast!
00:32:27.560 --> 00:32:33.040
KLUTTZ: You know, the pace of innovation, as you
just said, is just staggering. It is unbelievable.
00:32:33.040 --> 00:32:38.880
And sometimes it can feel overwhelming in my
space. But what I am most excited about is how we
00:32:38.880 --> 00:32:44.640
are building up this Emerging … I mentioned this
Emerging Technologies program in my team as a,
00:32:44.640 --> 00:32:51.280
sort of, formal program is relatively new. And I
really enjoy being able to take a step back and
00:32:51.280 --> 00:32:56.400
think a little bit more about the future and a
little bit more holistically. And I love working
00:32:56.400 --> 00:33:01.360
with engineering teams and sort of strategic
visionaries who are thinking about what we're
00:33:01.360 --> 00:33:05.920
doing a year from now or five years from now, or
even 10 years from now, and I get to be a part
00:33:05.920 --> 00:33:11.520
of those conversations. And that really gives me
energy and helps me … helps keep me grounded and
00:33:11.520 --> 00:33:19.040
not just dealing with the day to day, and, you
know, various fire drills that you may run. It's
00:33:19.040 --> 00:33:23.600
thinking strategically and having that foresight
about what's to come. And it's exciting.
00:33:23.600 --> 00:33:28.240
SULLIVAN: Great. Well, Daniel, just thanks so
much for being here. I had such a wonderful
00:33:28.240 --> 00:33:32.880
discussion with you, and I think the
thoughtfulness in our discussion today
00:33:32.880 --> 00:33:37.280
I hope resonates with our listeners. And
again, thanks to Alta for setting the stage
00:33:37.280 --> 00:33:42.460
and sharing her really amazing, insightful
thoughts here, as well. So thank you.
00:33:42.460 --> 00:33:43.000
[MUSIC]
00:33:43.000 --> 00:33:45.760
KLUTTZ: Thank you, Kathleen. I
appreciate it. It's been fun.
00:33:45.760 --> 00:33:51.040
SULLIVAN: And to our listeners, thanks for
tuning in. You can find resources related to
00:33:51.040 --> 00:33:54.800
this podcast in the show notes. And
if you want to learn more about how
00:33:54.800 --> 00:34:00.640
Microsoft approaches AI governance,
you can visit microsoft.com/RAI.
00:34:00.640 --> 00:34:17.680
See you next time!
00:34:17.680 --> 00:34:18.531
[MUSIC FADES]