1
00:00:00,000 --> 00:00:03,920
ALICIA PATTERSON: So even if no harm comes
to a particular person. So, for example,
2
00:00:03,920 --> 00:00:09,680
even if I am not, say, negatively impacted
— maybe I'm never detained by the police or
3
00:00:09,680 --> 00:00:15,665
wrongly identified — we can still see that
something is seemingly gone wrong here.
4
00:00:15,665 --> 00:00:20,240
HOUSSAM ABBAS: Sometimes the effective function
of the use of AI can be to focus attention on the
5
00:00:20,240 --> 00:00:26,240
technology and away from how it's used by whom
and to what ends. The ethical considerations
6
00:00:26,240 --> 00:00:30,080
don't start at the point after you've deployed
the system and now you're seeing some results.
7
00:00:30,080 --> 00:00:34,800
The ethical considerations ought to come early on.
And so, no one can hide behind their hand and say,
8
00:00:34,800 --> 00:00:41,214
oh, I didn't see this coming.
[MUSIC: “The Ether Bunny,” by
9
00:00:41,214 --> 00:00:42,720
Eyes Closed Audio, licensed under CC by 3.0]
RACHEL ROBERTSON: From the College of Engineering
10
00:00:42,720 --> 00:00:49,200
at Oregon State University,
this is Engineering Out Loud.
11
00:00:49,200 --> 00:00:54,480
Hi everyone. I’m Rachel Robertson. Today, on
Engineering Out Loud you will hear more from
12
00:00:54,480 --> 00:01:00,560
Alicia Patterson and Houssam Abbas. They are both
assistant professors at Oregon State University
13
00:01:00,560 --> 00:01:07,040
studying ethics in AI, but from very different
perspectives. Alicia has her Ph.D. in philosophy
14
00:01:07,040 --> 00:01:13,680
and her research focuses on the ethics of emerging
technology, especially privacy and data ethics.
15
00:01:13,680 --> 00:01:19,520
Houssam has a Ph.D. in electrical engineering and
his research interests are in the verification and
16
00:01:19,520 --> 00:01:25,440
control of cyber-physical systems and formal
ethical theories for autonomous agents like
17
00:01:25,440 --> 00:01:31,840
unpiloted ground and aerial vehicles. This
recording is from a talk entitled “Where do
18
00:01:31,840 --> 00:01:37,360
Ethics Belong in Artificial Intelligence?” They
designed this talk for a general audience for a
19
00:01:37,360 --> 00:01:43,760
presentation at OMSI, the Oregon Museum of Science
and Industry. This recording is from a talk at
20
00:01:43,760 --> 00:01:49,760
Oregon State University. Now, enjoy a deep dive
into how philosophers and engineers are thinking
21
00:01:49,760 --> 00:01:52,960
about ethics in artificial intelligence.
ABBAS:
22
00:01:52,960 --> 00:02:00,240
Where do ethics belong in artificial intelligence?
So, start with a story. And the story begins in
23
00:02:00,240 --> 00:02:06,560
the 1950s in the psychiatric ward of Bellevue
Hospital in New York where Bill McPhee was
24
00:02:06,560 --> 00:02:13,840
committed. While committed there, he developed
his idea for a voter prediction machine. The,
25
00:02:13,840 --> 00:02:19,040
the point of it was, and here I quote from Jill
Lepore's book, “If Then: How the Simulmatics
26
00:02:19,040 --> 00:02:25,280
Corporation invented the Future.” So, the
point of it was to create, literally quote,
27
00:02:25,280 --> 00:02:32,720
“a set of rules of thinking so that it is possible
to reproduce or closely approximate the thinking
28
00:02:32,720 --> 00:02:38,880
behaviors of voters.” Bill McPhee was then
released from the psychiatric ward at Bellevue and
29
00:02:38,880 --> 00:02:45,120
finished his dissertation at Columbia University
about a program for, to create a fully observable
30
00:02:45,120 --> 00:02:51,280
electorate. You would feed the machine massive
amounts, massive amounts of microscopic data about
31
00:02:51,280 --> 00:02:57,120
users and issues, voters and issues. And it would
be possible to assess the effect of any proposed
32
00:02:57,120 --> 00:03:05,280
course of action by political candidate based on
the likely reactions of the voters. Ed Greenfield,
33
00:03:05,280 --> 00:03:10,000
who's was an ad man and was one of the key
founders of the Simulmatics Corporation which
34
00:03:10,000 --> 00:03:15,600
monetized the ideas of Bill McPhee and sold them.
He put it like this, he said, suppose that a
35
00:03:15,600 --> 00:03:19,680
candidate was proposing to make a major civil
rights speech in the South. Remember this was
36
00:03:19,680 --> 00:03:29,600
the early sixties. We would be able, quote,
“through our model to predict what the meaning
37
00:03:29,600 --> 00:03:35,280
of that speech to over a thousand subgroups of
a given population. And thus, we'll be able to
38
00:03:35,280 --> 00:03:41,520
predict any the smallest percentage difference
in votes, that is, that such a speech would
39
00:03:41,520 --> 00:03:47,840
have.” The Simulmatics Corporation actually sold
their services to the Kennedy campaign. Kennedy
40
00:03:47,840 --> 00:03:53,200
then went on to win the presidential election that
year. Now, it's not clear how much influence they
41
00:03:53,200 --> 00:03:57,920
actually had, because the recommendations that
Simulmatics gave the campaign were largely seen
42
00:03:57,920 --> 00:04:04,800
to be common sense and would have been adopted
anyway. But I think you can already see the how
43
00:04:04,800 --> 00:04:14,960
that sort of foreshadows the general theme of
users/voters/people prediction and then control.
44
00:04:14,960 --> 00:04:19,840
The Cambridge Analytica — they basically sold a
more sophisticated version of what Simulmatics
45
00:04:19,840 --> 00:04:24,080
was trying to sell the Kennedy campaign to the
Trump campaign in 2016 and to the Brexit Leave
46
00:04:24,080 --> 00:04:32,000
campaign in 2015. And the reason I, I start with
this anecdote is to directly cut through to what
47
00:04:32,000 --> 00:04:36,960
perhaps is the biggest point of unease that
some people might feel when they fret about
48
00:04:36,960 --> 00:04:42,240
AI and ethics. The notion that human behavior
is predictable and then controllable. So,
49
00:04:42,240 --> 00:04:47,760
in particular, most of you have probably heard
of the Facebook experiment on playing basically
50
00:04:47,760 --> 00:04:53,440
and controlling user's emotions based
on what they bubble up in their feeds.
51
00:04:54,640 --> 00:05:04,160
Now, if users or people are predictable, at
least in the aggregate and controllable, then
52
00:05:04,160 --> 00:05:09,280
why not program them into robots? And that then
brings us to the notion of embodied intelligence,
53
00:05:09,280 --> 00:05:16,880
embodied artificial intelligence. One half of this
is a humanoid of its own inventor. I'll give you
54
00:05:16,880 --> 00:05:21,600
a couple of seconds to try to guess, which is the
humanoid and which is the flesh and blood human,
55
00:05:21,600 --> 00:05:26,400
right. This is the Geminiod DK by a
Danish inventor whose name I now forget,
56
00:05:26,400 --> 00:05:32,720
oh, it's in my notes. Henrik Scharf ,
right? And so, this reminds me of a famous,
57
00:05:32,720 --> 00:05:37,440
which I had on the slides that I removed of a
famous icon of Christ from the sixth century,
58
00:05:37,440 --> 00:05:43,520
which is supposed to portray the dual nature of
Christ as man and God. It sounds our ambitions
59
00:05:43,520 --> 00:05:52,800
have now reduced to becoming man and machine.
So, so hopefully this gives you sort of the
60
00:05:52,800 --> 00:05:58,640
corner cases or some of the corner cases when
discussing AI and ethics in, in the way that
61
00:05:58,640 --> 00:06:03,680
they relate mostly to us, like the notion that we
might be predictable and controllable. If you want
62
00:06:03,680 --> 00:06:07,920
to know what researchers are actually working
on you can literally pick up like any issue
63
00:06:07,920 --> 00:06:11,280
of the communications of the ACM from the last
couple of years. And there will be an article
64
00:06:11,280 --> 00:06:18,000
in there about AI and ethics, not ethics and CS in
general, but specifically AI from bias and credit
65
00:06:18,000 --> 00:06:23,840
rating to problems with certain problems with
computer vision systems, and so on and so forth.
66
00:06:23,840 --> 00:06:32,720
So rather than give a laundry list of that, I'll
hand it over to Alicia to help us think through
67
00:06:32,720 --> 00:06:39,200
a framework about these problems. Starting with a
cautionary note that ethics is the hottest product
68
00:06:39,200 --> 00:06:44,640
in Silicon Valley's hype cycle today. So, in order
to step away from the hype and perhaps think about
69
00:06:44,640 --> 00:06:47,760
it more productively: Alicia.
PATTERSON:
70
00:06:47,760 --> 00:06:54,720
Okay, so philosophers have a set of tools to think
about ethical problems that has been our bread and
71
00:06:54,720 --> 00:07:02,320
butter for forever, basically . And so,
we think maybe perhaps a little bit differently
72
00:07:02,320 --> 00:07:09,280
about how we spot ethical problems. So, one of
the key things, key sort of concepts we have
73
00:07:09,280 --> 00:07:14,880
in philosophy are the concepts of a harm and
a wrong. And we differentiate these because
74
00:07:14,880 --> 00:07:19,760
they sort of mean different things or pick
out different types of ethical problems. So,
75
00:07:19,760 --> 00:07:25,680
a harm is something that sets back your material
interests, it makes you worse off, right? We
76
00:07:25,680 --> 00:07:32,320
might think of like suffering or something like
that. So, if someone steps on your foot, right,
77
00:07:32,320 --> 00:07:37,120
we can say that they've harmed you because your
foot now hurts and maybe it's swollen. A wrong
78
00:07:37,120 --> 00:07:46,000
in contrast is a, a sort of a violation of rights
or some sort of disrespect of an important value
79
00:07:46,000 --> 00:07:53,280
or denying someone something they're owed, right?
So, if I say deny, deny someone the right to vote
80
00:07:53,280 --> 00:07:58,400
I've disrespected an important value and I've
denied something, they've that they've, they're
81
00:07:58,400 --> 00:08:06,080
owed, right? As a citizen, right? They're owed the
right to vote. Perhaps you know, if the person is
82
00:08:06,080 --> 00:08:10,400
in like, you know, not a swing state, their
vote maybe doesn't make much of a difference.
83
00:08:10,400 --> 00:08:15,360
So maybe I didn't harm them very much, but we can
still see that I've done something wrong to them,
84
00:08:15,360 --> 00:08:21,760
right? I've denied them something that they're
owed. So, harms and wrongs usually go together,
85
00:08:21,760 --> 00:08:26,560
but it's important to understand sort of their
different contours to be able to fully map
86
00:08:27,200 --> 00:08:36,560
ethical problems. So, some AI harms this when
we originally made this, this wasn't such a
87
00:08:36,560 --> 00:08:43,600
big thing in the news, but Cigna, a health
insurance company uses algorithms, right?
88
00:08:43,600 --> 00:08:50,320
To sort of batch deny claims. And doctors
are supposed to go through and, and review
89
00:08:50,320 --> 00:08:57,120
a patient's file, right? And see if the sort of
medical claim is medically a medical necessity.
90
00:08:57,680 --> 00:09:04,720
And according to research by ProPublica doctors
were spending an average of 1.2 seconds on each
91
00:09:04,720 --> 00:09:11,600
case. I don't know about you, but I, I feel like
I could speed read a little bit, but 1.2 seconds
92
00:09:11,600 --> 00:09:18,880
on a very serious medical decision seems that
seems pretty bad, right? So, in this instance,
93
00:09:18,880 --> 00:09:25,760
we get an algorithm, right, that caused a
particular kind of harm, right? It may probably
94
00:09:25,760 --> 00:09:32,480
denied thousands of people certain procedures.
That means patients, doctors, hospital staff have
95
00:09:32,480 --> 00:09:39,360
to do all of this extra labor to get the medical
care that they deserve. And some people just might
96
00:09:39,360 --> 00:09:45,680
not be able to like fully work the system and then
give up and have to pay for things out of pocket.
97
00:09:45,680 --> 00:09:52,160
So here we can see the way in which the
system made people worse off, it harmed them.
98
00:09:53,360 --> 00:10:03,920
In contrast a wrong may not necessarily
involve a concrete harm, right? So,
99
00:10:03,920 --> 00:10:11,040
an example of this is the use of facial
recognition by law enforcement. One of the,
100
00:10:11,040 --> 00:10:16,000
sort of, wrongs that we see is that people, a
lot of times in AI systems are treated sort of
101
00:10:16,000 --> 00:10:22,640
as near data in large data sets. And data is
scraped in mass from the internet, right? This
102
00:10:22,640 --> 00:10:29,280
is like all known to us. And there's no sort of
regard for sort of fundamental values about how
103
00:10:29,280 --> 00:10:36,400
these people might want to be treated, right? In
this instance, the value of privacy. So, this is
104
00:10:36,400 --> 00:10:41,840
a really interesting read, if you wanna check it
out put together by researchers at Georgetown. So,
105
00:10:41,840 --> 00:10:48,880
research at Georgetown found that one in two
Americans have their faces and data sets that
106
00:10:48,880 --> 00:10:54,320
are available to the police. And this is the
exact kind of thing that we should worry about,
107
00:10:54,320 --> 00:11:01,840
right? That we're sort of infringing on important
values like democratic freedoms or privacy.
108
00:11:01,840 --> 00:11:06,320
So even if no harm comes to a particular
person, so for example, even if I am not,
109
00:11:06,320 --> 00:11:14,320
say negatively impacted, maybe I'm never detained
by the police or wrongly identified we can still
110
00:11:14,320 --> 00:11:24,080
see that something is seemingly gone wrong here.
So, within the sort of scope of wrongs, injustice
111
00:11:24,080 --> 00:11:28,560
usually sits within it because one of the things
we want to pick out about injustice is that it's
112
00:11:28,560 --> 00:11:36,480
a certain form of disrespect. So, when someone's
committed an injustice, what's been … what value
113
00:11:36,480 --> 00:11:42,080
or what principle has been violated is the value
or principle of fairness or equality. Oftentimes,
114
00:11:42,080 --> 00:11:49,360
injustice involves both a harm and a wrong, right?
So, you're treated unfairly and your life is made
115
00:11:49,360 --> 00:11:54,560
worse. So, if you say, are denied a promotion
on the, say, the basis of your race or gender,
116
00:11:54,560 --> 00:11:59,200
we can say like, you're worse off. You don't have
the income that you would've had, right? The,
117
00:11:59,200 --> 00:12:03,440
the career opportunities you would've
had, but also, you've been disrespected,
118
00:12:03,440 --> 00:12:10,400
right? You've been treated in a way that you
shouldn't have. So those are some ways actually,
119
00:12:10,400 --> 00:12:15,040
lemme give an example of injustice first.
So, an example of injustice, there's too many
120
00:12:15,040 --> 00:12:22,560
to really even pick which one to talk about. But
one example of automated bias, right? Is in racial
121
00:12:22,560 --> 00:12:29,360
disparities in credit lending study by faculty
in Stanford and University of Chicago looked into
122
00:12:29,360 --> 00:12:35,840
this into sort of alternative models of credit
scoring and found that low income and minority
123
00:12:35,840 --> 00:12:43,280
groups were somewhere between five and 10% less,
less likely to receive a loan than high income or
124
00:12:43,280 --> 00:12:50,080
non-minority groups. So here we can see, right?
Being denied the alone on the basis of your race
125
00:12:50,080 --> 00:12:55,760
is both wrong and you're harmed, right? You're,
you don't get the house and you're treated in a,
126
00:12:55,760 --> 00:13:02,080
in a way that you should not have been treated.
So, this is sort of the background when we think
127
00:13:02,080 --> 00:13:09,040
about where is ethics coming into conversations
in, in engineering and design, this is sort of
128
00:13:09,040 --> 00:13:18,240
my like, toy example of the like design lifecycle.
Oftentimes, these sort of questions about ethics,
129
00:13:18,240 --> 00:13:22,720
about harms and wrongs and things like that
come in after a product has been designed,
130
00:13:22,720 --> 00:13:28,240
their research has been done in it for years,
and it's either launched or about to be launched,
131
00:13:28,240 --> 00:13:34,640
and some problems are surfacing and people ask
questions about you know, what sort of guardrail
132
00:13:34,640 --> 00:13:40,880
should be in place? And one of the problems with
this is that it's sort of too little too late,
133
00:13:40,880 --> 00:13:46,320
right? When we ask questions about ethics,
we should ask them from the very beginning,
134
00:13:46,320 --> 00:13:51,760
right? Because asking them from the beginning
allows us to ask a series of questions about
135
00:13:51,760 --> 00:13:56,400
values through the entire design lifecycle,
through the entire research process, right?
136
00:13:57,840 --> 00:14:03,520
And that allows us to ask a broader set of
questions, rather than this narrow question of
137
00:14:03,520 --> 00:14:09,120
what sort of guardrail should we put in place?
Or what sort of use cases should this be
138
00:14:09,120 --> 00:14:16,240
prohibited or encouraged for? So, a variety of
questions that we should ask are things like,
139
00:14:16,240 --> 00:14:20,560
before we even begin to develop this, is
this technology the right solution to the
140
00:14:20,560 --> 00:14:26,160
problem at all? Science and technology scholars
critique this question not being asked a lot,
141
00:14:26,160 --> 00:14:31,520
and they call this techno solutionism, the idea
that a technology can be sort of the solution
142
00:14:31,520 --> 00:14:36,640
for these big social problems that we're often
not sitting back and saying, is this the right,
143
00:14:36,640 --> 00:14:41,680
is this the right way to solve this problem?
The next question that needs to be asked,
144
00:14:41,680 --> 00:14:44,960
and this is, I'll say, I'm going through a
series of questions, but this is by no means
145
00:14:44,960 --> 00:14:51,920
an exhaustive list. Does the community want the
problem solved in this way? Whose input do we
146
00:14:51,920 --> 00:15:00,240
even have in thinking about what the problem is?
So, in design justice sort of like a canon text in
147
00:15:01,440 --> 00:15:08,560
in sort of intersection between ethics and design,
people have this phrase of nothing for us without
148
00:15:08,560 --> 00:15:13,200
us, right? So, do people who are impacted by the
technology have any sort of meaningful ways to
149
00:15:13,200 --> 00:15:22,160
shape the values in design of it? The second or
third, I guess we're on third we should foresee
150
00:15:22,160 --> 00:15:27,600
the, we should look into the potential unforeseen
consequences again, before we've like created a
151
00:15:27,600 --> 00:15:33,520
product or we've put a ton of research into a
certain type of algorithm, right? So feminist
152
00:15:33,520 --> 00:15:38,160
technology scholars, for example, often point
out that like the first public adoption of a
153
00:15:38,160 --> 00:15:44,800
technology usually comes in the form of harm for
women. So, one example, there's many I could give
154
00:15:44,800 --> 00:15:51,120
you, is that when DeepFakes were first, like,
open source available to people on the internet,
155
00:15:51,120 --> 00:15:56,960
they were used in mass for non-consensual
pornography or image-based sexual abuse.
156
00:15:56,960 --> 00:16:01,600
So, here we can see like being the subject
of pornography is something that women are
157
00:16:01,600 --> 00:16:06,080
significantly more likely to be subject to.
And so, we might have wanted the developers
158
00:16:06,080 --> 00:16:11,600
to have thought about that or grapple with that
before they released this technology, right? So,
159
00:16:11,600 --> 00:16:17,840
these are the sorts of things that maybe we can
solve for, prevent for early on. We also might
160
00:16:17,840 --> 00:16:22,800
wanna ask, is this technology compatible with
important social goods, like democratic values,
161
00:16:22,800 --> 00:16:29,120
right? When we are creating something that say,
involves a form of surveillance we might wanna
162
00:16:29,120 --> 00:16:35,360
understand, will this significantly impact, say,
people's ability to freely assemble. Freedom of
163
00:16:35,360 --> 00:16:40,800
assembly is considered say, a core democratic
value of value that you sort of a thing,
164
00:16:40,800 --> 00:16:46,160
a right that you need to be able to express to
be able to exist in a democracy at all. We also
165
00:16:46,160 --> 00:16:51,440
might wanna ask, are we getting data in a way that
respects important rights? As I mentioned before,
166
00:16:51,440 --> 00:16:57,680
like a common practice and machine learning.
It's just to do this mass web scraping. And we
167
00:16:57,680 --> 00:17:01,920
might wanna ask a variety of questions about
that. Like, are we respecting people's right
168
00:17:01,920 --> 00:17:09,760
to say not participate in oppressive systems?
Are we re respecting their rights to privacy?
169
00:17:10,960 --> 00:17:14,800
We might also wanna ask our, some people
going to be more harmed by this than others,
170
00:17:14,800 --> 00:17:18,880
right? Maybe this question comes up the most when
we're thinking about automating certain forms of
171
00:17:18,880 --> 00:17:26,560
labor. Who are the winners and losers? And who
might be especially vulnerable to the impacts of
172
00:17:26,560 --> 00:17:34,880
a system? We wanna ask questions about bias before
a product is launched. We wanna be able to grapple
173
00:17:34,880 --> 00:17:41,120
with that. We don't want something. So as so often
happens something gets launched and then users
174
00:17:41,120 --> 00:17:46,080
who are negatively affected, who are of that group
that the technology is biased against are the ones
175
00:17:46,080 --> 00:17:49,840
being like, this doesn't work for us. This has
happened again and again, right? So that's again,
176
00:17:49,840 --> 00:17:56,640
something that we wanna catch early and often.
We wanna ask what are the potential misuses,
177
00:17:56,640 --> 00:18:04,640
for example, have we created a tool that's like
really amazing, but also enables a lot of fraud?
178
00:18:04,640 --> 00:18:09,680
And lastly then we can come to these questions
about, say, guardrails. What sort of laws,
179
00:18:09,680 --> 00:18:15,600
regulation policies and norms need to be put
in place to protect people or, or to ensure
180
00:18:15,600 --> 00:18:18,760
this sort of ethical use?
ABBAS:
181
00:18:19,920 --> 00:18:28,240
Thank you. So, within that framework or against
the background of that framework, we can look
182
00:18:28,240 --> 00:18:36,000
at two major approaches so far in engineering to
tackle the question of how does one in a nutshell
183
00:18:36,000 --> 00:18:42,080
program certain ethical norms or social norms
more generally in engineering systems. This is
184
00:18:42,080 --> 00:18:48,240
a very high-level simplified view of that. One is
utilitarianism, the other is non-utilitarianism,
185
00:18:48,240 --> 00:18:56,000
or as sometimes referred to as deontology. I'll
start with Deontology from the Greek they own duty
186
00:18:56,000 --> 00:19:02,480
and laws for science or knowledge of. And whereas
most of us probably think of the right thing to do
187
00:19:02,480 --> 00:19:08,800
is the think that maximizes certain benefits,
maximizes certain goods. Deontology says, no,
188
00:19:08,800 --> 00:19:16,160
the way you assess whether some, there are certain
moral norms that the moral agent ought to obey,
189
00:19:16,160 --> 00:19:20,080
regardless of the consequences, regardless of
the amount of good that that action produces,
190
00:19:20,080 --> 00:19:25,440
or the amount of bad that that action leads to.
So, the deontology is a normative theory regarding
191
00:19:25,440 --> 00:19:31,440
which choices are morally required, forbidden,
or permitted. Those are three key categories. In
192
00:19:31,440 --> 00:19:36,240
contrast, for example, to moral theories, which
tell you what sort of person you ought to be,
193
00:19:36,240 --> 00:19:41,040
right? What makes a choice, right,
is its conformity with a moral norm.
194
00:19:41,840 --> 00:19:49,040
So, if we adopt such an approach, and that's sort
of the perspective I take in my own research on
195
00:19:49,040 --> 00:19:55,040
algorithmic ethics. One then asks, how do we
program duties into an AI agent, right? So,
196
00:19:55,040 --> 00:20:00,400
if you have statements of which choices are
morally required, forbidden or permitted, um, can
197
00:20:00,400 --> 00:20:06,080
we think of them as design requirements, uh, for a
subsystem of this overall system that we're trying
198
00:20:06,080 --> 00:20:11,520
to create? So, you have your objective function,
right? And this is the only math we're going to
199
00:20:11,520 --> 00:20:16,320
see. I think you have your objective function,
you want, uh, something to run fast, for example.
200
00:20:16,320 --> 00:20:22,240
And then you have those things that are permitted,
forbidden, and required, and you express those in
201
00:20:22,240 --> 00:20:27,680
some mathematical formalism. And the idea is that
once you've done, you've used these mathematical
202
00:20:27,680 --> 00:20:32,000
formalisms to express these two different things.
And that's sort of the point I wanna stress here.
203
00:20:32,000 --> 00:20:38,960
That object, the performance objectives are not
the normative constraints. Magic, magic, magic,
204
00:20:38,960 --> 00:20:43,920
it turns into zeros and ones, and then lo and
behold, Cassie can run a 5K without kicking a
205
00:20:43,920 --> 00:20:49,360
rabbit on the way, right? So, what is this
mathematical formalism that allows us to
206
00:20:49,360 --> 00:20:55,520
express normative requirements? This, I think
most of us are or ought to be familiar with,
207
00:20:55,520 --> 00:20:59,920
we need basically a mathematical language that
allows us to express such normative statements,
208
00:20:59,920 --> 00:21:04,400
which you and I can understand because
understanding is actually key here, right?
209
00:21:04,960 --> 00:21:09,840
An objective function with a thousand parameters
is not something you and I understand or anybody,
210
00:21:09,840 --> 00:21:14,640
and whoever tells you that is probably fooling
themselves and you. So, we want something that you
211
00:21:14,640 --> 00:21:18,400
and I can understand, but which is also amenable
to algorithmic, which is to say automatic,
212
00:21:18,400 --> 00:21:25,280
automatic or automated processing and reasoning.
And the, and the language most of us use is
213
00:21:25,280 --> 00:21:29,200
formal logic. You're all familiar with Boolean
logic, but there's a zoo of logics out there,
214
00:21:29,200 --> 00:21:34,080
some of which are appropriate or to some degree
appropriate for reasoning about normative
215
00:21:34,080 --> 00:21:40,400
statements. Once you do that, once you make
that move, then reasoning becomes a calculation,
216
00:21:40,400 --> 00:21:46,400
right? The dream of Leibniz was that when there
are disputes among persons, we can simply say, let
217
00:21:46,400 --> 00:21:52,320
us calculate, right? You, you run your argument,
I run my argument, and the machine tells us, well,
218
00:21:52,320 --> 00:21:57,440
which argument is a sound argument, right? Does
A follow from B? And so, without further ado,
219
00:21:57,440 --> 00:22:03,920
we can see who is right. We're nowhere near that,
obviously. But that, that's the overall idea. Now,
220
00:22:03,920 --> 00:22:09,680
formal methods and formal logic are heavily
used for, um, safety purposes, for designing
221
00:22:09,680 --> 00:22:15,920
safe systems, especially in safety critical
applications like avionics, for example, right?
222
00:22:16,800 --> 00:22:21,360
And in fact, in hardware, any hardware company
today makes heavy use of formal logic. But what
223
00:22:21,360 --> 00:22:26,800
we are talking about is using formal logics
in order to formalize normative requirements,
224
00:22:26,800 --> 00:22:31,280
ethical and social requirements. There's some work
I've done. There's work done by Matthias Scheutz
225
00:22:31,280 --> 00:22:40,240
at Tufts. Next week's AI seminar. Speaker Adnan
Darwiche also makes use heavy use of formal logic
226
00:22:40,240 --> 00:22:44,960
for explainable AI in his case. And I think
the center of gravity really of, of this sort
227
00:22:44,960 --> 00:22:49,840
of work still is in Europe especially
in Vienna. So there, if you're curious,
228
00:22:49,840 --> 00:22:54,320
you can look up that project that just launched
centered in Vienna, but with collaboration with
229
00:22:54,320 --> 00:23:00,080
Maryland. And the point one of the last thing
I wanna say here is that this formalism allows
230
00:23:00,080 --> 00:23:04,400
you to express and reason about normative
requirements. It doesn't prevent you from doing
231
00:23:04,400 --> 00:23:09,040
performance optimization the way we usually do.
The question is how do you do the two together?
232
00:23:09,040 --> 00:23:16,960
So, you know, you can still look at graphs,
. So finally, it's good not to drink
233
00:23:16,960 --> 00:23:22,240
one's own Kool-Aid. Are there really rules
that govern our thinking in general? Yes and
234
00:23:22,240 --> 00:23:27,840
no. You certainly cannot formalize all the
ways in which we actually think. Thinking
235
00:23:27,840 --> 00:23:32,400
is not always what we call rational thinking.
It is still different kind of thinking. So,
236
00:23:33,040 --> 00:23:42,080
you know, the other form is utilitarianism.
The other the other way of trying to program
237
00:23:42,960 --> 00:23:48,000
normative requirements is through utilitarianism,
where you have, you know, the good is a function
238
00:23:48,000 --> 00:23:52,880
of your actions. You have lots of optimizers
that you throw at this mysterious function,
239
00:23:52,880 --> 00:23:57,360
and it gives you the best action, which is
supposedly the most ethical action because it
240
00:23:57,360 --> 00:24:07,280
produces the most amount of good, right? It's,
it goes only so far in my view, because there,
241
00:24:07,280 --> 00:24:13,920
those are on some level inherently incommensurate
things, right? So even, let's take a very trivial
242
00:24:13,920 --> 00:24:19,360
example from this already old 2015 paper from
Chris Gerdes, right? So, say we want, you
243
00:24:19,360 --> 00:24:23,520
want the car to get to its destination, that's a
performance objective without collisions. That's
244
00:24:23,520 --> 00:24:28,480
a normative requirement. 'cause You, you know,
a car can collide with someone and keep going,
245
00:24:28,480 --> 00:24:32,400
right? So, it's not necessarily a performance
requirement while minimizing emissions,
246
00:24:32,400 --> 00:24:36,320
that's both performance and normative because
emissions are bad for people in general.
247
00:24:36,320 --> 00:24:40,640
So, what does it mean to weigh all these? Like
how do you weigh these in meters? What sort of
248
00:24:40,640 --> 00:24:43,440
objective function are you gonna put all
of these in? What sort of reward function
249
00:24:43,440 --> 00:24:48,800
are you gonna use in order to express all of
these? Not to mention that sometimes ethical
250
00:24:48,800 --> 00:24:52,960
requirements is not something that you can choose
more or less of something, you just have to do it,
251
00:24:52,960 --> 00:24:58,960
right? And so then you go into the multi-agent
case, which complicates the picture. Nonetheless,
252
00:24:58,960 --> 00:25:08,320
there is in fact the AI alignment literature.
Most of it is on the side of, let's see, how is it
253
00:25:08,320 --> 00:25:13,520
possible to design, for example, a reward function
for an RL agent such that it avoids catastrophic
254
00:25:13,520 --> 00:25:19,280
outcomes? Among that literature is reinforcement
learning from human feedback, right? Where the
255
00:25:19,280 --> 00:25:27,200
idea is suppose, you know, you want, for example,
chat GPT to produce the ethically normed answers.
256
00:25:27,200 --> 00:25:30,880
So what we do is we'll ask check GPT the same
question 10 times. It's a stochastic machine,
257
00:25:30,880 --> 00:25:34,320
it'll give us slightly different answers 10 times,
and then we give it feedback on which one is the
258
00:25:34,320 --> 00:25:40,160
best. And this way it tweaks its own parameters.
But, now that you might say, okay, sounds
259
00:25:40,160 --> 00:25:45,360
reasonable, but then it's, it's a market economy.
So, we'll say, oh, are we gonna really gonna hire
260
00:25:45,360 --> 00:25:50,720
a thousand people to do this a thousand times?
No, what do we do? We put another AI in there.
261
00:25:50,720 --> 00:25:57,560
So now you have an AI guiding another AI as, so
then you climb the AI ladder to nowhere .
262
00:25:59,440 --> 00:26:05,440
So, I'm not gonna pretend there's a good
segue, we're just switching gears. ,
263
00:26:05,440 --> 00:26:12,560
right? The problem of responsibility in autonomous
systems. There's two models of responsibility,
264
00:26:12,560 --> 00:26:18,240
fully autonomous systems, which I'm, I'm not going
to talk about because in my mind, there's no sense
265
00:26:18,240 --> 00:26:23,280
in which a fully autonomous machine is responsible
for something when there's a human behind it.
266
00:26:24,000 --> 00:26:28,560
So, we'll talk about . So, I just like
swiped a whole bunch of things on the side.
267
00:26:28,560 --> 00:26:33,280
We'll talk about semi-autonomous systems, okay?
Workers in the loop. Usually, humans in the loop
268
00:26:33,280 --> 00:26:37,040
is the terminology, but as Alicia was pointing out
this morning, this is actually creating whole new
269
00:26:37,040 --> 00:26:41,680
category of workers who have to watch terrible
videos all day and so on and so forth. Now you
270
00:26:41,680 --> 00:26:45,920
can have a worker in the loop for technical
purposes, like, you know, radiologists as
271
00:26:45,920 --> 00:26:50,400
checking the AI radiology algorithms, and you
can have them for legal and moral purposes.
272
00:26:50,400 --> 00:26:54,000
So even if this thing is great, you
still need for legal moral purposes,
273
00:26:54,000 --> 00:26:59,440
for example, a lawyer in the loop, a lawyer
in the war room, and I'll take an example
274
00:26:59,440 --> 00:27:05,200
from the Israeli Army's use of AI for lethal
targeting. So, this is from March 2023,
275
00:27:05,200 --> 00:27:13,200
so before the the latest war. So, so in from this
article, so several high ranking IDF officers
276
00:27:13,200 --> 00:27:19,520
informed the press that Israel is deploying AI
tools as part of its military arsenal. Skip, skip,
277
00:27:19,520 --> 00:27:24,080
skip. The public statements acknowledge that some
of the targets attacked by the IDF are produced
278
00:27:24,080 --> 00:27:29,920
by AI tools. Skip, skip. One encouraging aspect
is that it seems that the, the IDF seeks to use
279
00:27:29,920 --> 00:27:34,240
tools that complement human decision making rather
than a substitute for the human factor. So here,
280
00:27:34,240 --> 00:27:40,320
this is a human in the loop or a soldier
basically in the loop acting as a backstop to
281
00:27:40,320 --> 00:27:48,000
the decision or to to check the decision or the
recommendation of the AI tool. The question is,
282
00:27:48,000 --> 00:27:53,920
how likely is it that this person or this soldier
is actually doing due diligence? We saw earlier
283
00:27:53,920 --> 00:28:01,040
for the insurance claim that basically doctors are
spending 1.2 seconds checking an insurance claim.
284
00:28:01,040 --> 00:28:06,960
So, this is from the Guardian. And the point here
I'm making is that having a human in the loop,
285
00:28:06,960 --> 00:28:12,320
a worker in the loop is no panacea because that
human is subject to the same social forces that
286
00:28:12,320 --> 00:28:16,240
led to the creation of that AI system in the
first place. And that AI system has a purpose,
287
00:28:16,240 --> 00:28:20,720
and the purpose is to be automated. It is not
to be slowed down by, in this case, the human.
288
00:28:21,280 --> 00:28:28,480
So this is from a Guardian article in April, 2024
sourced on like six sources who worked with the
289
00:28:28,480 --> 00:28:33,600
Lavender system that the IDF was using in Gaza.
“We were constantly being pressured, bring us more
290
00:28:33,600 --> 00:28:39,600
targets. They really shouted at us. We were told,
now we have to f-up Hamas no matter what the cost.
291
00:28:39,600 --> 00:28:45,840
At its peak, the system managed to generate 37,000
people as potential human targets. But the numbers
292
00:28:45,840 --> 00:28:50,720
changed all the time because it depends on where
you set the bar and what a Hamas operative is.”
293
00:28:50,720 --> 00:28:55,040
This is a system with many parameters. It has many
knobs. So, depending on where you set the knobs,
294
00:28:55,040 --> 00:28:58,960
which is something as engineers we always do,
right? We're tweaking parameters for performance
295
00:28:58,960 --> 00:29:06,640
in this case for lethal targeting. So, the numbers
change all the time. So, guess in which direction
296
00:29:06,640 --> 00:29:12,640
the numbers are going to be trending in such
a situation. Now, who's going to vet 37,000
297
00:29:12,640 --> 00:29:19,520
targets or presumed targets. Skip to December 24.
This is from a Washington Post story. Soldiers
298
00:29:19,520 --> 00:29:23,360
who are poorly trained in using the technology
attacked human targets without corroborating
299
00:29:23,360 --> 00:29:27,280
Lavender's predictions at all. Lavender is
the name of the system. At certain times the
300
00:29:27,280 --> 00:29:32,640
only corroboration required was that the target
was a male. So, the point here is that if the,
301
00:29:32,640 --> 00:29:39,360
the worker in the loop, the soldier in the loop
and depending on how the, the scale of which the
302
00:29:39,360 --> 00:29:46,000
AI is being deployed is basically useless.
Even if that worker in the loop is a lawyer,
303
00:29:46,000 --> 00:29:52,160
there is a whole literature on so-called
war lawyers, right? Some legal experts
304
00:29:52,160 --> 00:29:55,760
skip, skip harness the law in the service of
the war effort. Now, the point being made in
305
00:29:55,760 --> 00:30:00,640
this book is not simply that the lawyers are
like a cog in the machine just basically giving
306
00:30:00,640 --> 00:30:04,800
blank checks to whoever wants to target. It's a
more nuanced point than that. But the point is,
307
00:30:04,800 --> 00:30:09,520
it's a system as a whole that exists. And too, so
to go back to Alicia's point earlier, the ethical
308
00:30:09,520 --> 00:30:13,360
considerations don't start at the point after
you've deployed the system and now you're seeing
309
00:30:13,360 --> 00:30:18,480
some results. The ethical considerations ought to
come early on. And so no one can hide behind their
310
00:30:18,480 --> 00:30:26,720
hand and say, oh, I didn't see this coming.
So, and I owe my thinking on this point to
311
00:30:26,720 --> 00:30:31,440
Zhanpei Fang who's here in the audience, AI is a
rhetorical tool. Sometimes the effective function
312
00:30:31,440 --> 00:30:37,680
of the use of AI can be to focus attention on
the technology and away from how it's used by
313
00:30:37,680 --> 00:30:43,840
whom and to what ends. It's like, oh, is the AI
parameter, are the AI parameters correctly set?
314
00:30:43,840 --> 00:30:49,040
Like, no, why is it being used? And this
isn't a question one asks only about AI,
315
00:30:49,040 --> 00:30:53,040
one can ask it about just technology in
general, right? But we're discussing AI here.
316
00:30:54,880 --> 00:30:56,320
Thank you.
PATTERSON:
317
00:30:56,320 --> 00:31:05,520
Okay, so continuing on this problem of
responsibility in AI systems. How many
318
00:31:05,520 --> 00:31:10,240
of you familiar with the concept of a crumple
zone? Excellent. This is so much different than
319
00:31:10,240 --> 00:31:15,360
a public talk . So right, the crumple zone,
lots of engineers, right? So, the crumple zone,
320
00:31:15,360 --> 00:31:21,840
right, we know is meant to crumple so that
the safety cage or where the passengers are,
321
00:31:21,840 --> 00:31:27,840
is protected from the force of a collision,
right? That's the point. The kinetic energy,
322
00:31:27,840 --> 00:31:34,880
right? Is transformative as like controlled
crumpling and it preserves the integrity.
323
00:31:36,320 --> 00:31:43,440
Well, Madeline Clare Elish argues that there's
a similar thing that happens for workers in the
324
00:31:43,440 --> 00:31:50,720
loop or humans in the loop where they are sort
of the thing that's supposed to crumple in order
325
00:31:50,720 --> 00:31:56,720
to protect the integrity of the technological
system, right? So, they are the one that takes,
326
00:31:56,720 --> 00:32:04,800
they're sort of like the fall guy in a sense.
And this is a bit strange, right? It's weird
327
00:32:04,800 --> 00:32:11,360
because human actors who are involved in these
systems tend to have very limited control and
328
00:32:11,360 --> 00:32:16,880
limited understanding of how the system works.
But they're often this like sort of last line of
329
00:32:16,880 --> 00:32:22,960
defense, right? Where they're like, the buck stops
here, I'm the last thing that can intervene. And
330
00:32:22,960 --> 00:32:27,600
humans in the loop or workers in the loop are
often there because there are all these sorts
331
00:32:27,600 --> 00:32:33,200
of edge cases or modes of failure that are hard
to predict or can't be ruled out. And so, we need
332
00:32:33,200 --> 00:32:40,080
someone there to try to stop these different modes
of failure. But as a result of being sort of the
333
00:32:40,080 --> 00:32:48,640
last one in the line of responsibility is that the
workers in the loop become this moral crumple zone
334
00:32:48,640 --> 00:32:54,640
where responsibility for the system is attributed
to the human actor who ultimately has limited
335
00:32:54,640 --> 00:33:01,920
control over the automated or autonomous system.
Ellish also describes this as liability sponges.
336
00:33:01,920 --> 00:33:09,280
Again, this sort of like absorbing the impact
metaphor. One prominent example of this is in 2018
337
00:33:10,000 --> 00:33:17,040
a self-driving car for Uber hit a pedestrian.
The self-driving feature failed to detect the
338
00:33:17,040 --> 00:33:24,080
pedestrian, right? And the safety driver was
intervened too late to stop the collision.
339
00:33:24,800 --> 00:33:29,840
However, it was only the driver that faced
criminal charges, criminal legal charges
340
00:33:29,840 --> 00:33:35,120
as a result of this incident, and Uber did not,
right? So, we can see here the driver absorbing
341
00:33:35,120 --> 00:33:41,280
the legal, the legal impact of the death of
a pedestrian. While the technological system,
342
00:33:42,240 --> 00:33:49,520
Uber, did not face any sort of penalty. So,
there's a couple different problems with putting
343
00:33:49,520 --> 00:33:56,720
people in this role. So, first is deskilling.
Increased automation leads to the atrophy
344
00:33:56,720 --> 00:34:02,240
of skills over time. This is why we require
pilots to do so much flight simulation, right?
345
00:34:02,240 --> 00:34:07,920
Because no one wants a deskilled pilot, but
actually most of flying, right, is automated is on
346
00:34:07,920 --> 00:34:16,160
autopilot. So once your skills have deskilled, say
you've got a doctor doing sort of a radiologist
347
00:34:16,160 --> 00:34:22,720
who's been reliant on a sort of MRI algorithm
for years, we might worry that, you know,
348
00:34:22,720 --> 00:34:28,640
their ability to sort of see and notice these sort
of edge cases is going to get worse over time.
349
00:34:29,280 --> 00:34:33,760
The other thing is what they call the handoff
problem. Maybe some of you're familiar with this,
350
00:34:33,760 --> 00:34:41,920
but this is the idea that you're sort of
doing nothing , you are superfluous,
351
00:34:41,920 --> 00:34:46,800
and then all of a sudden you need to intervene
in an emergency situation or you need to notice
352
00:34:46,800 --> 00:34:51,600
a failure that the system has not notified you of,
right? So, in the case of the doctor with the MRI
353
00:34:51,600 --> 00:34:59,360
machine, you need to notice that the diagnoses
or recommendations are somehow now inaccurate.
354
00:35:00,480 --> 00:35:06,240
In the case of self-driving cars, right? You're
sitting there, you have, you're not doing anything
355
00:35:06,240 --> 00:35:13,280
but all of a sudden you need to sort of like debug
in an emergency situation. And humans just aren't
356
00:35:13,280 --> 00:35:19,600
great at, at sort of like not having anything
to do and then intervening on, in sort of like
357
00:35:19,600 --> 00:35:26,080
a crisis or high stakes situation. So, this is
one of the things where we put people in these
358
00:35:26,080 --> 00:35:30,080
situations where they're sort of psychologically
not super gr... they're just not, we just don't
359
00:35:30,080 --> 00:35:38,240
have great capacities for this. And the last thing
is that people have a lot of trust in machines. My
360
00:35:38,240 --> 00:35:42,640
significant other always tells me not to trust
Google Maps autonomously. You know, like always
361
00:35:42,640 --> 00:35:49,520
check the route first because you might like
know things or have different ways to think about
362
00:35:50,720 --> 00:35:56,800
the route that the, that the map doesn't.
But people tend to be like, “oh, wow, like,
363
00:35:56,800 --> 00:36:02,080
look at all this math, like surely it knows
more than I can do.” And so, we're supposed
364
00:36:02,080 --> 00:36:08,480
to be having workers or people come in and bring
context or expertise, but their deference to the
365
00:36:08,480 --> 00:36:13,200
machine that they think knows more than them
makes it so that they might revisit their
366
00:36:13,200 --> 00:36:19,760
judgment or feel really scared to intervene when
they don't have say like all the numbers. And so,
367
00:36:19,760 --> 00:36:27,280
there's a social tendency to overestimate the, the
capabilities of machines. So, what's wrong with
368
00:36:27,280 --> 00:36:34,240
this? From a, from a philosophical perspective,
not just like this labor perspective. This seems
369
00:36:34,240 --> 00:36:40,000
like a crappy situation to put workers in. The
problem is that the existence of these moral
370
00:36:40,000 --> 00:36:45,520
crumple zones violates a basic moral principle
about blame worthiness. And this is what in
371
00:36:45,520 --> 00:36:51,520
philosophy we call the control principle.
So, to give an example of how the control
372
00:36:51,520 --> 00:36:57,120
principle works, it's highly intuitive. Let's say
you're on a bus and the bus driver breaks really
373
00:36:57,120 --> 00:37:01,200
hard and the person next to you has their coffee.
You know, it's the morning you're on your way to
374
00:37:01,200 --> 00:37:07,920
work and their coffee spills all over your shirt.
And you're like, oh god, now I have to go to work,
375
00:37:07,920 --> 00:37:13,920
and I have like coffee-stained shirt. In this
scenario you might be like really bummed about
376
00:37:13,920 --> 00:37:19,760
your shirt being stained or ruined, but you
are not gonna blame the person next to you,
377
00:37:19,760 --> 00:37:23,280
right? 'cause “You're like, oh, well, it's not
their fault that the coffee spilled. It was
378
00:37:23,280 --> 00:37:27,920
'cause the bus brake so aggressively.” And maybe,
I mean, depending on the scenario, you might not
379
00:37:27,920 --> 00:37:33,200
even blame the bus driver because they might not
have been able to do otherwise as well, right?
380
00:37:33,200 --> 00:37:39,120
They had no control over the situation. Maybe they
wanted to avoid a collision. But if you're like,
381
00:37:39,120 --> 00:37:43,280
whenever I teach this, my students are like,
no, the person with a coffee mug should have
382
00:37:43,280 --> 00:37:49,360
had a better coffee mug. You know, like, who are
these irresponsible people with their like coffee
383
00:37:49,360 --> 00:37:55,120
mugs that can spill on others? And even if you
think that in this scenario, what you're really
384
00:37:55,120 --> 00:38:00,400
arguing about is about what sorts of actions are
under your control, right? You're saying that the
385
00:38:00,400 --> 00:38:05,600
coffee-mug person is blameworthy because they
should have exerted more control than they did,
386
00:38:05,600 --> 00:38:10,080
even though I think that's like kind of
ridiculousness scenario. We agree in the end.
387
00:38:10,080 --> 00:38:16,080
Okay? So the control principle, right? Is the idea
that you should only be morally responsible for
388
00:38:16,080 --> 00:38:21,280
what's in your control. And there are like a bunch
of reasons why we might intuitively accept this.
389
00:38:21,280 --> 00:38:25,600
I mean, if you've ever been blamed for something
that your sibling did growing up, like you get the
390
00:38:25,600 --> 00:38:31,360
sense in which this is like unfair to be blamed
for things that are not your fault. But it's also
391
00:38:31,360 --> 00:38:37,280
really unproductive, right? If you are blamed for
things that are outside of your control, the thing
392
00:38:37,280 --> 00:38:44,240
that ultimately caused the problem or was the
source of failure might not sort of be accountable
393
00:38:44,240 --> 00:38:50,080
or punished or sort of changed in the right sorts
of ways, right? So, when humans in the loop get
394
00:38:50,080 --> 00:38:56,240
blamed for errors of a technological system it
leads to sort of disruption of, of accountability,
395
00:38:56,240 --> 00:39:03,440
which might make us sort of less safe long term,
but also, it's just unfair. So, going back to our
396
00:39:03,440 --> 00:39:11,760
harms and wrongs, right? It's wrong and also
a harm. So, in conclusion, we've given this
397
00:39:11,760 --> 00:39:18,400
whole talk about AI ethics, , and the note
that we wanna end on is actually we should all be
398
00:39:18,400 --> 00:39:27,600
suspicious of ethics because it's not a panacea.
Is anyone familiar with the phrase greenwashing?
399
00:39:27,600 --> 00:39:34,080
Greenwashing? Yeah. Yeah. Okay. So greenwashing
is the idea that companies often take on this
400
00:39:34,080 --> 00:39:41,760
superficial sort of commitment to environmental
action by, you know I don't know, putting some
401
00:39:41,760 --> 00:39:48,960
like part of their product that's recyclable or
doing some sort of superficial change to change
402
00:39:48,960 --> 00:39:51,960
their sort of public image or avoid from
ANONOMOUS SPEAKER: BP.
403
00:39:51,960 --> 00:39:57,200
PATTERSON:
Yeah, exactly. Exactly. So, ethics washing
404
00:39:57,200 --> 00:40:04,160
is something that we can also see in this sort of,
is it an analogous concept. And we see it a lot in
405
00:40:04,160 --> 00:40:09,440
this sort of hype cycle within ai, is this sort
of practice of feigning or engaging with ethical
406
00:40:09,440 --> 00:40:15,040
considerations in a superficial way to sort of
improve the perception of your organization,
407
00:40:15,040 --> 00:40:21,440
right? Or maybe to avoid regulation, like you
don't need to regulate us, we're regulating
408
00:40:21,440 --> 00:40:29,760
ourselves. So in this example, right? Open
AI talks about being a leader in safety about
409
00:40:29,760 --> 00:40:36,160
DeepFakes and private information, even though,
you know, they're creating this text of video
410
00:40:36,160 --> 00:40:42,640
generator and text to voice generator that propose
massive, massive problems, right? For privacy
411
00:40:43,680 --> 00:40:48,240
and sort of fraud or things like that. So, I
mean, how can, it's a little bit weird to be like,
412
00:40:48,240 --> 00:40:53,920
we're the leader in safety, but also we're like
heavily contributing to a certain type of problem.
413
00:40:53,920 --> 00:41:02,240
So the last thing I'll just say is one of the
big things about sort of the failure of ethics
414
00:41:02,240 --> 00:41:08,880
in AI is that so often they sort of end up
actually creating this sort of superficial
415
00:41:08,880 --> 00:41:15,920
reassurance while actually justifying the press
forward for innovation or whatever. And part
416
00:41:15,920 --> 00:41:22,240
of that is because these sort of the various
different ethical programs that are implemented
417
00:41:22,240 --> 00:41:29,040
are disconnected from power. So people don't
actually have a ton of power over corporations,
418
00:41:29,040 --> 00:41:37,920
over researchers, over sort of decisions. They get
made. And so it's very hard to actually enforce
419
00:41:37,920 --> 00:41:43,760
or sort of change the status quo when there's
giant power asymmetries between people and these
420
00:41:43,760 --> 00:41:48,320
forms of technology.
ABBAS:
421
00:41:48,320 --> 00:41:53,280
So we'll conclude on the following. It's
the question that Alicia and I off well
422
00:41:53,280 --> 00:41:58,400
often enough ask, which is, is there anything new
here when we talk about ethics and AI, is this any
423
00:41:58,400 --> 00:42:04,560
different from ethics and automation or ethics and
technology? And part of the reason that that is a
424
00:42:04,560 --> 00:42:10,720
question is one the understandable conflation
of AI and automation. So not all automation
425
00:42:10,720 --> 00:42:15,520
is AI. So some of the questions we are asking
about ethics and AI are really about questions
426
00:42:15,520 --> 00:42:20,000
about ethics and automation more generally.
So you know, who's going to lose their jobs,
427
00:42:20,000 --> 00:42:26,480
et cetera. That's been asked for the last 150
years. The understandable, or as Alicia pointed
428
00:42:26,480 --> 00:42:31,120
out intentional, sometimes confusion on what
constitutes AI intentional in the sense that there
429
00:42:31,120 --> 00:42:35,920
is a lot of hype in this domain from people who
want to sell products or who want to raise funds.
430
00:42:35,920 --> 00:42:43,040
So, you know, you buy an LED and that's like a
smart LED now . So so again, some of the
431
00:42:43,040 --> 00:42:48,080
questions we ask about those systems, which one
might debate whether it's AI or not. And that's,
432
00:42:48,080 --> 00:42:53,120
I don't know how interesting that question even
is, honestly. Is it AI or not? That also leads
433
00:42:53,120 --> 00:42:58,480
to a question of, well, what's new about ethics
of AI? Then there is the theory free ideal. I
434
00:42:58,480 --> 00:43:05,440
forget the name of philosopher again, this is
from Zhanpei’s paper on AI being presented as
435
00:43:05,440 --> 00:43:13,040
this new disruptive, unprecedented thing, which
is disconnected from the scientific, scientific
436
00:43:13,600 --> 00:43:18,160
endeavor as it preceded it, right? So that's,
oh, it's so new that the ethical questions we're
437
00:43:18,160 --> 00:43:23,440
asking about it are defacto new. I don't, that's
not a logical conclusion one can draw, I think
438
00:43:23,440 --> 00:43:29,680
just on the face of it. Finally, well, what does
it matter if the question is new or not? I mean,
439
00:43:29,680 --> 00:43:35,840
so suppose all of this is old hat, so what, in
a sense the questions still need to be asked and
440
00:43:35,840 --> 00:43:43,200
looked at. Thank you.
ROBERTSON:
441
00:43:43,200 --> 00:43:47,920
Thanks for listening. I hope you learned something
about how philosophers and engineers are thinking
442
00:43:47,920 --> 00:43:53,280
about this the topic of ethics in AI. This is
just a small sample of what goes on here at
443
00:43:53,280 --> 00:43:58,000
Oregon State University in the area of AI,
where we have a long history of research,
444
00:43:58,000 --> 00:44:03,520
reaching back decades. More recently, Oregon
State became the first institution in the
445
00:44:03,520 --> 00:44:09,680
country to offer both master’s and doctoral
degrees in AI. If you’d like to learn more
446
00:44:09,680 --> 00:44:15,440
check out the bonus content on your podcast
app or at engineeringoutloud.oregonstate.edu.
00:00:00,000 --> 00:00:03,920
ALICIA PATTERSON: So even if no harm comes
to a particular person. So, for example,
2
00:00:03,920 --> 00:00:09,680
even if I am not, say, negatively impacted
— maybe I'm never detained by the police or
3
00:00:09,680 --> 00:00:15,665
wrongly identified — we can still see that
something is seemingly gone wrong here.
4
00:00:15,665 --> 00:00:20,240
HOUSSAM ABBAS: Sometimes the effective function
of the use of AI can be to focus attention on the
5
00:00:20,240 --> 00:00:26,240
technology and away from how it's used by whom
and to what ends. The ethical considerations
6
00:00:26,240 --> 00:00:30,080
don't start at the point after you've deployed
the system and now you're seeing some results.
7
00:00:30,080 --> 00:00:34,800
The ethical considerations ought to come early on.
And so, no one can hide behind their hand and say,
8
00:00:34,800 --> 00:00:41,214
oh, I didn't see this coming.
[MUSIC: “The Ether Bunny,” by
9
00:00:41,214 --> 00:00:42,720
Eyes Closed Audio, licensed under CC by 3.0]
RACHEL ROBERTSON: From the College of Engineering
10
00:00:42,720 --> 00:00:49,200
at Oregon State University,
this is Engineering Out Loud.
11
00:00:49,200 --> 00:00:54,480
Hi everyone. I’m Rachel Robertson. Today, on
Engineering Out Loud you will hear more from
12
00:00:54,480 --> 00:01:00,560
Alicia Patterson and Houssam Abbas. They are both
assistant professors at Oregon State University
13
00:01:00,560 --> 00:01:07,040
studying ethics in AI, but from very different
perspectives. Alicia has her Ph.D. in philosophy
14
00:01:07,040 --> 00:01:13,680
and her research focuses on the ethics of emerging
technology, especially privacy and data ethics.
15
00:01:13,680 --> 00:01:19,520
Houssam has a Ph.D. in electrical engineering and
his research interests are in the verification and
16
00:01:19,520 --> 00:01:25,440
control of cyber-physical systems and formal
ethical theories for autonomous agents like
17
00:01:25,440 --> 00:01:31,840
unpiloted ground and aerial vehicles. This
recording is from a talk entitled “Where do
18
00:01:31,840 --> 00:01:37,360
Ethics Belong in Artificial Intelligence?” They
designed this talk for a general audience for a
19
00:01:37,360 --> 00:01:43,760
presentation at OMSI, the Oregon Museum of Science
and Industry. This recording is from a talk at
20
00:01:43,760 --> 00:01:49,760
Oregon State University. Now, enjoy a deep dive
into how philosophers and engineers are thinking
21
00:01:49,760 --> 00:01:52,960
about ethics in artificial intelligence.
ABBAS:
22
00:01:52,960 --> 00:02:00,240
Where do ethics belong in artificial intelligence?
So, start with a story. And the story begins in
23
00:02:00,240 --> 00:02:06,560
the 1950s in the psychiatric ward of Bellevue
Hospital in New York where Bill McPhee was
24
00:02:06,560 --> 00:02:13,840
committed. While committed there, he developed
his idea for a voter prediction machine. The,
25
00:02:13,840 --> 00:02:19,040
the point of it was, and here I quote from Jill
Lepore's book, “If Then: How the Simulmatics
26
00:02:19,040 --> 00:02:25,280
Corporation invented the Future.” So, the
point of it was to create, literally quote,
27
00:02:25,280 --> 00:02:32,720
“a set of rules of thinking so that it is possible
to reproduce or closely approximate the thinking
28
00:02:32,720 --> 00:02:38,880
behaviors of voters.” Bill McPhee was then
released from the psychiatric ward at Bellevue and
29
00:02:38,880 --> 00:02:45,120
finished his dissertation at Columbia University
about a program for, to create a fully observable
30
00:02:45,120 --> 00:02:51,280
electorate. You would feed the machine massive
amounts, massive amounts of microscopic data about
31
00:02:51,280 --> 00:02:57,120
users and issues, voters and issues. And it would
be possible to assess the effect of any proposed
32
00:02:57,120 --> 00:03:05,280
course of action by political candidate based on
the likely reactions of the voters. Ed Greenfield,
33
00:03:05,280 --> 00:03:10,000
who's was an ad man and was one of the key
founders of the Simulmatics Corporation which
34
00:03:10,000 --> 00:03:15,600
monetized the ideas of Bill McPhee and sold them.
He put it like this, he said, suppose that a
35
00:03:15,600 --> 00:03:19,680
candidate was proposing to make a major civil
rights speech in the South. Remember this was
36
00:03:19,680 --> 00:03:29,600
the early sixties. We would be able, quote,
“through our model to predict what the meaning
37
00:03:29,600 --> 00:03:35,280
of that speech to over a thousand subgroups of
a given population. And thus, we'll be able to
38
00:03:35,280 --> 00:03:41,520
predict any the smallest percentage difference
in votes, that is, that such a speech would
39
00:03:41,520 --> 00:03:47,840
have.” The Simulmatics Corporation actually sold
their services to the Kennedy campaign. Kennedy
40
00:03:47,840 --> 00:03:53,200
then went on to win the presidential election that
year. Now, it's not clear how much influence they
41
00:03:53,200 --> 00:03:57,920
actually had, because the recommendations that
Simulmatics gave the campaign were largely seen
42
00:03:57,920 --> 00:04:04,800
to be common sense and would have been adopted
anyway. But I think you can already see the how
43
00:04:04,800 --> 00:04:14,960
that sort of foreshadows the general theme of
users/voters/people prediction and then control.
44
00:04:14,960 --> 00:04:19,840
The Cambridge Analytica — they basically sold a
more sophisticated version of what Simulmatics
45
00:04:19,840 --> 00:04:24,080
was trying to sell the Kennedy campaign to the
Trump campaign in 2016 and to the Brexit Leave
46
00:04:24,080 --> 00:04:32,000
campaign in 2015. And the reason I, I start with
this anecdote is to directly cut through to what
47
00:04:32,000 --> 00:04:36,960
perhaps is the biggest point of unease that
some people might feel when they fret about
48
00:04:36,960 --> 00:04:42,240
AI and ethics. The notion that human behavior
is predictable and then controllable. So,
49
00:04:42,240 --> 00:04:47,760
in particular, most of you have probably heard
of the Facebook experiment on playing basically
50
00:04:47,760 --> 00:04:53,440
and controlling user's emotions based
on what they bubble up in their feeds.
51
00:04:54,640 --> 00:05:04,160
Now, if users or people are predictable, at
least in the aggregate and controllable, then
52
00:05:04,160 --> 00:05:09,280
why not program them into robots? And that then
brings us to the notion of embodied intelligence,
53
00:05:09,280 --> 00:05:16,880
embodied artificial intelligence. One half of this
is a humanoid of its own inventor. I'll give you
54
00:05:16,880 --> 00:05:21,600
a couple of seconds to try to guess, which is the
humanoid and which is the flesh and blood human,
55
00:05:21,600 --> 00:05:26,400
right. This is the Geminiod DK by a
Danish inventor whose name I now forget,
56
00:05:26,400 --> 00:05:32,720
oh, it's in my notes. Henrik Scharf ,
right? And so, this reminds me of a famous,
57
00:05:32,720 --> 00:05:37,440
which I had on the slides that I removed of a
famous icon of Christ from the sixth century,
58
00:05:37,440 --> 00:05:43,520
which is supposed to portray the dual nature of
Christ as man and God. It sounds our ambitions
59
00:05:43,520 --> 00:05:52,800
have now reduced to becoming man and machine.
So, so hopefully this gives you sort of the
60
00:05:52,800 --> 00:05:58,640
corner cases or some of the corner cases when
discussing AI and ethics in, in the way that
61
00:05:58,640 --> 00:06:03,680
they relate mostly to us, like the notion that we
might be predictable and controllable. If you want
62
00:06:03,680 --> 00:06:07,920
to know what researchers are actually working
on you can literally pick up like any issue
63
00:06:07,920 --> 00:06:11,280
of the communications of the ACM from the last
couple of years. And there will be an article
64
00:06:11,280 --> 00:06:18,000
in there about AI and ethics, not ethics and CS in
general, but specifically AI from bias and credit
65
00:06:18,000 --> 00:06:23,840
rating to problems with certain problems with
computer vision systems, and so on and so forth.
66
00:06:23,840 --> 00:06:32,720
So rather than give a laundry list of that, I'll
hand it over to Alicia to help us think through
67
00:06:32,720 --> 00:06:39,200
a framework about these problems. Starting with a
cautionary note that ethics is the hottest product
68
00:06:39,200 --> 00:06:44,640
in Silicon Valley's hype cycle today. So, in order
to step away from the hype and perhaps think about
69
00:06:44,640 --> 00:06:47,760
it more productively: Alicia.
PATTERSON:
70
00:06:47,760 --> 00:06:54,720
Okay, so philosophers have a set of tools to think
about ethical problems that has been our bread and
71
00:06:54,720 --> 00:07:02,320
butter for forever, basically . And so,
we think maybe perhaps a little bit differently
72
00:07:02,320 --> 00:07:09,280
about how we spot ethical problems. So, one of
the key things, key sort of concepts we have
73
00:07:09,280 --> 00:07:14,880
in philosophy are the concepts of a harm and
a wrong. And we differentiate these because
74
00:07:14,880 --> 00:07:19,760
they sort of mean different things or pick
out different types of ethical problems. So,
75
00:07:19,760 --> 00:07:25,680
a harm is something that sets back your material
interests, it makes you worse off, right? We
76
00:07:25,680 --> 00:07:32,320
might think of like suffering or something like
that. So, if someone steps on your foot, right,
77
00:07:32,320 --> 00:07:37,120
we can say that they've harmed you because your
foot now hurts and maybe it's swollen. A wrong
78
00:07:37,120 --> 00:07:46,000
in contrast is a, a sort of a violation of rights
or some sort of disrespect of an important value
79
00:07:46,000 --> 00:07:53,280
or denying someone something they're owed, right?
So, if I say deny, deny someone the right to vote
80
00:07:53,280 --> 00:07:58,400
I've disrespected an important value and I've
denied something, they've that they've, they're
81
00:07:58,400 --> 00:08:06,080
owed, right? As a citizen, right? They're owed the
right to vote. Perhaps you know, if the person is
82
00:08:06,080 --> 00:08:10,400
in like, you know, not a swing state, their
vote maybe doesn't make much of a difference.
83
00:08:10,400 --> 00:08:15,360
So maybe I didn't harm them very much, but we can
still see that I've done something wrong to them,
84
00:08:15,360 --> 00:08:21,760
right? I've denied them something that they're
owed. So, harms and wrongs usually go together,
85
00:08:21,760 --> 00:08:26,560
but it's important to understand sort of their
different contours to be able to fully map
86
00:08:27,200 --> 00:08:36,560
ethical problems. So, some AI harms this when
we originally made this, this wasn't such a
87
00:08:36,560 --> 00:08:43,600
big thing in the news, but Cigna, a health
insurance company uses algorithms, right?
88
00:08:43,600 --> 00:08:50,320
To sort of batch deny claims. And doctors
are supposed to go through and, and review
89
00:08:50,320 --> 00:08:57,120
a patient's file, right? And see if the sort of
medical claim is medically a medical necessity.
90
00:08:57,680 --> 00:09:04,720
And according to research by ProPublica doctors
were spending an average of 1.2 seconds on each
91
00:09:04,720 --> 00:09:11,600
case. I don't know about you, but I, I feel like
I could speed read a little bit, but 1.2 seconds
92
00:09:11,600 --> 00:09:18,880
on a very serious medical decision seems that
seems pretty bad, right? So, in this instance,
93
00:09:18,880 --> 00:09:25,760
we get an algorithm, right, that caused a
particular kind of harm, right? It may probably
94
00:09:25,760 --> 00:09:32,480
denied thousands of people certain procedures.
That means patients, doctors, hospital staff have
95
00:09:32,480 --> 00:09:39,360
to do all of this extra labor to get the medical
care that they deserve. And some people just might
96
00:09:39,360 --> 00:09:45,680
not be able to like fully work the system and then
give up and have to pay for things out of pocket.
97
00:09:45,680 --> 00:09:52,160
So here we can see the way in which the
system made people worse off, it harmed them.
98
00:09:53,360 --> 00:10:03,920
In contrast a wrong may not necessarily
involve a concrete harm, right? So,
99
00:10:03,920 --> 00:10:11,040
an example of this is the use of facial
recognition by law enforcement. One of the,
100
00:10:11,040 --> 00:10:16,000
sort of, wrongs that we see is that people, a
lot of times in AI systems are treated sort of
101
00:10:16,000 --> 00:10:22,640
as near data in large data sets. And data is
scraped in mass from the internet, right? This
102
00:10:22,640 --> 00:10:29,280
is like all known to us. And there's no sort of
regard for sort of fundamental values about how
103
00:10:29,280 --> 00:10:36,400
these people might want to be treated, right? In
this instance, the value of privacy. So, this is
104
00:10:36,400 --> 00:10:41,840
a really interesting read, if you wanna check it
out put together by researchers at Georgetown. So,
105
00:10:41,840 --> 00:10:48,880
research at Georgetown found that one in two
Americans have their faces and data sets that
106
00:10:48,880 --> 00:10:54,320
are available to the police. And this is the
exact kind of thing that we should worry about,
107
00:10:54,320 --> 00:11:01,840
right? That we're sort of infringing on important
values like democratic freedoms or privacy.
108
00:11:01,840 --> 00:11:06,320
So even if no harm comes to a particular
person, so for example, even if I am not,
109
00:11:06,320 --> 00:11:14,320
say negatively impacted, maybe I'm never detained
by the police or wrongly identified we can still
110
00:11:14,320 --> 00:11:24,080
see that something is seemingly gone wrong here.
So, within the sort of scope of wrongs, injustice
111
00:11:24,080 --> 00:11:28,560
usually sits within it because one of the things
we want to pick out about injustice is that it's
112
00:11:28,560 --> 00:11:36,480
a certain form of disrespect. So, when someone's
committed an injustice, what's been … what value
113
00:11:36,480 --> 00:11:42,080
or what principle has been violated is the value
or principle of fairness or equality. Oftentimes,
114
00:11:42,080 --> 00:11:49,360
injustice involves both a harm and a wrong, right?
So, you're treated unfairly and your life is made
115
00:11:49,360 --> 00:11:54,560
worse. So, if you say, are denied a promotion
on the, say, the basis of your race or gender,
116
00:11:54,560 --> 00:11:59,200
we can say like, you're worse off. You don't have
the income that you would've had, right? The,
117
00:11:59,200 --> 00:12:03,440
the career opportunities you would've
had, but also, you've been disrespected,
118
00:12:03,440 --> 00:12:10,400
right? You've been treated in a way that you
shouldn't have. So those are some ways actually,
119
00:12:10,400 --> 00:12:15,040
lemme give an example of injustice first.
So, an example of injustice, there's too many
120
00:12:15,040 --> 00:12:22,560
to really even pick which one to talk about. But
one example of automated bias, right? Is in racial
121
00:12:22,560 --> 00:12:29,360
disparities in credit lending study by faculty
in Stanford and University of Chicago looked into
122
00:12:29,360 --> 00:12:35,840
this into sort of alternative models of credit
scoring and found that low income and minority
123
00:12:35,840 --> 00:12:43,280
groups were somewhere between five and 10% less,
less likely to receive a loan than high income or
124
00:12:43,280 --> 00:12:50,080
non-minority groups. So here we can see, right?
Being denied the alone on the basis of your race
125
00:12:50,080 --> 00:12:55,760
is both wrong and you're harmed, right? You're,
you don't get the house and you're treated in a,
126
00:12:55,760 --> 00:13:02,080
in a way that you should not have been treated.
So, this is sort of the background when we think
127
00:13:02,080 --> 00:13:09,040
about where is ethics coming into conversations
in, in engineering and design, this is sort of
128
00:13:09,040 --> 00:13:18,240
my like, toy example of the like design lifecycle.
Oftentimes, these sort of questions about ethics,
129
00:13:18,240 --> 00:13:22,720
about harms and wrongs and things like that
come in after a product has been designed,
130
00:13:22,720 --> 00:13:28,240
their research has been done in it for years,
and it's either launched or about to be launched,
131
00:13:28,240 --> 00:13:34,640
and some problems are surfacing and people ask
questions about you know, what sort of guardrail
132
00:13:34,640 --> 00:13:40,880
should be in place? And one of the problems with
this is that it's sort of too little too late,
133
00:13:40,880 --> 00:13:46,320
right? When we ask questions about ethics,
we should ask them from the very beginning,
134
00:13:46,320 --> 00:13:51,760
right? Because asking them from the beginning
allows us to ask a series of questions about
135
00:13:51,760 --> 00:13:56,400
values through the entire design lifecycle,
through the entire research process, right?
136
00:13:57,840 --> 00:14:03,520
And that allows us to ask a broader set of
questions, rather than this narrow question of
137
00:14:03,520 --> 00:14:09,120
what sort of guardrail should we put in place?
Or what sort of use cases should this be
138
00:14:09,120 --> 00:14:16,240
prohibited or encouraged for? So, a variety of
questions that we should ask are things like,
139
00:14:16,240 --> 00:14:20,560
before we even begin to develop this, is
this technology the right solution to the
140
00:14:20,560 --> 00:14:26,160
problem at all? Science and technology scholars
critique this question not being asked a lot,
141
00:14:26,160 --> 00:14:31,520
and they call this techno solutionism, the idea
that a technology can be sort of the solution
142
00:14:31,520 --> 00:14:36,640
for these big social problems that we're often
not sitting back and saying, is this the right,
143
00:14:36,640 --> 00:14:41,680
is this the right way to solve this problem?
The next question that needs to be asked,
144
00:14:41,680 --> 00:14:44,960
and this is, I'll say, I'm going through a
series of questions, but this is by no means
145
00:14:44,960 --> 00:14:51,920
an exhaustive list. Does the community want the
problem solved in this way? Whose input do we
146
00:14:51,920 --> 00:15:00,240
even have in thinking about what the problem is?
So, in design justice sort of like a canon text in
147
00:15:01,440 --> 00:15:08,560
in sort of intersection between ethics and design,
people have this phrase of nothing for us without
148
00:15:08,560 --> 00:15:13,200
us, right? So, do people who are impacted by the
technology have any sort of meaningful ways to
149
00:15:13,200 --> 00:15:22,160
shape the values in design of it? The second or
third, I guess we're on third we should foresee
150
00:15:22,160 --> 00:15:27,600
the, we should look into the potential unforeseen
consequences again, before we've like created a
151
00:15:27,600 --> 00:15:33,520
product or we've put a ton of research into a
certain type of algorithm, right? So feminist
152
00:15:33,520 --> 00:15:38,160
technology scholars, for example, often point
out that like the first public adoption of a
153
00:15:38,160 --> 00:15:44,800
technology usually comes in the form of harm for
women. So, one example, there's many I could give
154
00:15:44,800 --> 00:15:51,120
you, is that when DeepFakes were first, like,
open source available to people on the internet,
155
00:15:51,120 --> 00:15:56,960
they were used in mass for non-consensual
pornography or image-based sexual abuse.
156
00:15:56,960 --> 00:16:01,600
So, here we can see like being the subject
of pornography is something that women are
157
00:16:01,600 --> 00:16:06,080
significantly more likely to be subject to.
And so, we might have wanted the developers
158
00:16:06,080 --> 00:16:11,600
to have thought about that or grapple with that
before they released this technology, right? So,
159
00:16:11,600 --> 00:16:17,840
these are the sorts of things that maybe we can
solve for, prevent for early on. We also might
160
00:16:17,840 --> 00:16:22,800
wanna ask, is this technology compatible with
important social goods, like democratic values,
161
00:16:22,800 --> 00:16:29,120
right? When we are creating something that say,
involves a form of surveillance we might wanna
162
00:16:29,120 --> 00:16:35,360
understand, will this significantly impact, say,
people's ability to freely assemble. Freedom of
163
00:16:35,360 --> 00:16:40,800
assembly is considered say, a core democratic
value of value that you sort of a thing,
164
00:16:40,800 --> 00:16:46,160
a right that you need to be able to express to
be able to exist in a democracy at all. We also
165
00:16:46,160 --> 00:16:51,440
might wanna ask, are we getting data in a way that
respects important rights? As I mentioned before,
166
00:16:51,440 --> 00:16:57,680
like a common practice and machine learning.
It's just to do this mass web scraping. And we
167
00:16:57,680 --> 00:17:01,920
might wanna ask a variety of questions about
that. Like, are we respecting people's right
168
00:17:01,920 --> 00:17:09,760
to say not participate in oppressive systems?
Are we re respecting their rights to privacy?
169
00:17:10,960 --> 00:17:14,800
We might also wanna ask our, some people
going to be more harmed by this than others,
170
00:17:14,800 --> 00:17:18,880
right? Maybe this question comes up the most when
we're thinking about automating certain forms of
171
00:17:18,880 --> 00:17:26,560
labor. Who are the winners and losers? And who
might be especially vulnerable to the impacts of
172
00:17:26,560 --> 00:17:34,880
a system? We wanna ask questions about bias before
a product is launched. We wanna be able to grapple
173
00:17:34,880 --> 00:17:41,120
with that. We don't want something. So as so often
happens something gets launched and then users
174
00:17:41,120 --> 00:17:46,080
who are negatively affected, who are of that group
that the technology is biased against are the ones
175
00:17:46,080 --> 00:17:49,840
being like, this doesn't work for us. This has
happened again and again, right? So that's again,
176
00:17:49,840 --> 00:17:56,640
something that we wanna catch early and often.
We wanna ask what are the potential misuses,
177
00:17:56,640 --> 00:18:04,640
for example, have we created a tool that's like
really amazing, but also enables a lot of fraud?
178
00:18:04,640 --> 00:18:09,680
And lastly then we can come to these questions
about, say, guardrails. What sort of laws,
179
00:18:09,680 --> 00:18:15,600
regulation policies and norms need to be put
in place to protect people or, or to ensure
180
00:18:15,600 --> 00:18:18,760
this sort of ethical use?
ABBAS:
181
00:18:19,920 --> 00:18:28,240
Thank you. So, within that framework or against
the background of that framework, we can look
182
00:18:28,240 --> 00:18:36,000
at two major approaches so far in engineering to
tackle the question of how does one in a nutshell
183
00:18:36,000 --> 00:18:42,080
program certain ethical norms or social norms
more generally in engineering systems. This is
184
00:18:42,080 --> 00:18:48,240
a very high-level simplified view of that. One is
utilitarianism, the other is non-utilitarianism,
185
00:18:48,240 --> 00:18:56,000
or as sometimes referred to as deontology. I'll
start with Deontology from the Greek they own duty
186
00:18:56,000 --> 00:19:02,480
and laws for science or knowledge of. And whereas
most of us probably think of the right thing to do
187
00:19:02,480 --> 00:19:08,800
is the think that maximizes certain benefits,
maximizes certain goods. Deontology says, no,
188
00:19:08,800 --> 00:19:16,160
the way you assess whether some, there are certain
moral norms that the moral agent ought to obey,
189
00:19:16,160 --> 00:19:20,080
regardless of the consequences, regardless of
the amount of good that that action produces,
190
00:19:20,080 --> 00:19:25,440
or the amount of bad that that action leads to.
So, the deontology is a normative theory regarding
191
00:19:25,440 --> 00:19:31,440
which choices are morally required, forbidden,
or permitted. Those are three key categories. In
192
00:19:31,440 --> 00:19:36,240
contrast, for example, to moral theories, which
tell you what sort of person you ought to be,
193
00:19:36,240 --> 00:19:41,040
right? What makes a choice, right,
is its conformity with a moral norm.
194
00:19:41,840 --> 00:19:49,040
So, if we adopt such an approach, and that's sort
of the perspective I take in my own research on
195
00:19:49,040 --> 00:19:55,040
algorithmic ethics. One then asks, how do we
program duties into an AI agent, right? So,
196
00:19:55,040 --> 00:20:00,400
if you have statements of which choices are
morally required, forbidden or permitted, um, can
197
00:20:00,400 --> 00:20:06,080
we think of them as design requirements, uh, for a
subsystem of this overall system that we're trying
198
00:20:06,080 --> 00:20:11,520
to create? So, you have your objective function,
right? And this is the only math we're going to
199
00:20:11,520 --> 00:20:16,320
see. I think you have your objective function,
you want, uh, something to run fast, for example.
200
00:20:16,320 --> 00:20:22,240
And then you have those things that are permitted,
forbidden, and required, and you express those in
201
00:20:22,240 --> 00:20:27,680
some mathematical formalism. And the idea is that
once you've done, you've used these mathematical
202
00:20:27,680 --> 00:20:32,000
formalisms to express these two different things.
And that's sort of the point I wanna stress here.
203
00:20:32,000 --> 00:20:38,960
That object, the performance objectives are not
the normative constraints. Magic, magic, magic,
204
00:20:38,960 --> 00:20:43,920
it turns into zeros and ones, and then lo and
behold, Cassie can run a 5K without kicking a
205
00:20:43,920 --> 00:20:49,360
rabbit on the way, right? So, what is this
mathematical formalism that allows us to
206
00:20:49,360 --> 00:20:55,520
express normative requirements? This, I think
most of us are or ought to be familiar with,
207
00:20:55,520 --> 00:20:59,920
we need basically a mathematical language that
allows us to express such normative statements,
208
00:20:59,920 --> 00:21:04,400
which you and I can understand because
understanding is actually key here, right?
209
00:21:04,960 --> 00:21:09,840
An objective function with a thousand parameters
is not something you and I understand or anybody,
210
00:21:09,840 --> 00:21:14,640
and whoever tells you that is probably fooling
themselves and you. So, we want something that you
211
00:21:14,640 --> 00:21:18,400
and I can understand, but which is also amenable
to algorithmic, which is to say automatic,
212
00:21:18,400 --> 00:21:25,280
automatic or automated processing and reasoning.
And the, and the language most of us use is
213
00:21:25,280 --> 00:21:29,200
formal logic. You're all familiar with Boolean
logic, but there's a zoo of logics out there,
214
00:21:29,200 --> 00:21:34,080
some of which are appropriate or to some degree
appropriate for reasoning about normative
215
00:21:34,080 --> 00:21:40,400
statements. Once you do that, once you make
that move, then reasoning becomes a calculation,
216
00:21:40,400 --> 00:21:46,400
right? The dream of Leibniz was that when there
are disputes among persons, we can simply say, let
217
00:21:46,400 --> 00:21:52,320
us calculate, right? You, you run your argument,
I run my argument, and the machine tells us, well,
218
00:21:52,320 --> 00:21:57,440
which argument is a sound argument, right? Does
A follow from B? And so, without further ado,
219
00:21:57,440 --> 00:22:03,920
we can see who is right. We're nowhere near that,
obviously. But that, that's the overall idea. Now,
220
00:22:03,920 --> 00:22:09,680
formal methods and formal logic are heavily
used for, um, safety purposes, for designing
221
00:22:09,680 --> 00:22:15,920
safe systems, especially in safety critical
applications like avionics, for example, right?
222
00:22:16,800 --> 00:22:21,360
And in fact, in hardware, any hardware company
today makes heavy use of formal logic. But what
223
00:22:21,360 --> 00:22:26,800
we are talking about is using formal logics
in order to formalize normative requirements,
224
00:22:26,800 --> 00:22:31,280
ethical and social requirements. There's some work
I've done. There's work done by Matthias Scheutz
225
00:22:31,280 --> 00:22:40,240
at Tufts. Next week's AI seminar. Speaker Adnan
Darwiche also makes use heavy use of formal logic
226
00:22:40,240 --> 00:22:44,960
for explainable AI in his case. And I think
the center of gravity really of, of this sort
227
00:22:44,960 --> 00:22:49,840
of work still is in Europe especially
in Vienna. So there, if you're curious,
228
00:22:49,840 --> 00:22:54,320
you can look up that project that just launched
centered in Vienna, but with collaboration with
229
00:22:54,320 --> 00:23:00,080
Maryland. And the point one of the last thing
I wanna say here is that this formalism allows
230
00:23:00,080 --> 00:23:04,400
you to express and reason about normative
requirements. It doesn't prevent you from doing
231
00:23:04,400 --> 00:23:09,040
performance optimization the way we usually do.
The question is how do you do the two together?
232
00:23:09,040 --> 00:23:16,960
So, you know, you can still look at graphs,
. So finally, it's good not to drink
233
00:23:16,960 --> 00:23:22,240
one's own Kool-Aid. Are there really rules
that govern our thinking in general? Yes and
234
00:23:22,240 --> 00:23:27,840
no. You certainly cannot formalize all the
ways in which we actually think. Thinking
235
00:23:27,840 --> 00:23:32,400
is not always what we call rational thinking.
It is still different kind of thinking. So,
236
00:23:33,040 --> 00:23:42,080
you know, the other form is utilitarianism.
The other the other way of trying to program
237
00:23:42,960 --> 00:23:48,000
normative requirements is through utilitarianism,
where you have, you know, the good is a function
238
00:23:48,000 --> 00:23:52,880
of your actions. You have lots of optimizers
that you throw at this mysterious function,
239
00:23:52,880 --> 00:23:57,360
and it gives you the best action, which is
supposedly the most ethical action because it
240
00:23:57,360 --> 00:24:07,280
produces the most amount of good, right? It's,
it goes only so far in my view, because there,
241
00:24:07,280 --> 00:24:13,920
those are on some level inherently incommensurate
things, right? So even, let's take a very trivial
242
00:24:13,920 --> 00:24:19,360
example from this already old 2015 paper from
Chris Gerdes, right? So, say we want, you
243
00:24:19,360 --> 00:24:23,520
want the car to get to its destination, that's a
performance objective without collisions. That's
244
00:24:23,520 --> 00:24:28,480
a normative requirement. 'cause You, you know,
a car can collide with someone and keep going,
245
00:24:28,480 --> 00:24:32,400
right? So, it's not necessarily a performance
requirement while minimizing emissions,
246
00:24:32,400 --> 00:24:36,320
that's both performance and normative because
emissions are bad for people in general.
247
00:24:36,320 --> 00:24:40,640
So, what does it mean to weigh all these? Like
how do you weigh these in meters? What sort of
248
00:24:40,640 --> 00:24:43,440
objective function are you gonna put all
of these in? What sort of reward function
249
00:24:43,440 --> 00:24:48,800
are you gonna use in order to express all of
these? Not to mention that sometimes ethical
250
00:24:48,800 --> 00:24:52,960
requirements is not something that you can choose
more or less of something, you just have to do it,
251
00:24:52,960 --> 00:24:58,960
right? And so then you go into the multi-agent
case, which complicates the picture. Nonetheless,
252
00:24:58,960 --> 00:25:08,320
there is in fact the AI alignment literature.
Most of it is on the side of, let's see, how is it
253
00:25:08,320 --> 00:25:13,520
possible to design, for example, a reward function
for an RL agent such that it avoids catastrophic
254
00:25:13,520 --> 00:25:19,280
outcomes? Among that literature is reinforcement
learning from human feedback, right? Where the
255
00:25:19,280 --> 00:25:27,200
idea is suppose, you know, you want, for example,
chat GPT to produce the ethically normed answers.
256
00:25:27,200 --> 00:25:30,880
So what we do is we'll ask check GPT the same
question 10 times. It's a stochastic machine,
257
00:25:30,880 --> 00:25:34,320
it'll give us slightly different answers 10 times,
and then we give it feedback on which one is the
258
00:25:34,320 --> 00:25:40,160
best. And this way it tweaks its own parameters.
But, now that you might say, okay, sounds
259
00:25:40,160 --> 00:25:45,360
reasonable, but then it's, it's a market economy.
So, we'll say, oh, are we gonna really gonna hire
260
00:25:45,360 --> 00:25:50,720
a thousand people to do this a thousand times?
No, what do we do? We put another AI in there.
261
00:25:50,720 --> 00:25:57,560
So now you have an AI guiding another AI as, so
then you climb the AI ladder to nowhere .
262
00:25:59,440 --> 00:26:05,440
So, I'm not gonna pretend there's a good
segue, we're just switching gears. ,
263
00:26:05,440 --> 00:26:12,560
right? The problem of responsibility in autonomous
systems. There's two models of responsibility,
264
00:26:12,560 --> 00:26:18,240
fully autonomous systems, which I'm, I'm not going
to talk about because in my mind, there's no sense
265
00:26:18,240 --> 00:26:23,280
in which a fully autonomous machine is responsible
for something when there's a human behind it.
266
00:26:24,000 --> 00:26:28,560
So, we'll talk about . So, I just like
swiped a whole bunch of things on the side.
267
00:26:28,560 --> 00:26:33,280
We'll talk about semi-autonomous systems, okay?
Workers in the loop. Usually, humans in the loop
268
00:26:33,280 --> 00:26:37,040
is the terminology, but as Alicia was pointing out
this morning, this is actually creating whole new
269
00:26:37,040 --> 00:26:41,680
category of workers who have to watch terrible
videos all day and so on and so forth. Now you
270
00:26:41,680 --> 00:26:45,920
can have a worker in the loop for technical
purposes, like, you know, radiologists as
271
00:26:45,920 --> 00:26:50,400
checking the AI radiology algorithms, and you
can have them for legal and moral purposes.
272
00:26:50,400 --> 00:26:54,000
So even if this thing is great, you
still need for legal moral purposes,
273
00:26:54,000 --> 00:26:59,440
for example, a lawyer in the loop, a lawyer
in the war room, and I'll take an example
274
00:26:59,440 --> 00:27:05,200
from the Israeli Army's use of AI for lethal
targeting. So, this is from March 2023,
275
00:27:05,200 --> 00:27:13,200
so before the the latest war. So, so in from this
article, so several high ranking IDF officers
276
00:27:13,200 --> 00:27:19,520
informed the press that Israel is deploying AI
tools as part of its military arsenal. Skip, skip,
277
00:27:19,520 --> 00:27:24,080
skip. The public statements acknowledge that some
of the targets attacked by the IDF are produced
278
00:27:24,080 --> 00:27:29,920
by AI tools. Skip, skip. One encouraging aspect
is that it seems that the, the IDF seeks to use
279
00:27:29,920 --> 00:27:34,240
tools that complement human decision making rather
than a substitute for the human factor. So here,
280
00:27:34,240 --> 00:27:40,320
this is a human in the loop or a soldier
basically in the loop acting as a backstop to
281
00:27:40,320 --> 00:27:48,000
the decision or to to check the decision or the
recommendation of the AI tool. The question is,
282
00:27:48,000 --> 00:27:53,920
how likely is it that this person or this soldier
is actually doing due diligence? We saw earlier
283
00:27:53,920 --> 00:28:01,040
for the insurance claim that basically doctors are
spending 1.2 seconds checking an insurance claim.
284
00:28:01,040 --> 00:28:06,960
So, this is from the Guardian. And the point here
I'm making is that having a human in the loop,
285
00:28:06,960 --> 00:28:12,320
a worker in the loop is no panacea because that
human is subject to the same social forces that
286
00:28:12,320 --> 00:28:16,240
led to the creation of that AI system in the
first place. And that AI system has a purpose,
287
00:28:16,240 --> 00:28:20,720
and the purpose is to be automated. It is not
to be slowed down by, in this case, the human.
288
00:28:21,280 --> 00:28:28,480
So this is from a Guardian article in April, 2024
sourced on like six sources who worked with the
289
00:28:28,480 --> 00:28:33,600
Lavender system that the IDF was using in Gaza.
“We were constantly being pressured, bring us more
290
00:28:33,600 --> 00:28:39,600
targets. They really shouted at us. We were told,
now we have to f-up Hamas no matter what the cost.
291
00:28:39,600 --> 00:28:45,840
At its peak, the system managed to generate 37,000
people as potential human targets. But the numbers
292
00:28:45,840 --> 00:28:50,720
changed all the time because it depends on where
you set the bar and what a Hamas operative is.”
293
00:28:50,720 --> 00:28:55,040
This is a system with many parameters. It has many
knobs. So, depending on where you set the knobs,
294
00:28:55,040 --> 00:28:58,960
which is something as engineers we always do,
right? We're tweaking parameters for performance
295
00:28:58,960 --> 00:29:06,640
in this case for lethal targeting. So, the numbers
change all the time. So, guess in which direction
296
00:29:06,640 --> 00:29:12,640
the numbers are going to be trending in such
a situation. Now, who's going to vet 37,000
297
00:29:12,640 --> 00:29:19,520
targets or presumed targets. Skip to December 24.
This is from a Washington Post story. Soldiers
298
00:29:19,520 --> 00:29:23,360
who are poorly trained in using the technology
attacked human targets without corroborating
299
00:29:23,360 --> 00:29:27,280
Lavender's predictions at all. Lavender is
the name of the system. At certain times the
300
00:29:27,280 --> 00:29:32,640
only corroboration required was that the target
was a male. So, the point here is that if the,
301
00:29:32,640 --> 00:29:39,360
the worker in the loop, the soldier in the loop
and depending on how the, the scale of which the
302
00:29:39,360 --> 00:29:46,000
AI is being deployed is basically useless.
Even if that worker in the loop is a lawyer,
303
00:29:46,000 --> 00:29:52,160
there is a whole literature on so-called
war lawyers, right? Some legal experts
304
00:29:52,160 --> 00:29:55,760
skip, skip harness the law in the service of
the war effort. Now, the point being made in
305
00:29:55,760 --> 00:30:00,640
this book is not simply that the lawyers are
like a cog in the machine just basically giving
306
00:30:00,640 --> 00:30:04,800
blank checks to whoever wants to target. It's a
more nuanced point than that. But the point is,
307
00:30:04,800 --> 00:30:09,520
it's a system as a whole that exists. And too, so
to go back to Alicia's point earlier, the ethical
308
00:30:09,520 --> 00:30:13,360
considerations don't start at the point after
you've deployed the system and now you're seeing
309
00:30:13,360 --> 00:30:18,480
some results. The ethical considerations ought to
come early on. And so no one can hide behind their
310
00:30:18,480 --> 00:30:26,720
hand and say, oh, I didn't see this coming.
So, and I owe my thinking on this point to
311
00:30:26,720 --> 00:30:31,440
Zhanpei Fang who's here in the audience, AI is a
rhetorical tool. Sometimes the effective function
312
00:30:31,440 --> 00:30:37,680
of the use of AI can be to focus attention on
the technology and away from how it's used by
313
00:30:37,680 --> 00:30:43,840
whom and to what ends. It's like, oh, is the AI
parameter, are the AI parameters correctly set?
314
00:30:43,840 --> 00:30:49,040
Like, no, why is it being used? And this
isn't a question one asks only about AI,
315
00:30:49,040 --> 00:30:53,040
one can ask it about just technology in
general, right? But we're discussing AI here.
316
00:30:54,880 --> 00:30:56,320
Thank you.
PATTERSON:
317
00:30:56,320 --> 00:31:05,520
Okay, so continuing on this problem of
responsibility in AI systems. How many
318
00:31:05,520 --> 00:31:10,240
of you familiar with the concept of a crumple
zone? Excellent. This is so much different than
319
00:31:10,240 --> 00:31:15,360
a public talk . So right, the crumple zone,
lots of engineers, right? So, the crumple zone,
320
00:31:15,360 --> 00:31:21,840
right, we know is meant to crumple so that
the safety cage or where the passengers are,
321
00:31:21,840 --> 00:31:27,840
is protected from the force of a collision,
right? That's the point. The kinetic energy,
322
00:31:27,840 --> 00:31:34,880
right? Is transformative as like controlled
crumpling and it preserves the integrity.
323
00:31:36,320 --> 00:31:43,440
Well, Madeline Clare Elish argues that there's
a similar thing that happens for workers in the
324
00:31:43,440 --> 00:31:50,720
loop or humans in the loop where they are sort
of the thing that's supposed to crumple in order
325
00:31:50,720 --> 00:31:56,720
to protect the integrity of the technological
system, right? So, they are the one that takes,
326
00:31:56,720 --> 00:32:04,800
they're sort of like the fall guy in a sense.
And this is a bit strange, right? It's weird
327
00:32:04,800 --> 00:32:11,360
because human actors who are involved in these
systems tend to have very limited control and
328
00:32:11,360 --> 00:32:16,880
limited understanding of how the system works.
But they're often this like sort of last line of
329
00:32:16,880 --> 00:32:22,960
defense, right? Where they're like, the buck stops
here, I'm the last thing that can intervene. And
330
00:32:22,960 --> 00:32:27,600
humans in the loop or workers in the loop are
often there because there are all these sorts
331
00:32:27,600 --> 00:32:33,200
of edge cases or modes of failure that are hard
to predict or can't be ruled out. And so, we need
332
00:32:33,200 --> 00:32:40,080
someone there to try to stop these different modes
of failure. But as a result of being sort of the
333
00:32:40,080 --> 00:32:48,640
last one in the line of responsibility is that the
workers in the loop become this moral crumple zone
334
00:32:48,640 --> 00:32:54,640
where responsibility for the system is attributed
to the human actor who ultimately has limited
335
00:32:54,640 --> 00:33:01,920
control over the automated or autonomous system.
Ellish also describes this as liability sponges.
336
00:33:01,920 --> 00:33:09,280
Again, this sort of like absorbing the impact
metaphor. One prominent example of this is in 2018
337
00:33:10,000 --> 00:33:17,040
a self-driving car for Uber hit a pedestrian.
The self-driving feature failed to detect the
338
00:33:17,040 --> 00:33:24,080
pedestrian, right? And the safety driver was
intervened too late to stop the collision.
339
00:33:24,800 --> 00:33:29,840
However, it was only the driver that faced
criminal charges, criminal legal charges
340
00:33:29,840 --> 00:33:35,120
as a result of this incident, and Uber did not,
right? So, we can see here the driver absorbing
341
00:33:35,120 --> 00:33:41,280
the legal, the legal impact of the death of
a pedestrian. While the technological system,
342
00:33:42,240 --> 00:33:49,520
Uber, did not face any sort of penalty. So,
there's a couple different problems with putting
343
00:33:49,520 --> 00:33:56,720
people in this role. So, first is deskilling.
Increased automation leads to the atrophy
344
00:33:56,720 --> 00:34:02,240
of skills over time. This is why we require
pilots to do so much flight simulation, right?
345
00:34:02,240 --> 00:34:07,920
Because no one wants a deskilled pilot, but
actually most of flying, right, is automated is on
346
00:34:07,920 --> 00:34:16,160
autopilot. So once your skills have deskilled, say
you've got a doctor doing sort of a radiologist
347
00:34:16,160 --> 00:34:22,720
who's been reliant on a sort of MRI algorithm
for years, we might worry that, you know,
348
00:34:22,720 --> 00:34:28,640
their ability to sort of see and notice these sort
of edge cases is going to get worse over time.
349
00:34:29,280 --> 00:34:33,760
The other thing is what they call the handoff
problem. Maybe some of you're familiar with this,
350
00:34:33,760 --> 00:34:41,920
but this is the idea that you're sort of
doing nothing , you are superfluous,
351
00:34:41,920 --> 00:34:46,800
and then all of a sudden you need to intervene
in an emergency situation or you need to notice
352
00:34:46,800 --> 00:34:51,600
a failure that the system has not notified you of,
right? So, in the case of the doctor with the MRI
353
00:34:51,600 --> 00:34:59,360
machine, you need to notice that the diagnoses
or recommendations are somehow now inaccurate.
354
00:35:00,480 --> 00:35:06,240
In the case of self-driving cars, right? You're
sitting there, you have, you're not doing anything
355
00:35:06,240 --> 00:35:13,280
but all of a sudden you need to sort of like debug
in an emergency situation. And humans just aren't
356
00:35:13,280 --> 00:35:19,600
great at, at sort of like not having anything
to do and then intervening on, in sort of like
357
00:35:19,600 --> 00:35:26,080
a crisis or high stakes situation. So, this is
one of the things where we put people in these
358
00:35:26,080 --> 00:35:30,080
situations where they're sort of psychologically
not super gr... they're just not, we just don't
359
00:35:30,080 --> 00:35:38,240
have great capacities for this. And the last thing
is that people have a lot of trust in machines. My
360
00:35:38,240 --> 00:35:42,640
significant other always tells me not to trust
Google Maps autonomously. You know, like always
361
00:35:42,640 --> 00:35:49,520
check the route first because you might like
know things or have different ways to think about
362
00:35:50,720 --> 00:35:56,800
the route that the, that the map doesn't.
But people tend to be like, “oh, wow, like,
363
00:35:56,800 --> 00:36:02,080
look at all this math, like surely it knows
more than I can do.” And so, we're supposed
364
00:36:02,080 --> 00:36:08,480
to be having workers or people come in and bring
context or expertise, but their deference to the
365
00:36:08,480 --> 00:36:13,200
machine that they think knows more than them
makes it so that they might revisit their
366
00:36:13,200 --> 00:36:19,760
judgment or feel really scared to intervene when
they don't have say like all the numbers. And so,
367
00:36:19,760 --> 00:36:27,280
there's a social tendency to overestimate the, the
capabilities of machines. So, what's wrong with
368
00:36:27,280 --> 00:36:34,240
this? From a, from a philosophical perspective,
not just like this labor perspective. This seems
369
00:36:34,240 --> 00:36:40,000
like a crappy situation to put workers in. The
problem is that the existence of these moral
370
00:36:40,000 --> 00:36:45,520
crumple zones violates a basic moral principle
about blame worthiness. And this is what in
371
00:36:45,520 --> 00:36:51,520
philosophy we call the control principle.
So, to give an example of how the control
372
00:36:51,520 --> 00:36:57,120
principle works, it's highly intuitive. Let's say
you're on a bus and the bus driver breaks really
373
00:36:57,120 --> 00:37:01,200
hard and the person next to you has their coffee.
You know, it's the morning you're on your way to
374
00:37:01,200 --> 00:37:07,920
work and their coffee spills all over your shirt.
And you're like, oh god, now I have to go to work,
375
00:37:07,920 --> 00:37:13,920
and I have like coffee-stained shirt. In this
scenario you might be like really bummed about
376
00:37:13,920 --> 00:37:19,760
your shirt being stained or ruined, but you
are not gonna blame the person next to you,
377
00:37:19,760 --> 00:37:23,280
right? 'cause “You're like, oh, well, it's not
their fault that the coffee spilled. It was
378
00:37:23,280 --> 00:37:27,920
'cause the bus brake so aggressively.” And maybe,
I mean, depending on the scenario, you might not
379
00:37:27,920 --> 00:37:33,200
even blame the bus driver because they might not
have been able to do otherwise as well, right?
380
00:37:33,200 --> 00:37:39,120
They had no control over the situation. Maybe they
wanted to avoid a collision. But if you're like,
381
00:37:39,120 --> 00:37:43,280
whenever I teach this, my students are like,
no, the person with a coffee mug should have
382
00:37:43,280 --> 00:37:49,360
had a better coffee mug. You know, like, who are
these irresponsible people with their like coffee
383
00:37:49,360 --> 00:37:55,120
mugs that can spill on others? And even if you
think that in this scenario, what you're really
384
00:37:55,120 --> 00:38:00,400
arguing about is about what sorts of actions are
under your control, right? You're saying that the
385
00:38:00,400 --> 00:38:05,600
coffee-mug person is blameworthy because they
should have exerted more control than they did,
386
00:38:05,600 --> 00:38:10,080
even though I think that's like kind of
ridiculousness scenario. We agree in the end.
387
00:38:10,080 --> 00:38:16,080
Okay? So the control principle, right? Is the idea
that you should only be morally responsible for
388
00:38:16,080 --> 00:38:21,280
what's in your control. And there are like a bunch
of reasons why we might intuitively accept this.
389
00:38:21,280 --> 00:38:25,600
I mean, if you've ever been blamed for something
that your sibling did growing up, like you get the
390
00:38:25,600 --> 00:38:31,360
sense in which this is like unfair to be blamed
for things that are not your fault. But it's also
391
00:38:31,360 --> 00:38:37,280
really unproductive, right? If you are blamed for
things that are outside of your control, the thing
392
00:38:37,280 --> 00:38:44,240
that ultimately caused the problem or was the
source of failure might not sort of be accountable
393
00:38:44,240 --> 00:38:50,080
or punished or sort of changed in the right sorts
of ways, right? So, when humans in the loop get
394
00:38:50,080 --> 00:38:56,240
blamed for errors of a technological system it
leads to sort of disruption of, of accountability,
395
00:38:56,240 --> 00:39:03,440
which might make us sort of less safe long term,
but also, it's just unfair. So, going back to our
396
00:39:03,440 --> 00:39:11,760
harms and wrongs, right? It's wrong and also
a harm. So, in conclusion, we've given this
397
00:39:11,760 --> 00:39:18,400
whole talk about AI ethics, , and the note
that we wanna end on is actually we should all be
398
00:39:18,400 --> 00:39:27,600
suspicious of ethics because it's not a panacea.
Is anyone familiar with the phrase greenwashing?
399
00:39:27,600 --> 00:39:34,080
Greenwashing? Yeah. Yeah. Okay. So greenwashing
is the idea that companies often take on this
400
00:39:34,080 --> 00:39:41,760
superficial sort of commitment to environmental
action by, you know I don't know, putting some
401
00:39:41,760 --> 00:39:48,960
like part of their product that's recyclable or
doing some sort of superficial change to change
402
00:39:48,960 --> 00:39:51,960
their sort of public image or avoid from
ANONOMOUS SPEAKER: BP.
403
00:39:51,960 --> 00:39:57,200
PATTERSON:
Yeah, exactly. Exactly. So, ethics washing
404
00:39:57,200 --> 00:40:04,160
is something that we can also see in this sort of,
is it an analogous concept. And we see it a lot in
405
00:40:04,160 --> 00:40:09,440
this sort of hype cycle within ai, is this sort
of practice of feigning or engaging with ethical
406
00:40:09,440 --> 00:40:15,040
considerations in a superficial way to sort of
improve the perception of your organization,
407
00:40:15,040 --> 00:40:21,440
right? Or maybe to avoid regulation, like you
don't need to regulate us, we're regulating
408
00:40:21,440 --> 00:40:29,760
ourselves. So in this example, right? Open
AI talks about being a leader in safety about
409
00:40:29,760 --> 00:40:36,160
DeepFakes and private information, even though,
you know, they're creating this text of video
410
00:40:36,160 --> 00:40:42,640
generator and text to voice generator that propose
massive, massive problems, right? For privacy
411
00:40:43,680 --> 00:40:48,240
and sort of fraud or things like that. So, I
mean, how can, it's a little bit weird to be like,
412
00:40:48,240 --> 00:40:53,920
we're the leader in safety, but also we're like
heavily contributing to a certain type of problem.
413
00:40:53,920 --> 00:41:02,240
So the last thing I'll just say is one of the
big things about sort of the failure of ethics
414
00:41:02,240 --> 00:41:08,880
in AI is that so often they sort of end up
actually creating this sort of superficial
415
00:41:08,880 --> 00:41:15,920
reassurance while actually justifying the press
forward for innovation or whatever. And part
416
00:41:15,920 --> 00:41:22,240
of that is because these sort of the various
different ethical programs that are implemented
417
00:41:22,240 --> 00:41:29,040
are disconnected from power. So people don't
actually have a ton of power over corporations,
418
00:41:29,040 --> 00:41:37,920
over researchers, over sort of decisions. They get
made. And so it's very hard to actually enforce
419
00:41:37,920 --> 00:41:43,760
or sort of change the status quo when there's
giant power asymmetries between people and these
420
00:41:43,760 --> 00:41:48,320
forms of technology.
ABBAS:
421
00:41:48,320 --> 00:41:53,280
So we'll conclude on the following. It's
the question that Alicia and I off well
422
00:41:53,280 --> 00:41:58,400
often enough ask, which is, is there anything new
here when we talk about ethics and AI, is this any
423
00:41:58,400 --> 00:42:04,560
different from ethics and automation or ethics and
technology? And part of the reason that that is a
424
00:42:04,560 --> 00:42:10,720
question is one the understandable conflation
of AI and automation. So not all automation
425
00:42:10,720 --> 00:42:15,520
is AI. So some of the questions we are asking
about ethics and AI are really about questions
426
00:42:15,520 --> 00:42:20,000
about ethics and automation more generally.
So you know, who's going to lose their jobs,
427
00:42:20,000 --> 00:42:26,480
et cetera. That's been asked for the last 150
years. The understandable, or as Alicia pointed
428
00:42:26,480 --> 00:42:31,120
out intentional, sometimes confusion on what
constitutes AI intentional in the sense that there
429
00:42:31,120 --> 00:42:35,920
is a lot of hype in this domain from people who
want to sell products or who want to raise funds.
430
00:42:35,920 --> 00:42:43,040
So, you know, you buy an LED and that's like a
smart LED now . So so again, some of the
431
00:42:43,040 --> 00:42:48,080
questions we ask about those systems, which one
might debate whether it's AI or not. And that's,
432
00:42:48,080 --> 00:42:53,120
I don't know how interesting that question even
is, honestly. Is it AI or not? That also leads
433
00:42:53,120 --> 00:42:58,480
to a question of, well, what's new about ethics
of AI? Then there is the theory free ideal. I
434
00:42:58,480 --> 00:43:05,440
forget the name of philosopher again, this is
from Zhanpei’s paper on AI being presented as
435
00:43:05,440 --> 00:43:13,040
this new disruptive, unprecedented thing, which
is disconnected from the scientific, scientific
436
00:43:13,600 --> 00:43:18,160
endeavor as it preceded it, right? So that's,
oh, it's so new that the ethical questions we're
437
00:43:18,160 --> 00:43:23,440
asking about it are defacto new. I don't, that's
not a logical conclusion one can draw, I think
438
00:43:23,440 --> 00:43:29,680
just on the face of it. Finally, well, what does
it matter if the question is new or not? I mean,
439
00:43:29,680 --> 00:43:35,840
so suppose all of this is old hat, so what, in
a sense the questions still need to be asked and
440
00:43:35,840 --> 00:43:43,200
looked at. Thank you.
ROBERTSON:
441
00:43:43,200 --> 00:43:47,920
Thanks for listening. I hope you learned something
about how philosophers and engineers are thinking
442
00:43:47,920 --> 00:43:53,280
about this the topic of ethics in AI. This is
just a small sample of what goes on here at
443
00:43:53,280 --> 00:43:58,000
Oregon State University in the area of AI,
where we have a long history of research,
444
00:43:58,000 --> 00:44:03,520
reaching back decades. More recently, Oregon
State became the first institution in the
445
00:44:03,520 --> 00:44:09,680
country to offer both master’s and doctoral
degrees in AI. If you’d like to learn more
446
00:44:09,680 --> 00:44:15,440
check out the bonus content on your podcast
app or at engineeringoutloud.oregonstate.edu.
