Ethics in AI presentation, S13E2

1
00:00:00,000 --> 00:00:03,920
ALICIA PATTERSON: So even if no harm comes 
to a particular person. So, for example,  
2
00:00:03,920 --> 00:00:09,680
even if I am not, say, negatively impacted 
— maybe I'm never detained by the police or  
3
00:00:09,680 --> 00:00:15,665
wrongly identified — we can still see that 
something is seemingly gone wrong here. 
4
00:00:15,665 --> 00:00:20,240
HOUSSAM ABBAS: Sometimes the effective function 
of the use of AI can be to focus attention on the  
5
00:00:20,240 --> 00:00:26,240
technology and away from how it's used by whom 
and to what ends. The ethical considerations  
6
00:00:26,240 --> 00:00:30,080
don't start at the point after you've deployed 
the system and now you're seeing some results.  
7
00:00:30,080 --> 00:00:34,800
The ethical considerations ought to come early on. 
And so, no one can hide behind their hand and say,  
8
00:00:34,800 --> 00:00:41,214
oh, I didn't see this coming.
[MUSIC: “The Ether Bunny,” by  
9
00:00:41,214 --> 00:00:42,720
Eyes Closed Audio, licensed under CC by 3.0] 
RACHEL ROBERTSON: From the College of Engineering  
10
00:00:42,720 --> 00:00:49,200
at Oregon State University, 
this is Engineering Out Loud.  
11
00:00:49,200 --> 00:00:54,480
Hi everyone. I’m Rachel Robertson. Today, on 
Engineering Out Loud you will hear more from  
12
00:00:54,480 --> 00:01:00,560
Alicia Patterson and Houssam Abbas. They are both 
assistant professors at Oregon State University  
13
00:01:00,560 --> 00:01:07,040
studying ethics in AI, but from very different 
perspectives. Alicia has her Ph.D. in philosophy  
14
00:01:07,040 --> 00:01:13,680
and her research focuses on the ethics of emerging 
technology, especially privacy and data ethics.  
15
00:01:13,680 --> 00:01:19,520
Houssam has a Ph.D. in electrical engineering and 
his research interests are in the verification and  
16
00:01:19,520 --> 00:01:25,440
control of cyber-physical systems and formal 
ethical theories for autonomous agents like  
17
00:01:25,440 --> 00:01:31,840
unpiloted ground and aerial vehicles. This 
recording is from a talk entitled “Where do  
18
00:01:31,840 --> 00:01:37,360
Ethics Belong in Artificial Intelligence?” They 
designed this talk for a general audience for a  
19
00:01:37,360 --> 00:01:43,760
presentation at OMSI, the Oregon Museum of Science 
and Industry. This recording is from a talk at  
20
00:01:43,760 --> 00:01:49,760
Oregon State University. Now, enjoy a deep dive 
into how philosophers and engineers are thinking  
21
00:01:49,760 --> 00:01:52,960
about ethics in artificial intelligence.
ABBAS: 
22
00:01:52,960 --> 00:02:00,240
Where do ethics belong in artificial intelligence? 
So, start with a story. And the story begins in  
23
00:02:00,240 --> 00:02:06,560
the 1950s in the psychiatric ward of Bellevue 
Hospital in New York where Bill McPhee was  
24
00:02:06,560 --> 00:02:13,840
committed. While committed there, he developed 
his idea for a voter prediction machine. The,  
25
00:02:13,840 --> 00:02:19,040
the point of it was, and here I quote from Jill 
Lepore's book, “If Then: How the Simulmatics  
26
00:02:19,040 --> 00:02:25,280
Corporation invented the Future.” So, the 
point of it was to create, literally quote,  
27
00:02:25,280 --> 00:02:32,720
“a set of rules of thinking so that it is possible 
to reproduce or closely approximate the thinking  
28
00:02:32,720 --> 00:02:38,880
behaviors of voters.” Bill McPhee was then 
released from the psychiatric ward at Bellevue and  
29
00:02:38,880 --> 00:02:45,120
finished his dissertation at Columbia University 
about a program for, to create a fully observable  
30
00:02:45,120 --> 00:02:51,280
electorate. You would feed the machine massive 
amounts, massive amounts of microscopic data about  
31
00:02:51,280 --> 00:02:57,120
users and issues, voters and issues. And it would 
be possible to assess the effect of any proposed  
32
00:02:57,120 --> 00:03:05,280
course of action by political candidate based on 
the likely reactions of the voters. Ed Greenfield,  
33
00:03:05,280 --> 00:03:10,000
who's was an ad man and was one of the key 
founders of the Simulmatics Corporation which  
34
00:03:10,000 --> 00:03:15,600
monetized the ideas of Bill McPhee and sold them.
He put it like this, he said, suppose that a  
35
00:03:15,600 --> 00:03:19,680
candidate was proposing to make a major civil 
rights speech in the South. Remember this was  
36
00:03:19,680 --> 00:03:29,600
the early sixties. We would be able, quote, 
“through our model to predict what the meaning  
37
00:03:29,600 --> 00:03:35,280
of that speech to over a thousand subgroups of 
a given population. And thus, we'll be able to  
38
00:03:35,280 --> 00:03:41,520
predict any the smallest percentage difference 
in votes, that is, that such a speech would  
39
00:03:41,520 --> 00:03:47,840
have.” The Simulmatics Corporation actually sold 
their services to the Kennedy campaign. Kennedy  
40
00:03:47,840 --> 00:03:53,200
then went on to win the presidential election that 
year. Now, it's not clear how much influence they  
41
00:03:53,200 --> 00:03:57,920
actually had, because the recommendations that 
Simulmatics gave the campaign were largely seen  
42
00:03:57,920 --> 00:04:04,800
to be common sense and would have been adopted 
anyway. But I think you can already see the how  
43
00:04:04,800 --> 00:04:14,960
that sort of foreshadows the general theme of 
users/voters/people prediction and then control. 
44
00:04:14,960 --> 00:04:19,840
The Cambridge Analytica — they basically sold a 
more sophisticated version of what Simulmatics  
45
00:04:19,840 --> 00:04:24,080
was trying to sell the Kennedy campaign to the 
Trump campaign in 2016 and to the Brexit Leave  
46
00:04:24,080 --> 00:04:32,000
campaign in 2015. And the reason I, I start with 
this anecdote is to directly cut through to what  
47
00:04:32,000 --> 00:04:36,960
perhaps is the biggest point of unease that 
some people might feel when they fret about  
48
00:04:36,960 --> 00:04:42,240
AI and ethics. The notion that human behavior 
is predictable and then controllable. So,  
49
00:04:42,240 --> 00:04:47,760
in particular, most of you have probably heard 
of the Facebook experiment on playing basically  
50
00:04:47,760 --> 00:04:53,440
and controlling user's emotions based 
on what they bubble up in their feeds. 
51
00:04:54,640 --> 00:05:04,160
Now, if users or people are predictable, at 
least in the aggregate and controllable, then  
52
00:05:04,160 --> 00:05:09,280
why not program them into robots? And that then 
brings us to the notion of embodied intelligence,  
53
00:05:09,280 --> 00:05:16,880
embodied artificial intelligence. One half of this 
is a humanoid of its own inventor. I'll give you  
54
00:05:16,880 --> 00:05:21,600
a couple of seconds to try to guess, which is the 
humanoid and which is the flesh and blood human,  
55
00:05:21,600 --> 00:05:26,400
right. This is the Geminiod DK by a 
Danish inventor whose name I now forget,  
56
00:05:26,400 --> 00:05:32,720
oh, it's in my notes. Henrik Scharf , 
right? And so, this reminds me of a famous,  
57
00:05:32,720 --> 00:05:37,440
which I had on the slides that I removed of a 
famous icon of Christ from the sixth century,  
58
00:05:37,440 --> 00:05:43,520
which is supposed to portray the dual nature of 
Christ as man and God. It sounds our ambitions  
59
00:05:43,520 --> 00:05:52,800
have now reduced to becoming man and machine.
So, so hopefully this gives you sort of the  
60
00:05:52,800 --> 00:05:58,640
corner cases or some of the corner cases when 
discussing AI and ethics in, in the way that  
61
00:05:58,640 --> 00:06:03,680
they relate mostly to us, like the notion that we 
might be predictable and controllable. If you want  
62
00:06:03,680 --> 00:06:07,920
to know what researchers are actually working 
on you can literally pick up like any issue  
63
00:06:07,920 --> 00:06:11,280
of the communications of the ACM from the last 
couple of years. And there will be an article  
64
00:06:11,280 --> 00:06:18,000
in there about AI and ethics, not ethics and CS in 
general, but specifically AI from bias and credit  
65
00:06:18,000 --> 00:06:23,840
rating to problems with certain problems with 
computer vision systems, and so on and so forth.  
66
00:06:23,840 --> 00:06:32,720
So rather than give a laundry list of that, I'll 
hand it over to Alicia to help us think through  
67
00:06:32,720 --> 00:06:39,200
a framework about these problems. Starting with a 
cautionary note that ethics is the hottest product  
68
00:06:39,200 --> 00:06:44,640
in Silicon Valley's hype cycle today. So, in order 
to step away from the hype and perhaps think about  
69
00:06:44,640 --> 00:06:47,760
it more productively: Alicia.
PATTERSON: 
70
00:06:47,760 --> 00:06:54,720
Okay, so philosophers have a set of tools to think 
about ethical problems that has been our bread and  
71
00:06:54,720 --> 00:07:02,320
butter for forever, basically . And so, 
we think maybe perhaps a little bit differently  
72
00:07:02,320 --> 00:07:09,280
about how we spot ethical problems. So, one of 
the key things, key sort of concepts we have  
73
00:07:09,280 --> 00:07:14,880
in philosophy are the concepts of a harm and 
a wrong. And we differentiate these because  
74
00:07:14,880 --> 00:07:19,760
they sort of mean different things or pick 
out different types of ethical problems. So,  
75
00:07:19,760 --> 00:07:25,680
a harm is something that sets back your material 
interests, it makes you worse off, right? We  
76
00:07:25,680 --> 00:07:32,320
might think of like suffering or something like 
that. So, if someone steps on your foot, right,  
77
00:07:32,320 --> 00:07:37,120
we can say that they've harmed you because your 
foot now hurts and maybe it's swollen. A wrong  
78
00:07:37,120 --> 00:07:46,000
in contrast is a, a sort of a violation of rights 
or some sort of disrespect of an important value  
79
00:07:46,000 --> 00:07:53,280
or denying someone something they're owed, right?
So, if I say deny, deny someone the right to vote  
80
00:07:53,280 --> 00:07:58,400
I've disrespected an important value and I've 
denied something, they've that they've, they're  
81
00:07:58,400 --> 00:08:06,080
owed, right? As a citizen, right? They're owed the 
right to vote. Perhaps you know, if the person is  
82
00:08:06,080 --> 00:08:10,400
in like, you know, not a swing state, their 
vote maybe doesn't make much of a difference.  
83
00:08:10,400 --> 00:08:15,360
So maybe I didn't harm them very much, but we can 
still see that I've done something wrong to them,  
84
00:08:15,360 --> 00:08:21,760
right? I've denied them something that they're 
owed. So, harms and wrongs usually go together,  
85
00:08:21,760 --> 00:08:26,560
but it's important to understand sort of their 
different contours to be able to fully map  
86
00:08:27,200 --> 00:08:36,560
ethical problems. So, some AI harms this when 
we originally made this, this wasn't such a  
87
00:08:36,560 --> 00:08:43,600
big thing in the news, but Cigna, a health 
insurance company uses algorithms, right? 
88
00:08:43,600 --> 00:08:50,320
To sort of batch deny claims. And doctors 
are supposed to go through and, and review  
89
00:08:50,320 --> 00:08:57,120
a patient's file, right? And see if the sort of 
medical claim is medically a medical necessity.  
90
00:08:57,680 --> 00:09:04,720
And according to research by ProPublica doctors 
were spending an average of 1.2 seconds on each  
91
00:09:04,720 --> 00:09:11,600
case. I don't know about you, but I, I feel like 
I could speed read a little bit, but 1.2 seconds  
92
00:09:11,600 --> 00:09:18,880
on a very serious medical decision seems that 
seems pretty bad, right? So, in this instance,  
93
00:09:18,880 --> 00:09:25,760
we get an algorithm, right, that caused a 
particular kind of harm, right? It may probably  
94
00:09:25,760 --> 00:09:32,480
denied thousands of people certain procedures. 
That means patients, doctors, hospital staff have  
95
00:09:32,480 --> 00:09:39,360
to do all of this extra labor to get the medical 
care that they deserve. And some people just might  
96
00:09:39,360 --> 00:09:45,680
not be able to like fully work the system and then 
give up and have to pay for things out of pocket. 
97
00:09:45,680 --> 00:09:52,160
So here we can see the way in which the 
system made people worse off, it harmed them.  
98
00:09:53,360 --> 00:10:03,920
In contrast a wrong may not necessarily 
involve a concrete harm, right? So,  
99
00:10:03,920 --> 00:10:11,040
an example of this is the use of facial 
recognition by law enforcement. One of the,  
100
00:10:11,040 --> 00:10:16,000
sort of, wrongs that we see is that people, a 
lot of times in AI systems are treated sort of  
101
00:10:16,000 --> 00:10:22,640
as near data in large data sets. And data is 
scraped in mass from the internet, right? This  
102
00:10:22,640 --> 00:10:29,280
is like all known to us. And there's no sort of 
regard for sort of fundamental values about how  
103
00:10:29,280 --> 00:10:36,400
these people might want to be treated, right? In 
this instance, the value of privacy. So, this is  
104
00:10:36,400 --> 00:10:41,840
a really interesting read, if you wanna check it 
out put together by researchers at Georgetown. So,  
105
00:10:41,840 --> 00:10:48,880
research at Georgetown found that one in two 
Americans have their faces and data sets that  
106
00:10:48,880 --> 00:10:54,320
are available to the police. And this is the 
exact kind of thing that we should worry about,  
107
00:10:54,320 --> 00:11:01,840
right? That we're sort of infringing on important 
values like democratic freedoms or privacy.  
108
00:11:01,840 --> 00:11:06,320
So even if no harm comes to a particular 
person, so for example, even if I am not,  
109
00:11:06,320 --> 00:11:14,320
say negatively impacted, maybe I'm never detained 
by the police or wrongly identified we can still  
110
00:11:14,320 --> 00:11:24,080
see that something is seemingly gone wrong here.
So, within the sort of scope of wrongs, injustice  
111
00:11:24,080 --> 00:11:28,560
usually sits within it because one of the things 
we want to pick out about injustice is that it's  
112
00:11:28,560 --> 00:11:36,480
a certain form of disrespect. So, when someone's 
committed an injustice, what's been … what value  
113
00:11:36,480 --> 00:11:42,080
or what principle has been violated is the value 
or principle of fairness or equality. Oftentimes,  
114
00:11:42,080 --> 00:11:49,360
injustice involves both a harm and a wrong, right? 
So, you're treated unfairly and your life is made  
115
00:11:49,360 --> 00:11:54,560
worse. So, if you say, are denied a promotion 
on the, say, the basis of your race or gender,  
116
00:11:54,560 --> 00:11:59,200
we can say like, you're worse off. You don't have 
the income that you would've had, right? The,  
117
00:11:59,200 --> 00:12:03,440
the career opportunities you would've 
had, but also, you've been disrespected,  
118
00:12:03,440 --> 00:12:10,400
right? You've been treated in a way that you 
shouldn't have. So those are some ways actually,  
119
00:12:10,400 --> 00:12:15,040
lemme give an example of injustice first.
So, an example of injustice, there's too many  
120
00:12:15,040 --> 00:12:22,560
to really even pick which one to talk about. But 
one example of automated bias, right? Is in racial  
121
00:12:22,560 --> 00:12:29,360
disparities in credit lending study by faculty 
in Stanford and University of Chicago looked into  
122
00:12:29,360 --> 00:12:35,840
this into sort of alternative models of credit 
scoring and found that low income and minority  
123
00:12:35,840 --> 00:12:43,280
groups were somewhere between five and 10% less, 
less likely to receive a loan than high income or  
124
00:12:43,280 --> 00:12:50,080
non-minority groups. So here we can see, right? 
Being denied the alone on the basis of your race  
125
00:12:50,080 --> 00:12:55,760
is both wrong and you're harmed, right? You're, 
you don't get the house and you're treated in a,  
126
00:12:55,760 --> 00:13:02,080
in a way that you should not have been treated. 
So, this is sort of the background when we think  
127
00:13:02,080 --> 00:13:09,040
about where is ethics coming into conversations 
in, in engineering and design, this is sort of  
128
00:13:09,040 --> 00:13:18,240
my like, toy example of the like design lifecycle.
Oftentimes, these sort of questions about ethics,  
129
00:13:18,240 --> 00:13:22,720
about harms and wrongs and things like that 
come in after a product has been designed,  
130
00:13:22,720 --> 00:13:28,240
their research has been done in it for years, 
and it's either launched or about to be launched,  
131
00:13:28,240 --> 00:13:34,640
and some problems are surfacing and people ask 
questions about you know, what sort of guardrail  
132
00:13:34,640 --> 00:13:40,880
should be in place? And one of the problems with 
this is that it's sort of too little too late,  
133
00:13:40,880 --> 00:13:46,320
right? When we ask questions about ethics, 
we should ask them from the very beginning,  
134
00:13:46,320 --> 00:13:51,760
right? Because asking them from the beginning 
allows us to ask a series of questions about  
135
00:13:51,760 --> 00:13:56,400
values through the entire design lifecycle, 
through the entire research process, right?  
136
00:13:57,840 --> 00:14:03,520
And that allows us to ask a broader set of 
questions, rather than this narrow question of  
137
00:14:03,520 --> 00:14:09,120
what sort of guardrail should we put in place?
Or what sort of use cases should this be  
138
00:14:09,120 --> 00:14:16,240
prohibited or encouraged for? So, a variety of 
questions that we should ask are things like,  
139
00:14:16,240 --> 00:14:20,560
before we even begin to develop this, is 
this technology the right solution to the  
140
00:14:20,560 --> 00:14:26,160
problem at all? Science and technology scholars 
critique this question not being asked a lot,  
141
00:14:26,160 --> 00:14:31,520
and they call this techno solutionism, the idea 
that a technology can be sort of the solution  
142
00:14:31,520 --> 00:14:36,640
for these big social problems that we're often 
not sitting back and saying, is this the right,  
143
00:14:36,640 --> 00:14:41,680
is this the right way to solve this problem? 
The next question that needs to be asked,  
144
00:14:41,680 --> 00:14:44,960
and this is, I'll say, I'm going through a 
series of questions, but this is by no means  
145
00:14:44,960 --> 00:14:51,920
an exhaustive list. Does the community want the 
problem solved in this way? Whose input do we  
146
00:14:51,920 --> 00:15:00,240
even have in thinking about what the problem is?
So, in design justice sort of like a canon text in  
147
00:15:01,440 --> 00:15:08,560
in sort of intersection between ethics and design, 
people have this phrase of nothing for us without  
148
00:15:08,560 --> 00:15:13,200
us, right? So, do people who are impacted by the 
technology have any sort of meaningful ways to  
149
00:15:13,200 --> 00:15:22,160
shape the values in design of it? The second or 
third, I guess we're on third we should foresee  
150
00:15:22,160 --> 00:15:27,600
the, we should look into the potential unforeseen 
consequences again, before we've like created a  
151
00:15:27,600 --> 00:15:33,520
product or we've put a ton of research into a 
certain type of algorithm, right? So feminist  
152
00:15:33,520 --> 00:15:38,160
technology scholars, for example, often point 
out that like the first public adoption of a  
153
00:15:38,160 --> 00:15:44,800
technology usually comes in the form of harm for 
women. So, one example, there's many I could give  
154
00:15:44,800 --> 00:15:51,120
you, is that when DeepFakes were first, like, 
open source available to people on the internet,  
155
00:15:51,120 --> 00:15:56,960
they were used in mass for non-consensual 
pornography or image-based sexual abuse. 
156
00:15:56,960 --> 00:16:01,600
So, here we can see like being the subject 
of pornography is something that women are  
157
00:16:01,600 --> 00:16:06,080
significantly more likely to be subject to. 
And so, we might have wanted the developers  
158
00:16:06,080 --> 00:16:11,600
to have thought about that or grapple with that 
before they released this technology, right? So,  
159
00:16:11,600 --> 00:16:17,840
these are the sorts of things that maybe we can 
solve for, prevent for early on. We also might  
160
00:16:17,840 --> 00:16:22,800
wanna ask, is this technology compatible with 
important social goods, like democratic values,  
161
00:16:22,800 --> 00:16:29,120
right? When we are creating something that say, 
involves a form of surveillance we might wanna  
162
00:16:29,120 --> 00:16:35,360
understand, will this significantly impact, say, 
people's ability to freely assemble. Freedom of  
163
00:16:35,360 --> 00:16:40,800
assembly is considered say, a core democratic 
value of value that you sort of a thing,  
164
00:16:40,800 --> 00:16:46,160
a right that you need to be able to express to 
be able to exist in a democracy at all. We also  
165
00:16:46,160 --> 00:16:51,440
might wanna ask, are we getting data in a way that 
respects important rights? As I mentioned before,  
166
00:16:51,440 --> 00:16:57,680
like a common practice and machine learning. 
It's just to do this mass web scraping. And we  
167
00:16:57,680 --> 00:17:01,920
might wanna ask a variety of questions about 
that. Like, are we respecting people's right  
168
00:17:01,920 --> 00:17:09,760
to say not participate in oppressive systems? 
Are we re respecting their rights to privacy? 
169
00:17:10,960 --> 00:17:14,800
We might also wanna ask our, some people 
going to be more harmed by this than others,  
170
00:17:14,800 --> 00:17:18,880
right? Maybe this question comes up the most when 
we're thinking about automating certain forms of  
171
00:17:18,880 --> 00:17:26,560
labor. Who are the winners and losers? And who 
might be especially vulnerable to the impacts of  
172
00:17:26,560 --> 00:17:34,880
a system? We wanna ask questions about bias before 
a product is launched. We wanna be able to grapple  
173
00:17:34,880 --> 00:17:41,120
with that. We don't want something. So as so often 
happens something gets launched and then users  
174
00:17:41,120 --> 00:17:46,080
who are negatively affected, who are of that group 
that the technology is biased against are the ones  
175
00:17:46,080 --> 00:17:49,840
being like, this doesn't work for us. This has 
happened again and again, right? So that's again,  
176
00:17:49,840 --> 00:17:56,640
something that we wanna catch early and often. 
We wanna ask what are the potential misuses,  
177
00:17:56,640 --> 00:18:04,640
for example, have we created a tool that's like 
really amazing, but also enables a lot of fraud?  
178
00:18:04,640 --> 00:18:09,680
And lastly then we can come to these questions 
about, say, guardrails. What sort of laws,  
179
00:18:09,680 --> 00:18:15,600
regulation policies and norms need to be put 
in place to protect people or, or to ensure  
180
00:18:15,600 --> 00:18:18,760
this sort of ethical use?
ABBAS: 
181
00:18:19,920 --> 00:18:28,240
Thank you. So, within that framework or against 
the background of that framework, we can look  
182
00:18:28,240 --> 00:18:36,000
at two major approaches so far in engineering to 
tackle the question of how does one in a nutshell  
183
00:18:36,000 --> 00:18:42,080
program certain ethical norms or social norms 
more generally in engineering systems. This is  
184
00:18:42,080 --> 00:18:48,240
a very high-level simplified view of that. One is 
utilitarianism, the other is non-utilitarianism,  
185
00:18:48,240 --> 00:18:56,000
or as sometimes referred to as deontology. I'll 
start with Deontology from the Greek they own duty  
186
00:18:56,000 --> 00:19:02,480
and laws for science or knowledge of. And whereas 
most of us probably think of the right thing to do  
187
00:19:02,480 --> 00:19:08,800
is the think that maximizes certain benefits, 
maximizes certain goods. Deontology says, no,  
188
00:19:08,800 --> 00:19:16,160
the way you assess whether some, there are certain 
moral norms that the moral agent ought to obey,  
189
00:19:16,160 --> 00:19:20,080
regardless of the consequences, regardless of 
the amount of good that that action produces,  
190
00:19:20,080 --> 00:19:25,440
or the amount of bad that that action leads to.
So, the deontology is a normative theory regarding  
191
00:19:25,440 --> 00:19:31,440
which choices are morally required, forbidden, 
or permitted. Those are three key categories. In  
192
00:19:31,440 --> 00:19:36,240
contrast, for example, to moral theories, which 
tell you what sort of person you ought to be,  
193
00:19:36,240 --> 00:19:41,040
right? What makes a choice, right, 
is its conformity with a moral norm.  
194
00:19:41,840 --> 00:19:49,040
So, if we adopt such an approach, and that's sort 
of the perspective I take in my own research on  
195
00:19:49,040 --> 00:19:55,040
algorithmic ethics. One then asks, how do we 
program duties into an AI agent, right? So,  
196
00:19:55,040 --> 00:20:00,400
if you have statements of which choices are 
morally required, forbidden or permitted, um, can  
197
00:20:00,400 --> 00:20:06,080
we think of them as design requirements, uh, for a 
subsystem of this overall system that we're trying  
198
00:20:06,080 --> 00:20:11,520
to create? So, you have your objective function, 
right? And this is the only math we're going to  
199
00:20:11,520 --> 00:20:16,320
see. I think you have your objective function, 
you want, uh, something to run fast, for example. 
200
00:20:16,320 --> 00:20:22,240
And then you have those things that are permitted, 
forbidden, and required, and you express those in  
201
00:20:22,240 --> 00:20:27,680
some mathematical formalism. And the idea is that 
once you've done, you've used these mathematical  
202
00:20:27,680 --> 00:20:32,000
formalisms to express these two different things. 
And that's sort of the point I wanna stress here.  
203
00:20:32,000 --> 00:20:38,960
That object, the performance objectives are not 
the normative constraints. Magic, magic, magic,  
204
00:20:38,960 --> 00:20:43,920
it turns into zeros and ones, and then lo and 
behold, Cassie can run a 5K without kicking a  
205
00:20:43,920 --> 00:20:49,360
rabbit on the way, right? So, what is this 
mathematical formalism that allows us to  
206
00:20:49,360 --> 00:20:55,520
express normative requirements? This, I think 
most of us are or ought to be familiar with,  
207
00:20:55,520 --> 00:20:59,920
we need basically a mathematical language that 
allows us to express such normative statements,  
208
00:20:59,920 --> 00:21:04,400
which you and I can understand because 
understanding is actually key here, right?  
209
00:21:04,960 --> 00:21:09,840
An objective function with a thousand parameters 
is not something you and I understand or anybody,  
210
00:21:09,840 --> 00:21:14,640
and whoever tells you that is probably fooling 
themselves and you. So, we want something that you  
211
00:21:14,640 --> 00:21:18,400
and I can understand, but which is also amenable 
to algorithmic, which is to say automatic,  
212
00:21:18,400 --> 00:21:25,280
automatic or automated processing and reasoning.
And the, and the language most of us use is  
213
00:21:25,280 --> 00:21:29,200
formal logic. You're all familiar with Boolean 
logic, but there's a zoo of logics out there,  
214
00:21:29,200 --> 00:21:34,080
some of which are appropriate or to some degree 
appropriate for reasoning about normative  
215
00:21:34,080 --> 00:21:40,400
statements. Once you do that, once you make 
that move, then reasoning becomes a calculation,  
216
00:21:40,400 --> 00:21:46,400
right? The dream of Leibniz was that when there 
are disputes among persons, we can simply say, let  
217
00:21:46,400 --> 00:21:52,320
us calculate, right? You, you run your argument, 
I run my argument, and the machine tells us, well,  
218
00:21:52,320 --> 00:21:57,440
which argument is a sound argument, right? Does 
A follow from B? And so, without further ado,  
219
00:21:57,440 --> 00:22:03,920
we can see who is right. We're nowhere near that, 
obviously. But that, that's the overall idea. Now,  
220
00:22:03,920 --> 00:22:09,680
formal methods and formal logic are heavily 
used for, um, safety purposes, for designing  
221
00:22:09,680 --> 00:22:15,920
safe systems, especially in safety critical 
applications like avionics, for example, right? 
222
00:22:16,800 --> 00:22:21,360
And in fact, in hardware, any hardware company 
today makes heavy use of formal logic. But what  
223
00:22:21,360 --> 00:22:26,800
we are talking about is using formal logics 
in order to formalize normative requirements,  
224
00:22:26,800 --> 00:22:31,280
ethical and social requirements. There's some work 
I've done. There's work done by Matthias Scheutz  
225
00:22:31,280 --> 00:22:40,240
at Tufts. Next week's AI seminar. Speaker Adnan 
Darwiche also makes use heavy use of formal logic  
226
00:22:40,240 --> 00:22:44,960
for explainable AI in his case. And I think 
the center of gravity really of, of this sort  
227
00:22:44,960 --> 00:22:49,840
of work still is in Europe especially 
in Vienna. So there, if you're curious,  
228
00:22:49,840 --> 00:22:54,320
you can look up that project that just launched 
centered in Vienna, but with collaboration with  
229
00:22:54,320 --> 00:23:00,080
Maryland. And the point one of the last thing 
I wanna say here is that this formalism allows  
230
00:23:00,080 --> 00:23:04,400
you to express and reason about normative 
requirements. It doesn't prevent you from doing  
231
00:23:04,400 --> 00:23:09,040
performance optimization the way we usually do.
The question is how do you do the two together?  
232
00:23:09,040 --> 00:23:16,960
So, you know, you can still look at graphs, 
. So finally, it's good not to drink  
233
00:23:16,960 --> 00:23:22,240
one's own Kool-Aid. Are there really rules 
that govern our thinking in general? Yes and  
234
00:23:22,240 --> 00:23:27,840
no. You certainly cannot formalize all the 
ways in which we actually think. Thinking  
235
00:23:27,840 --> 00:23:32,400
is not always what we call rational thinking. 
It is still different kind of thinking. So,  
236
00:23:33,040 --> 00:23:42,080
you know, the other form is utilitarianism.
The other the other way of trying to program  
237
00:23:42,960 --> 00:23:48,000
normative requirements is through utilitarianism, 
where you have, you know, the good is a function  
238
00:23:48,000 --> 00:23:52,880
of your actions. You have lots of optimizers 
that you throw at this mysterious function,  
239
00:23:52,880 --> 00:23:57,360
and it gives you the best action, which is 
supposedly the most ethical action because it  
240
00:23:57,360 --> 00:24:07,280
produces the most amount of good, right? It's, 
it goes only so far in my view, because there,  
241
00:24:07,280 --> 00:24:13,920
those are on some level inherently incommensurate 
things, right? So even, let's take a very trivial  
242
00:24:13,920 --> 00:24:19,360
example from this already old 2015 paper from 
Chris Gerdes, right? So, say we want, you  
243
00:24:19,360 --> 00:24:23,520
want the car to get to its destination, that's a 
performance objective without collisions. That's  
244
00:24:23,520 --> 00:24:28,480
a normative requirement. 'cause You, you know, 
a car can collide with someone and keep going,  
245
00:24:28,480 --> 00:24:32,400
right? So, it's not necessarily a performance 
requirement while minimizing emissions,  
246
00:24:32,400 --> 00:24:36,320
that's both performance and normative because 
emissions are bad for people in general. 
247
00:24:36,320 --> 00:24:40,640
So, what does it mean to weigh all these? Like 
how do you weigh these in meters? What sort of  
248
00:24:40,640 --> 00:24:43,440
objective function are you gonna put all 
of these in? What sort of reward function  
249
00:24:43,440 --> 00:24:48,800
are you gonna use in order to express all of 
these? Not to mention that sometimes ethical  
250
00:24:48,800 --> 00:24:52,960
requirements is not something that you can choose 
more or less of something, you just have to do it,  
251
00:24:52,960 --> 00:24:58,960
right? And so then you go into the multi-agent 
case, which complicates the picture. Nonetheless,  
252
00:24:58,960 --> 00:25:08,320
there is in fact the AI alignment literature. 
Most of it is on the side of, let's see, how is it  
253
00:25:08,320 --> 00:25:13,520
possible to design, for example, a reward function 
for an RL agent such that it avoids catastrophic  
254
00:25:13,520 --> 00:25:19,280
outcomes? Among that literature is reinforcement 
learning from human feedback, right? Where the  
255
00:25:19,280 --> 00:25:27,200
idea is suppose, you know, you want, for example, 
chat GPT to produce the ethically normed answers. 
256
00:25:27,200 --> 00:25:30,880
So what we do is we'll ask check GPT the same 
question 10 times. It's a stochastic machine,  
257
00:25:30,880 --> 00:25:34,320
it'll give us slightly different answers 10 times, 
and then we give it feedback on which one is the  
258
00:25:34,320 --> 00:25:40,160
best. And this way it tweaks its own parameters. 
But, now that you might say, okay, sounds  
259
00:25:40,160 --> 00:25:45,360
reasonable, but then it's, it's a market economy. 
So, we'll say, oh, are we gonna really gonna hire  
260
00:25:45,360 --> 00:25:50,720
a thousand people to do this a thousand times? 
No, what do we do? We put another AI in there.  
261
00:25:50,720 --> 00:25:57,560
So now you have an AI guiding another AI as, so 
then you climb the AI ladder to nowhere . 
262
00:25:59,440 --> 00:26:05,440
So, I'm not gonna pretend there's a good 
segue, we're just switching gears. ,  
263
00:26:05,440 --> 00:26:12,560
right? The problem of responsibility in autonomous 
systems. There's two models of responsibility,  
264
00:26:12,560 --> 00:26:18,240
fully autonomous systems, which I'm, I'm not going 
to talk about because in my mind, there's no sense  
265
00:26:18,240 --> 00:26:23,280
in which a fully autonomous machine is responsible 
for something when there's a human behind it.  
266
00:26:24,000 --> 00:26:28,560
So, we'll talk about . So, I just like 
swiped a whole bunch of things on the side.  
267
00:26:28,560 --> 00:26:33,280
We'll talk about semi-autonomous systems, okay? 
Workers in the loop. Usually, humans in the loop  
268
00:26:33,280 --> 00:26:37,040
is the terminology, but as Alicia was pointing out 
this morning, this is actually creating whole new  
269
00:26:37,040 --> 00:26:41,680
category of workers who have to watch terrible 
videos all day and so on and so forth. Now you  
270
00:26:41,680 --> 00:26:45,920
can have a worker in the loop for technical 
purposes, like, you know, radiologists as  
271
00:26:45,920 --> 00:26:50,400
checking the AI radiology algorithms, and you 
can have them for legal and moral purposes. 
272
00:26:50,400 --> 00:26:54,000
So even if this thing is great, you 
still need for legal moral purposes,  
273
00:26:54,000 --> 00:26:59,440
for example, a lawyer in the loop, a lawyer 
in the war room, and I'll take an example  
274
00:26:59,440 --> 00:27:05,200
from the Israeli Army's use of AI for lethal 
targeting. So, this is from March 2023,  
275
00:27:05,200 --> 00:27:13,200
so before the the latest war. So, so in from this 
article, so several high ranking IDF officers  
276
00:27:13,200 --> 00:27:19,520
informed the press that Israel is deploying AI 
tools as part of its military arsenal. Skip, skip,  
277
00:27:19,520 --> 00:27:24,080
skip. The public statements acknowledge that some 
of the targets attacked by the IDF are produced  
278
00:27:24,080 --> 00:27:29,920
by AI tools. Skip, skip. One encouraging aspect 
is that it seems that the, the IDF seeks to use  
279
00:27:29,920 --> 00:27:34,240
tools that complement human decision making rather 
than a substitute for the human factor. So here,  
280
00:27:34,240 --> 00:27:40,320
this is a human in the loop or a soldier 
basically in the loop acting as a backstop to  
281
00:27:40,320 --> 00:27:48,000
the decision or to to check the decision or the 
recommendation of the AI tool. The question is,  
282
00:27:48,000 --> 00:27:53,920
how likely is it that this person or this soldier 
is actually doing due diligence? We saw earlier  
283
00:27:53,920 --> 00:28:01,040
for the insurance claim that basically doctors are 
spending 1.2 seconds checking an insurance claim. 
284
00:28:01,040 --> 00:28:06,960
So, this is from the Guardian. And the point here 
I'm making is that having a human in the loop,  
285
00:28:06,960 --> 00:28:12,320
a worker in the loop is no panacea because that 
human is subject to the same social forces that  
286
00:28:12,320 --> 00:28:16,240
led to the creation of that AI system in the 
first place. And that AI system has a purpose,  
287
00:28:16,240 --> 00:28:20,720
and the purpose is to be automated. It is not 
to be slowed down by, in this case, the human.  
288
00:28:21,280 --> 00:28:28,480
So this is from a Guardian article in April, 2024 
sourced on like six sources who worked with the  
289
00:28:28,480 --> 00:28:33,600
Lavender system that the IDF was using in Gaza. 
“We were constantly being pressured, bring us more  
290
00:28:33,600 --> 00:28:39,600
targets. They really shouted at us. We were told, 
now we have to f-up Hamas no matter what the cost.  
291
00:28:39,600 --> 00:28:45,840
At its peak, the system managed to generate 37,000 
people as potential human targets. But the numbers  
292
00:28:45,840 --> 00:28:50,720
changed all the time because it depends on where 
you set the bar and what a Hamas operative is.” 
293
00:28:50,720 --> 00:28:55,040
This is a system with many parameters. It has many 
knobs. So, depending on where you set the knobs,  
294
00:28:55,040 --> 00:28:58,960
which is something as engineers we always do, 
right? We're tweaking parameters for performance  
295
00:28:58,960 --> 00:29:06,640
in this case for lethal targeting. So, the numbers 
change all the time. So, guess in which direction  
296
00:29:06,640 --> 00:29:12,640
the numbers are going to be trending in such 
a situation. Now, who's going to vet 37,000  
297
00:29:12,640 --> 00:29:19,520
targets or presumed targets. Skip to December 24. 
This is from a Washington Post story. Soldiers  
298
00:29:19,520 --> 00:29:23,360
who are poorly trained in using the technology 
attacked human targets without corroborating  
299
00:29:23,360 --> 00:29:27,280
Lavender's predictions at all. Lavender is 
the name of the system. At certain times the  
300
00:29:27,280 --> 00:29:32,640
only corroboration required was that the target 
was a male. So, the point here is that if the,  
301
00:29:32,640 --> 00:29:39,360
the worker in the loop, the soldier in the loop 
and depending on how the, the scale of which the  
302
00:29:39,360 --> 00:29:46,000
AI is being deployed is basically useless.
Even if that worker in the loop is a lawyer,  
303
00:29:46,000 --> 00:29:52,160
there is a whole literature on so-called 
war lawyers, right? Some legal experts  
304
00:29:52,160 --> 00:29:55,760
skip, skip harness the law in the service of 
the war effort. Now, the point being made in  
305
00:29:55,760 --> 00:30:00,640
this book is not simply that the lawyers are 
like a cog in the machine just basically giving  
306
00:30:00,640 --> 00:30:04,800
blank checks to whoever wants to target. It's a 
more nuanced point than that. But the point is,  
307
00:30:04,800 --> 00:30:09,520
it's a system as a whole that exists. And too, so 
to go back to Alicia's point earlier, the ethical  
308
00:30:09,520 --> 00:30:13,360
considerations don't start at the point after 
you've deployed the system and now you're seeing  
309
00:30:13,360 --> 00:30:18,480
some results. The ethical considerations ought to 
come early on. And so no one can hide behind their  
310
00:30:18,480 --> 00:30:26,720
hand and say, oh, I didn't see this coming.
So, and I owe my thinking on this point to  
311
00:30:26,720 --> 00:30:31,440
Zhanpei Fang who's here in the audience, AI is a 
rhetorical tool. Sometimes the effective function  
312
00:30:31,440 --> 00:30:37,680
of the use of AI can be to focus attention on 
the technology and away from how it's used by  
313
00:30:37,680 --> 00:30:43,840
whom and to what ends. It's like, oh, is the AI 
parameter, are the AI parameters correctly set?  
314
00:30:43,840 --> 00:30:49,040
Like, no, why is it being used? And this 
isn't a question one asks only about AI,  
315
00:30:49,040 --> 00:30:53,040
one can ask it about just technology in 
general, right? But we're discussing AI here. 
316
00:30:54,880 --> 00:30:56,320
Thank you.
PATTERSON: 
317
00:30:56,320 --> 00:31:05,520
Okay, so continuing on this problem of 
responsibility in AI systems. How many  
318
00:31:05,520 --> 00:31:10,240
of you familiar with the concept of a crumple 
zone? Excellent. This is so much different than  
319
00:31:10,240 --> 00:31:15,360
a public talk . So right, the crumple zone, 
lots of engineers, right? So, the crumple zone,  
320
00:31:15,360 --> 00:31:21,840
right, we know is meant to crumple so that 
the safety cage or where the passengers are,  
321
00:31:21,840 --> 00:31:27,840
is protected from the force of a collision, 
right? That's the point. The kinetic energy,  
322
00:31:27,840 --> 00:31:34,880
right? Is transformative as like controlled 
crumpling and it preserves the integrity.  
323
00:31:36,320 --> 00:31:43,440
Well, Madeline Clare Elish argues that there's 
a similar thing that happens for workers in the  
324
00:31:43,440 --> 00:31:50,720
loop or humans in the loop where they are sort 
of the thing that's supposed to crumple in order  
325
00:31:50,720 --> 00:31:56,720
to protect the integrity of the technological 
system, right? So, they are the one that takes,  
326
00:31:56,720 --> 00:32:04,800
they're sort of like the fall guy in a sense.
And this is a bit strange, right? It's weird  
327
00:32:04,800 --> 00:32:11,360
because human actors who are involved in these 
systems tend to have very limited control and  
328
00:32:11,360 --> 00:32:16,880
limited understanding of how the system works. 
But they're often this like sort of last line of  
329
00:32:16,880 --> 00:32:22,960
defense, right? Where they're like, the buck stops 
here, I'm the last thing that can intervene. And  
330
00:32:22,960 --> 00:32:27,600
humans in the loop or workers in the loop are 
often there because there are all these sorts  
331
00:32:27,600 --> 00:32:33,200
of edge cases or modes of failure that are hard 
to predict or can't be ruled out. And so, we need  
332
00:32:33,200 --> 00:32:40,080
someone there to try to stop these different modes 
of failure. But as a result of being sort of the  
333
00:32:40,080 --> 00:32:48,640
last one in the line of responsibility is that the 
workers in the loop become this moral crumple zone  
334
00:32:48,640 --> 00:32:54,640
where responsibility for the system is attributed 
to the human actor who ultimately has limited  
335
00:32:54,640 --> 00:33:01,920
control over the automated or autonomous system.
Ellish also describes this as liability sponges.  
336
00:33:01,920 --> 00:33:09,280
Again, this sort of like absorbing the impact 
metaphor. One prominent example of this is in 2018  
337
00:33:10,000 --> 00:33:17,040
a self-driving car for Uber hit a pedestrian. 
The self-driving feature failed to detect the  
338
00:33:17,040 --> 00:33:24,080
pedestrian, right? And the safety driver was 
intervened too late to stop the collision.  
339
00:33:24,800 --> 00:33:29,840
However, it was only the driver that faced 
criminal charges, criminal legal charges  
340
00:33:29,840 --> 00:33:35,120
as a result of this incident, and Uber did not, 
right? So, we can see here the driver absorbing  
341
00:33:35,120 --> 00:33:41,280
the legal, the legal impact of the death of 
a pedestrian. While the technological system,  
342
00:33:42,240 --> 00:33:49,520
Uber, did not face any sort of penalty. So, 
there's a couple different problems with putting  
343
00:33:49,520 --> 00:33:56,720
people in this role. So, first is deskilling. 
Increased automation leads to the atrophy  
344
00:33:56,720 --> 00:34:02,240
of skills over time. This is why we require 
pilots to do so much flight simulation, right? 
345
00:34:02,240 --> 00:34:07,920
Because no one wants a deskilled pilot, but 
actually most of flying, right, is automated is on  
346
00:34:07,920 --> 00:34:16,160
autopilot. So once your skills have deskilled, say 
you've got a doctor doing sort of a radiologist  
347
00:34:16,160 --> 00:34:22,720
who's been reliant on a sort of MRI algorithm 
for years, we might worry that, you know,  
348
00:34:22,720 --> 00:34:28,640
their ability to sort of see and notice these sort 
of edge cases is going to get worse over time.  
349
00:34:29,280 --> 00:34:33,760
The other thing is what they call the handoff 
problem. Maybe some of you're familiar with this,  
350
00:34:33,760 --> 00:34:41,920
but this is the idea that you're sort of 
doing nothing , you are superfluous,  
351
00:34:41,920 --> 00:34:46,800
and then all of a sudden you need to intervene 
in an emergency situation or you need to notice  
352
00:34:46,800 --> 00:34:51,600
a failure that the system has not notified you of, 
right? So, in the case of the doctor with the MRI  
353
00:34:51,600 --> 00:34:59,360
machine, you need to notice that the diagnoses 
or recommendations are somehow now inaccurate. 
354
00:35:00,480 --> 00:35:06,240
In the case of self-driving cars, right? You're 
sitting there, you have, you're not doing anything  
355
00:35:06,240 --> 00:35:13,280
but all of a sudden you need to sort of like debug 
in an emergency situation. And humans just aren't  
356
00:35:13,280 --> 00:35:19,600
great at, at sort of like not having anything 
to do and then intervening on, in sort of like  
357
00:35:19,600 --> 00:35:26,080
a crisis or high stakes situation. So, this is 
one of the things where we put people in these  
358
00:35:26,080 --> 00:35:30,080
situations where they're sort of psychologically 
not super gr... they're just not, we just don't  
359
00:35:30,080 --> 00:35:38,240
have great capacities for this. And the last thing 
is that people have a lot of trust in machines. My  
360
00:35:38,240 --> 00:35:42,640
significant other always tells me not to trust 
Google Maps autonomously. You know, like always  
361
00:35:42,640 --> 00:35:49,520
check the route first because you might like 
know things or have different ways to think about  
362
00:35:50,720 --> 00:35:56,800
the route that the, that the map doesn't.
But people tend to be like, “oh, wow, like,  
363
00:35:56,800 --> 00:36:02,080
look at all this math, like surely it knows 
more than I can do.” And so, we're supposed  
364
00:36:02,080 --> 00:36:08,480
to be having workers or people come in and bring 
context or expertise, but their deference to the  
365
00:36:08,480 --> 00:36:13,200
machine that they think knows more than them 
makes it so that they might revisit their  
366
00:36:13,200 --> 00:36:19,760
judgment or feel really scared to intervene when 
they don't have say like all the numbers. And so,  
367
00:36:19,760 --> 00:36:27,280
there's a social tendency to overestimate the, the 
capabilities of machines. So, what's wrong with  
368
00:36:27,280 --> 00:36:34,240
this? From a, from a philosophical perspective, 
not just like this labor perspective. This seems  
369
00:36:34,240 --> 00:36:40,000
like a crappy situation to put workers in. The 
problem is that the existence of these moral  
370
00:36:40,000 --> 00:36:45,520
crumple zones violates a basic moral principle 
about blame worthiness. And this is what in  
371
00:36:45,520 --> 00:36:51,520
philosophy we call the control principle.
So, to give an example of how the control  
372
00:36:51,520 --> 00:36:57,120
principle works, it's highly intuitive. Let's say 
you're on a bus and the bus driver breaks really  
373
00:36:57,120 --> 00:37:01,200
hard and the person next to you has their coffee. 
You know, it's the morning you're on your way to  
374
00:37:01,200 --> 00:37:07,920
work and their coffee spills all over your shirt. 
And you're like, oh god, now I have to go to work,  
375
00:37:07,920 --> 00:37:13,920
and I have like coffee-stained shirt. In this 
scenario you might be like really bummed about  
376
00:37:13,920 --> 00:37:19,760
your shirt being stained or ruined, but you 
are not gonna blame the person next to you,  
377
00:37:19,760 --> 00:37:23,280
right? 'cause “You're like, oh, well, it's not 
their fault that the coffee spilled. It was  
378
00:37:23,280 --> 00:37:27,920
'cause the bus brake so aggressively.” And maybe, 
I mean, depending on the scenario, you might not  
379
00:37:27,920 --> 00:37:33,200
even blame the bus driver because they might not 
have been able to do otherwise as well, right? 
380
00:37:33,200 --> 00:37:39,120
They had no control over the situation. Maybe they 
wanted to avoid a collision. But if you're like,  
381
00:37:39,120 --> 00:37:43,280
whenever I teach this, my students are like, 
no, the person with a coffee mug should have  
382
00:37:43,280 --> 00:37:49,360
had a better coffee mug. You know, like, who are 
these irresponsible people with their like coffee  
383
00:37:49,360 --> 00:37:55,120
mugs that can spill on others? And even if you 
think that in this scenario, what you're really  
384
00:37:55,120 --> 00:38:00,400
arguing about is about what sorts of actions are 
under your control, right? You're saying that the  
385
00:38:00,400 --> 00:38:05,600
coffee-mug person is blameworthy because they 
should have exerted more control than they did,  
386
00:38:05,600 --> 00:38:10,080
even though I think that's like kind of 
ridiculousness scenario. We agree in the end.  
387
00:38:10,080 --> 00:38:16,080
Okay? So the control principle, right? Is the idea 
that you should only be morally responsible for  
388
00:38:16,080 --> 00:38:21,280
what's in your control. And there are like a bunch 
of reasons why we might intuitively accept this. 
389
00:38:21,280 --> 00:38:25,600
I mean, if you've ever been blamed for something 
that your sibling did growing up, like you get the  
390
00:38:25,600 --> 00:38:31,360
sense in which this is like unfair to be blamed 
for things that are not your fault. But it's also  
391
00:38:31,360 --> 00:38:37,280
really unproductive, right? If you are blamed for 
things that are outside of your control, the thing  
392
00:38:37,280 --> 00:38:44,240
that ultimately caused the problem or was the 
source of failure might not sort of be accountable  
393
00:38:44,240 --> 00:38:50,080
or punished or sort of changed in the right sorts 
of ways, right? So, when humans in the loop get  
394
00:38:50,080 --> 00:38:56,240
blamed for errors of a technological system it 
leads to sort of disruption of, of accountability,  
395
00:38:56,240 --> 00:39:03,440
which might make us sort of less safe long term, 
but also, it's just unfair. So, going back to our  
396
00:39:03,440 --> 00:39:11,760
harms and wrongs, right? It's wrong and also 
a harm. So, in conclusion, we've given this  
397
00:39:11,760 --> 00:39:18,400
whole talk about AI ethics, , and the note 
that we wanna end on is actually we should all be  
398
00:39:18,400 --> 00:39:27,600
suspicious of ethics because it's not a panacea. 
Is anyone familiar with the phrase greenwashing? 
399
00:39:27,600 --> 00:39:34,080
Greenwashing? Yeah. Yeah. Okay. So greenwashing 
is the idea that companies often take on this  
400
00:39:34,080 --> 00:39:41,760
superficial sort of commitment to environmental 
action by, you know I don't know, putting some  
401
00:39:41,760 --> 00:39:48,960
like part of their product that's recyclable or 
doing some sort of superficial change to change  
402
00:39:48,960 --> 00:39:51,960
their sort of public image or avoid from
ANONOMOUS SPEAKER: BP. 
403
00:39:51,960 --> 00:39:57,200
PATTERSON:
Yeah, exactly. Exactly. So, ethics washing  
404
00:39:57,200 --> 00:40:04,160
is something that we can also see in this sort of, 
is it an analogous concept. And we see it a lot in  
405
00:40:04,160 --> 00:40:09,440
this sort of hype cycle within ai, is this sort 
of practice of feigning or engaging with ethical  
406
00:40:09,440 --> 00:40:15,040
considerations in a superficial way to sort of 
improve the perception of your organization,  
407
00:40:15,040 --> 00:40:21,440
right? Or maybe to avoid regulation, like you 
don't need to regulate us, we're regulating  
408
00:40:21,440 --> 00:40:29,760
ourselves. So in this example, right? Open 
AI talks about being a leader in safety about  
409
00:40:29,760 --> 00:40:36,160
DeepFakes and private information, even though, 
you know, they're creating this text of video  
410
00:40:36,160 --> 00:40:42,640
generator and text to voice generator that propose 
massive, massive problems, right? For privacy  
411
00:40:43,680 --> 00:40:48,240
and sort of fraud or things like that. So, I 
mean, how can, it's a little bit weird to be like,  
412
00:40:48,240 --> 00:40:53,920
we're the leader in safety, but also we're like 
heavily contributing to a certain type of problem. 
413
00:40:53,920 --> 00:41:02,240
So the last thing I'll just say is one of the 
big things about sort of the failure of ethics  
414
00:41:02,240 --> 00:41:08,880
in AI is that so often they sort of end up 
actually creating this sort of superficial  
415
00:41:08,880 --> 00:41:15,920
reassurance while actually justifying the press 
forward for innovation or whatever. And part  
416
00:41:15,920 --> 00:41:22,240
of that is because these sort of the various 
different ethical programs that are implemented  
417
00:41:22,240 --> 00:41:29,040
are disconnected from power. So people don't 
actually have a ton of power over corporations,  
418
00:41:29,040 --> 00:41:37,920
over researchers, over sort of decisions. They get 
made. And so it's very hard to actually enforce  
419
00:41:37,920 --> 00:41:43,760
or sort of change the status quo when there's 
giant power asymmetries between people and these  
420
00:41:43,760 --> 00:41:48,320
forms of technology.
ABBAS: 
421
00:41:48,320 --> 00:41:53,280
So we'll conclude on the following. It's 
the question that Alicia and I off well  
422
00:41:53,280 --> 00:41:58,400
often enough ask, which is, is there anything new 
here when we talk about ethics and AI, is this any  
423
00:41:58,400 --> 00:42:04,560
different from ethics and automation or ethics and 
technology? And part of the reason that that is a  
424
00:42:04,560 --> 00:42:10,720
question is one the understandable conflation 
of AI and automation. So not all automation  
425
00:42:10,720 --> 00:42:15,520
is AI. So some of the questions we are asking 
about ethics and AI are really about questions  
426
00:42:15,520 --> 00:42:20,000
about ethics and automation more generally. 
So you know, who's going to lose their jobs,  
427
00:42:20,000 --> 00:42:26,480
et cetera. That's been asked for the last 150 
years. The understandable, or as Alicia pointed  
428
00:42:26,480 --> 00:42:31,120
out intentional, sometimes confusion on what 
constitutes AI intentional in the sense that there  
429
00:42:31,120 --> 00:42:35,920
is a lot of hype in this domain from people who 
want to sell products or who want to raise funds. 
430
00:42:35,920 --> 00:42:43,040
So, you know, you buy an LED and that's like a 
smart LED now . So so again, some of the  
431
00:42:43,040 --> 00:42:48,080
questions we ask about those systems, which one 
might debate whether it's AI or not. And that's,  
432
00:42:48,080 --> 00:42:53,120
I don't know how interesting that question even 
is, honestly. Is it AI or not? That also leads  
433
00:42:53,120 --> 00:42:58,480
to a question of, well, what's new about ethics 
of AI? Then there is the theory free ideal. I  
434
00:42:58,480 --> 00:43:05,440
forget the name of philosopher again, this is 
from Zhanpei’s paper on AI being presented as  
435
00:43:05,440 --> 00:43:13,040
this new disruptive, unprecedented thing, which 
is disconnected from the scientific, scientific  
436
00:43:13,600 --> 00:43:18,160
endeavor as it preceded it, right? So that's, 
oh, it's so new that the ethical questions we're  
437
00:43:18,160 --> 00:43:23,440
asking about it are defacto new. I don't, that's 
not a logical conclusion one can draw, I think  
438
00:43:23,440 --> 00:43:29,680
just on the face of it. Finally, well, what does 
it matter if the question is new or not? I mean,  
439
00:43:29,680 --> 00:43:35,840
so suppose all of this is old hat, so what, in 
a sense the questions still need to be asked and  
440
00:43:35,840 --> 00:43:43,200
looked at. Thank you.
ROBERTSON: 
441
00:43:43,200 --> 00:43:47,920
Thanks for listening. I hope you learned something 
about how philosophers and engineers are thinking  
442
00:43:47,920 --> 00:43:53,280
about this the topic of ethics in AI. This is 
just a small sample of what goes on here at  
443
00:43:53,280 --> 00:43:58,000
Oregon State University in the area of AI, 
where we have a long history of research,  
444
00:43:58,000 --> 00:44:03,520
reaching back decades. More recently, Oregon 
State became the first institution in the  
445
00:44:03,520 --> 00:44:09,680
country to offer both master’s and doctoral 
degrees in AI. If you’d like to learn more  
446
00:44:09,680 --> 00:44:15,440
check out the bonus content on your podcast 
app or at engineeringoutloud.oregonstate.edu.