1
00:00:00,000 --> 00:00:08,240
[MUSIC]
2
00:00:08,240 --> 00:00:11,680
Hello and welcome to Computer Seance, a podcast featuring
3
00:00:11,680 --> 00:00:14,320
spirited discussion about AI and popular culture.
4
00:00:14,320 --> 00:00:15,320
I'm Justin.
5
00:00:15,320 --> 00:00:16,040
I'm Ben.
6
00:00:16,040 --> 00:00:16,960
And I'm Tracie.
7
00:00:16,960 --> 00:00:22,080
Each episode we take a seat at the table to conjure the spirits of AI past, present and future.
8
00:00:22,080 --> 00:00:24,400
Trying to find meaning in the technological wonders,
9
00:00:24,400 --> 00:00:25,840
imagined in popular fictions.
10
00:00:27,680 --> 00:00:31,280
All ideas presented in this podcast are understood to be the presenters' own.
11
00:00:31,280 --> 00:00:36,480
They do not reflect the thoughts, ideas, opinions or positions of our employers,
12
00:00:36,480 --> 00:00:38,560
their programs or their representatives.
13
00:00:38,560 --> 00:00:42,080
Ben, this week you're taking the helm.
14
00:00:42,080 --> 00:00:44,160
What do you have in the store for us?
15
00:00:44,160 --> 00:00:51,120
So for today I've selected War Games, the 1983 film where US high school student computer
16
00:00:51,120 --> 00:00:54,560
games enthusiast and hacker David, played by Matthew Broderick,
17
00:00:55,120 --> 00:00:57,920
starts to play global thermonuclear war.
18
00:00:57,920 --> 00:01:00,400
I think it brings up lots of interesting material for today.
19
00:01:00,400 --> 00:01:04,960
Before we go any further, spoiler warning, if you haven't seen War Games,
20
00:01:04,960 --> 00:01:08,720
you may want to hit pause and go watch it before continuing on with this episode.
21
00:01:08,720 --> 00:01:15,920
In the film, David attempts to hack a computer games company in California to get access
22
00:01:15,920 --> 00:01:17,200
to their upcoming titles.
23
00:01:17,200 --> 00:01:22,960
But he ends up accidentally hacking into NORAD, the US Strategic Nuclear Command.
24
00:01:22,960 --> 00:01:28,400
He accesses a list of games available on their AI supercomputer, the wonderfully named WOPR,
25
00:01:28,400 --> 00:01:34,480
or War Operation Plan Response, a machine that continually plays out World War 3 scenarios as
26
00:01:34,480 --> 00:01:36,320
war games in order to learn from them.
27
00:01:36,320 --> 00:01:41,680
With the help of his computer programmer friends, David continues the hack by finding a backdoor
28
00:01:41,680 --> 00:01:47,440
left by the original programmer Stephen Falken, the password being Joshua, the name of his son,
29
00:01:47,440 --> 00:01:49,440
who had died in a car crash.
30
00:01:49,440 --> 00:01:53,760
And this is the name given to the AI that's embodied within WOPR.
31
00:01:53,760 --> 00:01:57,120
David starts playing the game, global thermonuclear war,
32
00:01:57,120 --> 00:02:01,680
selecting from a long list of games that also includes chess and backgammon.
33
00:02:01,680 --> 00:02:07,120
WOPR has been plugged into the operational infrastructure of NORAD and the US missile silos,
34
00:02:07,120 --> 00:02:12,800
which has been done to automate operations and take the fallible, or apparently fallible humans
35
00:02:12,800 --> 00:02:17,680
out of the loop. Because of this, the lines between the games that are in the computer simulation
36
00:02:17,680 --> 00:02:21,600
and in the war room become blurred, bringing us to the edge of nuclear war.
37
00:02:21,600 --> 00:02:25,360
Before we go any further, Tracie, Justin, I just want to turn to you and ask,
38
00:02:25,360 --> 00:02:27,600
how did you find watching this film for today?
39
00:02:27,600 --> 00:02:31,200
I was actually surprised by how much of the film that I remembered.
40
00:02:31,200 --> 00:02:32,400
I really enjoyed it, actually.
41
00:02:32,400 --> 00:02:33,760
Good movie.
42
00:02:33,760 --> 00:02:34,800
I was the opposite.
43
00:02:34,800 --> 00:02:38,960
I think I'd forgotten most of this film, but I enjoyed Allie Sheedy specifically.
44
00:02:38,960 --> 00:02:41,520
I think I might have a lesbian love affair with her one day.
45
00:02:41,520 --> 00:02:43,840
Somehow I really love her character.
46
00:02:43,840 --> 00:02:47,440
I think she just brought like just about everything you possibly can to that
47
00:02:47,440 --> 00:02:47,920
Role
48
00:02:47,920 --> 00:02:50,800
Including aerobics, which is great.
49
00:02:50,800 --> 00:02:52,400
Including aerobics
50
00:02:52,400 --> 00:02:56,160
And can you both sum up the film in just one sentence?
51
00:02:56,160 --> 00:02:58,400
The only winning move is not to play.
52
00:02:58,400 --> 00:03:01,360
Ooh, that was good one, Tracie.
53
00:03:01,360 --> 00:03:02,720
That's a line from the movie.
54
00:03:02,720 --> 00:03:05,280
Yeah, mine would be one in doubt, Tic Tac Toe.
55
00:03:05,280 --> 00:03:10,240
So one of the things I was thinking about watching it again was what sort of film it is,
56
00:03:10,240 --> 00:03:12,560
because it's quite hard to categorize it.
57
00:03:12,560 --> 00:03:18,400
So sometimes it feels like the vibe is like a 1980s high school film sometimes.
58
00:03:18,400 --> 00:03:22,080
And then it's also a Cold War thriller.
59
00:03:22,080 --> 00:03:27,440
I think it kind of belongs to me to this genre that feels very 80s and 90s about
60
00:03:27,440 --> 00:03:30,720
game becomes the reality, the deadly game kind of scenario.
61
00:03:30,720 --> 00:03:34,720
Is that science fiction then?
62
00:03:34,720 --> 00:03:35,040
Or no?
63
00:03:35,040 --> 00:03:38,720
I mean, I think it's a techno thriller in some ways, you know?
64
00:03:39,280 --> 00:03:43,680
I was thinking this film was like, Ferris Bueller meets Dr. Strangelove on his day off.
65
00:03:43,680 --> 00:03:47,920
When we were watching this movie,
66
00:03:47,920 --> 00:03:53,200
my wife was also really annoyed at what she called a Matthew Broderick trope of
67
00:03:53,200 --> 00:03:56,160
the rules don't apply to me, you know?
68
00:03:56,160 --> 00:03:59,760
Like just performing this kind of screw everybody else.
69
00:03:59,760 --> 00:04:04,800
Like I have knowledge that nobody else has and I can do what I want.
70
00:04:05,680 --> 00:04:11,040
And I think to a certain extent that is an attitude that persists today as well, you know?
71
00:04:11,040 --> 00:04:15,040
When we talk about tech bro, that is exactly the type of person we're talking about.
72
00:04:15,040 --> 00:04:21,360
Yeah, I do mean it's not science fiction, but it's a adventure dramedy with Ally Sheedy
73
00:04:21,360 --> 00:04:23,440
That's all it matters is Ally Sheedy
74
00:04:23,440 --> 00:04:26,720
And Ally Sheedy is in Short Circuit
75
00:04:26,720 --> 00:04:28,720
Yeah, that's right. which is quite similar.
76
00:04:28,720 --> 00:04:32,880
Yeah, you can tell that the studios were like, let's make three or four of these with these, a key,
77
00:04:33,760 --> 00:04:36,720
cute, delicious little nuggets of actors.
78
00:04:36,720 --> 00:04:41,920
But like there is this thing about technology putting us into proximity with power and ways
79
00:04:41,920 --> 00:04:44,640
that we didn't have before in the movie Space Camp.
80
00:04:44,640 --> 00:04:50,480
You have these kids basically sitting inside a rocket ship and through technology.
81
00:04:50,480 --> 00:04:53,440
They actually have the ability to launch it and wind up in space.
82
00:04:53,440 --> 00:04:55,920
It's amazing. It has Val Kilmer in it.
83
00:04:55,920 --> 00:04:58,400
But I feel like they're kind of part of the fear we had.
84
00:04:58,400 --> 00:05:03,440
Like even a kid could break into our war gaming technology and mess around with it.
85
00:05:03,440 --> 00:05:08,800
Like there's this fear that kind of goes along with having that proximity being able to
86
00:05:08,800 --> 00:05:10,800
infiltrate power through technology.
87
00:05:10,800 --> 00:05:13,600
That makes me think of the way in which innocence is being exploited here.
88
00:05:13,600 --> 00:05:17,840
But also it's a great trope to have these innocent bodies accidentally engage with something that's
89
00:05:17,840 --> 00:05:18,880
well beyond them.
90
00:05:18,880 --> 00:05:21,680
But actually it's training young kids that you too can change the world.
91
00:05:21,680 --> 00:05:26,560
But by going in and playing thermonuclear war instead of tic-tac-toe.
92
00:05:26,560 --> 00:05:29,600
He's literally teaching people how to hack using a phone line.
93
00:05:29,600 --> 00:05:30,560
Unlike in hackers.
94
00:05:32,080 --> 00:05:34,080
Which you can learn no skills from.
95
00:05:34,080 --> 00:05:36,480
It's a very how to make out with everybody.
96
00:05:36,480 --> 00:05:43,600
To begin with I want to just focus on the initial decision to put WOPR or Joshua in control
97
00:05:43,600 --> 00:05:46,080
of the operations of the US nuclear arsenal.
98
00:05:46,080 --> 00:05:51,600
And to take humans out of the loop as they say in the film.
99
00:05:51,600 --> 00:05:55,360
And I think that's a scenario that really relates strongly today.
100
00:05:55,360 --> 00:05:57,440
So we're going to just begin by running through
101
00:05:57,440 --> 00:06:03,120
of opening scenes that set up the scenario.
102
00:06:03,120 --> 00:06:06,160
This first one which is the opening scene of the film which is in and around the US nuclear
103
00:06:06,160 --> 00:06:10,960
missile silo and has a lot of foreshadowing in that I think is interesting to pull out.
104
00:06:10,960 --> 00:06:13,760
Does someone want to take us through that opening sequence?
105
00:06:13,760 --> 00:06:17,040
You get the sense of like a shift change.
106
00:06:17,040 --> 00:06:19,200
Shift change that might occur at any old job.
107
00:06:19,200 --> 00:06:21,520
Income to the two new people.
108
00:06:21,520 --> 00:06:25,360
Outgo the two people who've been there for however long.
109
00:06:25,360 --> 00:06:29,760
They walk into like a regular looking house and a dusty kind of environment.
110
00:06:29,760 --> 00:06:34,160
And then they stand in front of I think it's a mirror or something like that.
111
00:06:34,160 --> 00:06:36,400
And they stand there and hold up their passes.
112
00:06:36,400 --> 00:06:39,920
And you realize that that house is the front to like a deep nuclear bunker.
113
00:06:39,920 --> 00:06:42,000
So these guys go in.
114
00:06:42,000 --> 00:06:45,520
They're running down their protocol.
115
00:06:45,520 --> 00:06:49,440
It obviously they have to have a lot of things done in pairs and agreement.
116
00:06:49,440 --> 00:06:55,120
And then I believe they think that they're actually undergoing a real situation.
117
00:06:55,120 --> 00:06:57,680
They don't actually know that it's a drill.
118
00:06:57,680 --> 00:07:01,360
And when push comes to shove and they're all the way up to the very end,
119
00:07:01,360 --> 00:07:04,000
we're just like one other guy has to turn his key.
120
00:07:04,000 --> 00:07:05,200
He just can't do it.
121
00:07:05,200 --> 00:07:07,680
There's like 20 million lives hanging in the balance.
122
00:07:07,680 --> 00:07:09,840
And he can't do it.
123
00:07:09,840 --> 00:07:12,400
And I think that's where that scene ends there, right?
124
00:07:12,400 --> 00:07:14,160
Before we go back to the bridge.
125
00:07:14,160 --> 00:07:17,360
So there's several bits of foreshadowing I think in the sequence.
126
00:07:17,360 --> 00:07:18,720
They arrive in a storm.
127
00:07:18,720 --> 00:07:19,680
They can't see anything.
128
00:07:19,680 --> 00:07:20,800
They can't see the roads.
129
00:07:20,800 --> 00:07:23,360
So we have a kind of foreshadowing of the fog of war.
130
00:07:23,360 --> 00:07:27,360
And it really expresses the isolation of the decision making.
131
00:07:27,360 --> 00:07:30,320
When they sit down, there's a warning light that comes on
132
00:07:30,320 --> 00:07:31,840
on their panel, big red light.
133
00:07:31,840 --> 00:07:33,920
And that like we've got a warning.
134
00:07:33,920 --> 00:07:34,560
And so what is it?
135
00:07:34,560 --> 00:07:36,000
Oh, it's just the warhead.
136
00:07:36,000 --> 00:07:38,640
The officer says, just give it a tap.
137
00:07:38,640 --> 00:07:40,640
And he gives it a tap and the warning light goes off.
138
00:07:40,640 --> 00:07:44,480
So you see there just the idea of kind of false signals.
139
00:07:44,480 --> 00:07:47,360
And needing to interpret this information, not just what it says,
140
00:07:47,360 --> 00:07:49,600
which I think obviously comes up a lot later.
141
00:07:49,600 --> 00:07:53,280
And then they get interrupted by the radio with the launch order.
142
00:07:53,280 --> 00:07:56,160
One of the two men wants to check with a human
143
00:07:56,160 --> 00:07:57,680
by getting them on the phone.
144
00:07:57,680 --> 00:08:01,040
So we cut from that to a scene in Norad,
145
00:08:01,040 --> 00:08:04,320
which is the North American Aerospace Defense Command.
146
00:08:04,320 --> 00:08:06,320
And it turns out the whole thing was an exercise
147
00:08:06,320 --> 00:08:09,280
to test the functioning of their procedures.
148
00:08:09,280 --> 00:08:10,480
There's people from the government.
149
00:08:10,480 --> 00:08:11,360
There's a general.
150
00:08:11,360 --> 00:08:13,760
There's a computer scientist guy, McKittrick
151
00:08:13,760 --> 00:08:15,280
who we'll come onto.
152
00:08:15,280 --> 00:08:17,600
They're really worried about how to explain to the president
153
00:08:17,600 --> 00:08:20,800
that so many of the missile commanders didn't launch their missiles.
154
00:08:20,800 --> 00:08:22,560
They didn't follow their orders, essentially.
155
00:08:23,520 --> 00:08:24,880
And there's an interesting exchange.
156
00:08:24,880 --> 00:08:28,880
The general, who's the sort of top military person there,
157
00:08:28,880 --> 00:08:31,360
he proposes better psychological screening.
158
00:08:31,360 --> 00:08:34,000
But the computer scientist, McKittrick
159
00:08:34,000 --> 00:08:36,880
says you can't screen out human response.
160
00:08:36,880 --> 00:08:39,360
And instead they should take the humans out of the loop,
161
00:08:39,360 --> 00:08:40,800
saying in a nuclear war,
162
00:08:40,800 --> 00:08:43,040
we can't afford to have the missiles lying dormant
163
00:08:43,040 --> 00:08:44,400
in those silos,
164
00:08:44,400 --> 00:08:46,240
because those men refuse to turn the keys
165
00:08:46,240 --> 00:08:48,400
when the computers tell them to.
166
00:08:48,400 --> 00:08:52,000
And we move from that to introducing the WOPR computer,
167
00:08:52,000 --> 00:08:55,920
which McKittrick proposes as replacing
168
00:08:55,920 --> 00:08:57,840
the human links and the command chain.
169
00:08:57,840 --> 00:08:59,360
So I'm going to play you a little clip
170
00:08:59,360 --> 00:09:02,880
that follows just after this meeting exchange in Norad,
171
00:09:02,880 --> 00:09:04,960
where McKittrick, the computer scientist,
172
00:09:04,960 --> 00:09:08,160
is taking all the others through to see the WAPA computer.
173
00:09:08,160 --> 00:09:10,560
This is Mr. Richter.
174
00:09:10,560 --> 00:09:12,560
Oh, would you like to tell these gentlemen about the WOPR?
175
00:09:12,560 --> 00:09:17,920
Well, the WOPR spends all its time thinking about World War III.
176
00:09:18,720 --> 00:09:23,040
24 hours a day, 365 days a year,
177
00:09:23,040 --> 00:09:25,680
plays an endless series of war games
178
00:09:25,680 --> 00:09:28,400
using all available information on the state of the world.
179
00:09:28,400 --> 00:09:32,720
The WOPR has already fought World War III as a game,
180
00:09:32,720 --> 00:09:34,640
time and time again.
181
00:09:34,640 --> 00:09:37,920
Picking out a few things I think are important,
182
00:09:37,920 --> 00:09:41,200
it spends all its time thinking about World War III.
183
00:09:41,200 --> 00:09:44,800
It is estimating Soviet responses to U.S.
184
00:09:44,800 --> 00:09:47,040
responses to Soviet responses and so on.
185
00:09:47,040 --> 00:09:52,080
So it's got a sophistication to how it's playing the game, let's say.
186
00:09:52,080 --> 00:09:55,840
And there's a really interesting tension, I think,
187
00:09:55,840 --> 00:09:59,280
between McKittrick and the other computer scientist.
188
00:09:59,280 --> 00:10:03,440
McKittrick is a bit frustrated with him talking too much about the theory,
189
00:10:03,440 --> 00:10:07,360
if you like, and he just wants to get in there and say how practical and useful it's going to be,
190
00:10:07,360 --> 00:10:08,800
which I think is a theme that will come up.
191
00:10:08,800 --> 00:10:15,040
Gentlemen, I wouldn't trust this overgrown pile of microchips
192
00:10:15,040 --> 00:10:16,160
further than i could throw it.
193
00:10:16,160 --> 00:10:19,440
And I don't know if you want to trust the safety of our country
194
00:10:19,440 --> 00:10:21,600
to some silicone dial.
195
00:10:21,600 --> 00:10:25,280
General, nobody is talking about entrusting safety
196
00:10:25,280 --> 00:10:27,280
the nation to a machine for God's sake.
197
00:10:27,280 --> 00:10:31,120
We'll keep control, but we'll keep it here at the top where it belongs.
198
00:10:31,120 --> 00:10:34,160
So the general doesn't trust the machine, obviously,
199
00:10:34,160 --> 00:10:38,720
but McKittrick doesn't think he's ceding control to the machine either.
200
00:10:38,720 --> 00:10:44,000
So maybe first we'll take the question of humans in and out of the loop as a topic.
201
00:10:44,000 --> 00:10:45,040
What do we think about that?
202
00:10:45,040 --> 00:10:52,960
I know that it is, let's say, an argument or a selling point of AI that I hear quite often that
203
00:10:52,960 --> 00:10:55,200
humans are biased.
204
00:10:55,200 --> 00:11:01,520
You know, machines might make decisions in ways that actually help to mitigate some of the biases.
205
00:11:01,520 --> 00:11:06,080
I think we need to continue looking at that kind of a topic because machines also introduce
206
00:11:06,080 --> 00:11:12,480
other types of biases. But I did think a lot about some recent research that has been about
207
00:11:12,480 --> 00:11:16,720
decision support systems and how people relying on decision support systems
208
00:11:16,720 --> 00:11:23,920
do start eventually following the judgment of what the machine has advised them.
209
00:11:23,920 --> 00:11:26,160
But I think that there's something to be said for
210
00:11:26,160 --> 00:11:29,920
having the ability to refuse orders.
211
00:11:29,920 --> 00:11:34,800
There have been other moments in history where people working in hierarchical organizations
212
00:11:34,800 --> 00:11:40,960
like the military would be refusing to follow orders as a measure of kind of showing that the
213
00:11:40,960 --> 00:11:46,640
people have changed. The people don't want whatever the people at the top are trying to lead them toward.
214
00:11:46,640 --> 00:11:52,880
And the fact that he finds that a human failing that this person was thinking and reflecting over
215
00:11:52,880 --> 00:12:00,240
the idea of killing 20 million people, that's not a failing. And it just this whole idea about that
216
00:12:00,240 --> 00:12:06,080
where he says, keep it at the top where it belongs. Actually, that is a fear of if you keep the humans
217
00:12:06,080 --> 00:12:08,720
out of the loop, there's nobody checking on the top.
218
00:12:08,720 --> 00:12:11,920
For some reason, this film makes me think a lot about speech act theory.
219
00:12:11,920 --> 00:12:15,760
As a performance study scholar, I'm pulling this from people like
220
00:12:15,760 --> 00:12:20,640
J.L. Austin and Benaviste, scholars who thought a lot about the ways in which our language systems
221
00:12:20,640 --> 00:12:25,520
can make us do things in the real world who will take the command, who will give the command,
222
00:12:25,520 --> 00:12:28,960
and what that command will happen once it is enacted or not.
223
00:12:28,960 --> 00:12:34,160
And this film is really working on this question of command. This is imperative, this short
224
00:12:34,160 --> 00:12:40,080
structure of language to enforce an action when that action is fires, when the kind of
225
00:12:40,080 --> 00:12:45,600
illucutionary act or what we might say speech act of misfires, what is the effect of that in the
226
00:12:45,600 --> 00:12:51,040
material world? Here we have that brought into high relief. So this is more of a design question,
227
00:12:51,040 --> 00:12:56,640
but maybe it kind of relates to what you're saying, like the affordances for thinking and speaking
228
00:12:56,640 --> 00:13:03,360
that an act gives. So when they're about to turn the nuclear thing, it's two people turning a key,
229
00:13:03,360 --> 00:13:07,120
like that's a very specific action. It's not just jamming a red button.
230
00:13:07,120 --> 00:13:14,160
It's turning something, you know, for me, it seems to indicate that whoever designed that kind of
231
00:13:14,160 --> 00:13:18,960
system would actually like a human being to reflect on what they're doing. It made me also think
232
00:13:18,960 --> 00:13:24,960
of course about the early fear of LLMs as they became available through OpenAI, the chat GPT structures
233
00:13:24,960 --> 00:13:29,920
that we've all kind of gotten to. The first thing that humans do is to ask it questions about
234
00:13:29,920 --> 00:13:35,280
suicide and death and they want to know the gross stuff and they also want to be calmed in the fears
235
00:13:35,280 --> 00:13:39,920
that this computer won't tell them to kill themselves or enact war. They need to know that there's a
236
00:13:39,920 --> 00:13:44,160
fell safe in the command. The first guy points a gun at the other guy because he's not following the
237
00:13:44,160 --> 00:13:50,800
command and that is their job. He's willing to kill his teammate so that the goal of the war game,
238
00:13:50,800 --> 00:13:56,640
which is to win, can be played out. There's real effects to the way in which our commands are
239
00:13:56,640 --> 00:14:02,640
out of our night. Human in the loop is often kind of portrayed as like, oh don't worry, we've got a
240
00:14:02,640 --> 00:14:09,680
human in the loop or we've still got a human place within what AI is doing in questions of automation
241
00:14:09,680 --> 00:14:15,520
in the workplace and so on. I was really reminded of this article I read recently by Cory Doctorow
242
00:14:15,520 --> 00:14:22,960
that talks about centaurs and reverse centaurs. A centaur is the idea of a human being assisted by a
243
00:14:22,960 --> 00:14:30,560
machine so the centaur being the kind of mythological animal that's like a human noise, upper body,
244
00:14:30,560 --> 00:14:37,840
on a horse's lower body. In an AI situation, the human mind is sort of extended through the kind
245
00:14:37,840 --> 00:14:44,240
of technological infrastructure and that's how we're often sold AI. Oh it'll expand your abilities
246
00:14:44,240 --> 00:14:50,240
to do this. But what Doctorow is saying is that the kind of business case for AI is the opposite of
247
00:14:50,240 --> 00:14:58,560
that so he calls it a reverse centaur. So where the humans are serving as appendages to the AI,
248
00:14:58,560 --> 00:15:04,000
so rather than kind of AI is an extra layer double checking the work that actually makes things more
249
00:15:04,000 --> 00:15:09,040
expensive and introduces more processes. Even if that makes things better but the business case for
250
00:15:09,040 --> 00:15:13,600
AI is not about making things better it's about making things cheaper. So actually you end up with
251
00:15:13,600 --> 00:15:18,320
kind of many less humans who are in the loop of all these AI is doing lots of work and then the
252
00:15:18,320 --> 00:15:25,600
humans become responsible for checking and for any mistakes that the AI's make so they take the fall
253
00:15:25,600 --> 00:15:33,920
effectively. So I have this friend who is an editor and she was telling me recently that she's like
254
00:15:33,920 --> 00:15:40,080
I don't I don't want to just work with AI that's her main job has now become about checking the
255
00:15:40,080 --> 00:15:46,640
edits that AI has done and for her that is not why she went into that work she used to love working
256
00:15:46,640 --> 00:15:52,480
with an author to really think about how to bring their work to life talk about some of the stylistic
257
00:15:52,480 --> 00:15:58,880
and personal choices and word use like really like a wordsmith type of person. And so like there's
258
00:15:58,880 --> 00:16:06,720
this idea that in work where we become like a helpmate to AI we're also being disconnected from
259
00:16:06,720 --> 00:16:12,000
other human beings within our job role. Thinking about the human in the loop the fear is that there
260
00:16:12,000 --> 00:16:16,480
will be no human loop right and this one reminds us the ways in which humans will be required to
261
00:16:16,480 --> 00:16:24,320
always be in the loop and or as humans will always kind of intervene in any system whether it's
262
00:16:24,320 --> 00:16:29,520
desired or not. I'm also interested in the way in which Cory Doctorow article does miss out early
263
00:16:29,520 --> 00:16:34,480
feminist thinking around social tech and the cyborg it might be more useful to think about the ways
264
00:16:34,480 --> 00:16:38,960
in which we were always already cyborg or if you don't want to use that term anymore the way in which
265
00:16:38,960 --> 00:16:46,080
our sociotechnical capacities have grown and the way in which that that growth has been
266
00:16:46,080 --> 00:16:51,840
mythologized allowing us to fully understand our engagement with the systems of AI. I don't know
267
00:16:51,840 --> 00:16:58,160
if either of you saw Sam Altman's recent comments about how expensive it is to train AI did you hear
268
00:16:58,160 --> 00:17:04,800
him talking about this? He was saying I don't think that's fair because think about how much energy
269
00:17:04,800 --> 00:17:14,720
it takes to train a human you've got 20 to 30 years of eating how like ecologically taxing humanity is
270
00:17:15,600 --> 00:17:22,640
and I don't even know if he sees how perverse that line of thinking is you know. Oh that guy my god
271
00:17:22,640 --> 00:17:28,960
and I guess this is the question is this entire turn of ever increasing forced engagement with AI
272
00:17:28,960 --> 00:17:35,600
systems part of a larger command control. I'm sound like a conspiracy theorist here but I actually
273
00:17:35,600 --> 00:17:41,200
think this is reasonable strategy. Yeah to get everybody thinking and doing along the same lines
274
00:17:41,200 --> 00:17:46,160
in addition to the research I mentioned earlier about people kind of trusting decision support
275
00:17:46,160 --> 00:17:51,280
algorithms over their own professional judgment. There was this MIT report recently that's been
276
00:17:51,280 --> 00:17:56,320
all over the media and of course it's a single study things need to be repeated we need to understand
277
00:17:56,320 --> 00:18:03,280
exactly how these things are working but that there could be some cognitive deficits involved in
278
00:18:03,280 --> 00:18:09,440
relying a lot on the use of large language models and how we learn and how we engage. I'm
279
00:18:09,440 --> 00:18:17,040
really struck by the Sam Altman quote essentially if machines were doing labor for free then it sort of
280
00:18:17,040 --> 00:18:24,560
becomes a slave labor market right and I feel like that's kind of what I get from how Altman's
281
00:18:24,560 --> 00:18:31,120
thinking that we're eating food just to kind of be laborers working for people like him. There was a
282
00:18:31,120 --> 00:18:37,840
comment to those comments that he made that was basically like he sees us all as like inefficient
283
00:18:37,840 --> 00:18:45,200
neat computers and I thought that is exactly what it sounds like. We talked about this in an
284
00:18:45,200 --> 00:18:52,000
episode on the podcast about virtuosity this idea that the fact that humans are human is a flaw
285
00:18:52,000 --> 00:19:00,720
that they are not perfect as a flaw and even just going back to war games the idea of mercy being a flaw
286
00:19:00,720 --> 00:19:06,240
if you're thinking about this from like a machine perspective and yes does every boss
287
00:19:07,200 --> 00:19:14,320
I'm guessing like every master wishes that everybody underneath them would just do exactly as they're
288
00:19:14,320 --> 00:19:21,280
told and every colonel in the army just wants their soldiers to do what they're told but the fact
289
00:19:21,280 --> 00:19:27,680
that we don't do that is actually a very important fail safe I think in human society.
290
00:19:27,680 --> 00:19:37,120
So in the film we have a theme which is around the military and there's also a theme of learning
291
00:19:37,120 --> 00:19:44,160
and the learning in military themes really come together in games so games as simulation and models
292
00:19:44,160 --> 00:19:51,120
and play the conjunction of these things has like an interesting history but it's also something that
293
00:19:51,120 --> 00:19:56,640
is really present right now I think. One of the clearest but probably oldest examples of this
294
00:19:56,640 --> 00:20:03,120
would be in the so-called unmanned aircraft where it's remote controlled that's back in the early
295
00:20:03,120 --> 00:20:08,720
days it be you have somebody in a remote location that's controlling this and there's something very
296
00:20:08,720 --> 00:20:13,840
gay-me about this and already back in those days people were talking about how that creates a
297
00:20:13,840 --> 00:20:19,520
psychological distance between soldiers and whatever their targets are whatever they're attacking
298
00:20:19,520 --> 00:20:26,640
that kind of removes some personal accountability and now we have autonomous unmanned aircraft that are
299
00:20:27,440 --> 00:20:34,480
flying into places doing reconnaissance choosing targets and executing their strikes on those targets
300
00:20:34,480 --> 00:20:43,440
without the involvement of any human that is in unique development that we're heading into right now
301
00:20:43,440 --> 00:20:49,840
and then you have some of the major heads of the large tech corporations stereo MOA from Anthropic.
302
00:20:50,800 --> 00:20:57,760
His vision is that the western nations like that what he's calling like the western democratic
303
00:20:57,760 --> 00:21:03,440
allies should be working together to share this technology and to build it out so that they can
304
00:21:03,440 --> 00:21:08,720
vanquish their enemies there's online games like that too where we live in worlds and we work together
305
00:21:08,720 --> 00:21:15,600
with people to vanquish our enemies and now this incredibly powerful technology that is reducing
306
00:21:15,600 --> 00:21:23,680
the time frame that it used to take us to think about whether or not and how and where to just obliterate
307
00:21:23,680 --> 00:21:29,840
a large number of targets and I think that all builds up this narrative of like AI as defense
308
00:21:29,840 --> 00:21:36,000
and where now offence as defense that I think is really dangerous in the current climate you have
309
00:21:36,000 --> 00:21:42,560
countries blowing up entire civilian infrastructure as a defense mechanism and I wonder if that goes
310
00:21:42,560 --> 00:21:49,520
back to this whole distance and removal that with every step that we're out of that loop more
311
00:21:49,520 --> 00:21:55,040
and more humanity gets lost and in war that's a really serious consequence. I was reading about
312
00:21:55,040 --> 00:22:03,040
the use of drones but also other kinds of robots in the war in Ukraine and the kind of training
313
00:22:03,040 --> 00:22:10,560
for the operators and one of the remarks was being an experienced gamer was better prep than having
314
00:22:10,560 --> 00:22:16,960
military experience and I think this is quite interesting because there’s this intersection with games
315
00:22:16,960 --> 00:22:21,520
which are often you know a lot a lot of computer games are around shooting that's a whole kind of
316
00:22:21,520 --> 00:22:28,800
genre but I think it blurs a line for me between like soldiering and killing. In the first person
317
00:22:28,800 --> 00:22:36,400
shooting game the aim is to kill things whereas soldiering isn't primarily about that or maybe
318
00:22:36,400 --> 00:22:41,520
hasn't always been about that you might fire a gun as a soldier to suppress people you might
319
00:22:41,520 --> 00:22:47,920
not aim to hit them but you deny an area or something like this and so this that sort of feels to me
320
00:22:47,920 --> 00:22:54,720
like the there's a kind of blurring of or change of what it means to be in the military to be more like
321
00:22:54,720 --> 00:23:01,600
the games that have been built right yeah so we're seeing kind of military actions that are much more
322
00:23:01,600 --> 00:23:08,080
like murder right or killing and perhaps the kind of nuclear war theme that's in the film like
323
00:23:08,080 --> 00:23:14,560
is a version of that but now rather than being the kind of one-off fear of world war three this is
324
00:23:14,560 --> 00:23:20,480
now becoming a more everyday kind of activity and more localized I mean we just had the President
325
00:23:20,480 --> 00:23:25,440
of the United States recently threatened to wipe an entire civilization off the face of the earth
326
00:23:26,160 --> 00:23:33,280
that's nuts more than nuts you know and the fact that his press secretary just wiped it away as
327
00:23:33,280 --> 00:23:40,400
blustery rhetoric you know and the problem is this collapsing between defense and offensive
328
00:23:40,400 --> 00:23:46,480
technologies I think is you know this I learned this term related to drones called
329
00:23:46,480 --> 00:23:53,600
loitering munitions and just that whole term the idea that it's basically a missile that
330
00:23:54,320 --> 00:23:59,840
is already up in the air and sits there waiting to find a target and then strikes it it's kind of
331
00:23:59,840 --> 00:24:06,720
changed the whole temporality of warfare as well it both speeds things up and extends them longer
332
00:24:06,720 --> 00:24:12,240
when we're fighting trench warfare how long do these battles go on people clash loads of
333
00:24:12,240 --> 00:24:17,760
casualties on both sides and then you need some time to go and recoup and bring your people back to
334
00:24:17,760 --> 00:24:25,040
battle now it's like this continual loop of highly destructive warfare there's also something
335
00:24:25,040 --> 00:24:32,320
around the temporal aspects of it I've heard this horrible phrase shortening the kill chain
336
00:24:32,320 --> 00:24:38,880
but how AI is being used for target selection for drone strikes or missile strikes and so on
337
00:24:38,880 --> 00:24:45,440
that it can analyze data much more quickly and it can mean the decisions about attacks can be made
338
00:24:45,440 --> 00:24:51,120
much more quickly and part of how they present that is that well the final decision will always be
339
00:24:51,120 --> 00:24:57,600
made by humans still so we have this humans in the loop idea here which is still the human being
340
00:24:57,600 --> 00:25:03,200
placed in a position of responsibility for the failure of AI right but also like removing all
341
00:25:03,200 --> 00:25:09,440
space for judgment removing time for judgment because if as you say like the Israeli I think it's
342
00:25:09,440 --> 00:25:16,640
called gospel that basically it would take a human analyst let's say a year to come up with 50 targets
343
00:25:16,640 --> 00:25:23,520
and this is coming up with thousands of targets in a very limited amount of time what human in the loop
344
00:25:23,520 --> 00:25:30,800
can do the appropriate checks and balances there to I mean we're already seeing this you know war is
345
00:25:30,800 --> 00:25:36,800
definitely targeting civilian infrastructure much more you know so clearly AI is not capable of
346
00:25:36,800 --> 00:25:42,560
making those nuanced decisions and I think what's worse is that governments don't care
347
00:25:42,560 --> 00:25:50,480
to bring just the discussion back to games before we move on games obviously run through all kinds
348
00:25:50,480 --> 00:25:58,400
of aspects of computing right so we're in so it's you know computer science and games design could be
349
00:25:58,400 --> 00:26:05,200
next to each other in a university department and so on we've seen games like chess as important
350
00:26:05,200 --> 00:26:12,880
tests of certain kinds of AI through the decades also we have game theory so game theory coming from
351
00:26:12,880 --> 00:26:19,280
von Neumann who's a pioneer of the modern computer let's say does come into kind of scenario
352
00:26:19,280 --> 00:26:25,360
planning and quite explicitly into kind of US nuclear strategy we see a little bit of that in the
353
00:26:25,360 --> 00:26:32,400
film when David's doing the research about Falcon he discovers a article called poker and Armageddon
354
00:26:33,440 --> 00:26:40,320
the role of bluffing in a nuclear standoff so if either of you to come across this games in the
355
00:26:40,320 --> 00:26:48,960
context of nuclear war no not me my my relationship to games is in terms of feeder games
356
00:26:48,960 --> 00:26:56,160
I come from the performance world games can break down and then reinforce social norms and
357
00:26:56,160 --> 00:27:00,560
that's what the tool is for when you're teaching within through games where we see a cognate with
358
00:27:00,560 --> 00:27:05,600
war games and theatre games of course is in the Campancino Theater and Theater of the Oppressed
359
00:27:05,600 --> 00:27:14,640
style from Boal etc where all of the kind of choreographic and strategic structures of games
360
00:27:14,640 --> 00:27:21,040
or things you can learn within a game are also a way to kind of push back on cultural repression
361
00:27:21,040 --> 00:27:26,080
that's happening within your own systems so yeah not in terms of nuclear war but definitely in terms
362
00:27:26,080 --> 00:27:33,440
of the ways in which gaming as a social interactive structure can help a sport the repressive apparatus
363
00:27:33,440 --> 00:27:40,080
of the state damn I'm not even sure how to follow that that was really brilliant when I think about
364
00:27:40,080 --> 00:27:45,040
like the kind of game theory that you're talking about the kind that really heavily influenced like
365
00:27:45,040 --> 00:27:51,840
the field of behavioral economics I've got so many criticisms of that kind of way of looking at it
366
00:27:51,840 --> 00:27:59,520
you know this ultra a very specific idea of rationality is involved in those games somewhat of a
367
00:27:59,520 --> 00:28:07,680
lack of respect for context and I don't know I kind of see that mirrored in the way that these
368
00:28:07,680 --> 00:28:12,960
kinds of games are envisioned as well these kinds of war games there's a scene where really kind of
369
00:28:12,960 --> 00:28:19,120
brings that out I think where David is a founder computer terminals talking to Joshua
370
00:28:20,080 --> 00:28:28,240
and is realizing that Joshua is still playing and Joshua is still playing because the primary goal
371
00:28:28,240 --> 00:28:34,480
in Joshua's words has not been achieved David asks Joshua what is the primary goal and the primary
372
00:28:34,480 --> 00:28:41,760
goal is quote to win the game right so Joshua doesn't have the concept of playing for fun which is a
373
00:28:41,760 --> 00:28:49,120
more complex goal it's a it's an embodied goal and I think that's kind of critical you know doing
374
00:28:49,120 --> 00:28:54,720
something for pleasure or doing something for fun yeah so it makes sense that a machine can't connect
375
00:28:54,720 --> 00:29:01,600
with that so one of the themes that's in the film is a kind of simulation versus reality binary
376
00:29:01,600 --> 00:29:07,520
for instance David asks is this a game or is it real to Joshua and Joshua replies what's the
377
00:29:07,520 --> 00:29:16,720
difference I should reach deathcon 1 and launch night missiles in 28 hours would you like to see some
378
00:29:16,720 --> 00:29:24,640
projected kill ratios 69% of the housing destroyed 72 million people dead
379
00:29:24,640 --> 00:29:37,200
is this a game or is it real what's the difference oh wow
380
00:29:37,200 --> 00:29:45,280
and I'm just wondering if you have some reactions to that when I watched yesterday I
381
00:29:46,560 --> 00:29:52,000
wasn't thinking about the interiority of Joshua slash whoppers kind of experience can we
382
00:29:52,000 --> 00:29:58,320
came up aware of that kind of misstep of mind what if the AI is learning if Joshua has the capacity
383
00:29:58,320 --> 00:30:04,160
to learn what has he learned in those 10 years potentially of stasis that he does not play in the
384
00:30:04,160 --> 00:30:09,760
game with with Falcon anymore and how fast can he learn given this and I guess one of the questions
385
00:30:09,760 --> 00:30:14,400
is this a version of hyper reality does Joshua exist in hyper reality and what does that mean for us
386
00:30:14,400 --> 00:30:21,440
hyper reality being a state whether the situation between the real and its representation collapses
387
00:30:21,440 --> 00:30:26,880
norad becomes a hyper real space once the game is in play but also this interior world
388
00:30:26,880 --> 00:30:32,800
the Joshua experiences must be hyper real I was thinking about how Falcon says that
389
00:30:32,800 --> 00:30:40,560
Joshua never learned futility you know giving up on something and the fact that it's called Joshua
390
00:30:40,560 --> 00:30:45,920
the fact that Joshua was the name of his son who died in the car accident this idea of like the
391
00:30:45,920 --> 00:30:52,720
machine as the baby of the developer comes up a lot and therefore also this idea of how you parent
392
00:30:52,720 --> 00:30:58,880
your baby machine how you raise it the things you teach it what's important to it or not this idea of
393
00:30:58,880 --> 00:31:04,960
you know having that responsibility but then also at some point you no longer have any
394
00:31:04,960 --> 00:31:09,520
and what Falcon is kind of explaining in that scene really reminds me a lot of actually a
395
00:31:09,520 --> 00:31:14,960
perspective from the 80s Michael Crichton had it too that it's sort of like oh well you know humans
396
00:31:14,960 --> 00:31:20,320
will eventually destroy themselves they probably deserve it let let everything take its course you
397
00:31:20,320 --> 00:31:25,280
know and I think that like when Falcon sitting there and he's like I don't know if I should intervene
398
00:31:25,280 --> 00:31:31,120
to stop it and that that was an argument back in the day and still is actually about climate change
399
00:31:31,120 --> 00:31:36,960
I feel like it's the futility issue would not be what I think is the biggest issue in terms of
400
00:31:36,960 --> 00:31:41,040
you know how to raise your machine baby it's the fact that machines don't understand any kind of
401
00:31:41,040 --> 00:31:48,080
vulnerability drones target children and non-combatants you know like Mercy is a human failing like I
402
00:31:48,080 --> 00:31:55,360
said earlier like that comes from somewhere the environment in which machines are taught to conceive
403
00:31:55,360 --> 00:32:02,960
of vulnerability in this way. A couple more clips to play which is from the final sequence and here we
404
00:32:02,960 --> 00:32:09,600
have you know the scene where Joshua is still playing the game and there's a trying to get the codes to
405
00:32:09,600 --> 00:32:16,480
launch the unique the missiles which is all going to be automated and so on and Falcon and David
406
00:32:16,480 --> 00:32:24,480
together gets the idea to get Joshua to play lots and crosses or take Tektoe to try to gain access
407
00:32:24,480 --> 00:32:38,080
to Joshua and also to teach it that it can't win which is its goal. And through that realizing the only
408
00:32:38,080 --> 00:32:45,760
winning move is not to play and deciding not to initiate the attack so lots to discuss in the scene
409
00:32:45,760 --> 00:32:51,200
that I think is about learning with machines for me there's something about different types of
410
00:32:51,200 --> 00:32:58,960
learning relating to the kind of game simulation the kind of learning that McKittrick wants Joshua to
411
00:32:58,960 --> 00:33:06,720
do is to learn how to be better at Nick the War measured in amounts of destruction and so on and
412
00:33:06,720 --> 00:33:13,120
Falcon says that there are people like McKittrick who think you can win this whereas what Joshua
413
00:33:13,120 --> 00:33:18,320
manages to learn at the end they learn something that's of a different type so they learn something
414
00:33:18,320 --> 00:33:23,360
that's a constraint of the game they've been learning to play better and I think that's a really
415
00:33:23,360 --> 00:33:28,640
interesting kind of juxtaposition the different kinds of learning that Joshua does the different
416
00:33:28,640 --> 00:33:34,800
kind of learning that Falcon wants Joshua to do versus McKittrick and taking that to today's AI
417
00:33:34,800 --> 00:33:40,400
and so on like I think everyone's under pressure in the workplace and then everyday life and so on
418
00:33:40,400 --> 00:33:45,920
to learn how to use AI that's given to us whereas one of the things we need to learn is more about
419
00:33:45,920 --> 00:33:51,280
what it's like what it's constraints are what it does yeah exactly yeah and it would be easy to
420
00:33:51,280 --> 00:33:57,760
miss it out by concentrating on what the McKittrick surround us want us to do for me I thought that
421
00:33:57,760 --> 00:34:03,440
like the realities that they all inhabit are kind of based around what they're perceiving as relevant
422
00:34:03,440 --> 00:34:10,320
and in the process of learning they're expanding that horizon of what's relevant to consider
423
00:34:10,320 --> 00:34:15,680
McKittrick isn't considering when he's suggesting that the machine run everything that a scenario
424
00:34:15,680 --> 00:34:21,680
such as this could could happen Joshua isn't considering the idea that there are some games you cannot
425
00:34:21,680 --> 00:34:28,720
win Falcon is not considering a lot of things you know his own personal accountability and responsibility
426
00:34:28,720 --> 00:34:34,800
for having created this thing which he might have done so to examine these loftier questions that he
427
00:34:34,800 --> 00:34:40,960
has in mind for himself but should he have considered the practical application of the things that he built maybe
428
00:34:41,840 --> 00:34:48,720
it's just a matter of your process of learning is your horizon being opened up to you about what
429
00:34:48,720 --> 00:34:54,880
things matter in the context do you want to share a final thought on what to take away from watching
430
00:34:54,880 --> 00:35:04,080
the film in relationship to today I'm really struck by how even in this early framing of AI
431
00:35:04,640 --> 00:35:13,280
and in relation to our globalized world we're hyper aware of the human capacity for pain and pleasure
432
00:35:13,280 --> 00:35:18,480
and knowledge and that seems to be stripped out once we've mechanized or automated learning and
433
00:35:18,480 --> 00:35:26,240
knowledge systems and how much time it would take to bring that type of embodied affective
434
00:35:26,240 --> 00:35:31,680
relationships knowledge into these other systems so yeah just really thinking and reflecting on
435
00:35:31,680 --> 00:35:39,680
the deep time of learning maybe in a similar vein thinking about how much like human experience and
436
00:35:39,680 --> 00:35:48,160
maybe even human biases in a positive sense how things like human interactive emotions and
437
00:35:48,160 --> 00:35:55,680
attitudes such as mercy and compassion and these are all also sort of embodied parts of our
438
00:35:55,680 --> 00:36:01,040
learning and relating to the world and things that machines can't understand or at least don't
439
00:36:01,040 --> 00:36:07,360
understand at the moment and so I'm very I'm very dubious about the application of AI in any kind
440
00:36:07,360 --> 00:36:13,120
of military setting I don't think it should be done I don't think war should be optimized made
441
00:36:13,120 --> 00:36:15,680
more efficient yeah none of that
442
00:36:15,680 --> 00:36:24,160
okay so Tracie Justin thank you both so much for your thoughts and thank you to everyone for joining
443
00:36:24,160 --> 00:36:28,640
us to listen to this podcast it's been a pleasure thinking about War Games with you. For more
444
00:36:28,640 --> 00:36:32,800
detail and references for our discussion in this episode please do check out the show notes and if
445
00:36:32,800 --> 00:36:38,080
you enjoy the episode please do subscribe or text it to a friend you can follow us on Instagram at
446
00:36:38,080 --> 00:36:44,880
computerseance.podcast that's at computerseance.podcast we want to hear your thoughts the more people we get into
447
00:36:44,880 --> 00:36:49,840
the conversation the smarter and honestly the more fun it gets
448
00:36:49,840 --> 00:37:08,240
will you be Ally again? thank you for listening. could you do this in the voice of Joshua please thank you for listening thank you for listening
449
00:37:10,400 --> 00:37:17,360
That was beautiful
450
00:37:17,360 --> 00:37:24,800
this podcast is funded by a UK research and innovation future leaders fellowship
00:00:00,000 --> 00:00:08,240
[MUSIC]
2
00:00:08,240 --> 00:00:11,680
Hello and welcome to Computer Seance, a podcast featuring
3
00:00:11,680 --> 00:00:14,320
spirited discussion about AI and popular culture.
4
00:00:14,320 --> 00:00:15,320
I'm Justin.
5
00:00:15,320 --> 00:00:16,040
I'm Ben.
6
00:00:16,040 --> 00:00:16,960
And I'm Tracie.
7
00:00:16,960 --> 00:00:22,080
Each episode we take a seat at the table to conjure the spirits of AI past, present and future.
8
00:00:22,080 --> 00:00:24,400
Trying to find meaning in the technological wonders,
9
00:00:24,400 --> 00:00:25,840
imagined in popular fictions.
10
00:00:27,680 --> 00:00:31,280
All ideas presented in this podcast are understood to be the presenters' own.
11
00:00:31,280 --> 00:00:36,480
They do not reflect the thoughts, ideas, opinions or positions of our employers,
12
00:00:36,480 --> 00:00:38,560
their programs or their representatives.
13
00:00:38,560 --> 00:00:42,080
Ben, this week you're taking the helm.
14
00:00:42,080 --> 00:00:44,160
What do you have in the store for us?
15
00:00:44,160 --> 00:00:51,120
So for today I've selected War Games, the 1983 film where US high school student computer
16
00:00:51,120 --> 00:00:54,560
games enthusiast and hacker David, played by Matthew Broderick,
17
00:00:55,120 --> 00:00:57,920
starts to play global thermonuclear war.
18
00:00:57,920 --> 00:01:00,400
I think it brings up lots of interesting material for today.
19
00:01:00,400 --> 00:01:04,960
Before we go any further, spoiler warning, if you haven't seen War Games,
20
00:01:04,960 --> 00:01:08,720
you may want to hit pause and go watch it before continuing on with this episode.
21
00:01:08,720 --> 00:01:15,920
In the film, David attempts to hack a computer games company in California to get access
22
00:01:15,920 --> 00:01:17,200
to their upcoming titles.
23
00:01:17,200 --> 00:01:22,960
But he ends up accidentally hacking into NORAD, the US Strategic Nuclear Command.
24
00:01:22,960 --> 00:01:28,400
He accesses a list of games available on their AI supercomputer, the wonderfully named WOPR,
25
00:01:28,400 --> 00:01:34,480
or War Operation Plan Response, a machine that continually plays out World War 3 scenarios as
26
00:01:34,480 --> 00:01:36,320
war games in order to learn from them.
27
00:01:36,320 --> 00:01:41,680
With the help of his computer programmer friends, David continues the hack by finding a backdoor
28
00:01:41,680 --> 00:01:47,440
left by the original programmer Stephen Falken, the password being Joshua, the name of his son,
29
00:01:47,440 --> 00:01:49,440
who had died in a car crash.
30
00:01:49,440 --> 00:01:53,760
And this is the name given to the AI that's embodied within WOPR.
31
00:01:53,760 --> 00:01:57,120
David starts playing the game, global thermonuclear war,
32
00:01:57,120 --> 00:02:01,680
selecting from a long list of games that also includes chess and backgammon.
33
00:02:01,680 --> 00:02:07,120
WOPR has been plugged into the operational infrastructure of NORAD and the US missile silos,
34
00:02:07,120 --> 00:02:12,800
which has been done to automate operations and take the fallible, or apparently fallible humans
35
00:02:12,800 --> 00:02:17,680
out of the loop. Because of this, the lines between the games that are in the computer simulation
36
00:02:17,680 --> 00:02:21,600
and in the war room become blurred, bringing us to the edge of nuclear war.
37
00:02:21,600 --> 00:02:25,360
Before we go any further, Tracie, Justin, I just want to turn to you and ask,
38
00:02:25,360 --> 00:02:27,600
how did you find watching this film for today?
39
00:02:27,600 --> 00:02:31,200
I was actually surprised by how much of the film that I remembered.
40
00:02:31,200 --> 00:02:32,400
I really enjoyed it, actually.
41
00:02:32,400 --> 00:02:33,760
Good movie.
42
00:02:33,760 --> 00:02:34,800
I was the opposite.
43
00:02:34,800 --> 00:02:38,960
I think I'd forgotten most of this film, but I enjoyed Allie Sheedy specifically.
44
00:02:38,960 --> 00:02:41,520
I think I might have a lesbian love affair with her one day.
45
00:02:41,520 --> 00:02:43,840
Somehow I really love her character.
46
00:02:43,840 --> 00:02:47,440
I think she just brought like just about everything you possibly can to that
47
00:02:47,440 --> 00:02:47,920
Role
48
00:02:47,920 --> 00:02:50,800
Including aerobics, which is great.
49
00:02:50,800 --> 00:02:52,400
Including aerobics
50
00:02:52,400 --> 00:02:56,160
And can you both sum up the film in just one sentence?
51
00:02:56,160 --> 00:02:58,400
The only winning move is not to play.
52
00:02:58,400 --> 00:03:01,360
Ooh, that was good one, Tracie.
53
00:03:01,360 --> 00:03:02,720
That's a line from the movie.
54
00:03:02,720 --> 00:03:05,280
Yeah, mine would be one in doubt, Tic Tac Toe.
55
00:03:05,280 --> 00:03:10,240
So one of the things I was thinking about watching it again was what sort of film it is,
56
00:03:10,240 --> 00:03:12,560
because it's quite hard to categorize it.
57
00:03:12,560 --> 00:03:18,400
So sometimes it feels like the vibe is like a 1980s high school film sometimes.
58
00:03:18,400 --> 00:03:22,080
And then it's also a Cold War thriller.
59
00:03:22,080 --> 00:03:27,440
I think it kind of belongs to me to this genre that feels very 80s and 90s about
60
00:03:27,440 --> 00:03:30,720
game becomes the reality, the deadly game kind of scenario.
61
00:03:30,720 --> 00:03:34,720
Is that science fiction then?
62
00:03:34,720 --> 00:03:35,040
Or no?
63
00:03:35,040 --> 00:03:38,720
I mean, I think it's a techno thriller in some ways, you know?
64
00:03:39,280 --> 00:03:43,680
I was thinking this film was like, Ferris Bueller meets Dr. Strangelove on his day off.
65
00:03:43,680 --> 00:03:47,920
When we were watching this movie,
66
00:03:47,920 --> 00:03:53,200
my wife was also really annoyed at what she called a Matthew Broderick trope of
67
00:03:53,200 --> 00:03:56,160
the rules don't apply to me, you know?
68
00:03:56,160 --> 00:03:59,760
Like just performing this kind of screw everybody else.
69
00:03:59,760 --> 00:04:04,800
Like I have knowledge that nobody else has and I can do what I want.
70
00:04:05,680 --> 00:04:11,040
And I think to a certain extent that is an attitude that persists today as well, you know?
71
00:04:11,040 --> 00:04:15,040
When we talk about tech bro, that is exactly the type of person we're talking about.
72
00:04:15,040 --> 00:04:21,360
Yeah, I do mean it's not science fiction, but it's a adventure dramedy with Ally Sheedy
73
00:04:21,360 --> 00:04:23,440
That's all it matters is Ally Sheedy
74
00:04:23,440 --> 00:04:26,720
And Ally Sheedy is in Short Circuit
75
00:04:26,720 --> 00:04:28,720
Yeah, that's right. which is quite similar.
76
00:04:28,720 --> 00:04:32,880
Yeah, you can tell that the studios were like, let's make three or four of these with these, a key,
77
00:04:33,760 --> 00:04:36,720
cute, delicious little nuggets of actors.
78
00:04:36,720 --> 00:04:41,920
But like there is this thing about technology putting us into proximity with power and ways
79
00:04:41,920 --> 00:04:44,640
that we didn't have before in the movie Space Camp.
80
00:04:44,640 --> 00:04:50,480
You have these kids basically sitting inside a rocket ship and through technology.
81
00:04:50,480 --> 00:04:53,440
They actually have the ability to launch it and wind up in space.
82
00:04:53,440 --> 00:04:55,920
It's amazing. It has Val Kilmer in it.
83
00:04:55,920 --> 00:04:58,400
But I feel like they're kind of part of the fear we had.
84
00:04:58,400 --> 00:05:03,440
Like even a kid could break into our war gaming technology and mess around with it.
85
00:05:03,440 --> 00:05:08,800
Like there's this fear that kind of goes along with having that proximity being able to
86
00:05:08,800 --> 00:05:10,800
infiltrate power through technology.
87
00:05:10,800 --> 00:05:13,600
That makes me think of the way in which innocence is being exploited here.
88
00:05:13,600 --> 00:05:17,840
But also it's a great trope to have these innocent bodies accidentally engage with something that's
89
00:05:17,840 --> 00:05:18,880
well beyond them.
90
00:05:18,880 --> 00:05:21,680
But actually it's training young kids that you too can change the world.
91
00:05:21,680 --> 00:05:26,560
But by going in and playing thermonuclear war instead of tic-tac-toe.
92
00:05:26,560 --> 00:05:29,600
He's literally teaching people how to hack using a phone line.
93
00:05:29,600 --> 00:05:30,560
Unlike in hackers.
94
00:05:32,080 --> 00:05:34,080
Which you can learn no skills from.
95
00:05:34,080 --> 00:05:36,480
It's a very how to make out with everybody.
96
00:05:36,480 --> 00:05:43,600
To begin with I want to just focus on the initial decision to put WOPR or Joshua in control
97
00:05:43,600 --> 00:05:46,080
of the operations of the US nuclear arsenal.
98
00:05:46,080 --> 00:05:51,600
And to take humans out of the loop as they say in the film.
99
00:05:51,600 --> 00:05:55,360
And I think that's a scenario that really relates strongly today.
100
00:05:55,360 --> 00:05:57,440
So we're going to just begin by running through
101
00:05:57,440 --> 00:06:03,120
of opening scenes that set up the scenario.
102
00:06:03,120 --> 00:06:06,160
This first one which is the opening scene of the film which is in and around the US nuclear
103
00:06:06,160 --> 00:06:10,960
missile silo and has a lot of foreshadowing in that I think is interesting to pull out.
104
00:06:10,960 --> 00:06:13,760
Does someone want to take us through that opening sequence?
105
00:06:13,760 --> 00:06:17,040
You get the sense of like a shift change.
106
00:06:17,040 --> 00:06:19,200
Shift change that might occur at any old job.
107
00:06:19,200 --> 00:06:21,520
Income to the two new people.
108
00:06:21,520 --> 00:06:25,360
Outgo the two people who've been there for however long.
109
00:06:25,360 --> 00:06:29,760
They walk into like a regular looking house and a dusty kind of environment.
110
00:06:29,760 --> 00:06:34,160
And then they stand in front of I think it's a mirror or something like that.
111
00:06:34,160 --> 00:06:36,400
And they stand there and hold up their passes.
112
00:06:36,400 --> 00:06:39,920
And you realize that that house is the front to like a deep nuclear bunker.
113
00:06:39,920 --> 00:06:42,000
So these guys go in.
114
00:06:42,000 --> 00:06:45,520
They're running down their protocol.
115
00:06:45,520 --> 00:06:49,440
It obviously they have to have a lot of things done in pairs and agreement.
116
00:06:49,440 --> 00:06:55,120
And then I believe they think that they're actually undergoing a real situation.
117
00:06:55,120 --> 00:06:57,680
They don't actually know that it's a drill.
118
00:06:57,680 --> 00:07:01,360
And when push comes to shove and they're all the way up to the very end,
119
00:07:01,360 --> 00:07:04,000
we're just like one other guy has to turn his key.
120
00:07:04,000 --> 00:07:05,200
He just can't do it.
121
00:07:05,200 --> 00:07:07,680
There's like 20 million lives hanging in the balance.
122
00:07:07,680 --> 00:07:09,840
And he can't do it.
123
00:07:09,840 --> 00:07:12,400
And I think that's where that scene ends there, right?
124
00:07:12,400 --> 00:07:14,160
Before we go back to the bridge.
125
00:07:14,160 --> 00:07:17,360
So there's several bits of foreshadowing I think in the sequence.
126
00:07:17,360 --> 00:07:18,720
They arrive in a storm.
127
00:07:18,720 --> 00:07:19,680
They can't see anything.
128
00:07:19,680 --> 00:07:20,800
They can't see the roads.
129
00:07:20,800 --> 00:07:23,360
So we have a kind of foreshadowing of the fog of war.
130
00:07:23,360 --> 00:07:27,360
And it really expresses the isolation of the decision making.
131
00:07:27,360 --> 00:07:30,320
When they sit down, there's a warning light that comes on
132
00:07:30,320 --> 00:07:31,840
on their panel, big red light.
133
00:07:31,840 --> 00:07:33,920
And that like we've got a warning.
134
00:07:33,920 --> 00:07:34,560
And so what is it?
135
00:07:34,560 --> 00:07:36,000
Oh, it's just the warhead.
136
00:07:36,000 --> 00:07:38,640
The officer says, just give it a tap.
137
00:07:38,640 --> 00:07:40,640
And he gives it a tap and the warning light goes off.
138
00:07:40,640 --> 00:07:44,480
So you see there just the idea of kind of false signals.
139
00:07:44,480 --> 00:07:47,360
And needing to interpret this information, not just what it says,
140
00:07:47,360 --> 00:07:49,600
which I think obviously comes up a lot later.
141
00:07:49,600 --> 00:07:53,280
And then they get interrupted by the radio with the launch order.
142
00:07:53,280 --> 00:07:56,160
One of the two men wants to check with a human
143
00:07:56,160 --> 00:07:57,680
by getting them on the phone.
144
00:07:57,680 --> 00:08:01,040
So we cut from that to a scene in Norad,
145
00:08:01,040 --> 00:08:04,320
which is the North American Aerospace Defense Command.
146
00:08:04,320 --> 00:08:06,320
And it turns out the whole thing was an exercise
147
00:08:06,320 --> 00:08:09,280
to test the functioning of their procedures.
148
00:08:09,280 --> 00:08:10,480
There's people from the government.
149
00:08:10,480 --> 00:08:11,360
There's a general.
150
00:08:11,360 --> 00:08:13,760
There's a computer scientist guy, McKittrick
151
00:08:13,760 --> 00:08:15,280
who we'll come onto.
152
00:08:15,280 --> 00:08:17,600
They're really worried about how to explain to the president
153
00:08:17,600 --> 00:08:20,800
that so many of the missile commanders didn't launch their missiles.
154
00:08:20,800 --> 00:08:22,560
They didn't follow their orders, essentially.
155
00:08:23,520 --> 00:08:24,880
And there's an interesting exchange.
156
00:08:24,880 --> 00:08:28,880
The general, who's the sort of top military person there,
157
00:08:28,880 --> 00:08:31,360
he proposes better psychological screening.
158
00:08:31,360 --> 00:08:34,000
But the computer scientist, McKittrick
159
00:08:34,000 --> 00:08:36,880
says you can't screen out human response.
160
00:08:36,880 --> 00:08:39,360
And instead they should take the humans out of the loop,
161
00:08:39,360 --> 00:08:40,800
saying in a nuclear war,
162
00:08:40,800 --> 00:08:43,040
we can't afford to have the missiles lying dormant
163
00:08:43,040 --> 00:08:44,400
in those silos,
164
00:08:44,400 --> 00:08:46,240
because those men refuse to turn the keys
165
00:08:46,240 --> 00:08:48,400
when the computers tell them to.
166
00:08:48,400 --> 00:08:52,000
And we move from that to introducing the WOPR computer,
167
00:08:52,000 --> 00:08:55,920
which McKittrick proposes as replacing
168
00:08:55,920 --> 00:08:57,840
the human links and the command chain.
169
00:08:57,840 --> 00:08:59,360
So I'm going to play you a little clip
170
00:08:59,360 --> 00:09:02,880
that follows just after this meeting exchange in Norad,
171
00:09:02,880 --> 00:09:04,960
where McKittrick, the computer scientist,
172
00:09:04,960 --> 00:09:08,160
is taking all the others through to see the WAPA computer.
173
00:09:08,160 --> 00:09:10,560
This is Mr. Richter.
174
00:09:10,560 --> 00:09:12,560
Oh, would you like to tell these gentlemen about the WOPR?
175
00:09:12,560 --> 00:09:17,920
Well, the WOPR spends all its time thinking about World War III.
176
00:09:18,720 --> 00:09:23,040
24 hours a day, 365 days a year,
177
00:09:23,040 --> 00:09:25,680
plays an endless series of war games
178
00:09:25,680 --> 00:09:28,400
using all available information on the state of the world.
179
00:09:28,400 --> 00:09:32,720
The WOPR has already fought World War III as a game,
180
00:09:32,720 --> 00:09:34,640
time and time again.
181
00:09:34,640 --> 00:09:37,920
Picking out a few things I think are important,
182
00:09:37,920 --> 00:09:41,200
it spends all its time thinking about World War III.
183
00:09:41,200 --> 00:09:44,800
It is estimating Soviet responses to U.S.
184
00:09:44,800 --> 00:09:47,040
responses to Soviet responses and so on.
185
00:09:47,040 --> 00:09:52,080
So it's got a sophistication to how it's playing the game, let's say.
186
00:09:52,080 --> 00:09:55,840
And there's a really interesting tension, I think,
187
00:09:55,840 --> 00:09:59,280
between McKittrick and the other computer scientist.
188
00:09:59,280 --> 00:10:03,440
McKittrick is a bit frustrated with him talking too much about the theory,
189
00:10:03,440 --> 00:10:07,360
if you like, and he just wants to get in there and say how practical and useful it's going to be,
190
00:10:07,360 --> 00:10:08,800
which I think is a theme that will come up.
191
00:10:08,800 --> 00:10:15,040
Gentlemen, I wouldn't trust this overgrown pile of microchips
192
00:10:15,040 --> 00:10:16,160
further than i could throw it.
193
00:10:16,160 --> 00:10:19,440
And I don't know if you want to trust the safety of our country
194
00:10:19,440 --> 00:10:21,600
to some silicone dial.
195
00:10:21,600 --> 00:10:25,280
General, nobody is talking about entrusting safety
196
00:10:25,280 --> 00:10:27,280
the nation to a machine for God's sake.
197
00:10:27,280 --> 00:10:31,120
We'll keep control, but we'll keep it here at the top where it belongs.
198
00:10:31,120 --> 00:10:34,160
So the general doesn't trust the machine, obviously,
199
00:10:34,160 --> 00:10:38,720
but McKittrick doesn't think he's ceding control to the machine either.
200
00:10:38,720 --> 00:10:44,000
So maybe first we'll take the question of humans in and out of the loop as a topic.
201
00:10:44,000 --> 00:10:45,040
What do we think about that?
202
00:10:45,040 --> 00:10:52,960
I know that it is, let's say, an argument or a selling point of AI that I hear quite often that
203
00:10:52,960 --> 00:10:55,200
humans are biased.
204
00:10:55,200 --> 00:11:01,520
You know, machines might make decisions in ways that actually help to mitigate some of the biases.
205
00:11:01,520 --> 00:11:06,080
I think we need to continue looking at that kind of a topic because machines also introduce
206
00:11:06,080 --> 00:11:12,480
other types of biases. But I did think a lot about some recent research that has been about
207
00:11:12,480 --> 00:11:16,720
decision support systems and how people relying on decision support systems
208
00:11:16,720 --> 00:11:23,920
do start eventually following the judgment of what the machine has advised them.
209
00:11:23,920 --> 00:11:26,160
But I think that there's something to be said for
210
00:11:26,160 --> 00:11:29,920
having the ability to refuse orders.
211
00:11:29,920 --> 00:11:34,800
There have been other moments in history where people working in hierarchical organizations
212
00:11:34,800 --> 00:11:40,960
like the military would be refusing to follow orders as a measure of kind of showing that the
213
00:11:40,960 --> 00:11:46,640
people have changed. The people don't want whatever the people at the top are trying to lead them toward.
214
00:11:46,640 --> 00:11:52,880
And the fact that he finds that a human failing that this person was thinking and reflecting over
215
00:11:52,880 --> 00:12:00,240
the idea of killing 20 million people, that's not a failing. And it just this whole idea about that
216
00:12:00,240 --> 00:12:06,080
where he says, keep it at the top where it belongs. Actually, that is a fear of if you keep the humans
217
00:12:06,080 --> 00:12:08,720
out of the loop, there's nobody checking on the top.
218
00:12:08,720 --> 00:12:11,920
For some reason, this film makes me think a lot about speech act theory.
219
00:12:11,920 --> 00:12:15,760
As a performance study scholar, I'm pulling this from people like
220
00:12:15,760 --> 00:12:20,640
J.L. Austin and Benaviste, scholars who thought a lot about the ways in which our language systems
221
00:12:20,640 --> 00:12:25,520
can make us do things in the real world who will take the command, who will give the command,
222
00:12:25,520 --> 00:12:28,960
and what that command will happen once it is enacted or not.
223
00:12:28,960 --> 00:12:34,160
And this film is really working on this question of command. This is imperative, this short
224
00:12:34,160 --> 00:12:40,080
structure of language to enforce an action when that action is fires, when the kind of
225
00:12:40,080 --> 00:12:45,600
illucutionary act or what we might say speech act of misfires, what is the effect of that in the
226
00:12:45,600 --> 00:12:51,040
material world? Here we have that brought into high relief. So this is more of a design question,
227
00:12:51,040 --> 00:12:56,640
but maybe it kind of relates to what you're saying, like the affordances for thinking and speaking
228
00:12:56,640 --> 00:13:03,360
that an act gives. So when they're about to turn the nuclear thing, it's two people turning a key,
229
00:13:03,360 --> 00:13:07,120
like that's a very specific action. It's not just jamming a red button.
230
00:13:07,120 --> 00:13:14,160
It's turning something, you know, for me, it seems to indicate that whoever designed that kind of
231
00:13:14,160 --> 00:13:18,960
system would actually like a human being to reflect on what they're doing. It made me also think
232
00:13:18,960 --> 00:13:24,960
of course about the early fear of LLMs as they became available through OpenAI, the chat GPT structures
233
00:13:24,960 --> 00:13:29,920
that we've all kind of gotten to. The first thing that humans do is to ask it questions about
234
00:13:29,920 --> 00:13:35,280
suicide and death and they want to know the gross stuff and they also want to be calmed in the fears
235
00:13:35,280 --> 00:13:39,920
that this computer won't tell them to kill themselves or enact war. They need to know that there's a
236
00:13:39,920 --> 00:13:44,160
fell safe in the command. The first guy points a gun at the other guy because he's not following the
237
00:13:44,160 --> 00:13:50,800
command and that is their job. He's willing to kill his teammate so that the goal of the war game,
238
00:13:50,800 --> 00:13:56,640
which is to win, can be played out. There's real effects to the way in which our commands are
239
00:13:56,640 --> 00:14:02,640
out of our night. Human in the loop is often kind of portrayed as like, oh don't worry, we've got a
240
00:14:02,640 --> 00:14:09,680
human in the loop or we've still got a human place within what AI is doing in questions of automation
241
00:14:09,680 --> 00:14:15,520
in the workplace and so on. I was really reminded of this article I read recently by Cory Doctorow
242
00:14:15,520 --> 00:14:22,960
that talks about centaurs and reverse centaurs. A centaur is the idea of a human being assisted by a
243
00:14:22,960 --> 00:14:30,560
machine so the centaur being the kind of mythological animal that's like a human noise, upper body,
244
00:14:30,560 --> 00:14:37,840
on a horse's lower body. In an AI situation, the human mind is sort of extended through the kind
245
00:14:37,840 --> 00:14:44,240
of technological infrastructure and that's how we're often sold AI. Oh it'll expand your abilities
246
00:14:44,240 --> 00:14:50,240
to do this. But what Doctorow is saying is that the kind of business case for AI is the opposite of
247
00:14:50,240 --> 00:14:58,560
that so he calls it a reverse centaur. So where the humans are serving as appendages to the AI,
248
00:14:58,560 --> 00:15:04,000
so rather than kind of AI is an extra layer double checking the work that actually makes things more
249
00:15:04,000 --> 00:15:09,040
expensive and introduces more processes. Even if that makes things better but the business case for
250
00:15:09,040 --> 00:15:13,600
AI is not about making things better it's about making things cheaper. So actually you end up with
251
00:15:13,600 --> 00:15:18,320
kind of many less humans who are in the loop of all these AI is doing lots of work and then the
252
00:15:18,320 --> 00:15:25,600
humans become responsible for checking and for any mistakes that the AI's make so they take the fall
253
00:15:25,600 --> 00:15:33,920
effectively. So I have this friend who is an editor and she was telling me recently that she's like
254
00:15:33,920 --> 00:15:40,080
I don't I don't want to just work with AI that's her main job has now become about checking the
255
00:15:40,080 --> 00:15:46,640
edits that AI has done and for her that is not why she went into that work she used to love working
256
00:15:46,640 --> 00:15:52,480
with an author to really think about how to bring their work to life talk about some of the stylistic
257
00:15:52,480 --> 00:15:58,880
and personal choices and word use like really like a wordsmith type of person. And so like there's
258
00:15:58,880 --> 00:16:06,720
this idea that in work where we become like a helpmate to AI we're also being disconnected from
259
00:16:06,720 --> 00:16:12,000
other human beings within our job role. Thinking about the human in the loop the fear is that there
260
00:16:12,000 --> 00:16:16,480
will be no human loop right and this one reminds us the ways in which humans will be required to
261
00:16:16,480 --> 00:16:24,320
always be in the loop and or as humans will always kind of intervene in any system whether it's
262
00:16:24,320 --> 00:16:29,520
desired or not. I'm also interested in the way in which Cory Doctorow article does miss out early
263
00:16:29,520 --> 00:16:34,480
feminist thinking around social tech and the cyborg it might be more useful to think about the ways
264
00:16:34,480 --> 00:16:38,960
in which we were always already cyborg or if you don't want to use that term anymore the way in which
265
00:16:38,960 --> 00:16:46,080
our sociotechnical capacities have grown and the way in which that that growth has been
266
00:16:46,080 --> 00:16:51,840
mythologized allowing us to fully understand our engagement with the systems of AI. I don't know
267
00:16:51,840 --> 00:16:58,160
if either of you saw Sam Altman's recent comments about how expensive it is to train AI did you hear
268
00:16:58,160 --> 00:17:04,800
him talking about this? He was saying I don't think that's fair because think about how much energy
269
00:17:04,800 --> 00:17:14,720
it takes to train a human you've got 20 to 30 years of eating how like ecologically taxing humanity is
270
00:17:15,600 --> 00:17:22,640
and I don't even know if he sees how perverse that line of thinking is you know. Oh that guy my god
271
00:17:22,640 --> 00:17:28,960
and I guess this is the question is this entire turn of ever increasing forced engagement with AI
272
00:17:28,960 --> 00:17:35,600
systems part of a larger command control. I'm sound like a conspiracy theorist here but I actually
273
00:17:35,600 --> 00:17:41,200
think this is reasonable strategy. Yeah to get everybody thinking and doing along the same lines
274
00:17:41,200 --> 00:17:46,160
in addition to the research I mentioned earlier about people kind of trusting decision support
275
00:17:46,160 --> 00:17:51,280
algorithms over their own professional judgment. There was this MIT report recently that's been
276
00:17:51,280 --> 00:17:56,320
all over the media and of course it's a single study things need to be repeated we need to understand
277
00:17:56,320 --> 00:18:03,280
exactly how these things are working but that there could be some cognitive deficits involved in
278
00:18:03,280 --> 00:18:09,440
relying a lot on the use of large language models and how we learn and how we engage. I'm
279
00:18:09,440 --> 00:18:17,040
really struck by the Sam Altman quote essentially if machines were doing labor for free then it sort of
280
00:18:17,040 --> 00:18:24,560
becomes a slave labor market right and I feel like that's kind of what I get from how Altman's
281
00:18:24,560 --> 00:18:31,120
thinking that we're eating food just to kind of be laborers working for people like him. There was a
282
00:18:31,120 --> 00:18:37,840
comment to those comments that he made that was basically like he sees us all as like inefficient
283
00:18:37,840 --> 00:18:45,200
neat computers and I thought that is exactly what it sounds like. We talked about this in an
284
00:18:45,200 --> 00:18:52,000
episode on the podcast about virtuosity this idea that the fact that humans are human is a flaw
285
00:18:52,000 --> 00:19:00,720
that they are not perfect as a flaw and even just going back to war games the idea of mercy being a flaw
286
00:19:00,720 --> 00:19:06,240
if you're thinking about this from like a machine perspective and yes does every boss
287
00:19:07,200 --> 00:19:14,320
I'm guessing like every master wishes that everybody underneath them would just do exactly as they're
288
00:19:14,320 --> 00:19:21,280
told and every colonel in the army just wants their soldiers to do what they're told but the fact
289
00:19:21,280 --> 00:19:27,680
that we don't do that is actually a very important fail safe I think in human society.
290
00:19:27,680 --> 00:19:37,120
So in the film we have a theme which is around the military and there's also a theme of learning
291
00:19:37,120 --> 00:19:44,160
and the learning in military themes really come together in games so games as simulation and models
292
00:19:44,160 --> 00:19:51,120
and play the conjunction of these things has like an interesting history but it's also something that
293
00:19:51,120 --> 00:19:56,640
is really present right now I think. One of the clearest but probably oldest examples of this
294
00:19:56,640 --> 00:20:03,120
would be in the so-called unmanned aircraft where it's remote controlled that's back in the early
295
00:20:03,120 --> 00:20:08,720
days it be you have somebody in a remote location that's controlling this and there's something very
296
00:20:08,720 --> 00:20:13,840
gay-me about this and already back in those days people were talking about how that creates a
297
00:20:13,840 --> 00:20:19,520
psychological distance between soldiers and whatever their targets are whatever they're attacking
298
00:20:19,520 --> 00:20:26,640
that kind of removes some personal accountability and now we have autonomous unmanned aircraft that are
299
00:20:27,440 --> 00:20:34,480
flying into places doing reconnaissance choosing targets and executing their strikes on those targets
300
00:20:34,480 --> 00:20:43,440
without the involvement of any human that is in unique development that we're heading into right now
301
00:20:43,440 --> 00:20:49,840
and then you have some of the major heads of the large tech corporations stereo MOA from Anthropic.
302
00:20:50,800 --> 00:20:57,760
His vision is that the western nations like that what he's calling like the western democratic
303
00:20:57,760 --> 00:21:03,440
allies should be working together to share this technology and to build it out so that they can
304
00:21:03,440 --> 00:21:08,720
vanquish their enemies there's online games like that too where we live in worlds and we work together
305
00:21:08,720 --> 00:21:15,600
with people to vanquish our enemies and now this incredibly powerful technology that is reducing
306
00:21:15,600 --> 00:21:23,680
the time frame that it used to take us to think about whether or not and how and where to just obliterate
307
00:21:23,680 --> 00:21:29,840
a large number of targets and I think that all builds up this narrative of like AI as defense
308
00:21:29,840 --> 00:21:36,000
and where now offence as defense that I think is really dangerous in the current climate you have
309
00:21:36,000 --> 00:21:42,560
countries blowing up entire civilian infrastructure as a defense mechanism and I wonder if that goes
310
00:21:42,560 --> 00:21:49,520
back to this whole distance and removal that with every step that we're out of that loop more
311
00:21:49,520 --> 00:21:55,040
and more humanity gets lost and in war that's a really serious consequence. I was reading about
312
00:21:55,040 --> 00:22:03,040
the use of drones but also other kinds of robots in the war in Ukraine and the kind of training
313
00:22:03,040 --> 00:22:10,560
for the operators and one of the remarks was being an experienced gamer was better prep than having
314
00:22:10,560 --> 00:22:16,960
military experience and I think this is quite interesting because there’s this intersection with games
315
00:22:16,960 --> 00:22:21,520
which are often you know a lot a lot of computer games are around shooting that's a whole kind of
316
00:22:21,520 --> 00:22:28,800
genre but I think it blurs a line for me between like soldiering and killing. In the first person
317
00:22:28,800 --> 00:22:36,400
shooting game the aim is to kill things whereas soldiering isn't primarily about that or maybe
318
00:22:36,400 --> 00:22:41,520
hasn't always been about that you might fire a gun as a soldier to suppress people you might
319
00:22:41,520 --> 00:22:47,920
not aim to hit them but you deny an area or something like this and so this that sort of feels to me
320
00:22:47,920 --> 00:22:54,720
like the there's a kind of blurring of or change of what it means to be in the military to be more like
321
00:22:54,720 --> 00:23:01,600
the games that have been built right yeah so we're seeing kind of military actions that are much more
322
00:23:01,600 --> 00:23:08,080
like murder right or killing and perhaps the kind of nuclear war theme that's in the film like
323
00:23:08,080 --> 00:23:14,560
is a version of that but now rather than being the kind of one-off fear of world war three this is
324
00:23:14,560 --> 00:23:20,480
now becoming a more everyday kind of activity and more localized I mean we just had the President
325
00:23:20,480 --> 00:23:25,440
of the United States recently threatened to wipe an entire civilization off the face of the earth
326
00:23:26,160 --> 00:23:33,280
that's nuts more than nuts you know and the fact that his press secretary just wiped it away as
327
00:23:33,280 --> 00:23:40,400
blustery rhetoric you know and the problem is this collapsing between defense and offensive
328
00:23:40,400 --> 00:23:46,480
technologies I think is you know this I learned this term related to drones called
329
00:23:46,480 --> 00:23:53,600
loitering munitions and just that whole term the idea that it's basically a missile that
330
00:23:54,320 --> 00:23:59,840
is already up in the air and sits there waiting to find a target and then strikes it it's kind of
331
00:23:59,840 --> 00:24:06,720
changed the whole temporality of warfare as well it both speeds things up and extends them longer
332
00:24:06,720 --> 00:24:12,240
when we're fighting trench warfare how long do these battles go on people clash loads of
333
00:24:12,240 --> 00:24:17,760
casualties on both sides and then you need some time to go and recoup and bring your people back to
334
00:24:17,760 --> 00:24:25,040
battle now it's like this continual loop of highly destructive warfare there's also something
335
00:24:25,040 --> 00:24:32,320
around the temporal aspects of it I've heard this horrible phrase shortening the kill chain
336
00:24:32,320 --> 00:24:38,880
but how AI is being used for target selection for drone strikes or missile strikes and so on
337
00:24:38,880 --> 00:24:45,440
that it can analyze data much more quickly and it can mean the decisions about attacks can be made
338
00:24:45,440 --> 00:24:51,120
much more quickly and part of how they present that is that well the final decision will always be
339
00:24:51,120 --> 00:24:57,600
made by humans still so we have this humans in the loop idea here which is still the human being
340
00:24:57,600 --> 00:25:03,200
placed in a position of responsibility for the failure of AI right but also like removing all
341
00:25:03,200 --> 00:25:09,440
space for judgment removing time for judgment because if as you say like the Israeli I think it's
342
00:25:09,440 --> 00:25:16,640
called gospel that basically it would take a human analyst let's say a year to come up with 50 targets
343
00:25:16,640 --> 00:25:23,520
and this is coming up with thousands of targets in a very limited amount of time what human in the loop
344
00:25:23,520 --> 00:25:30,800
can do the appropriate checks and balances there to I mean we're already seeing this you know war is
345
00:25:30,800 --> 00:25:36,800
definitely targeting civilian infrastructure much more you know so clearly AI is not capable of
346
00:25:36,800 --> 00:25:42,560
making those nuanced decisions and I think what's worse is that governments don't care
347
00:25:42,560 --> 00:25:50,480
to bring just the discussion back to games before we move on games obviously run through all kinds
348
00:25:50,480 --> 00:25:58,400
of aspects of computing right so we're in so it's you know computer science and games design could be
349
00:25:58,400 --> 00:26:05,200
next to each other in a university department and so on we've seen games like chess as important
350
00:26:05,200 --> 00:26:12,880
tests of certain kinds of AI through the decades also we have game theory so game theory coming from
351
00:26:12,880 --> 00:26:19,280
von Neumann who's a pioneer of the modern computer let's say does come into kind of scenario
352
00:26:19,280 --> 00:26:25,360
planning and quite explicitly into kind of US nuclear strategy we see a little bit of that in the
353
00:26:25,360 --> 00:26:32,400
film when David's doing the research about Falcon he discovers a article called poker and Armageddon
354
00:26:33,440 --> 00:26:40,320
the role of bluffing in a nuclear standoff so if either of you to come across this games in the
355
00:26:40,320 --> 00:26:48,960
context of nuclear war no not me my my relationship to games is in terms of feeder games
356
00:26:48,960 --> 00:26:56,160
I come from the performance world games can break down and then reinforce social norms and
357
00:26:56,160 --> 00:27:00,560
that's what the tool is for when you're teaching within through games where we see a cognate with
358
00:27:00,560 --> 00:27:05,600
war games and theatre games of course is in the Campancino Theater and Theater of the Oppressed
359
00:27:05,600 --> 00:27:14,640
style from Boal etc where all of the kind of choreographic and strategic structures of games
360
00:27:14,640 --> 00:27:21,040
or things you can learn within a game are also a way to kind of push back on cultural repression
361
00:27:21,040 --> 00:27:26,080
that's happening within your own systems so yeah not in terms of nuclear war but definitely in terms
362
00:27:26,080 --> 00:27:33,440
of the ways in which gaming as a social interactive structure can help a sport the repressive apparatus
363
00:27:33,440 --> 00:27:40,080
of the state damn I'm not even sure how to follow that that was really brilliant when I think about
364
00:27:40,080 --> 00:27:45,040
like the kind of game theory that you're talking about the kind that really heavily influenced like
365
00:27:45,040 --> 00:27:51,840
the field of behavioral economics I've got so many criticisms of that kind of way of looking at it
366
00:27:51,840 --> 00:27:59,520
you know this ultra a very specific idea of rationality is involved in those games somewhat of a
367
00:27:59,520 --> 00:28:07,680
lack of respect for context and I don't know I kind of see that mirrored in the way that these
368
00:28:07,680 --> 00:28:12,960
kinds of games are envisioned as well these kinds of war games there's a scene where really kind of
369
00:28:12,960 --> 00:28:19,120
brings that out I think where David is a founder computer terminals talking to Joshua
370
00:28:20,080 --> 00:28:28,240
and is realizing that Joshua is still playing and Joshua is still playing because the primary goal
371
00:28:28,240 --> 00:28:34,480
in Joshua's words has not been achieved David asks Joshua what is the primary goal and the primary
372
00:28:34,480 --> 00:28:41,760
goal is quote to win the game right so Joshua doesn't have the concept of playing for fun which is a
373
00:28:41,760 --> 00:28:49,120
more complex goal it's a it's an embodied goal and I think that's kind of critical you know doing
374
00:28:49,120 --> 00:28:54,720
something for pleasure or doing something for fun yeah so it makes sense that a machine can't connect
375
00:28:54,720 --> 00:29:01,600
with that so one of the themes that's in the film is a kind of simulation versus reality binary
376
00:29:01,600 --> 00:29:07,520
for instance David asks is this a game or is it real to Joshua and Joshua replies what's the
377
00:29:07,520 --> 00:29:16,720
difference I should reach deathcon 1 and launch night missiles in 28 hours would you like to see some
378
00:29:16,720 --> 00:29:24,640
projected kill ratios 69% of the housing destroyed 72 million people dead
379
00:29:24,640 --> 00:29:37,200
is this a game or is it real what's the difference oh wow
380
00:29:37,200 --> 00:29:45,280
and I'm just wondering if you have some reactions to that when I watched yesterday I
381
00:29:46,560 --> 00:29:52,000
wasn't thinking about the interiority of Joshua slash whoppers kind of experience can we
382
00:29:52,000 --> 00:29:58,320
came up aware of that kind of misstep of mind what if the AI is learning if Joshua has the capacity
383
00:29:58,320 --> 00:30:04,160
to learn what has he learned in those 10 years potentially of stasis that he does not play in the
384
00:30:04,160 --> 00:30:09,760
game with with Falcon anymore and how fast can he learn given this and I guess one of the questions
385
00:30:09,760 --> 00:30:14,400
is this a version of hyper reality does Joshua exist in hyper reality and what does that mean for us
386
00:30:14,400 --> 00:30:21,440
hyper reality being a state whether the situation between the real and its representation collapses
387
00:30:21,440 --> 00:30:26,880
norad becomes a hyper real space once the game is in play but also this interior world
388
00:30:26,880 --> 00:30:32,800
the Joshua experiences must be hyper real I was thinking about how Falcon says that
389
00:30:32,800 --> 00:30:40,560
Joshua never learned futility you know giving up on something and the fact that it's called Joshua
390
00:30:40,560 --> 00:30:45,920
the fact that Joshua was the name of his son who died in the car accident this idea of like the
391
00:30:45,920 --> 00:30:52,720
machine as the baby of the developer comes up a lot and therefore also this idea of how you parent
392
00:30:52,720 --> 00:30:58,880
your baby machine how you raise it the things you teach it what's important to it or not this idea of
393
00:30:58,880 --> 00:31:04,960
you know having that responsibility but then also at some point you no longer have any
394
00:31:04,960 --> 00:31:09,520
and what Falcon is kind of explaining in that scene really reminds me a lot of actually a
395
00:31:09,520 --> 00:31:14,960
perspective from the 80s Michael Crichton had it too that it's sort of like oh well you know humans
396
00:31:14,960 --> 00:31:20,320
will eventually destroy themselves they probably deserve it let let everything take its course you
397
00:31:20,320 --> 00:31:25,280
know and I think that like when Falcon sitting there and he's like I don't know if I should intervene
398
00:31:25,280 --> 00:31:31,120
to stop it and that that was an argument back in the day and still is actually about climate change
399
00:31:31,120 --> 00:31:36,960
I feel like it's the futility issue would not be what I think is the biggest issue in terms of
400
00:31:36,960 --> 00:31:41,040
you know how to raise your machine baby it's the fact that machines don't understand any kind of
401
00:31:41,040 --> 00:31:48,080
vulnerability drones target children and non-combatants you know like Mercy is a human failing like I
402
00:31:48,080 --> 00:31:55,360
said earlier like that comes from somewhere the environment in which machines are taught to conceive
403
00:31:55,360 --> 00:32:02,960
of vulnerability in this way. A couple more clips to play which is from the final sequence and here we
404
00:32:02,960 --> 00:32:09,600
have you know the scene where Joshua is still playing the game and there's a trying to get the codes to
405
00:32:09,600 --> 00:32:16,480
launch the unique the missiles which is all going to be automated and so on and Falcon and David
406
00:32:16,480 --> 00:32:24,480
together gets the idea to get Joshua to play lots and crosses or take Tektoe to try to gain access
407
00:32:24,480 --> 00:32:38,080
to Joshua and also to teach it that it can't win which is its goal. And through that realizing the only
408
00:32:38,080 --> 00:32:45,760
winning move is not to play and deciding not to initiate the attack so lots to discuss in the scene
409
00:32:45,760 --> 00:32:51,200
that I think is about learning with machines for me there's something about different types of
410
00:32:51,200 --> 00:32:58,960
learning relating to the kind of game simulation the kind of learning that McKittrick wants Joshua to
411
00:32:58,960 --> 00:33:06,720
do is to learn how to be better at Nick the War measured in amounts of destruction and so on and
412
00:33:06,720 --> 00:33:13,120
Falcon says that there are people like McKittrick who think you can win this whereas what Joshua
413
00:33:13,120 --> 00:33:18,320
manages to learn at the end they learn something that's of a different type so they learn something
414
00:33:18,320 --> 00:33:23,360
that's a constraint of the game they've been learning to play better and I think that's a really
415
00:33:23,360 --> 00:33:28,640
interesting kind of juxtaposition the different kinds of learning that Joshua does the different
416
00:33:28,640 --> 00:33:34,800
kind of learning that Falcon wants Joshua to do versus McKittrick and taking that to today's AI
417
00:33:34,800 --> 00:33:40,400
and so on like I think everyone's under pressure in the workplace and then everyday life and so on
418
00:33:40,400 --> 00:33:45,920
to learn how to use AI that's given to us whereas one of the things we need to learn is more about
419
00:33:45,920 --> 00:33:51,280
what it's like what it's constraints are what it does yeah exactly yeah and it would be easy to
420
00:33:51,280 --> 00:33:57,760
miss it out by concentrating on what the McKittrick surround us want us to do for me I thought that
421
00:33:57,760 --> 00:34:03,440
like the realities that they all inhabit are kind of based around what they're perceiving as relevant
422
00:34:03,440 --> 00:34:10,320
and in the process of learning they're expanding that horizon of what's relevant to consider
423
00:34:10,320 --> 00:34:15,680
McKittrick isn't considering when he's suggesting that the machine run everything that a scenario
424
00:34:15,680 --> 00:34:21,680
such as this could could happen Joshua isn't considering the idea that there are some games you cannot
425
00:34:21,680 --> 00:34:28,720
win Falcon is not considering a lot of things you know his own personal accountability and responsibility
426
00:34:28,720 --> 00:34:34,800
for having created this thing which he might have done so to examine these loftier questions that he
427
00:34:34,800 --> 00:34:40,960
has in mind for himself but should he have considered the practical application of the things that he built maybe
428
00:34:41,840 --> 00:34:48,720
it's just a matter of your process of learning is your horizon being opened up to you about what
429
00:34:48,720 --> 00:34:54,880
things matter in the context do you want to share a final thought on what to take away from watching
430
00:34:54,880 --> 00:35:04,080
the film in relationship to today I'm really struck by how even in this early framing of AI
431
00:35:04,640 --> 00:35:13,280
and in relation to our globalized world we're hyper aware of the human capacity for pain and pleasure
432
00:35:13,280 --> 00:35:18,480
and knowledge and that seems to be stripped out once we've mechanized or automated learning and
433
00:35:18,480 --> 00:35:26,240
knowledge systems and how much time it would take to bring that type of embodied affective
434
00:35:26,240 --> 00:35:31,680
relationships knowledge into these other systems so yeah just really thinking and reflecting on
435
00:35:31,680 --> 00:35:39,680
the deep time of learning maybe in a similar vein thinking about how much like human experience and
436
00:35:39,680 --> 00:35:48,160
maybe even human biases in a positive sense how things like human interactive emotions and
437
00:35:48,160 --> 00:35:55,680
attitudes such as mercy and compassion and these are all also sort of embodied parts of our
438
00:35:55,680 --> 00:36:01,040
learning and relating to the world and things that machines can't understand or at least don't
439
00:36:01,040 --> 00:36:07,360
understand at the moment and so I'm very I'm very dubious about the application of AI in any kind
440
00:36:07,360 --> 00:36:13,120
of military setting I don't think it should be done I don't think war should be optimized made
441
00:36:13,120 --> 00:36:15,680
more efficient yeah none of that
442
00:36:15,680 --> 00:36:24,160
okay so Tracie Justin thank you both so much for your thoughts and thank you to everyone for joining
443
00:36:24,160 --> 00:36:28,640
us to listen to this podcast it's been a pleasure thinking about War Games with you. For more
444
00:36:28,640 --> 00:36:32,800
detail and references for our discussion in this episode please do check out the show notes and if
445
00:36:32,800 --> 00:36:38,080
you enjoy the episode please do subscribe or text it to a friend you can follow us on Instagram at
446
00:36:38,080 --> 00:36:44,880
computerseance.podcast that's at computerseance.podcast we want to hear your thoughts the more people we get into
447
00:36:44,880 --> 00:36:49,840
the conversation the smarter and honestly the more fun it gets
448
00:36:49,840 --> 00:37:08,240
will you be Ally again? thank you for listening. could you do this in the voice of Joshua please thank you for listening thank you for listening
449
00:37:10,400 --> 00:37:17,360
That was beautiful
450
00:37:17,360 --> 00:37:24,800
this podcast is funded by a UK research and innovation future leaders fellowship
