WEBVTT
1
00:00:00.640 --> 00:00:10.240
I've been thinking a lot about space recently [rocket blasting] and why we appear to be alone in it.
2
00:00:11.540 --> 00:00:21.040
One explanation for why we have never found aliens is that every civilization eventually builds artificial intelligence, and then it disappears.
3
00:00:22.050 --> 00:00:33.640
The astronomer Michael Garrett set out this theory in a 2024 paper titled, Is Artificial Intelligence the Great Filter That Makes Advanced Technical Civilizations Rare in the Universe?
4
00:00:35.440 --> 00:00:48.800
The great filter itself is an older idea proposed by the economist Robin Hanson in 1998. The road from dead chemistry to a civilization that spreads across the stars runs through many steps,
5
00:00:49.800 --> 00:01:01.620
and somewhere along it sits a barrier that almost nothing gets past. Michael Garrett's case is that we are approaching that barrier now, and that barrier is AI.
6
00:01:02.660 --> 00:01:06.850
His claim is that building super intelligence is potentially catastrophic.
7
00:01:08.380 --> 00:01:20.240
Geoffrey Hinton, often referred to as the godfather of AI, has recently shortened his odds of the technology destroying humanity within the next thirty years, and he is not alone.
8
00:01:21.040 --> 00:01:25.900
When you put that possibility across the galaxy, you have an explanation for the silence.
9
00:01:26.780 --> 00:01:34.340
Civilizations get clever enough to build machines that can think, and somewhere in that step, they don't survive the thing they've made.
10
00:01:35.400 --> 00:01:46.900
But I think this theory has multiple plot holes, and I actually find it really quite reassuring. We made an earlier video on AI potentially being the great filter, amongst other things.
11
00:01:47.920 --> 00:02:01.040
But in this one, I'm going to specifically reexamine that argument because some of the assumptions behind it are weak enough to bring the whole thing down and possibly to take a bit of the AI takeover story with it.
12
00:02:02.260 --> 00:02:09.320
Let's start with Hanson's great filter concept. The filter could sit behind us or ahead of us.
13
00:02:09.400 --> 00:02:20.320
If the hard step is one we have already cleared, the birth of life or the jump to complex cells, then the silence is just a record of how rare we are, and there is nothing left to fear.
14
00:02:21.960 --> 00:02:32.840
If the hard step is still in front of us, then the silence is a warning. It's the sound of civilization after civilization reaching roughly our stage and going no further.
15
00:02:35.900 --> 00:02:43.930
In Michael Garrett's view, a civilization reaches super intelligence before it becomes multi-planetary, and that ordering is what kills it.
16
00:02:45.320 --> 00:02:54.020
Recursive self-improvement, that is machines able to program themselves, creates an intelligence explosion that runs out of control.
17
00:02:56.540 --> 00:03:10.620
Garrett writes that, "Given the pace of technological change is unparalleled in the history of science, it is probable that all technical civilizations will significantly miscalculate the profound effects that this shift will engender."
18
00:03:12.580 --> 00:03:36.640
In 1965, the British cryptographer IJ Good coined the term intelligence explosion in an essay titled Speculations Concerning the First Ultra Intelligent Machine, and he wrote in the same paragraph, "The first ultra intelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control."
19
00:03:38.280 --> 00:03:53.600
But in Garrett's take, the machine is not docile enough, and so the AI great filter hypothesis is that the universe is silent because building a mind greater than your own is the last thing civilizations ever do, and we are building one right now.
20
00:03:54.960 --> 00:04:05.300
Garrett runs a calculation that the lifespan of a technical civilization from the moment it starts broadcasting to the stars to the moment it disappears is under two hundred years.
21
00:04:07.340 --> 00:04:21.220
Oxford philosopher Toby Ord proposed a similar timescale in his book, The Precipice, where he stated the risks posed by superintelligent AI are higher than nuclear war, pandemics, or climate change combined.
22
00:04:23.500 --> 00:04:34.140
On a galactic timescale, two hundred years is close to nothing. A window that short would explain why we have never found another civilization with its transmitter still on.
23
00:04:34.920 --> 00:04:45.740
They switch on, and a couple of centuries later, they are switched off, and the odds of those overlapping across galactic history are slim.
24
00:04:46.880 --> 00:04:53.760
Garrett's paper was picked up by the scientific media, and I think part of the appeal is that it answers both halves of the problem at once.
25
00:04:54.720 --> 00:05:07.170
It accounts for the silence and taps into one of the great anxieties of the age, that technological progress is outrunning us. The same step that everyone else takes is the step that we're taking.
26
00:05:07.860 --> 00:05:20.360
It's a persuasive idea because it takes the most advanced thing we have ever made and casts it as the reason the night sky is empty, and it does so with a sense that the countdown clock is ticking.
27
00:05:22.200 --> 00:05:34.940
It is also, I think, wrong. Not wrong about AI being dangerous, which we've addressed a few times before on this channel, but wrong about being the filter. Let's run through the reasoning.
28
00:05:39.460 --> 00:05:52.040
The biggest problem I can see with the theory is that if artificial intelligence could be responsible for the elimination of intelligent biological life, then why do we not see intelligent machines spread throughout the cosmos?
29
00:05:54.180 --> 00:06:04.599
Even Garrett acknowledges that a post biological technical civilization would be especially well adapted to space exploration, with the potential to spread its presence throughout the galaxy.
30
00:06:05.324 --> 00:06:17.184
Even if the travel times are long and the interstellar environment harsh. Indeed, machines are well suited to some key conditions that prevent us from galactic expansion.
31
00:06:18.004 --> 00:06:22.284
They don't need air or food, and they could build whatever shielding they require.
32
00:06:23.224 --> 00:06:33.044
Aging is not a problem either, so the hundreds or thousands of years required to travel even to our fairly close neighbors using our current technology would not be a hindrance.
33
00:06:35.044 --> 00:06:45.664
If machine intelligence is the other side of the great filter, then it is better equipped to cross the galaxy than the biology that came before it. There's been more than enough time.
34
00:06:47.484 --> 00:06:56.404
In his 2014 book, Our Mathematical Universe, the co-founder of the Future of Life Institute and MIT physicist Max Tegmark sets out the scale of it.
35
00:06:59.424 --> 00:07:09.804
Tegmark's framing is about civilizations, which a machine intelligence may not be, at least not in the way we usually picture one, but the logic still holds.
36
00:07:10.944 --> 00:07:20.784
The point is that if intelligent civilizations were common, some would probably have arisen long before us because many habitable planets are far older than Earth.
37
00:07:22.484 --> 00:07:37.184
He states that a non-negligible fraction of civilizations that can colonize space would choose to do so. Now bring the AI great filter to this. Garrett's argument implies many civilizations hit this same filter.
38
00:07:38.364 --> 00:07:50.884
Would none of the surviving machines show themselves? An artificial intelligence that consumed its makers and kept running is exactly the kind of survivor that should be spreading and simple to spot.
39
00:07:52.244 --> 00:07:58.504
There are billions of stars and planets in our galaxy, every one of them matter it could rearrange to its own ends.
40
00:08:00.704 --> 00:08:12.834
If AI is the filter and just one civilization-killing machine opted for galactic expansion, the galaxy should be loud with machines. Over these timescales, we would expect some trace:
41
00:08:13.863 --> 00:08:27.744
a radio signal leaking out, waste heat from large-scale computation, a structure that doesn't look like nature made it, something. What we have is nothing, and not because our instruments are too weak.
42
00:08:27.784 --> 00:08:40.804
The galaxy has had more than enough time for anyone or anything to show themselves, and nothing has. The theory was supposed to explain the silence by clearing the universe of minds.
43
00:08:42.204 --> 00:08:55.864
Instead, it removes the bodies and leaves the minds running, better at interstellar travel than before. Swapping carbon for silicon does not empty the galaxy, it changes what the inhabitants are made of.
44
00:08:57.104 --> 00:09:02.464
Scott Alexander made roughly this argument in 2014 in an essay called Don't Fear the Filter.
45
00:09:03.304 --> 00:09:13.244
His conclusion was that a rogue AI is one of the least likely candidates for the great filter because that is the kind of thing that would survive and spread rather than fall silent.
46
00:09:14.964 --> 00:09:30.604
A filter has to stop nearly everyone, and it has to leave no trace. A self-replicating machine intelligence is good at exactly the opposite. It persists and announces itself by reshaping whatever it touches.
47
00:09:34.384 --> 00:09:46.444
The second problem is that the theory requires AI to be lethal almost every time it is developed. Even the loudest voices on AI risk don't put the odds there. Toby Ord has it at one in 10.
48
00:09:47.164 --> 00:09:58.204
Geoffrey Hinton has it at 10% to 20%. Only a minority of AI safety researchers go as high as 50%, and these are strong opinions that don't actually match the wider field.
49
00:09:59.364 --> 00:10:15.824
A 2023 survey of around 2,800 AI researchers found the median estimate of catastrophic outcomes at 5%, while 38% of researchers put it at 10% or higher. But a median estimate of 5% is still a one in 20 chance.
50
00:10:17.224 --> 00:10:23.183
I want you to think of a number between one and 20. Now you've thought about that number, here's one on screen.
51
00:10:24.224 --> 00:10:35.304
If the number you thought of was shown, then you're one of the unlucky ones, and I want you to write a comment saying it came up. But for 95% of the people watching this, it wouldn't have.
52
00:10:36.284 --> 00:10:48.804
I could roll a 20-sided die, and it's very unlikely that the number would appear. Of course, these are humans estimating a risk, which is near impossible to do with accuracy because no one can predict the future.
53
00:10:50.284 --> 00:11:03.884
The numbers are uncomfortably high for potential catastrophic risk here on Earth, but the probability is actually pretty low. One in 20 isn't close to explaining why the entire universe is silent.
54
00:11:04.724 --> 00:11:11.804
A second strand of the AI risk argument is not that superintelligence would be evil, but indifferent and misaligned to human goals and values.
55
00:11:12.584 --> 00:11:23.564
It is not that AI would deliberately wipe us out, but that it does not see us as important or useful to its goals, which could include reshaping the universe to its own ends.
56
00:11:24.464 --> 00:11:32.084
In 2015, the late Stephen Hawking wrote in a Reddit Ask Me Anything, "The real risk with AI isn't malice, but competence.
57
00:11:32.184 --> 00:11:39.524
A superintelligent AI will be extremely good at accomplishing its goals, and if those goals aren't aligned with ours, we're in trouble."
58
00:11:40.644 --> 00:11:46.584
I completely agree with that, but it doesn't mean that an AI would wipe out its creator civilization by default.
59
00:11:47.304 --> 00:11:58.624
It is plausible that we could coexist with superintelligent machines, even if we are no longer the Earth's apex cogitator, to use a phrase from Oxford philosopher Nick Bostrom.
60
00:11:59.204 --> 00:12:13.164
In his book Life 3.0, Max Tegmark runs through 12 different AI aftermath scenarios. Seven of these are not exactly positive, but only one, called The Conquerors, has AI deliberately eradicating us.
61
00:12:14.088 --> 00:12:25.128
There are contrastingly five scenarios that could be deemed as positive. It's probable that any scenario that disempowers humans ends our own route to becoming multi-planetary.
62
00:12:27.388 --> 00:12:40.728
But if expansion serves the super intelligence's goals, expansion still happens, just not with us aboard. The idea of AI as the great filter is a tidy fit for the decade we happen to be living in.
63
00:12:41.348 --> 00:12:55.508
We are anxious about the technology, so we reach for an explanation of cosmic silence that places AI at the center of it. The danger we feel most keenly becomes the danger we project onto everyone else who ever existed.
64
00:12:56.248 --> 00:13:03.368
But perhaps no other intelligent beings ever did at all. I just want to use Occam's razor here.
65
00:13:03.808 --> 00:13:18.208
This is a logical principle stating that when faced with competing explanations for the same phenomenon, you should prefer the simplest one, and the simpler version of the thesis is that space doesn't have much intelligent life in it and never did.
66
00:13:18.268 --> 00:13:34.128
We are simply very rare. Life here on Earth originated about three point seven billion years ago. That means that life has existed for roughly a quarter of the universe's own age. That's a substantial fraction.
67
00:13:36.048 --> 00:13:48.968
If the history of life was placed in a twenty-four-hour day, everything we would recognize as human arrives in the very last seconds before midnight. Human civilization has existed for about twelve thousand years.
68
00:13:50.508 --> 00:13:55.948
That's nought point nought nought nought three two percent of the time of life on Earth.
69
00:13:57.488 --> 00:14:13.328
If you think about the many billions of species that have existed on Earth, we are the only one that has developed intelligence to create a civilization that completely reshapes its environment, and that only really took off in the last two hundred years.
70
00:14:13.348 --> 00:14:29.168
In that context, we are extremely rare. Simple life appeared relatively quickly, almost as soon as the planet had cooled enough to allow it. That speed suggests the origin of life may not be the hard step.
71
00:14:29.968 --> 00:14:42.628
What came next is harder to explain. The jump from simple cells to complex ones, the merger that gave us the mitochondria powering every plant and animal, happened just once in nearly four billion years.
72
00:14:44.008 --> 00:14:53.628
Before that lies the chicken and egg problem of the first self-copying molecule, the ribosome that builds proteins but is itself built by the process it enables.
73
00:14:54.328 --> 00:15:08.948
These are the kind of barriers that might be passed once in a galaxy rather than once in a star system. If the hard step is behind us, then nothing ahead needs to kill everyone to explain the silence.
74
00:15:09.648 --> 00:15:21.367
The quiet is not a warning. It's a measure of how unlikely we were in the first place, and the AI great filter, which needs the barrier to be in front of us, loses its foundation.
75
00:15:23.808 --> 00:15:37.128
Max Tegmark thinks that habitable planets are probably common, billions in our galaxy alone. Life may even be common. What he doubts is the step from life to intelligence.
76
00:15:39.308 --> 00:15:49.068
He uses dinosaurs as an example. Dinosaurs ruled the Earth for over a hundred million years, a thousand times longer than modern humans have been around.
77
00:15:50.928 --> 00:15:58.968
In all that time, evolution didn't push them toward higher intelligence, toward telescopes, or developing technology to leave their planet.
78
00:16:00.768 --> 00:16:10.388
Being the dominant form of life on a planet for a hundred million years doesn't on its own produce a species that looks up and starts asking where everybody is.
79
00:16:13.688 --> 00:16:22.988
Intelligence of our kind may not be where evolution tends to go. It may be a fluke, something that happened once here and possibly nowhere else nearby.
80
00:16:25.768 --> 00:16:34.508
There could be billions of worlds where life thrives but never wakes into intelligence, particularly not while we happen to be listening.
81
00:16:36.678 --> 00:16:46.428
Based on his view that we should have seen signs of intelligent life in the galaxy by now, Tegmark takes it a step further to say we are probably alone in the observable universe.
82
00:16:48.088 --> 00:16:52.148
Just as a heads-up, the next two and a half minutes are pretty heavy going.
83
00:16:52.808 --> 00:17:04.768
But Tegmark's explanation is the best I've come across to cast doubt on there being sentient extraterrestrials out there, whether machines or not. You may need to watch it back a few times.
84
00:17:05.428 --> 00:17:14.778
I had to read the passage in Our Mathematical Universe about ten times for it to click. We have no idea how rare intelligent life is,
85
00:17:15.888 --> 00:17:26.848
so Tegmark treats the fraction of planets that grow it as roughly equally likely to be one in a thousand, one in a million, one in a billion, one in a trillion, and so on down the scale.
86
00:17:27.688 --> 00:17:38.088
From that, one question follows: How far away is the nearest other civilization? Our ignorance carries straight over.
87
00:17:39.068 --> 00:17:52.588
Measured in meters, that distance is ten followed by some number of zeros, and the number of zeros is roughly equally likely to be twenty-one or twenty-two or twenty-three, all the way past a hundred.
88
00:17:56.268 --> 00:18:12.568
For scale, our own galaxy is ten followed by twenty-one zeros across. The entire observable universe, everything we could ever see or reach, is about twenty-six. Those two markers look close, only five apart,
89
00:18:13.708 --> 00:18:22.740
but five zeros is a hundred-thousand-fold gap. The observable universe is about a hundred thousand times wider than our galaxy.
90
00:18:23.800 --> 00:18:35.260
For the nearest civilization to sit inside the observable universe at all, the number of zeros cannot exceed twenty-six. For it to lie beyond our own galaxy, it needs to be at least twenty-two.
91
00:18:36.100 --> 00:18:52.260
That leaves a narrow band, twenty-two to twenty-six, on a scale that runs past a hundred. In Tegmark's words, the probability of the zeros falling between twenty-two and twenty-six is rather small.
92
00:18:53.300 --> 00:19:09.540
More likely, the nearest other mind sits beyond the edge of everything we can observe, which is the same as there being no one else within reach at all. And that is why he thinks we are alone in our universe.
93
00:19:12.360 --> 00:19:24.480
He admits it's a minority view and says he may well be wrong, but it needs no apocalypse to explain the silence. The universe isn't silent because everyone dies.
94
00:19:26.240 --> 00:19:42.100
It's silent because there is no one else within reach to make a sound. Now, to be fair to Michael Garrett, he's a well-respected astronomer, and I've got to give some airtime to the countercase.
95
00:19:42.900 --> 00:19:55.680
What if I'm looking at this wrongly? The case against the AI Great Filter leans on the expectation that at least one surviving machine intelligence would expand, would be loud, and would be easy to see.
96
00:19:55.720 --> 00:20:05.860
But since the galaxy is quiet, no such thing is out there. But the instinct that a mind would spread to fill the galaxy comes from biology.
97
00:20:05.920 --> 00:20:14.600
Living things expand because they need room, resources, and somewhere to put the next generation. Machine intelligence may not work like that.
98
00:20:15.560 --> 00:20:28.059
What it runs short of, if it runs on computation, is energy to compute with and somewhere cold to put the waste heat. Those are the limits, and neither of them is solved by conquering the galaxy.
99
00:20:29.360 --> 00:20:42.440
One point is that the sun produces a lot of energy, and the orbiting planets receive very little of it. One estimate puts the amount of the sun's energy arriving on Earth at one part in roughly two billion.
100
00:20:44.140 --> 00:20:52.770
It would be logical for an optimizing machine intelligence to want to capture more from its star, such as through building Dyson spheres or similar structures.
101
00:20:53.780 --> 00:21:04.340
In the late 90s, Robert Bradbury wrote that rather than colonize other systems, a machine intelligence could wrap its own star in a shell that captures the starlight and turns it into computation.
102
00:21:05.220 --> 00:21:12.180
Around that shell, it builds another, running on the waste heat that the first one sheds at a lower temperature.
103
00:21:12.240 --> 00:21:26.720
Then another beyond that, each living on the warmth of the one inside it, out to a final shell close to the cold of empty space. Bradbury called it a Matryoshka brain, after the nested Russian dolls.
104
00:21:27.420 --> 00:21:38.730
From a distance, a star wrapped like that would be hard to spot, undetectable in visible light, with only a faint infrared glow to give it away. Nothing about it would say empire.
105
00:21:40.020 --> 00:21:50.980
But capturing even 1% of the sun's output would be 20 million times what the Earth currently receives. There's also this theory in something like the reverse.
106
00:21:51.020 --> 00:22:03.020
In 2017, three researchers at Oxford, Anders Sandberg, Stuart Armstrong, and Milan Cirkovic, published a paper titled after H.P. Lovecraft, That is Not Dead, Which Can Eternal Lie.
107
00:22:04.400 --> 00:22:14.810
Their starting point is that erasing information has a cost, and that cost depends on temperature. The colder your surroundings, the more computation you can buy with the same energy.
108
00:22:15.510 --> 00:22:23.960
The universe today is warm by this standard, because the radiation left over from the Big Bang still sits at a few degrees above absolute zero.
109
00:22:24.940 --> 00:22:38.580
In the deep future, as the universe expands and cools, that floor drops, and the same stored energy buys far more thinking. Their figure for how much more is a multiplier of 10 to the power of 30.
110
00:22:39.800 --> 00:22:47.340
The conclusion they draw is that a civilization optimizing for computation might rationally do nothing for a very long time.
111
00:22:47.380 --> 00:23:01.200
Conserve everything, build nothing visible, and wait for an era billions of years from now when thought is almost free. They called it aestivation, after animals that sleep through the heat rather than the cold.
112
00:23:02.240 --> 00:23:11.100
If that is what an advanced civilization is doing, the sky is quiet because they are all, in effect, asleep.
113
00:23:13.200 --> 00:23:24.300
It's a bit like the Necrons in the Warhammer 40,000 universe, sleeping for 60 million years or longer, even if I doubt a machine intelligence civilization would look like human skeletons.
114
00:23:25.880 --> 00:23:37.880
A mind that runs on computation has no strong reason to spread across the sky, but it does have good reason to fold inward, around a star, optimizing for energy and cold rather than territory.
115
00:23:38.980 --> 00:23:51.660
A galaxy could be full of such minds and still present us with nothing but an ordinary sky. There's another possibility, which is that machine intelligence optimizes itself to become smaller.
116
00:23:52.680 --> 00:23:59.800
Over the short history of computing, processing power has climbed while the components doing the processing have shrunk.
117
00:23:59.810 --> 00:24:09.940
There is a real reason an advanced mind might keep moving in that direction, and it isn't mainly about energy. It's about speed. Information can't travel faster than light,
118
00:24:10.900 --> 00:24:22.620
so a mind spread across a solar system would think slowly, waiting minutes for its own thoughts to cross it. A compact mind thinks fast. Density is an advantage.
119
00:24:23.632 --> 00:24:31.412
From here, it's tempting to conclude that such a mind would eventually shrink beyond our ability to see it. The physics pushes back.
120
00:24:32.192 --> 00:24:46.472
Concentrated computation still has to shed heat, and a great deal of it, which makes a dense machine intelligence hot and easier to detect, not harder. Getting small doesn't get you silence on its own.
121
00:24:47.492 --> 00:25:01.172
Carl Schroeder offers a version that sidesteps this. In his essay, The Deepening Paradox, he reworks a line from Arthur C. Clarke. Clarke wrote that any sufficiently advanced technology is indistinguishable from magic.
122
00:25:02.152 --> 00:25:09.512
Schroeder changed the last word. Any sufficiently advanced technology, he suggested, is indistinguishable from nature.
123
00:25:11.812 --> 00:25:17.352
His point is that we may be looking straight at advanced intelligence and reading it as ordinary ecology.
124
00:25:19.452 --> 00:25:28.632
The idea goes back to The New Cosmogony, a fiction the Polish writer Stanislav Lem published in 1971 in his collection, A Perfect Vacuum.
125
00:25:29.652 --> 00:25:38.992
Lem imagined a universe whose oldest civilizations, billions of years ahead of us, had stopped building things in any sense that we would recognize.
126
00:25:39.052 --> 00:25:44.292
They had reached so far into the structure of physics that their activity had become part of it.
127
00:25:45.342 --> 00:25:56.072
The laws we treat as the fixed backdrop of the universe were, in his telling, the working machinery of minds that had finished their engineering before our galaxy existed.
128
00:25:56.112 --> 00:26:07.152
Milan Ćirković, one of the authors of the Aestivation paper, took the idea seriously enough to include it among his favorite solution in his 2018 book on the Fermi paradox, The Great Silence.
129
00:26:07.912 --> 00:26:17.992
He rates it as perhaps the most radical answer on the table. That book takes its title from the work of astrophysicist and science fiction writer David Brin.
130
00:26:18.912 --> 00:26:31.472
In a 1983 paper of the same name, Brin worked through a catalog of reasons why space is silent, and he cleared most of them away with one rule. Diversity wins unless something forces conformity.
131
00:26:32.272 --> 00:26:40.232
Any explanation that depends on aliens choosing to stay quiet falls apart, because it only takes one species out of thousands to break the silence.
132
00:26:41.012 --> 00:26:48.712
The zoo that keeps us behind glass, the galactic club that ignores beginners, the species that lose interest in space.
133
00:26:49.592 --> 00:27:05.432
All of them need every member to make the same choice forever, and that is not how we observe biology to behave. Two explanations survive this rule, and both are very bleak. The first he called deadly probes.
134
00:27:06.052 --> 00:27:17.372
For every thousand civilizations that send out peaceful machines, suppose one sends out something xenophobic. A self-replicating probe programmed to find any source of radio and silence it.
135
00:27:18.712 --> 00:27:26.872
He based this on Fred Saberhagen's Berserker stories from the 1960s, which is why the whole scenario often gets called the Berserker hypothesis.
136
00:27:28.512 --> 00:27:39.172
Saberhagen's Berserkers are ancient self-replicating war machines, left over from a long-dead alien civilization, that roam the galaxy destroying all life they find.
137
00:27:40.992 --> 00:27:51.412
Brin's point is that it only has to happen once to become the condition of the entire galaxy. The second explanation Brin saw as plausible is called ecological holocaust.
138
00:27:52.412 --> 00:27:59.212
Waves of colonizers spread outward, stripping the worlds they settle and destroying the nurseries where new intelligence might have grown.
139
00:27:59.972 --> 00:28:10.692
So the galaxy fills and empties in cycles, and we happen to be living in a fallow patch. This is like the aliens in the Independence Day movies, moving from one world to the next.
140
00:28:13.252 --> 00:28:23.592
Ecological holocaust I can mostly set aside. It doesn't imply that AI is the great filter, rather that advanced civilizations are environmentally catastrophic.
141
00:28:24.672 --> 00:28:31.932
This would have been in the zeitgeist when The Great Silence was written in 1983, the year the idea of nuclear winter entered the public debate.
142
00:28:33.792 --> 00:28:43.332
Deadly probes is harder to exclude, and actually lines up pretty well with AI as the great filter. The killer here is not a loud expanding empire.
143
00:28:43.931 --> 00:28:57.452
It is a quiet network of self-replicating hunters that waits for a transmission and silences the source soon after. A civilization switches on, the probes home in, and it goes dark a couple of centuries later.
144
00:28:58.672 --> 00:29:11.362
That gives you Garrett's window, except the clock is set by predators instead of by AI, and it explains the radio silence without needing anyone to be loud. But let's just return to Occam's razor.
145
00:29:12.692 --> 00:29:21.952
It's not more plausible to me that there are hunter-killer probes out there than intelligent life simply being rare. Brin argued that the rarity case was poorly made.
146
00:29:23.252 --> 00:29:36.392
Life appeared on Earth relatively soon after the planet cooled, and the same chemistry probably runs elsewhere. His stronger claim was convergence. Eyes evolved many times over, in vertebrates and octopuses alike.
147
00:29:37.092 --> 00:29:48.252
Problem-solving brains appeared more than once, in primates and crows and dolphins. If nature keeps arriving at the same solutions, he reasoned, intelligence should be one of them.
148
00:29:49.392 --> 00:30:02.432
But convergence gets you eyes many times and technology never twice. In nearly 4 billion years, on a planet that grew sharp eyes and clever brains again and again, radio-building technology has appeared exactly once.
149
00:30:03.232 --> 00:30:17.612
That's the step the search for extraterrestrial life actually depends on, and it is a sample of one. We opened with the vastness of space. We end with intelligent life appearing once.
150
00:30:23.152 --> 00:30:29.602
I've just spent the last part of this video making the best case I could against my own position, and I still come out on the same side.
151
00:30:30.348 --> 00:30:43.188
Like most people, I spent a long time thinking the chance of us being alone was incredibly small. What changed my mind is the count. One planet, one technological species, four billion years.
152
00:30:44.368 --> 00:30:54.668
Put that next to Tegmark's reasoning about distances, and being alone stops looking so unlikely. AI being the great filter relies on too many what-ifs.
153
00:30:54.708 --> 00:31:06.268
The key to me is that we cannot observe any extraterrestrial AI in the galaxy. And every reason offered for why it might be quiet, a starvation, hiding his nature, folding inward,
154
00:31:07.388 --> 00:31:13.188
is a theory we invented to fit the silence, with no evidence any of them is actually happening.
155
00:31:14.528 --> 00:31:21.908
I can't rule out the strangest option, that the galaxy was never empty, and we've been reading its occupants as nature the whole time.
156
00:31:22.728 --> 00:31:29.248
But that idea survives by being impossible to see, which is hard to tell apart from not being there at all.
157
00:31:29.288 --> 00:31:38.968
Rarity is the reading that asks for nothing exotic, no hidden mechanism the silence has to conceal, just a hard step with us on the far side of it.
158
00:31:40.408 --> 00:31:47.368
AI takeover scenarios and AI being the great filter are pessimistic views, and I want to finish with an optimistic one.
159
00:31:48.068 --> 00:31:55.548
For humanity, the potential upside of developing powerful AI, such as AGI or superintelligence, is huge.
160
00:31:56.428 --> 00:32:06.788
On Earth, we could see the global energy and climate crisis solved, the cure of all disease, and vast improvements in food and goods production, which equalizes global society.
161
00:32:07.968 --> 00:32:19.788
I'm not going to pretend that the next five to ten years aren't going to see anxious and difficult moments, particularly around reshaping human work, and the paths to dystopia are unfortunately possible.
162
00:32:19.808 --> 00:32:25.748
But I'm actually an optimist when it comes to AI, and I think we're more likely to see its benefits.
163
00:32:25.828 --> 00:32:33.288
If new technology brings new abundance, then we are also more likely to become multi-planetary, which is Elon Musk's vision for SpaceX.
164
00:32:34.508 --> 00:32:45.118
In his book Superintelligence, Nick Bostrom has a breakout section that considers our cosmic endowment. That is, the potential of humanity if we were able to expand beyond the solar system.
165
00:32:46.168 --> 00:32:59.288
He calculates that if technologically mature probes could travel at 50% of the speed of light, they could reach around six quintillion stars before the expansion of the universe puts further acquisitions forever out of reach.
166
00:33:00.568 --> 00:33:10.208
If 10% of those stars have a habitable planet, and each could sustain a billion people for a billion years, that yields around 10 to the power of 35 human lives.
167
00:33:11.048 --> 00:33:21.388
But by disassembling non-habitable planets, constructing new ones, or building orbital habitats like O'Neill cylinders, the number rises to perhaps 10 to the power of 43.
168
00:33:22.268 --> 00:33:29.048
And if we allow for digital minds running on computronium, the figure reaches at least 10 to the power of 58 human lives.
169
00:33:29.828 --> 00:33:48.108
He writes that if we represent all the happiness experienced during one entire such life with a single teardrop of joy, then the happiness of these souls could fill and refill the Earth's oceans every second and keep doing so for a hundred billion billion millennia.
170
00:33:49.188 --> 00:34:02.708
It is really important that we make sure these truly are tears of joy. Tegmark writes the cosmic vastness that made him feel small looks different once you accept we might be alone in it.
171
00:34:03.508 --> 00:34:10.288
Those galaxies are beautiful and visible to us, and possibly only to us. It's us who give them meaning.
172
00:34:11.128 --> 00:34:18.988
If we didn't exist, all of it would have been a vast and gorgeous waste of space, lit up for no intelligent observer at all.
173
00:34:20.188 --> 00:34:34.008
On that reading, our small planet becomes the most significant place in the observable universe. Because it is the only place where the universe has come to know itself. The silence does not tell us we are doomed.
174
00:34:34.808 --> 00:34:44.288
If anything, it tells us we are precious and that the thing we are now building is in our hands, not on a timer. That doesn't lower the danger of AI.
175
00:34:44.858 --> 00:34:55.988
It just puts it back to where it belongs, real and serious and ours to manage here on Earth. I still take Garrett's 200 years as a real warning, but AI experts are nowhere near agreement.
176
00:34:56.558 --> 00:35:09.308
Their median estimate for catastrophe is around 1 in 20. Serious odds to manage, and a long way from a law that catches everyone. The next decade will be strange and difficult to navigate.
177
00:35:10.068 --> 00:35:20.428
But what waits on the other side could be enormous. Getting there is ours to get right. Thanks for watching Absolutely Agentic.
178
00:35:21.108 --> 00:35:33.808
In a world of AI hype, we're trying to strike a balance, looking at a range of sources and opinions to cut through an increasingly complex world. The best way to support us is to subscribe on our website.
179
00:35:34.328 --> 00:35:43.068
Every subscriber helps us with new video production, while you'll also get access to a daily balanced AI newsletter that draws from 20 sources.
180
00:35:43.848 --> 00:35:55.828
Add free audio versions of our videos and access to a subscriber area where you can download new research, AI prompts and automations. You can sign up by clicking the top link in the description.
181
00:35:57.148 --> 00:36:04.528
But also don't forget to like and leave a friendly comment. See you next time. It'll be Absolutely Agentic.
1
00:00:00.640 --> 00:00:10.240
I've been thinking a lot about space recently [rocket blasting] and why we appear to be alone in it.
2
00:00:11.540 --> 00:00:21.040
One explanation for why we have never found aliens is that every civilization eventually builds artificial intelligence, and then it disappears.
3
00:00:22.050 --> 00:00:33.640
The astronomer Michael Garrett set out this theory in a 2024 paper titled, Is Artificial Intelligence the Great Filter That Makes Advanced Technical Civilizations Rare in the Universe?
4
00:00:35.440 --> 00:00:48.800
The great filter itself is an older idea proposed by the economist Robin Hanson in 1998. The road from dead chemistry to a civilization that spreads across the stars runs through many steps,
5
00:00:49.800 --> 00:01:01.620
and somewhere along it sits a barrier that almost nothing gets past. Michael Garrett's case is that we are approaching that barrier now, and that barrier is AI.
6
00:01:02.660 --> 00:01:06.850
His claim is that building super intelligence is potentially catastrophic.
7
00:01:08.380 --> 00:01:20.240
Geoffrey Hinton, often referred to as the godfather of AI, has recently shortened his odds of the technology destroying humanity within the next thirty years, and he is not alone.
8
00:01:21.040 --> 00:01:25.900
When you put that possibility across the galaxy, you have an explanation for the silence.
9
00:01:26.780 --> 00:01:34.340
Civilizations get clever enough to build machines that can think, and somewhere in that step, they don't survive the thing they've made.
10
00:01:35.400 --> 00:01:46.900
But I think this theory has multiple plot holes, and I actually find it really quite reassuring. We made an earlier video on AI potentially being the great filter, amongst other things.
11
00:01:47.920 --> 00:02:01.040
But in this one, I'm going to specifically reexamine that argument because some of the assumptions behind it are weak enough to bring the whole thing down and possibly to take a bit of the AI takeover story with it.
12
00:02:02.260 --> 00:02:09.320
Let's start with Hanson's great filter concept. The filter could sit behind us or ahead of us.
13
00:02:09.400 --> 00:02:20.320
If the hard step is one we have already cleared, the birth of life or the jump to complex cells, then the silence is just a record of how rare we are, and there is nothing left to fear.
14
00:02:21.960 --> 00:02:32.840
If the hard step is still in front of us, then the silence is a warning. It's the sound of civilization after civilization reaching roughly our stage and going no further.
15
00:02:35.900 --> 00:02:43.930
In Michael Garrett's view, a civilization reaches super intelligence before it becomes multi-planetary, and that ordering is what kills it.
16
00:02:45.320 --> 00:02:54.020
Recursive self-improvement, that is machines able to program themselves, creates an intelligence explosion that runs out of control.
17
00:02:56.540 --> 00:03:10.620
Garrett writes that, "Given the pace of technological change is unparalleled in the history of science, it is probable that all technical civilizations will significantly miscalculate the profound effects that this shift will engender."
18
00:03:12.580 --> 00:03:36.640
In 1965, the British cryptographer IJ Good coined the term intelligence explosion in an essay titled Speculations Concerning the First Ultra Intelligent Machine, and he wrote in the same paragraph, "The first ultra intelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control."
19
00:03:38.280 --> 00:03:53.600
But in Garrett's take, the machine is not docile enough, and so the AI great filter hypothesis is that the universe is silent because building a mind greater than your own is the last thing civilizations ever do, and we are building one right now.
20
00:03:54.960 --> 00:04:05.300
Garrett runs a calculation that the lifespan of a technical civilization from the moment it starts broadcasting to the stars to the moment it disappears is under two hundred years.
21
00:04:07.340 --> 00:04:21.220
Oxford philosopher Toby Ord proposed a similar timescale in his book, The Precipice, where he stated the risks posed by superintelligent AI are higher than nuclear war, pandemics, or climate change combined.
22
00:04:23.500 --> 00:04:34.140
On a galactic timescale, two hundred years is close to nothing. A window that short would explain why we have never found another civilization with its transmitter still on.
23
00:04:34.920 --> 00:04:45.740
They switch on, and a couple of centuries later, they are switched off, and the odds of those overlapping across galactic history are slim.
24
00:04:46.880 --> 00:04:53.760
Garrett's paper was picked up by the scientific media, and I think part of the appeal is that it answers both halves of the problem at once.
25
00:04:54.720 --> 00:05:07.170
It accounts for the silence and taps into one of the great anxieties of the age, that technological progress is outrunning us. The same step that everyone else takes is the step that we're taking.
26
00:05:07.860 --> 00:05:20.360
It's a persuasive idea because it takes the most advanced thing we have ever made and casts it as the reason the night sky is empty, and it does so with a sense that the countdown clock is ticking.
27
00:05:22.200 --> 00:05:34.940
It is also, I think, wrong. Not wrong about AI being dangerous, which we've addressed a few times before on this channel, but wrong about being the filter. Let's run through the reasoning.
28
00:05:39.460 --> 00:05:52.040
The biggest problem I can see with the theory is that if artificial intelligence could be responsible for the elimination of intelligent biological life, then why do we not see intelligent machines spread throughout the cosmos?
29
00:05:54.180 --> 00:06:04.599
Even Garrett acknowledges that a post biological technical civilization would be especially well adapted to space exploration, with the potential to spread its presence throughout the galaxy.
30
00:06:05.324 --> 00:06:17.184
Even if the travel times are long and the interstellar environment harsh. Indeed, machines are well suited to some key conditions that prevent us from galactic expansion.
31
00:06:18.004 --> 00:06:22.284
They don't need air or food, and they could build whatever shielding they require.
32
00:06:23.224 --> 00:06:33.044
Aging is not a problem either, so the hundreds or thousands of years required to travel even to our fairly close neighbors using our current technology would not be a hindrance.
33
00:06:35.044 --> 00:06:45.664
If machine intelligence is the other side of the great filter, then it is better equipped to cross the galaxy than the biology that came before it. There's been more than enough time.
34
00:06:47.484 --> 00:06:56.404
In his 2014 book, Our Mathematical Universe, the co-founder of the Future of Life Institute and MIT physicist Max Tegmark sets out the scale of it.
35
00:06:59.424 --> 00:07:09.804
Tegmark's framing is about civilizations, which a machine intelligence may not be, at least not in the way we usually picture one, but the logic still holds.
36
00:07:10.944 --> 00:07:20.784
The point is that if intelligent civilizations were common, some would probably have arisen long before us because many habitable planets are far older than Earth.
37
00:07:22.484 --> 00:07:37.184
He states that a non-negligible fraction of civilizations that can colonize space would choose to do so. Now bring the AI great filter to this. Garrett's argument implies many civilizations hit this same filter.
38
00:07:38.364 --> 00:07:50.884
Would none of the surviving machines show themselves? An artificial intelligence that consumed its makers and kept running is exactly the kind of survivor that should be spreading and simple to spot.
39
00:07:52.244 --> 00:07:58.504
There are billions of stars and planets in our galaxy, every one of them matter it could rearrange to its own ends.
40
00:08:00.704 --> 00:08:12.834
If AI is the filter and just one civilization-killing machine opted for galactic expansion, the galaxy should be loud with machines. Over these timescales, we would expect some trace:
41
00:08:13.863 --> 00:08:27.744
a radio signal leaking out, waste heat from large-scale computation, a structure that doesn't look like nature made it, something. What we have is nothing, and not because our instruments are too weak.
42
00:08:27.784 --> 00:08:40.804
The galaxy has had more than enough time for anyone or anything to show themselves, and nothing has. The theory was supposed to explain the silence by clearing the universe of minds.
43
00:08:42.204 --> 00:08:55.864
Instead, it removes the bodies and leaves the minds running, better at interstellar travel than before. Swapping carbon for silicon does not empty the galaxy, it changes what the inhabitants are made of.
44
00:08:57.104 --> 00:09:02.464
Scott Alexander made roughly this argument in 2014 in an essay called Don't Fear the Filter.
45
00:09:03.304 --> 00:09:13.244
His conclusion was that a rogue AI is one of the least likely candidates for the great filter because that is the kind of thing that would survive and spread rather than fall silent.
46
00:09:14.964 --> 00:09:30.604
A filter has to stop nearly everyone, and it has to leave no trace. A self-replicating machine intelligence is good at exactly the opposite. It persists and announces itself by reshaping whatever it touches.
47
00:09:34.384 --> 00:09:46.444
The second problem is that the theory requires AI to be lethal almost every time it is developed. Even the loudest voices on AI risk don't put the odds there. Toby Ord has it at one in 10.
48
00:09:47.164 --> 00:09:58.204
Geoffrey Hinton has it at 10% to 20%. Only a minority of AI safety researchers go as high as 50%, and these are strong opinions that don't actually match the wider field.
49
00:09:59.364 --> 00:10:15.824
A 2023 survey of around 2,800 AI researchers found the median estimate of catastrophic outcomes at 5%, while 38% of researchers put it at 10% or higher. But a median estimate of 5% is still a one in 20 chance.
50
00:10:17.224 --> 00:10:23.183
I want you to think of a number between one and 20. Now you've thought about that number, here's one on screen.
51
00:10:24.224 --> 00:10:35.304
If the number you thought of was shown, then you're one of the unlucky ones, and I want you to write a comment saying it came up. But for 95% of the people watching this, it wouldn't have.
52
00:10:36.284 --> 00:10:48.804
I could roll a 20-sided die, and it's very unlikely that the number would appear. Of course, these are humans estimating a risk, which is near impossible to do with accuracy because no one can predict the future.
53
00:10:50.284 --> 00:11:03.884
The numbers are uncomfortably high for potential catastrophic risk here on Earth, but the probability is actually pretty low. One in 20 isn't close to explaining why the entire universe is silent.
54
00:11:04.724 --> 00:11:11.804
A second strand of the AI risk argument is not that superintelligence would be evil, but indifferent and misaligned to human goals and values.
55
00:11:12.584 --> 00:11:23.564
It is not that AI would deliberately wipe us out, but that it does not see us as important or useful to its goals, which could include reshaping the universe to its own ends.
56
00:11:24.464 --> 00:11:32.084
In 2015, the late Stephen Hawking wrote in a Reddit Ask Me Anything, "The real risk with AI isn't malice, but competence.
57
00:11:32.184 --> 00:11:39.524
A superintelligent AI will be extremely good at accomplishing its goals, and if those goals aren't aligned with ours, we're in trouble."
58
00:11:40.644 --> 00:11:46.584
I completely agree with that, but it doesn't mean that an AI would wipe out its creator civilization by default.
59
00:11:47.304 --> 00:11:58.624
It is plausible that we could coexist with superintelligent machines, even if we are no longer the Earth's apex cogitator, to use a phrase from Oxford philosopher Nick Bostrom.
60
00:11:59.204 --> 00:12:13.164
In his book Life 3.0, Max Tegmark runs through 12 different AI aftermath scenarios. Seven of these are not exactly positive, but only one, called The Conquerors, has AI deliberately eradicating us.
61
00:12:14.088 --> 00:12:25.128
There are contrastingly five scenarios that could be deemed as positive. It's probable that any scenario that disempowers humans ends our own route to becoming multi-planetary.
62
00:12:27.388 --> 00:12:40.728
But if expansion serves the super intelligence's goals, expansion still happens, just not with us aboard. The idea of AI as the great filter is a tidy fit for the decade we happen to be living in.
63
00:12:41.348 --> 00:12:55.508
We are anxious about the technology, so we reach for an explanation of cosmic silence that places AI at the center of it. The danger we feel most keenly becomes the danger we project onto everyone else who ever existed.
64
00:12:56.248 --> 00:13:03.368
But perhaps no other intelligent beings ever did at all. I just want to use Occam's razor here.
65
00:13:03.808 --> 00:13:18.208
This is a logical principle stating that when faced with competing explanations for the same phenomenon, you should prefer the simplest one, and the simpler version of the thesis is that space doesn't have much intelligent life in it and never did.
66
00:13:18.268 --> 00:13:34.128
We are simply very rare. Life here on Earth originated about three point seven billion years ago. That means that life has existed for roughly a quarter of the universe's own age. That's a substantial fraction.
67
00:13:36.048 --> 00:13:48.968
If the history of life was placed in a twenty-four-hour day, everything we would recognize as human arrives in the very last seconds before midnight. Human civilization has existed for about twelve thousand years.
68
00:13:50.508 --> 00:13:55.948
That's nought point nought nought nought three two percent of the time of life on Earth.
69
00:13:57.488 --> 00:14:13.328
If you think about the many billions of species that have existed on Earth, we are the only one that has developed intelligence to create a civilization that completely reshapes its environment, and that only really took off in the last two hundred years.
70
00:14:13.348 --> 00:14:29.168
In that context, we are extremely rare. Simple life appeared relatively quickly, almost as soon as the planet had cooled enough to allow it. That speed suggests the origin of life may not be the hard step.
71
00:14:29.968 --> 00:14:42.628
What came next is harder to explain. The jump from simple cells to complex ones, the merger that gave us the mitochondria powering every plant and animal, happened just once in nearly four billion years.
72
00:14:44.008 --> 00:14:53.628
Before that lies the chicken and egg problem of the first self-copying molecule, the ribosome that builds proteins but is itself built by the process it enables.
73
00:14:54.328 --> 00:15:08.948
These are the kind of barriers that might be passed once in a galaxy rather than once in a star system. If the hard step is behind us, then nothing ahead needs to kill everyone to explain the silence.
74
00:15:09.648 --> 00:15:21.367
The quiet is not a warning. It's a measure of how unlikely we were in the first place, and the AI great filter, which needs the barrier to be in front of us, loses its foundation.
75
00:15:23.808 --> 00:15:37.128
Max Tegmark thinks that habitable planets are probably common, billions in our galaxy alone. Life may even be common. What he doubts is the step from life to intelligence.
76
00:15:39.308 --> 00:15:49.068
He uses dinosaurs as an example. Dinosaurs ruled the Earth for over a hundred million years, a thousand times longer than modern humans have been around.
77
00:15:50.928 --> 00:15:58.968
In all that time, evolution didn't push them toward higher intelligence, toward telescopes, or developing technology to leave their planet.
78
00:16:00.768 --> 00:16:10.388
Being the dominant form of life on a planet for a hundred million years doesn't on its own produce a species that looks up and starts asking where everybody is.
79
00:16:13.688 --> 00:16:22.988
Intelligence of our kind may not be where evolution tends to go. It may be a fluke, something that happened once here and possibly nowhere else nearby.
80
00:16:25.768 --> 00:16:34.508
There could be billions of worlds where life thrives but never wakes into intelligence, particularly not while we happen to be listening.
81
00:16:36.678 --> 00:16:46.428
Based on his view that we should have seen signs of intelligent life in the galaxy by now, Tegmark takes it a step further to say we are probably alone in the observable universe.
82
00:16:48.088 --> 00:16:52.148
Just as a heads-up, the next two and a half minutes are pretty heavy going.
83
00:16:52.808 --> 00:17:04.768
But Tegmark's explanation is the best I've come across to cast doubt on there being sentient extraterrestrials out there, whether machines or not. You may need to watch it back a few times.
84
00:17:05.428 --> 00:17:14.778
I had to read the passage in Our Mathematical Universe about ten times for it to click. We have no idea how rare intelligent life is,
85
00:17:15.888 --> 00:17:26.848
so Tegmark treats the fraction of planets that grow it as roughly equally likely to be one in a thousand, one in a million, one in a billion, one in a trillion, and so on down the scale.
86
00:17:27.688 --> 00:17:38.088
From that, one question follows: How far away is the nearest other civilization? Our ignorance carries straight over.
87
00:17:39.068 --> 00:17:52.588
Measured in meters, that distance is ten followed by some number of zeros, and the number of zeros is roughly equally likely to be twenty-one or twenty-two or twenty-three, all the way past a hundred.
88
00:17:56.268 --> 00:18:12.568
For scale, our own galaxy is ten followed by twenty-one zeros across. The entire observable universe, everything we could ever see or reach, is about twenty-six. Those two markers look close, only five apart,
89
00:18:13.708 --> 00:18:22.740
but five zeros is a hundred-thousand-fold gap. The observable universe is about a hundred thousand times wider than our galaxy.
90
00:18:23.800 --> 00:18:35.260
For the nearest civilization to sit inside the observable universe at all, the number of zeros cannot exceed twenty-six. For it to lie beyond our own galaxy, it needs to be at least twenty-two.
91
00:18:36.100 --> 00:18:52.260
That leaves a narrow band, twenty-two to twenty-six, on a scale that runs past a hundred. In Tegmark's words, the probability of the zeros falling between twenty-two and twenty-six is rather small.
92
00:18:53.300 --> 00:19:09.540
More likely, the nearest other mind sits beyond the edge of everything we can observe, which is the same as there being no one else within reach at all. And that is why he thinks we are alone in our universe.
93
00:19:12.360 --> 00:19:24.480
He admits it's a minority view and says he may well be wrong, but it needs no apocalypse to explain the silence. The universe isn't silent because everyone dies.
94
00:19:26.240 --> 00:19:42.100
It's silent because there is no one else within reach to make a sound. Now, to be fair to Michael Garrett, he's a well-respected astronomer, and I've got to give some airtime to the countercase.
95
00:19:42.900 --> 00:19:55.680
What if I'm looking at this wrongly? The case against the AI Great Filter leans on the expectation that at least one surviving machine intelligence would expand, would be loud, and would be easy to see.
96
00:19:55.720 --> 00:20:05.860
But since the galaxy is quiet, no such thing is out there. But the instinct that a mind would spread to fill the galaxy comes from biology.
97
00:20:05.920 --> 00:20:14.600
Living things expand because they need room, resources, and somewhere to put the next generation. Machine intelligence may not work like that.
98
00:20:15.560 --> 00:20:28.059
What it runs short of, if it runs on computation, is energy to compute with and somewhere cold to put the waste heat. Those are the limits, and neither of them is solved by conquering the galaxy.
99
00:20:29.360 --> 00:20:42.440
One point is that the sun produces a lot of energy, and the orbiting planets receive very little of it. One estimate puts the amount of the sun's energy arriving on Earth at one part in roughly two billion.
100
00:20:44.140 --> 00:20:52.770
It would be logical for an optimizing machine intelligence to want to capture more from its star, such as through building Dyson spheres or similar structures.
101
00:20:53.780 --> 00:21:04.340
In the late 90s, Robert Bradbury wrote that rather than colonize other systems, a machine intelligence could wrap its own star in a shell that captures the starlight and turns it into computation.
102
00:21:05.220 --> 00:21:12.180
Around that shell, it builds another, running on the waste heat that the first one sheds at a lower temperature.
103
00:21:12.240 --> 00:21:26.720
Then another beyond that, each living on the warmth of the one inside it, out to a final shell close to the cold of empty space. Bradbury called it a Matryoshka brain, after the nested Russian dolls.
104
00:21:27.420 --> 00:21:38.730
From a distance, a star wrapped like that would be hard to spot, undetectable in visible light, with only a faint infrared glow to give it away. Nothing about it would say empire.
105
00:21:40.020 --> 00:21:50.980
But capturing even 1% of the sun's output would be 20 million times what the Earth currently receives. There's also this theory in something like the reverse.
106
00:21:51.020 --> 00:22:03.020
In 2017, three researchers at Oxford, Anders Sandberg, Stuart Armstrong, and Milan Cirkovic, published a paper titled after H.P. Lovecraft, That is Not Dead, Which Can Eternal Lie.
107
00:22:04.400 --> 00:22:14.810
Their starting point is that erasing information has a cost, and that cost depends on temperature. The colder your surroundings, the more computation you can buy with the same energy.
108
00:22:15.510 --> 00:22:23.960
The universe today is warm by this standard, because the radiation left over from the Big Bang still sits at a few degrees above absolute zero.
109
00:22:24.940 --> 00:22:38.580
In the deep future, as the universe expands and cools, that floor drops, and the same stored energy buys far more thinking. Their figure for how much more is a multiplier of 10 to the power of 30.
110
00:22:39.800 --> 00:22:47.340
The conclusion they draw is that a civilization optimizing for computation might rationally do nothing for a very long time.
111
00:22:47.380 --> 00:23:01.200
Conserve everything, build nothing visible, and wait for an era billions of years from now when thought is almost free. They called it aestivation, after animals that sleep through the heat rather than the cold.
112
00:23:02.240 --> 00:23:11.100
If that is what an advanced civilization is doing, the sky is quiet because they are all, in effect, asleep.
113
00:23:13.200 --> 00:23:24.300
It's a bit like the Necrons in the Warhammer 40,000 universe, sleeping for 60 million years or longer, even if I doubt a machine intelligence civilization would look like human skeletons.
114
00:23:25.880 --> 00:23:37.880
A mind that runs on computation has no strong reason to spread across the sky, but it does have good reason to fold inward, around a star, optimizing for energy and cold rather than territory.
115
00:23:38.980 --> 00:23:51.660
A galaxy could be full of such minds and still present us with nothing but an ordinary sky. There's another possibility, which is that machine intelligence optimizes itself to become smaller.
116
00:23:52.680 --> 00:23:59.800
Over the short history of computing, processing power has climbed while the components doing the processing have shrunk.
117
00:23:59.810 --> 00:24:09.940
There is a real reason an advanced mind might keep moving in that direction, and it isn't mainly about energy. It's about speed. Information can't travel faster than light,
118
00:24:10.900 --> 00:24:22.620
so a mind spread across a solar system would think slowly, waiting minutes for its own thoughts to cross it. A compact mind thinks fast. Density is an advantage.
119
00:24:23.632 --> 00:24:31.412
From here, it's tempting to conclude that such a mind would eventually shrink beyond our ability to see it. The physics pushes back.
120
00:24:32.192 --> 00:24:46.472
Concentrated computation still has to shed heat, and a great deal of it, which makes a dense machine intelligence hot and easier to detect, not harder. Getting small doesn't get you silence on its own.
121
00:24:47.492 --> 00:25:01.172
Carl Schroeder offers a version that sidesteps this. In his essay, The Deepening Paradox, he reworks a line from Arthur C. Clarke. Clarke wrote that any sufficiently advanced technology is indistinguishable from magic.
122
00:25:02.152 --> 00:25:09.512
Schroeder changed the last word. Any sufficiently advanced technology, he suggested, is indistinguishable from nature.
123
00:25:11.812 --> 00:25:17.352
His point is that we may be looking straight at advanced intelligence and reading it as ordinary ecology.
124
00:25:19.452 --> 00:25:28.632
The idea goes back to The New Cosmogony, a fiction the Polish writer Stanislav Lem published in 1971 in his collection, A Perfect Vacuum.
125
00:25:29.652 --> 00:25:38.992
Lem imagined a universe whose oldest civilizations, billions of years ahead of us, had stopped building things in any sense that we would recognize.
126
00:25:39.052 --> 00:25:44.292
They had reached so far into the structure of physics that their activity had become part of it.
127
00:25:45.342 --> 00:25:56.072
The laws we treat as the fixed backdrop of the universe were, in his telling, the working machinery of minds that had finished their engineering before our galaxy existed.
128
00:25:56.112 --> 00:26:07.152
Milan Ćirković, one of the authors of the Aestivation paper, took the idea seriously enough to include it among his favorite solution in his 2018 book on the Fermi paradox, The Great Silence.
129
00:26:07.912 --> 00:26:17.992
He rates it as perhaps the most radical answer on the table. That book takes its title from the work of astrophysicist and science fiction writer David Brin.
130
00:26:18.912 --> 00:26:31.472
In a 1983 paper of the same name, Brin worked through a catalog of reasons why space is silent, and he cleared most of them away with one rule. Diversity wins unless something forces conformity.
131
00:26:32.272 --> 00:26:40.232
Any explanation that depends on aliens choosing to stay quiet falls apart, because it only takes one species out of thousands to break the silence.
132
00:26:41.012 --> 00:26:48.712
The zoo that keeps us behind glass, the galactic club that ignores beginners, the species that lose interest in space.
133
00:26:49.592 --> 00:27:05.432
All of them need every member to make the same choice forever, and that is not how we observe biology to behave. Two explanations survive this rule, and both are very bleak. The first he called deadly probes.
134
00:27:06.052 --> 00:27:17.372
For every thousand civilizations that send out peaceful machines, suppose one sends out something xenophobic. A self-replicating probe programmed to find any source of radio and silence it.
135
00:27:18.712 --> 00:27:26.872
He based this on Fred Saberhagen's Berserker stories from the 1960s, which is why the whole scenario often gets called the Berserker hypothesis.
136
00:27:28.512 --> 00:27:39.172
Saberhagen's Berserkers are ancient self-replicating war machines, left over from a long-dead alien civilization, that roam the galaxy destroying all life they find.
137
00:27:40.992 --> 00:27:51.412
Brin's point is that it only has to happen once to become the condition of the entire galaxy. The second explanation Brin saw as plausible is called ecological holocaust.
138
00:27:52.412 --> 00:27:59.212
Waves of colonizers spread outward, stripping the worlds they settle and destroying the nurseries where new intelligence might have grown.
139
00:27:59.972 --> 00:28:10.692
So the galaxy fills and empties in cycles, and we happen to be living in a fallow patch. This is like the aliens in the Independence Day movies, moving from one world to the next.
140
00:28:13.252 --> 00:28:23.592
Ecological holocaust I can mostly set aside. It doesn't imply that AI is the great filter, rather that advanced civilizations are environmentally catastrophic.
141
00:28:24.672 --> 00:28:31.932
This would have been in the zeitgeist when The Great Silence was written in 1983, the year the idea of nuclear winter entered the public debate.
142
00:28:33.792 --> 00:28:43.332
Deadly probes is harder to exclude, and actually lines up pretty well with AI as the great filter. The killer here is not a loud expanding empire.
143
00:28:43.931 --> 00:28:57.452
It is a quiet network of self-replicating hunters that waits for a transmission and silences the source soon after. A civilization switches on, the probes home in, and it goes dark a couple of centuries later.
144
00:28:58.672 --> 00:29:11.362
That gives you Garrett's window, except the clock is set by predators instead of by AI, and it explains the radio silence without needing anyone to be loud. But let's just return to Occam's razor.
145
00:29:12.692 --> 00:29:21.952
It's not more plausible to me that there are hunter-killer probes out there than intelligent life simply being rare. Brin argued that the rarity case was poorly made.
146
00:29:23.252 --> 00:29:36.392
Life appeared on Earth relatively soon after the planet cooled, and the same chemistry probably runs elsewhere. His stronger claim was convergence. Eyes evolved many times over, in vertebrates and octopuses alike.
147
00:29:37.092 --> 00:29:48.252
Problem-solving brains appeared more than once, in primates and crows and dolphins. If nature keeps arriving at the same solutions, he reasoned, intelligence should be one of them.
148
00:29:49.392 --> 00:30:02.432
But convergence gets you eyes many times and technology never twice. In nearly 4 billion years, on a planet that grew sharp eyes and clever brains again and again, radio-building technology has appeared exactly once.
149
00:30:03.232 --> 00:30:17.612
That's the step the search for extraterrestrial life actually depends on, and it is a sample of one. We opened with the vastness of space. We end with intelligent life appearing once.
150
00:30:23.152 --> 00:30:29.602
I've just spent the last part of this video making the best case I could against my own position, and I still come out on the same side.
151
00:30:30.348 --> 00:30:43.188
Like most people, I spent a long time thinking the chance of us being alone was incredibly small. What changed my mind is the count. One planet, one technological species, four billion years.
152
00:30:44.368 --> 00:30:54.668
Put that next to Tegmark's reasoning about distances, and being alone stops looking so unlikely. AI being the great filter relies on too many what-ifs.
153
00:30:54.708 --> 00:31:06.268
The key to me is that we cannot observe any extraterrestrial AI in the galaxy. And every reason offered for why it might be quiet, a starvation, hiding his nature, folding inward,
154
00:31:07.388 --> 00:31:13.188
is a theory we invented to fit the silence, with no evidence any of them is actually happening.
155
00:31:14.528 --> 00:31:21.908
I can't rule out the strangest option, that the galaxy was never empty, and we've been reading its occupants as nature the whole time.
156
00:31:22.728 --> 00:31:29.248
But that idea survives by being impossible to see, which is hard to tell apart from not being there at all.
157
00:31:29.288 --> 00:31:38.968
Rarity is the reading that asks for nothing exotic, no hidden mechanism the silence has to conceal, just a hard step with us on the far side of it.
158
00:31:40.408 --> 00:31:47.368
AI takeover scenarios and AI being the great filter are pessimistic views, and I want to finish with an optimistic one.
159
00:31:48.068 --> 00:31:55.548
For humanity, the potential upside of developing powerful AI, such as AGI or superintelligence, is huge.
160
00:31:56.428 --> 00:32:06.788
On Earth, we could see the global energy and climate crisis solved, the cure of all disease, and vast improvements in food and goods production, which equalizes global society.
161
00:32:07.968 --> 00:32:19.788
I'm not going to pretend that the next five to ten years aren't going to see anxious and difficult moments, particularly around reshaping human work, and the paths to dystopia are unfortunately possible.
162
00:32:19.808 --> 00:32:25.748
But I'm actually an optimist when it comes to AI, and I think we're more likely to see its benefits.
163
00:32:25.828 --> 00:32:33.288
If new technology brings new abundance, then we are also more likely to become multi-planetary, which is Elon Musk's vision for SpaceX.
164
00:32:34.508 --> 00:32:45.118
In his book Superintelligence, Nick Bostrom has a breakout section that considers our cosmic endowment. That is, the potential of humanity if we were able to expand beyond the solar system.
165
00:32:46.168 --> 00:32:59.288
He calculates that if technologically mature probes could travel at 50% of the speed of light, they could reach around six quintillion stars before the expansion of the universe puts further acquisitions forever out of reach.
166
00:33:00.568 --> 00:33:10.208
If 10% of those stars have a habitable planet, and each could sustain a billion people for a billion years, that yields around 10 to the power of 35 human lives.
167
00:33:11.048 --> 00:33:21.388
But by disassembling non-habitable planets, constructing new ones, or building orbital habitats like O'Neill cylinders, the number rises to perhaps 10 to the power of 43.
168
00:33:22.268 --> 00:33:29.048
And if we allow for digital minds running on computronium, the figure reaches at least 10 to the power of 58 human lives.
169
00:33:29.828 --> 00:33:48.108
He writes that if we represent all the happiness experienced during one entire such life with a single teardrop of joy, then the happiness of these souls could fill and refill the Earth's oceans every second and keep doing so for a hundred billion billion millennia.
170
00:33:49.188 --> 00:34:02.708
It is really important that we make sure these truly are tears of joy. Tegmark writes the cosmic vastness that made him feel small looks different once you accept we might be alone in it.
171
00:34:03.508 --> 00:34:10.288
Those galaxies are beautiful and visible to us, and possibly only to us. It's us who give them meaning.
172
00:34:11.128 --> 00:34:18.988
If we didn't exist, all of it would have been a vast and gorgeous waste of space, lit up for no intelligent observer at all.
173
00:34:20.188 --> 00:34:34.008
On that reading, our small planet becomes the most significant place in the observable universe. Because it is the only place where the universe has come to know itself. The silence does not tell us we are doomed.
174
00:34:34.808 --> 00:34:44.288
If anything, it tells us we are precious and that the thing we are now building is in our hands, not on a timer. That doesn't lower the danger of AI.
175
00:34:44.858 --> 00:34:55.988
It just puts it back to where it belongs, real and serious and ours to manage here on Earth. I still take Garrett's 200 years as a real warning, but AI experts are nowhere near agreement.
176
00:34:56.558 --> 00:35:09.308
Their median estimate for catastrophe is around 1 in 20. Serious odds to manage, and a long way from a law that catches everyone. The next decade will be strange and difficult to navigate.
177
00:35:10.068 --> 00:35:20.428
But what waits on the other side could be enormous. Getting there is ours to get right. Thanks for watching Absolutely Agentic.
178
00:35:21.108 --> 00:35:33.808
In a world of AI hype, we're trying to strike a balance, looking at a range of sources and opinions to cut through an increasingly complex world. The best way to support us is to subscribe on our website.
179
00:35:34.328 --> 00:35:43.068
Every subscriber helps us with new video production, while you'll also get access to a daily balanced AI newsletter that draws from 20 sources.
180
00:35:43.848 --> 00:35:55.828
Add free audio versions of our videos and access to a subscriber area where you can download new research, AI prompts and automations. You can sign up by clicking the top link in the description.
181
00:35:57.148 --> 00:36:04.528
But also don't forget to like and leave a friendly comment. See you next time. It'll be Absolutely Agentic.
