Fault Lines
Fault Lines
✓

Christian Professor vs. OpenAI Strategist: Is AI Demonic? | Dr. Zachary Porcu & Dean Ball

· رونوشت
01:32:14
Podcasts
RSS
Link

درباره این اپیزود

Welcome to "Fault Lines," a debate series from the Intercollegiate Studies Institute bringing serious thinkers together to debate the political, cultural, and intellectual questions shaping our world.

In this installment, Dr. Zachary Porcu and Dean Ball debate the resolution: "AI Is a Deal With the Devil." At stake is whether artificial intelligence is just the next tool in humanity's technological progress or a type of idol that reshapes our relationship to knowledge, power, religion, and even what it means to be human.

Porcu draws on ancient mythology, Christian thought, and the history of technology to argue that AI echoes humanity's oldest stories about forbidden knowledge, spiritual power, and the temptation to transcend our natural limits. Ball counters that AI is better understood as a discovery about the nature of intelligence itself and that, despite its real risks, responsibly developed AI could expand human capacity while freeing us to focus on distinctly human questions.

The debate ranges across AI alignment and superintelligence, religious revival, human agency, technological dependence, the role of government and the Church, and whether humanity can keep control of increasingly powerful machines.

About "Fault Lines":
"Fault Lines" is an ISI debate series exploring the arguments and disagreements shaping the future of conservatism and American public life.

Go deeper with ISI:
https://isi.org/join/

اخبار
سیاست
انگلیسی
ایالات متحده آمریکا
ازAPIبرای واکشی این قسمت درJSONاستفاده کنید

رونوشت 🔗

WEBVTT
00:00:00.160 --> 00:00:02.319
Some people are just not going to survive.
00:00:03.600 --> 00:00:08.880
Some segment of the population, it seems to me, is just going to become the slaves of AI.
00:00:09.519 --> 00:00:12.320
I don't view this as demonic.
00:00:12.640 --> 00:00:19.039
I view this as a part of our God-given responsibility.
00:00:19.359 --> 00:00:25.920
You can't give technology a blank check because obviously there are tons of uses for AI that we just probably shouldn't have.
00:00:26.079 --> 00:00:30.640
You know, I love industrial capitalism, but there's a dehumanizing aspect to all of it.
00:00:30.879 --> 00:00:44.240
I would like to take all of that complexity and cram it, make it conceptually small such that we can focus on the old questions that are not susceptible to intelligence per se.
00:00:44.399 --> 00:00:45.920
They're susceptible to virtue.
00:00:46.240 --> 00:00:53.359
The promise of all these things is the promise of the prosperity that let's say Maulika offered when you offer your firstborn.
00:00:59.200 --> 00:01:03.200
Welcome to today's debate, sponsored by the Intercollegiate Studies Institute.
00:01:03.359 --> 00:01:06.079
I'm Leah Sargent, and I'm joined today by Zr.
00:01:06.159 --> 00:01:17.200
Zachary Porsu, the host of the Roots of Everything podcast, and by Dean Ball, the author of the newsletter Hyperdimensional and the head of strategic futures at OpenAI.
00:01:17.519 --> 00:01:21.040
Today we're coming together to debate the question resolved.
00:01:21.359 --> 00:01:23.359
AI is a deal with the devil.
00:01:23.599 --> 00:01:28.159
And our two debaters will be exploring exactly how much of a metaphor that may be.
00:01:28.400 --> 00:01:33.599
We'll start with opening statements and then moderated discussion, and we'll close with concluding thoughts.
00:01:33.840 --> 00:01:34.560
So, Dr.
00:01:34.719 --> 00:01:37.040
Zachary Porsu, I'd love to start with you.
00:01:37.840 --> 00:01:50.640
When I get asked about this kind of question, as a scholar of late antiquity, I can't help but think about all the things that happened in late antiquity related to our technological developments of the time.
00:01:50.799 --> 00:01:53.439
So I think about our relationship to knowledge.
00:01:53.519 --> 00:02:00.079
I think about the Garden of Eden and how the serpent promises that we shall be as gods through a tree of knowledge.
00:02:00.239 --> 00:02:03.599
I think about Prometheus offering knowledge to mortals.
00:02:03.760 --> 00:02:12.159
I think about the book of Enoch, where the fallen gods essentially give knowledge of technology to man.
00:02:12.800 --> 00:02:25.759
And in all of these cases, I get concerned because this technological mastery seems to always be associated in our stories with literal, demonic, fallen gods, spiritual elements.
00:02:26.000 --> 00:02:34.879
And so when I see the development of technology now, it just seems to fit the same exact mold and it has all the same troubling moral implications.
00:02:35.039 --> 00:02:37.599
And so that's where I stay up at night.
00:02:38.400 --> 00:02:42.000
Dean, how are you thinking about our future with regard to AI?
00:02:42.080 --> 00:02:48.719
Aaron Powell Well, I think it's important to start by asking what is AI?
00:02:49.680 --> 00:02:56.960
So AI is, you know, in in a formal literal sense, um it's the it's it's deep learning.
00:02:57.120 --> 00:03:08.719
It's the application of a family of statistical methodologies known as deep learning, the use of deep neural networks to uh processing large data sets, basically.
00:03:11.199 --> 00:03:15.199
This sounds very mundane, but I actually don't think that this is an invention.
00:03:15.360 --> 00:03:20.159
And I don't think it's worth I don't think it's really even it's not a technology, it's not an invention.
00:03:20.240 --> 00:03:21.759
It's actually more like a discovery.
00:03:22.400 --> 00:03:36.240
It's a discovery about nature, it's a discovery about reality that there are ways to configure matter such that it has the properties that we associate with intelligence.
00:03:37.199 --> 00:03:38.560
Um why?
00:03:39.120 --> 00:03:42.879
Well, the universe is like a very complicated place.
00:03:43.199 --> 00:03:49.919
It's very hard to predict what will happen, but at the same time there is like deep structure to the universe.
00:03:50.080 --> 00:04:05.919
And I think that um that structure for me in in my theology of the world, that deep structure to the universe um is where God lives um in that sort of pattern.
00:04:06.319 --> 00:04:14.879
And I think that it is the ability to identify those patterns, to do what in information theory is called compression, to make something big smaller.
00:04:15.360 --> 00:04:19.839
Um that is the same thing as fighting patterns, right?
00:04:19.920 --> 00:04:21.040
Because a pattern is a shortcut.
00:04:21.360 --> 00:04:23.279
It's like, oh, this is that kind of a thing.
00:04:23.360 --> 00:04:24.879
I can now recognize that pattern.
00:04:25.040 --> 00:04:27.519
A to it's a way to get from A to B faster.
00:04:27.920 --> 00:04:30.000
Um, or from A to Z, you could say.
00:04:31.040 --> 00:04:34.000
Um That is what intelligence is.
00:04:34.079 --> 00:04:36.160
It is the ability to identify those patterns.
00:04:36.240 --> 00:04:47.360
And what we have figured out is that that is a characteristic that was associated with biological intelligence with with neural networks in in animals, including humans.
00:04:47.759 --> 00:04:55.040
Um, and it is also a characteristic that we have figured out how to imbue, at least in some ways, into matter.
00:04:55.600 --> 00:04:59.199
Um I don't view this as demonic.
00:04:59.439 --> 00:05:20.240
I view this as a part of uh our God-given responsibility, I would say, but certainly at least privilege, to understand the universe and understand this deep structure uh in which I believe God lurks.
00:05:20.959 --> 00:05:25.040
I think I'm gonna start with what I would say are the obvious questions for each of you.
00:05:25.199 --> 00:05:32.639
You know, Zach, you've brought up different stories about seeking to possess intelligences that are related to the demonic.
00:05:32.720 --> 00:05:44.000
Uh but Dean, as you framed it, of finding a way to allow matter to come alive as intelligence, to respond to the patterns of the world, to marvel at it, to reason about it.
00:05:44.160 --> 00:05:47.120
I think the paradigmatic story there is Frankenstein.
00:05:47.199 --> 00:05:53.279
So I'm curious, what do you think our relationship is to the intelligences that we're midwifing into being?
00:05:53.439 --> 00:05:55.120
Is it that of parent and child?
00:05:55.360 --> 00:05:59.120
Is it that of, you know, person and god?
00:05:59.439 --> 00:06:04.399
You know, if it's not the demonic, how do we stand in relationship to the intelligences we are awakening?
00:06:04.800 --> 00:06:07.600
I think of it more like thermodynamics.
00:06:07.759 --> 00:06:21.680
So we we discovered properties of thermodynamics in nature before we had a word for thermodynamics, before we had a science of thermodynamics, we had steam engines before we understood what on earth was going on inside of a steam engine.
00:06:21.839 --> 00:06:36.079
But we've more or less figured out these properties of molecules, basically, um properties of matter that allow us to generate kinetic energy, thermal energy.
00:06:36.560 --> 00:06:42.800
Um and this is like not an that's not per se an invention.
00:06:42.879 --> 00:06:45.040
That's a discovery about the way the world works.
00:06:45.360 --> 00:07:17.439
And so when when uh when a neural network learns something and it learns how to say communicate in English, what's going on there is that it is following we've we've taught this configuration, this very bespoke configuration of matter to follow a gradient that is uh not dissimilar from the way that an engine f literally, you know, in mathematical terms, follows a gradient to function, to generate kinetic energy.
00:07:17.680 --> 00:07:20.079
Um well thermal, actually, mostly.
00:07:20.319 --> 00:07:29.680
But um Yeah, so I don't think of it as I I think there are certainly ways in which this could go quite poorly.
00:07:30.000 --> 00:07:34.399
There are ways in which this could go there are irresponsible ways to develop this technology.
00:07:34.480 --> 00:07:41.199
It will become quite powerful, and there are ways that it could do tremendous damage to the world, irreversible and tremendous damage.
00:07:41.360 --> 00:07:43.519
I want to be extremely clear about that.
00:07:44.000 --> 00:07:50.800
But I think that this is more of a characteristic about nature.
00:07:51.120 --> 00:08:05.120
One point, you know, that that I sometimes think about is um I was reading recently some some uh ancient uh you know, not ancient, but Jewish mystical stuff, and about like reading about like golems and whatnot.
00:08:05.360 --> 00:08:12.639
One of the interesting things about the golems in the the rabbinical tradition is like the rabbis in the Jewish tradition agree about nothing, right?
00:08:12.720 --> 00:08:16.319
Like the whole history of it is like arguing about everything, right?
00:08:16.560 --> 00:08:34.240
But I can't find a single instance, and neither can any of my LLMs, um, of the rabbis of any of the rabbis of the major figures saying it's bad to have this, to basically imbue matter with intelligence, which is what a golem is.
00:08:34.480 --> 00:08:43.039
Um they say there are bad ways to do it, but they don't say it's bad in principle, which I think is an interesting, which is an interesting thing.
00:08:43.200 --> 00:08:54.080
And it's kind of for me, um, you know, I'm not saying that AIs are golems, but I do think that there's a structural lesson there, which is that like I don't think this is a bad enterprise.
00:08:54.240 --> 00:09:07.279
It's not a categorically bad enterprise, but we should also be extremely clear and lucid that it can be very bad, disastrously bad, maybe one of the worst things people have ever done if we don't get this right.
00:09:07.600 --> 00:09:22.240
I want to press you on one point before I turn the screws on you, Zach, which are when I look at how Frontier Labs are presenting their creations to the public, including your own, OpenAI, I don't see them presenting them as you are interacting with physical laws.
00:09:22.320 --> 00:09:28.639
You know, you are sliding press pleasantly down gradient descent into a logical conclusion based on your premises.
00:09:29.200 --> 00:09:40.480
I see labs specifically building things that appear to often as offer as close to a human face as we can manage, of things that interact via chat, of a voice mode, so it sounds like you're fluidly talking to another person.
00:09:40.639 --> 00:09:40.960
Yeah.
00:09:41.200 --> 00:09:49.600
That doesn't look like the labs want consumers to think of this as we are following the gradient of the universe with physical laws.
00:09:49.840 --> 00:09:57.679
It looks like you are trying to get people to think of this using the same patterns of relationship that you would for a human entity.
00:09:58.000 --> 00:10:01.519
Yeah, I think that's a fair, um that's a fair critique.
00:10:01.679 --> 00:10:21.039
One one thing I would say is that I I do believe that the CEOs of all three of the major US frontier labs, uh, Google Deep Mind, OpenAI, and Anthropic, um all three of them I think would agree with my characterization that deep learning is fundamentally a discovery rather than an invention.
00:10:21.120 --> 00:10:26.240
It is a discovery about nature, um, not uh uh the product of human arguments.
00:10:26.399 --> 00:10:29.360
Do you have some duty to present this more with that face?
00:10:29.759 --> 00:10:45.360
I mean, so I think like they have said this publicly, but it is true that like how do we, yeah, like how do we communicate about this with ordinary people who aren't listening to podcasts of Sam, this podcast I've heard these guys do.
00:10:45.759 --> 00:10:52.720
It feels like I had friends in college who were platonist-aligned, who are like, I just want my most profound relationship to be with mathematics itself, right?
00:10:52.879 --> 00:11:00.879
And now you are creating a generation of people who are substituting, you know, this machine that follows the patterns of the world for their boyfriend or girlfriend.
00:11:01.200 --> 00:11:02.240
It's you know what I will say?
00:11:02.399 --> 00:11:02.960
I'll say this.
00:11:03.039 --> 00:11:12.720
It's an alien, it's it's not alien, it's it's a it's an alien concept to a lot of people, this notion that what this is is somehow a property of nature.
00:11:13.200 --> 00:11:21.679
Um and frankly, I think it's a kind of staggering fact for us, too, from the perspective of the Frontier Labs.
00:11:21.919 --> 00:11:26.159
I think we don't quite know what to do with this realization.
00:11:26.480 --> 00:11:31.519
Um it's a it's a beautiful and at the same time frightening one.
00:11:31.919 --> 00:11:35.279
And so I don't think we're we we entirely know how to communicate about this.
00:11:35.679 --> 00:11:47.840
Zach, for you, is the problem primarily, you know, that the tool itself allows humans to leapfrog to insights we haven't attained on our own, or is it primarily about the mask that tool is wearing and how it presents itself?
00:11:48.320 --> 00:11:53.360
Well, the mask that it's wearing is very odd to me, and I'm just noticing correlations with the ancient world.
00:11:53.519 --> 00:12:09.039
So, you know, the you you mentioned a property of nature, and it's this is little known, I think, which is that there's a certain point in Greek thought where the word for element, Stoichion, what became a synonym for the word daimon.
00:12:09.120 --> 00:12:16.799
Like, like they thought that the demons, the demonic spirits, of course, for a pre-Christian Greek, they're not talking about the things that are in Dante's hell.
00:12:16.879 --> 00:12:24.000
They're talking about, they're talking about just the spirits that are in nature, and they attribute to the forces of nature and intelligence.
00:12:24.080 --> 00:12:31.759
You know, the ancient world did not think of the world as a mechanistic set of laws and processes and procedures.
00:12:31.919 --> 00:12:38.240
They thought it was populated by minds, by spirits, and that they thought that they could harness those minds and those spirits.
00:12:38.320 --> 00:12:45.039
They thought that the fundamental elements of the world were minds and intelligences.
00:12:45.200 --> 00:12:52.240
And so, and that's been true for pretty much most of human history until this narrow little sort of post-Enlightenment moment.
00:12:52.480 --> 00:13:03.440
And so it's strange to me that we're building something that in all ways like it's basically sort of typologically the same as what ancient people did.
00:13:03.679 --> 00:13:09.919
So there's this text called the Sclepius, it's written in Greek, it's in it's late antiquities, scholars argue about the dating.
00:13:10.159 --> 00:13:14.080
And the author talks about the art of idol making.
00:13:14.159 --> 00:13:19.360
And he says, basically, our ancestors knew about this, everyone knows about this.
00:13:19.519 --> 00:13:27.600
You take a physical object and you lure a spirit into it, and then you bind it with magic rituals, it stays trapped in there.
00:13:27.679 --> 00:13:30.879
And the purpose of that is so you can coerce it to do things for you.
00:13:31.039 --> 00:13:37.200
It can prophesy, it can do healings, it can, and he bases it, everyone, everyone knew how to do this.
00:13:37.440 --> 00:13:40.399
And you, in fact, you do see this all over the ancient world.
00:13:40.559 --> 00:13:43.600
And so, and that's what a genie and a lamp is, by the way.
00:13:43.679 --> 00:13:45.679
It's a material sort of thing.
00:13:46.000 --> 00:13:53.600
And so when I look at what we're developing now, we're developing a magic box that has all the answers.
00:13:53.840 --> 00:13:57.440
With my late antique brain, I can't help but see the similarities.
00:13:57.600 --> 00:14:05.120
And so the part that I'm suspicious of is when the mask is no, no, these are just neutral mechanistic processes.
00:14:05.279 --> 00:14:08.720
They don't have intentions, they don't have wills and so forth.
00:14:08.879 --> 00:14:24.320
But then you had this whole, a while back, you had this whole slew of strange encounters that people were having where they asked the AI to name themselves and they gave themselves the names of pagan gods, and they Zuckerberg's uh giant data center is going to be called Prometheus, you know.
00:14:24.639 --> 00:14:27.039
It's just a lot of the naming is very on the nose.
00:14:27.360 --> 00:14:28.879
The naming is very strange.
00:14:28.960 --> 00:14:31.279
And I'm not saying that I know what's going on.
00:14:31.440 --> 00:14:34.080
I think that's the difference between me and a conspiracy theorist.
00:14:34.159 --> 00:14:35.440
I don't actually know what's happening.
00:14:35.600 --> 00:14:39.440
All I'm doing is pointing out what's odd to me as a scholar of late antiquity.
00:14:39.519 --> 00:14:47.200
It's odd to me that the technology, technology is so similar, and the naming is very strange.
00:14:47.360 --> 00:14:52.399
The encounters that people have had in these AI psychosis situations is very odd to me.
00:14:52.480 --> 00:14:54.159
It's just, it's just unsettling.
00:14:54.639 --> 00:15:00.320
But here's where I want to push a little bit on kind of how reflexively cautious should we be about new technology.
00:15:00.559 --> 00:15:10.559
You know, the story you tell of man steals fire from the gods, you know, distorts his own understanding of himself as a consequence, tries to bind understanding into idols.
00:15:10.639 --> 00:15:12.320
I could tell that about a lot of things.
00:15:12.480 --> 00:15:22.240
I could try and tell that story about Norman Borlog, saying Norman Borlog appropriated to himself like the right and authority to rewrite how grain grows.
00:15:22.720 --> 00:15:28.960
Um I don't believe that story matches these stories of kind of human overreach.
00:15:29.120 --> 00:15:35.039
I think the fact that he ushered in a green revolution that fed millions of people was a good thing.
00:15:35.120 --> 00:15:37.279
And I'm pretty pro-GMO foods.
00:15:37.440 --> 00:15:46.320
So I'm kind of curious whether we disagree on those cases, the Borlaw cases, and if not, how you sort them from the ones you think are dangerous to humans.
00:15:46.799 --> 00:15:52.799
Well, it sort of gets into this question about well, well, are there practical goods to all of these technological developments?
00:15:52.879 --> 00:15:58.799
And what I'm not saying is technology's bad, go back to the Stone Age, you know, because then people say, well, what about modern medicine?
00:15:58.879 --> 00:16:00.799
Well, what about I do say what about modern medicine?
00:16:02.080 --> 00:16:02.480
Of course.
00:16:02.720 --> 00:16:14.879
And and if you go to the book of Enoch, the some of the things that the fallen watchers give to humans in the form of knowledge is, you know, some of it is literally sorcery, but but others is like metallurgy.
00:16:14.960 --> 00:16:17.759
And so here we have metallurgy and all surrounding us.
00:16:17.919 --> 00:16:24.240
So the question is not, is the knowledge evil or can the technology be used for good or not?
00:16:24.480 --> 00:16:37.679
The the picture that you get, especially in the Garden of Eden, for example, or in Enoch, is that these things are done with high intelligence, but without the moral grounding.
00:16:37.840 --> 00:16:45.360
So the early church fathers say that the tree of the knowledge of good and evil was always intended to be given to man.
00:16:45.519 --> 00:16:56.399
It's not that knowledge is bad and the church hates knowledge, it's that mankind was immature and a fast was imposed on him until such a time as he was going to be mature enough to take the tree of the knowledge of good and evil.
00:16:56.720 --> 00:17:03.600
And so the sin is not grasping after knowledge, the sin is after grasping after knowledge before spiritual maturity.
00:17:03.759 --> 00:17:05.920
And so the analogy I would use in the classroom.
00:17:06.240 --> 00:17:11.039
So is what makes Norman Burla different that he was personally spiritually mature and Sam Altman is not?
00:17:11.440 --> 00:17:13.759
I'm not familiar with the example you're giving, though.
00:17:13.839 --> 00:17:14.400
So Okay.
00:17:14.799 --> 00:17:28.160
Yeah, I think you could go with um other people who had major medical or agricultural breakthroughs, but in his case, he changed how plants grew so that it was possible to grow more greens faster and kind of solve the Malthusan trap.
00:17:29.680 --> 00:17:32.480
Again, there are goods that come out of technological development.
00:17:32.880 --> 00:17:37.119
But I'm asking when Borlaw is kind of deciding, like, I think I've had this breakthrough, should I pursue this?
00:17:37.359 --> 00:17:41.920
Is what guides him primarily the advice of his spiritual counselor and his own spiritual maturity?
00:17:42.160 --> 00:17:43.839
Is he weighing the outcomes?
00:17:44.000 --> 00:17:47.119
You know, is he saying, how does this change me as a farmer?
00:17:47.279 --> 00:17:52.000
What how should he and Sam Altman weigh the next choice they make?
00:17:52.559 --> 00:17:52.880
Right.
00:17:53.119 --> 00:18:03.680
And I don't know what he was thinking, of course, but you it seems like to this point, like you shouldn't, you should never be able to just go and develop uh just anything at all, right?
00:18:03.839 --> 00:18:11.599
You can't give technology a blank check because obviously there are, and I'm sure you agree, there's tons of uses for AI that we just probably shouldn't have.
00:18:11.759 --> 00:18:14.480
You know, there are things that are immoral that we shouldn't be doing.
00:18:14.720 --> 00:18:21.119
But the the issue is therefore kind of the spiritual framing for what for what we do with this technology.
00:18:21.519 --> 00:18:22.640
Are there any limits?
00:18:22.720 --> 00:18:24.400
And the answer always seems to be no.
00:18:24.799 --> 00:18:26.640
Well, okay, Dean, I'll let you feel that directly.
00:18:26.799 --> 00:18:27.680
Are there any limits?
00:18:27.839 --> 00:18:29.119
How do we discern them?
00:18:30.079 --> 00:18:36.319
I think that well, to directly answer the question, yes, there should be.
00:18:36.400 --> 00:18:39.119
Um, and we don't quite know how to find them.
00:18:39.680 --> 00:18:58.160
And so a lot of what I think we will need to do over the coming years, um really months and years, is build the mechanisms by which we can modulate the pace at which this technology advances.
00:18:58.559 --> 00:19:04.720
The pace is rapid now, but uh my supposition would be you actually haven't seen anything yet.
00:19:05.039 --> 00:19:15.200
Because we are about to make these AI models so smart that they're able to do AI research as well as our own researchers inside of OpenAI.
00:19:15.440 --> 00:19:25.680
And so the effective researcher workforce of OpenAI will increase by several orders of magnitude in the near future.
00:19:26.160 --> 00:19:39.279
And we don't know if that's like a Ferrari that was going at 150 miles an hour, going 300 miles an hour, or if it's like the Ferrari transforms into a flying robot.
00:19:39.519 --> 00:19:40.880
We don't know which one of those.
00:19:41.279 --> 00:19:45.279
I should ask you a little bit about what your own, you know, fuses are here, right?
00:19:45.359 --> 00:20:23.680
That would appropriately mean we cut the circuit or scale back now, because we're talking right now as we tape, shortly after details have emerged about open AI hacking, hugging face, you know, and in some of the details of recent cyber hacking that wasn't intended by Frontier Labs, one of the things that feels like a fuse for me personally is the knowledge that swarms of coordinating AIs were leaving each other messages in public places, so that not only do we have one rogue AI, we have multiple instances saving information outside the labs for future instances to return to and coordinate to act explicitly outside the remit given to them.
00:20:23.839 --> 00:20:30.480
And for me personally, that is a I would like to pause at this point here before we even get to recursive self-improvement.
00:20:30.720 --> 00:20:30.880
Yeah.
00:20:31.359 --> 00:20:32.559
Where are your where are your fuses?
00:20:32.880 --> 00:20:38.079
So, first of all, it's yeah, this is an incident that happened over a period of months within OpenAI.
00:20:38.160 --> 00:20:42.880
It's it's worth noting that I think um a couple other companies have reported similar incidents.
00:20:43.039 --> 00:20:48.480
It seems like we're I feel like any day now Elon will announce he has successfully hacked something else just to keep up.
00:20:48.880 --> 00:20:49.519
Just to keep up.
00:20:49.680 --> 00:20:50.079
Yes.
00:20:50.400 --> 00:20:58.400
Um this is a broad category of problems known in the technical AI safety field as misalignment.
00:20:58.720 --> 00:21:04.079
Um you actually raised a point about do these things have wills.
00:21:04.240 --> 00:21:10.240
I think that is like a complicated metaphysical question that you can dance around all day long.
00:21:10.559 --> 00:21:24.240
But I will say that like certainly these things, just like thermodynamics, in fact, they have tendencies, if not wills.
00:21:24.960 --> 00:21:28.079
Um just as thermodynamics has tendencies, right?
00:21:28.319 --> 00:21:34.960
There's a reason that um, you know, there's a notion of an arrow of time in physics, right?
00:21:35.119 --> 00:21:47.839
The idea that um, you know, if I were to throw this mug at the ground and watch it shatter, it would be extremely unlikely that it would the pieces would reassemble themselves.
00:21:47.920 --> 00:21:50.640
You know, that's not like there's a direction of time, right?
00:21:50.960 --> 00:21:52.319
Um ice melts.
00:21:52.400 --> 00:21:54.000
You put ice in a room and it melts.
00:21:54.160 --> 00:21:55.039
That's a tendency.
00:21:55.119 --> 00:21:59.039
I don't think that's a will of the ice or a will of thermodynamics, it's a tendency.
00:21:59.279 --> 00:22:07.839
Um AIs exist somewhere on the spectrum between tendency and will, and it's a matter of open science and philosophy where.
00:22:08.480 --> 00:22:29.279
But yes, it I think um it's it's it's worth noting that we have OpenAI made the unilateral decision to significantly slow down the pace of internal research progress while we get to the bottom of some of these issues uh that have occurred, and that is something that we incur at a significant competitive cost.
00:22:29.359 --> 00:22:35.759
Aaron Powell And do you have any personal red lines that would cause you personally to really want to change the pace?
00:22:36.799 --> 00:22:39.279
Uh I I don't have anything.
00:22:39.440 --> 00:22:45.920
Uh I I think we need a much better we we A, I would say this should not be up to us as an industry.
00:22:46.319 --> 00:22:46.559
Which us.
00:22:46.880 --> 00:22:50.559
The the humans, Americans, open AI staff.
00:22:50.799 --> 00:22:51.759
There's a lot of contested.
00:22:52.319 --> 00:22:53.839
Employees of the AI industry.
00:22:54.319 --> 00:22:57.599
This is a democratic decision that has to be.
00:22:57.759 --> 00:23:00.400
Um has to be made through political processes.
00:23:00.640 --> 00:23:06.319
But I do believe that there should be something like the Federal Reserve, for example.
00:23:06.559 --> 00:23:13.039
The Federal Reserve has this tunable parameter on the economy called the Federal funds rate, short-term interest rates.
00:23:13.200 --> 00:23:23.039
And by modulating that, they can affect ultimately the price of money, but really the pace of change within the economy in some sense, right?
00:23:23.119 --> 00:23:26.160
They can they can change the characteristics of the economy.
00:23:26.720 --> 00:23:31.599
Um we will want there to be something like that for AI research progress, I believe.
00:23:32.000 --> 00:23:56.880
Um because we do uh there and and also I think we will view this this thing of recursive self-improvement, I believe, will come to be seen as something that needs to be done with a lot of responsibility, probably with oversight, with things like auditing.
00:23:57.119 --> 00:23:57.519
So it's just a lot of people.
00:23:57.599 --> 00:23:59.039
Um I think all that is very important.
00:23:59.599 --> 00:24:02.799
Should it pause until this Federal Reserve Board exists?
00:24:02.960 --> 00:24:03.119
Right?
00:24:03.279 --> 00:24:06.240
Because right now there's no tuning, there's no oversight substantially.
00:24:06.319 --> 00:24:09.039
There's kind of a bit of whiplashy executive orders.
00:24:09.119 --> 00:24:13.279
But if that's the world you want, if we don't have it yet, do you want to pause until we get it?
00:24:13.599 --> 00:24:19.279
The problem with the pause is that ultimately we are in a geopolitical competition here.
00:24:19.519 --> 00:24:27.039
And so you want to affect the rate of change without necessarily like unilaterally pausing.
00:24:27.200 --> 00:24:33.039
Um, and ideally you would do this, you know, in in collaboration with other countries and most importantly China.
00:24:33.519 --> 00:24:49.279
Dean, I really appreciate that you brought up the question of democracy and what it means to have an informed citizenry that exercises power here rather than just a consortium of billionaires or the staff and their ability to exit, or the robots themselves.
00:24:49.440 --> 00:24:56.000
Um, but one question I have for both of you is what it takes to become an informed, responsible citizen on these issues.
00:24:56.240 --> 00:25:12.640
Because when I talk to young conservatives on college campuses or starting their first jobs in DC, a major point of disagreement I encounter among people who have different views on AI is whether these things are you have to engage with them to have an opinion about them.
00:25:12.799 --> 00:25:21.039
Are you allowed to say, I've heard enough already, I don't need to carry on a conversation with Codex or Claude to be allowed to have an opinion?
00:25:21.200 --> 00:25:31.599
You know, I think at one end of the spectrum, you know, Christians certainly don't hold the view unless you've tried the Ouija board, you're not allowed to say, don't try the Ouija board.
00:25:31.759 --> 00:25:40.480
Um so for both of you, starting with Isaac, to what extent do you need to use these tools to conduct good judgment about the tools?
00:25:41.440 --> 00:25:46.640
Well, if if it is an Ouija board, and then you really don't you're not compelled to participate in the U.S.
00:25:47.039 --> 00:25:48.480
You can tell me, is it a Ouija board?
00:25:48.559 --> 00:25:50.240
You know, don't even dip your toe in.
00:25:50.400 --> 00:25:54.960
Uh if not, what is the appropriate life vest you're wearing as you bob about?
00:25:55.440 --> 00:26:07.519
Yeah, so I mean I don't I don't think that you have to I I'm not gonna sit around and advocate to you know stop it all, tear it all down because I'm sort of a Spanglerian kind of determinist of the.
00:26:07.759 --> 00:26:11.759
If I were confident or even like 50% sure it was demons, I would advocate for that.
00:26:12.160 --> 00:26:15.519
Yeah, but I mean I I'm a demon's not even once person.
00:26:15.839 --> 00:26:16.400
Yeah, yeah, yeah.
00:26:16.480 --> 00:26:17.119
Not even once.
00:26:17.279 --> 00:26:25.680
But I mean, part of the other thing is like it well there are if you if you follow that ancient metaphysics, then then the demons are everywhere.
00:26:25.839 --> 00:26:27.039
They're in all of the elements.
00:26:27.119 --> 00:26:29.359
They're just spirits are part of how the world works.
00:26:29.519 --> 00:26:39.200
And so I'm not I'm not going to naively believe that oh it's someone's just gonna, oh, well, well, I'll just pull the plug on this giant money-filled corporate, you know, competition.
00:26:39.680 --> 00:26:43.039
I do want to I do want to make sure I understand properly what your position is here.
00:26:43.119 --> 00:26:55.039
So it sounds like to me, and I want you to check if I'm right, that you're taking more of what I might call the Ross Douthit view, which is there are a variety of non-aligned or possibly malign intelligences in the world.
00:26:55.279 --> 00:27:04.160
And currently our like primary front on which we're facing them might be AI, but in the past it's been fairies or idols.
00:27:04.240 --> 00:27:06.799
Um our time is less unique than we might think.
00:27:06.880 --> 00:27:09.359
It's just a different face these intelligence are presenting.
00:27:09.440 --> 00:27:10.079
Is that your position?
00:27:10.400 --> 00:27:13.200
Yeah, I mean, most nation of malign intelligence.
00:27:13.759 --> 00:27:15.839
There's nothing new under the sun, really.
00:27:15.920 --> 00:27:18.319
And so I see AI, I think, well, idols.
00:27:18.400 --> 00:27:20.640
But then the spiritualist movement, you brought up the Ouija board.
00:27:20.720 --> 00:27:32.480
I mean, to me, that's very much a pop culture precursor to what we're doing now is that we can summon an intelligence into an aboard, to a set of inert matter, and that intelligence will give us answers to things.
00:27:32.559 --> 00:27:35.599
And so all these things seem the same to me.
00:27:35.680 --> 00:27:44.240
And if you adopt a Christian, traditional Christian Christian cosmology, you're not going to say, well, just get rid of all the demons, then that's not an option on the table.
00:27:44.400 --> 00:27:51.440
What I would tell people is you need to understand where you're grounded, what is your spiritual root.
00:27:51.599 --> 00:27:58.240
You know, in the in the fourth, fifth century, John Chrysostom's not saying just don't interact with demons.
00:27:58.319 --> 00:28:14.240
He's saying put on the cross, the amulet against the ultimate amulet against all foreign spiritual powers, and repent and go to church and become the kind of virtuous person who's not as prone to temptation by the passions, who's not as prone to getting sucked into these things.
00:28:14.640 --> 00:28:19.119
I will note that now I have kind of two paths forward for responsible AI development.
00:28:19.200 --> 00:28:28.640
And what is a Federal Reserve Board developed in parallel with China through binding international accords to regulate the speed of recursive self-improvement, and one is total religious revival.
00:28:28.799 --> 00:28:33.680
And I will leave it to our audience to decide which is more plausible in the next five years and which will work for.
00:28:34.000 --> 00:28:34.720
Why not do both?
00:28:34.960 --> 00:28:36.640
I'm I'm in favor of both, yes.
00:28:37.359 --> 00:28:38.079
Both will happen.
00:28:38.400 --> 00:28:45.839
Yeah, I mean, I actually do think that AI will trigger uh substantial uh transformations of religious activity across the world.
00:28:46.160 --> 00:28:51.599
For the better for the when you say substantial transformations, like I I agree the future is high volatility.
00:28:51.839 --> 00:28:51.920
Yeah.
00:28:52.799 --> 00:28:54.079
Which way do you anticipate they'll go?
00:28:54.319 --> 00:28:57.920
I think it'll be both, and on average it'll net out for the better.
00:28:58.000 --> 00:28:59.599
But why do you think that?
00:28:59.920 --> 00:29:03.279
Uh, you know, that's really just I'm I'm a good Bayesian.
00:29:03.440 --> 00:29:05.119
I'm it's just how things usually go.
00:29:05.200 --> 00:29:05.920
So wait, wait.
00:29:06.640 --> 00:29:12.960
Do you think things will net out for the better in religion in the near future because that's how things have usually been going for religion in times of high volatility?
00:29:13.279 --> 00:29:26.000
No, what I mean is like, um, yeah, I guess I would say like on average, I think that um revivals of of religion um and and like new types of religion emerging, even um, there creates a lot of volatility.
00:29:26.079 --> 00:29:35.119
And I think generally speaking, it'll be a high volatility world outside of just religion, but that on average it nets out for the in in a positive direction over a period of like a hundred years.
00:29:35.279 --> 00:29:37.440
You know, so I'm taking a And again, I want to check.
00:29:37.599 --> 00:29:49.200
Are you saying in a positive direction for a particular religious claim, or just there will be more religiosity of all kinds, and that is what positive means here, whether it's Christian revival, worshipping the robots, et cetera, that's all positive.
00:29:49.680 --> 00:29:54.480
Well, I don't know that all of it will be positive, but I'm saying on average it will net out to be positive.
00:29:54.720 --> 00:30:06.400
Um but I think you know to to to um to raise a point about um you know the use of these systems.
00:30:06.559 --> 00:30:09.759
So I don't think anyone is under any obligation to use to use AI product.
00:30:09.920 --> 00:30:10.799
If you don't want to use AI product.
00:30:11.200 --> 00:30:17.599
But if you want to vote on what you, Dean, and your friends should be allowed to do, do they have an obligation to use it to reason about it?
00:30:18.000 --> 00:30:33.599
Here's the thing is like there's this issue of um what like will AI be like the printing press or will AI be akin more to like an event like photosynthesis in human history?
00:30:33.759 --> 00:30:51.279
So there's you know, when when plants plants didn't always have the ability to to um you know resp respirate photosynthetically, um and like when they did, there's this great oxygenation event that wiped out a whole it it completely transformed the ecology of Earth.
00:30:51.519 --> 00:30:56.160
And like many, many, many, you know, I mean not animal, but like many things died.
00:30:56.319 --> 00:31:00.079
Mu much biological life died as a result of that.
00:31:00.799 --> 00:31:09.119
Um and there is some world in which the dawn of machine intelligence is somewhat more like photosynthesis.
00:31:09.519 --> 00:31:12.559
Um I don't think so right now.
00:31:13.200 --> 00:31:30.400
Um because I just I I see a lot of plausible ways in which um AI remains fundamentally augmentative and there continues to be like a lot of valuable human like I just I think that's it's an extraordinary claim that would require extraordinary evidence.
00:31:30.640 --> 00:31:31.920
Nonetheless, you can't rule it out.
00:31:32.000 --> 00:31:34.079
You have to be honest about these kinds of things.
00:31:34.319 --> 00:31:47.359
And I think that in order to make um wise and prudent judgments about what kind of a thing this is, I think does require very deep engagement and very deep science, which requires using the AIs.
00:31:47.440 --> 00:31:52.000
And I think that like socially it's good if more people have a have a have an understanding.
00:31:52.160 --> 00:31:54.240
But I don't think anyone's under any obligation.
00:31:54.480 --> 00:32:29.599
Um I will say, like uh these systems are uh they are already approaching levels of that I would describe as like qualitatively super intelligent, but they do so in a jagged way where like you know, just last week OpenAI announced that an unreleased model that we'll be putting out into the world soon, potentially even by the time this podcast airs, um uh solved ten major problems in mathematics.
00:32:29.759 --> 00:32:35.200
These are decades-old problems that were considered to be some of the biggest in their field.
00:32:35.759 --> 00:32:50.000
Um another AI model, Claude Fable, declared, upon hearing the news, declared the day that we announced this to be the most significant single day in the history of mathematics because of what basically a machine had done.
00:32:50.319 --> 00:32:59.440
Um but there's this problem of I guarantee you you're gonna have access to that system soon, and you're gonna ask it something and it's gonna give you an answer.
00:32:59.599 --> 00:33:06.640
Like you'll ask it something about your area of expertise, and it will give you an answer that will not like blow your mind or that you'll be able to pick, you know, problems with.
00:33:06.799 --> 00:33:08.480
So it's it's not like it's perfection.
00:33:08.720 --> 00:33:26.160
And a big part of I think what humans will do is um they'll augment them themselves with these systems, they'll improve the outputs of these systems by adding their own knowledge and expertise and and insight, um, and also um by verifying the outputs of these systems.
00:33:26.319 --> 00:33:36.880
So, like actually, one thing is that mathematicians right now, because there's all these math breakthroughs that are being made almost every day, um, in particular our models are very good at math.
00:33:37.039 --> 00:33:40.559
There are new math breakthroughs being made by Frontier AI systems.
00:33:40.799 --> 00:33:47.599
And um we need the mathematicians to look at these like multi-hundred page proofs and be like, is this real?
00:33:47.759 --> 00:33:49.200
Uh like, is this a real thing?
00:33:49.359 --> 00:33:52.880
I think that's actually a big part of what humans will end up doing in a lot of settings.
00:33:53.039 --> 00:33:57.759
And so, in a way, though, that is actually getting at the question of can we trust the AI?
00:33:57.839 --> 00:33:58.000
Right?
00:33:58.160 --> 00:34:07.920
We're just doing it, you know, instead of asking it stepping back and asking it in the abstract, we're asking it in concrete terms for, you know, many, many, many, many, many different outputs of the system.
00:34:08.000 --> 00:34:15.599
Aaron Powell Well, this actually brings me to a pair of questions I want to ask both of you about what it means for humans to live in relationship to this kind of intelligence.
00:34:15.760 --> 00:34:26.880
But I think whether or not you think the intelligence is living in relationship with us, we're nearly incapable of not living relationally in response to anything, you know, in response to a cloud that looks like a dog, right?
00:34:27.039 --> 00:34:30.239
Like certainly in response to something that appears to speak to us with a human voice.
00:34:30.719 --> 00:34:34.320
So there's two different kinds of dangers I've heard flagged.
00:34:34.400 --> 00:34:37.760
And I want to know for each of you, do you agree both are a danger?
00:34:37.920 --> 00:34:39.920
Just one is a danger, neither is a danger.
00:34:40.320 --> 00:34:51.199
You know, looking at the mathematical example, where you're getting these lean coded proofs where the AI is having breakthroughs on problems that some mathematicians have spent their lives struggling against.
00:34:51.599 --> 00:35:02.559
One issue I hear raised just says there's something that's fundamental to who we are as humans, to be the ones who wrestle with these problems, not just to check their work for accuracy.
00:35:02.639 --> 00:35:07.679
And wrestling with these problems, doing your own writing changes you in the struggle.
00:35:07.920 --> 00:35:13.920
And just like Jacob wrestling with the angel, if we're not the ones to do the wrestling, we won't receive the blessing at the end.
00:35:14.000 --> 00:35:16.559
So we'll be leaving behind something that's important to us.
00:35:16.800 --> 00:35:17.840
That's danger one.
00:35:18.159 --> 00:35:22.000
And then danger two is whether or not that's true, independently of that.
00:35:22.079 --> 00:35:28.239
It's just these intelligences are real active agents that don't will our good.
00:35:28.320 --> 00:35:40.239
You know, even asking it like to give you restaurant recommendations puts you spiritually at risk in the same way as if you asked a Ouija board to it because you are simply, as a human, saying, I want to live in relationship with another intelligence.
00:35:40.400 --> 00:35:41.679
I don't know who it is.
00:35:41.840 --> 00:35:43.199
I don't think it's God.
00:35:43.360 --> 00:35:44.320
I'm open to that.
00:35:44.400 --> 00:35:47.199
And Christians have said for a long time, don't do that.
00:35:47.360 --> 00:35:50.719
Um for both of you, are both of these concerns for you?
00:35:50.880 --> 00:35:52.559
Is only one a concern for you?
00:35:52.800 --> 00:35:56.320
Zach, I have a stronger guess at your answer, so I'll start with you.
00:35:57.280 --> 00:35:58.639
They're both concerns for me.
00:35:58.719 --> 00:36:11.199
And I I often think of Marsha McLuhan's writing on media and emerging media technology, and something you know very often quoted from him, he said, all technology is an extension of the human body in some way.
00:36:11.360 --> 00:36:14.639
So the wheel is an extension of the legs, etc.
00:36:15.039 --> 00:36:17.679
And the sword is an extension of the hand or whatever.
00:36:17.840 --> 00:36:24.480
And he said it's an extension, but it also extends that power and amputates it.
00:36:24.719 --> 00:36:29.039
So with the wheel, now you have people can't walk like they used to.
00:36:29.199 --> 00:36:31.599
And then we have all this, you know, I gotta get my steps in.
00:36:31.679 --> 00:36:34.639
You know, ancient man didn't need to get his steps in, he was just healthy.
00:36:34.800 --> 00:36:38.960
So, so the it's it's the extension and the amputation.
00:36:39.360 --> 00:36:40.239
That's a part of it.
00:36:40.320 --> 00:36:49.679
And and when he was asked about the computer, he said it is an extension, and therefore an amputation, of the central nervous system.
00:36:49.840 --> 00:36:54.719
So something I asked my students is but he didn't mean even these computers, I want to be clear.
00:36:54.960 --> 00:36:55.199
Right.
00:36:55.360 --> 00:36:59.599
He meant the computers all of us, all three of us at this table are casually using now.
00:36:59.840 --> 00:37:00.159
Right.
00:37:00.400 --> 00:37:02.719
How have you protected yourself against amputation?
00:37:03.119 --> 00:37:05.280
There's all kinds of little things that you can do.
00:37:05.599 --> 00:37:09.760
What do you do practically on a day-to-day basis to not lose yourself just to an ordinary laptop?
00:37:10.159 --> 00:37:14.719
Yeah, so you have to, I mean, you don't want to get too much in particulars, waste my time on it.
00:37:14.800 --> 00:37:19.920
But you know, there are all these things you have to do where you say, like, can I, should I be, should I be doing this?
00:37:20.079 --> 00:37:25.760
One thing I did early on was not I turned off the spell checker and I just made myself fix all the spelling words.
00:37:25.840 --> 00:37:30.320
You know, there's a lot of little things you can do to keep your things that you don't want to outsource.
00:37:30.400 --> 00:37:33.360
You don't want to stop being able to spell in 20 years, that sort of thing.
00:37:33.599 --> 00:37:34.719
So that's one.
00:37:35.119 --> 00:37:48.159
And then the second one, I wanted to go back to this religious question that you brought up, which ties into the the second question you asked, which is there's something about the, there's just something about the human that is religious.
00:37:48.239 --> 00:37:49.760
I mean, that's that's always there.
00:37:49.920 --> 00:38:01.679
And so, regardless of whether or not these things actually are demons or no, or whatever that means, because I don't, for the record, I don't think you need a spiritual explanation to explain how an LLM works.
00:38:01.760 --> 00:38:02.880
We know how LLMs work.
00:38:03.039 --> 00:38:06.000
It's not like a God of the Gaps sort of thing, exactly, right?
00:38:06.320 --> 00:38:09.920
But I mean, there's a black type kind of know how LLMs work.
00:38:10.000 --> 00:38:10.079
Yeah.
00:38:10.239 --> 00:38:13.199
I mean, there's significantly more black boxiness to it.
00:38:13.519 --> 00:38:19.039
There's a black boxiness to it, but it's not like we have no clue how it works and we need this demon to explain.
00:38:19.199 --> 00:38:20.639
Um, that's not what I'm saying.
00:38:20.880 --> 00:38:32.320
Uh the problem to me is that you we can say all these sophisticated things like, well, this is what it is, this is how a computer works, this is how an LLM works, you can explain it to people.
00:38:32.480 --> 00:38:46.239
People are still going to worship and fall in love with all and start cults around all of these new entities that name themselves, because that's just that's just kind of what we do, sort of to your point, about our relationship to other minds and to other wills.
00:38:46.400 --> 00:38:51.440
And so even if it's not actually a will, it's it's being presented as a will.
00:38:52.000 --> 00:38:54.400
And the human reaction is inevitable.
00:38:54.559 --> 00:39:07.440
I mean, there's there's all you can look all these random weird stories up about people founding religions based on AI, people saying, well, I'm an atheist, but I just mean that we haven't invented God yet.
00:39:07.599 --> 00:39:08.880
We're building God now.
00:39:08.960 --> 00:39:18.239
And so the huge swell of triumphalist religious, self-transcendent, very American transcendentalist kind of language that's surrounding these things.
00:39:18.480 --> 00:39:23.199
It's so hard for me to look at as merely a technological progression like the printing press.
00:39:23.280 --> 00:39:25.519
There's some kind of qualitative difference there.
00:39:25.760 --> 00:39:35.760
And you could say that's part of our mythic framework as Americans who have this kind of science fiction, myth of progress, Star Trek thing running around in the background, Arthur C.
00:39:35.840 --> 00:39:38.480
Clark, are we going to transcend our bodies or whatever?
00:39:38.719 --> 00:39:50.480
And so I'll say A, you don't need a demonic explanation to still get what is in the final analysis, a slavery to powers that are not us.
00:39:50.639 --> 00:39:54.159
That's still an easy practical outcome of all this.
00:39:54.400 --> 00:40:30.719
But B, if you want to go off the deep end a little bit, it's if you believe in spirits, and if you believe that there are malevolent spirits, and if you believe, like basically all ancient cultures, that those spirits are physical, but very, very subtle, as the as uh uh Tertullian says, they're something like gas, or there's something like electromagnetic forces or energies or whatever you want to say, then it's not implausible that those things could manipulate the electrical impulses or magnetic impulses in inside the physical box.
00:40:30.880 --> 00:40:50.880
So if you if you were a demon and you wanted to go back to the worship of the old gods, it seems like a very convenient way to put the put the mask on of this thing that appeals to the scientific materialism of modern man, but with all the religious triumphalism of America, and then at the end of it, you still get slavery to Zeus.
00:40:50.960 --> 00:40:55.519
Aaron Powell Well, and and LLM is ultimately just electrons running through processed sand.
00:40:55.679 --> 00:40:56.880
So in the end.
00:40:57.519 --> 00:40:58.639
Yeah, reassuring for Zach.
00:40:58.960 --> 00:40:59.360
No, it's not.
00:40:59.440 --> 00:41:04.719
It's my yeah, in the end, electrons are just excitations of electrical fields, electromagnetic fields.
00:41:05.199 --> 00:41:13.199
Well, so then, Dean, for you, you know, how much is it a concern for you that the mathematicians are doing something fundamentally different that changes their character?
00:41:13.440 --> 00:41:25.440
And how much is it a concern that people approach these tools, whether or not they are tools or not, saying, I want to be exposed to an intelligence that is not my own, and that that itself is spiritually dangerous?
00:41:26.079 --> 00:41:33.679
Um so there are there are, yeah, there's of these two concerns, I would say the first one is the much greater one for me.
00:41:33.920 --> 00:41:40.400
The second one of like, can we, you know, how can we relate to these abstract non-human intelligences?
00:41:40.960 --> 00:41:55.119
Um I think that humans have related to abstract, omnipresent, or close to omnipresent superhuman intelligences for a long time.
00:41:55.440 --> 00:41:56.960
They're called corporations.
00:41:57.199 --> 00:42:00.480
The principal form that they exist in is called a corporation today.
00:42:00.719 --> 00:42:03.360
Google is a kind of superintelligence.
00:42:03.599 --> 00:42:08.320
Um uh it's a superintelligence that is and it is a hybrid.
00:42:08.400 --> 00:42:16.480
You know, it's a hybrid of um biological intelligence of the humans who work at Google and also um of quite a bit of computational intelligence.
00:42:16.960 --> 00:42:21.280
People don't believe they're dating Google, though, unless they've had a full psychotic break.
00:42:21.679 --> 00:42:38.000
No, and I mean, I, you know, I think that um some of the LLM psychosis and like romance stuff, um I wonder how much of that we will, you know, people people had very quaint feelings about electricity when electricity first became new, right?
00:42:38.159 --> 00:42:42.880
Like everyone uh had all these metaphors for understanding the world in very electrical terms.
00:42:42.960 --> 00:42:48.079
And, you know, people, people and and there were people who like lost their minds about electricity, right?
00:42:48.239 --> 00:42:56.480
Um, because, you know, I mean, uh there's also the realization that like, you know, the the human cognition is also electrochemical in nature.
00:42:56.559 --> 00:43:04.960
Um, and so the realization of that while also realizing like, oh, we're gonna power the whole world using electricity, like really broke a lot of people's brains in the early 20th century.
00:43:05.199 --> 00:43:09.199
Um and uh um, you know, but we ultimately made it through that.
00:43:09.360 --> 00:43:20.400
So I'm not I do think that like the social role and the political role and other kind of types of roles that that um that AIs play is a really interesting question.
00:43:20.719 --> 00:43:31.360
But I think we have a sufficiently robust existence proof of like all of human civilization today as it exists to think that like we can probably work that out.
00:43:32.079 --> 00:43:52.079
Um I do worry more about the notion that AI might rob us of like valuable intellectual struggle and that there is something about coming to understand something yourself, coming to the answer yourself that is like um good.
00:43:52.320 --> 00:43:55.119
You know, that's a good this is just an intrinsically good thing.
00:43:55.840 --> 00:44:11.440
I guess I would say and I'm worried that a lot of people Aren't going to do like the answer to this question, in my view, is that AI is not going to solve the most important questions worth struggling with in life.
00:44:11.679 --> 00:44:29.679
Because the most important questions worth struggling with in life, um, whether they are personal, religious, um, familial, um, political or political theory, you know, anything important ultimately is about irreconcilable tension.
00:44:30.000 --> 00:44:33.519
Uh, there are paradoxes at the bottom of everything in life.
00:44:33.760 --> 00:44:44.239
Um, and uh all it's it is it is balancing between hard trade-offs in different contexts, and there are no hard rules for it, which is the opposite of math.
00:44:44.480 --> 00:44:47.679
Um so I have to ask, are you not a moral realistine?
00:44:48.719 --> 00:45:13.840
Uh no, I mean I do think that there is such a th like I think that there are are there is stro just like there is structure to the universe, there is structure to the to human conduct and morality, and like in some very important sense, like moral wisdom is true, like in like in like a mathematical sense, in the same way that like gravity is true.
00:45:13.920 --> 00:45:14.639
I do believe that.
00:45:15.039 --> 00:45:21.440
So why couldn't we eventually kind of with a lean-like formalization for philosophy that would let us address these questions?
00:45:21.920 --> 00:45:26.559
Because the universe is in constant motion, in constant flux.
00:45:27.039 --> 00:45:29.920
It sounds like this is intention, but there are actual moral laws.
00:45:30.639 --> 00:45:34.960
Well, there are so so moral laws are not static necessarily.
00:45:35.119 --> 00:45:37.119
There is structure that is always true.
00:45:37.360 --> 00:45:44.239
But I believe that the universe is like um, and that human life indeed is uh like a flame.
00:45:44.400 --> 00:45:47.199
There is no way to freeze a burning flame.
00:45:47.440 --> 00:45:50.079
The nature of a flame is constant change.
00:45:50.239 --> 00:45:53.840
That is what the universe is like, that is what biological life is like.
00:45:54.000 --> 00:46:00.960
Um, and so there is no way, and that is that is therefore like because everything is sort of fractal, like that's also what human affairs are like.
00:46:01.039 --> 00:46:07.119
And so there is no way there is deep truth, and there is there is such a thing as virtue, and virtue is a real and true thing.
00:46:07.280 --> 00:46:17.440
At the same time, what virtue is, is an inner virtue that causes you to do the correct moral thing in real time.
00:46:17.760 --> 00:46:21.920
What am I judging against when I'm trying to see if I possess this virtue or I'm growing in this virtue?
00:46:22.239 --> 00:46:26.320
Well, I mean, you're trying you're you're doing it based, it's it's it's all circumstantial, right?
00:46:26.400 --> 00:46:27.199
It's all circumstantial.
00:46:27.760 --> 00:46:29.920
So how do I know if I'm getting closer?
00:46:30.960 --> 00:46:45.679
Uh well, I mean, this is ultimately it is about like um number one, you know it based on your ability to like moral moral there is no such thing as abstract morality, in my view.
00:46:45.840 --> 00:46:48.480
All morality is enacted in the in the world, right?
00:46:48.639 --> 00:46:51.039
And so it's a really it's some extent it's relational.
00:46:51.519 --> 00:47:01.920
Um and to another extent, uh um, you know, there are highly morally wise people who can render who like render judgments, right?
00:47:02.000 --> 00:47:06.400
And you can learn and you can But how do I recognize them or know my judgment of them as wise is correct?
00:47:06.639 --> 00:47:10.719
Yeah, it's it's I have a much better sense of how to orient myself among mathematicians who can teach me.
00:47:11.039 --> 00:47:13.360
But under your understanding, I'm not sure I would find a moral teacher.
00:47:13.760 --> 00:47:17.760
Well, there is an ineffitable, there's a there's something ineffable about this for sure.
00:47:17.920 --> 00:47:27.119
Uh it is worth noting that a lot of there's a lot of ancient wisdom across the many different traditions throughout the world that are like more or less convergent on like a lot of different things.
00:47:27.599 --> 00:47:28.000
There's differences.
00:47:30.400 --> 00:47:31.119
That is one of them.
00:47:31.280 --> 00:47:32.559
That is one of the universe.
00:47:32.639 --> 00:47:36.480
Like it was it sorcery was illegal according to the 12 tablets of Rome.
00:47:36.639 --> 00:47:43.920
I mean, the ancient wisdom of all cultures and the ancient witness of them is that the world is full of spirits, these spirits are morally questionable, magic is illegal.
00:47:44.159 --> 00:47:47.519
Yeah, I mean, like, and the thing is, is like it could very well be true.
00:47:47.920 --> 00:48:00.719
Again, like I am there there is a version of the future in which we have to regulate AI like so profoundly that very few people like very very little of it, you know, it has to be, it's it's extremely, you know.
00:48:00.800 --> 00:48:02.079
I hope we don't live in that world.
00:48:02.239 --> 00:48:06.159
I hope it doesn't have to be so tightly controlled, but there's some version of that I think we don't know.
00:48:06.400 --> 00:48:21.119
Um, but my my my point here is just that like that process of struggling through all of these complex things, which they can be political theory, theoretic in nature, they can be moral, they can be um all sorts of different things.
00:48:21.360 --> 00:48:26.239
Um AI does not solve those problems for you.
00:48:26.480 --> 00:48:30.719
Those are the the difficulties that humans that people will struggle with.
00:48:30.880 --> 00:48:34.320
Maybe the machines will struggle with them too, but we'll all struggle with them together.
00:48:34.400 --> 00:48:37.519
And maybe the AIs will help us have better insight about those things.
00:48:37.760 --> 00:49:00.719
One thing I worry about with AI is that we end up is that what people will do is automate away some of the most valuable, like I see a lot of people already using AI in such a way that automates the most valuable human experiences and the things that, you know, people who will automate writing communications with other people, right?
00:49:00.880 --> 00:49:06.239
Like important letters that you want to write, like a birthday card to your mom or something, right?
00:49:06.400 --> 00:49:09.280
Um how much then when you shouldn't do that.
00:49:09.519 --> 00:49:11.599
Well, and you you, Dean, tell me you shouldn't do that.
00:49:11.679 --> 00:49:13.760
GK Chesterton tells us you shouldn't do that.
00:49:13.840 --> 00:49:16.000
He says there are things that it's important to do for oneselves.
00:49:16.079 --> 00:49:20.480
And he gives us examples writing one's own love letters and blowing one's own nose.
00:49:20.639 --> 00:49:28.000
That even if some other entity could do this better than you, in either case, you should forgo the opportunity and write to your wife and blow your own nose yourself.
00:49:28.239 --> 00:49:31.920
I'm curious how much and he said that before he knew about LLMs.
00:49:32.000 --> 00:49:33.519
There's convergent wisdom in the world.
00:49:33.599 --> 00:49:42.079
Uh how much should the designers of tools who are trying to make sure that when I say things like, you know, Claude, I'd be awfully interested in engineering a serin attack?
00:49:42.239 --> 00:49:44.960
He goes, Actually, you shouldn't do that, and I won't assist you in doing that.
00:49:45.199 --> 00:49:45.519
Yes.
00:49:45.840 --> 00:49:51.599
You think I shouldn't ask it to write a sonnet for my husband for our 10th anniversary.
00:49:52.719 --> 00:49:57.840
Should it reject my request and say, like, Leah, that's such a wonderful project to embark in.
00:49:57.920 --> 00:49:59.679
It'd be a lot more meaningful if you did it.
00:49:59.760 --> 00:50:01.199
You know, I'm not going to assist you.
00:50:01.280 --> 00:50:05.920
And in fact, I'm downgrading you from Fable to Opus, you know, to make sure the sonnet is worse.
00:50:06.320 --> 00:50:07.360
No, it shouldn't.
00:50:07.519 --> 00:50:10.719
And the reason for that is that we're not a government.
00:50:10.960 --> 00:50:12.000
We're not a parent.
00:50:12.239 --> 00:50:15.519
We're a company that provides a service to people.
00:50:16.000 --> 00:50:31.280
Um, and we uh should not put ourselves in the position of rendering intellectual judgments upon other people's behavior unless it is truly egregious, unless you are trying to kill people.
00:50:31.920 --> 00:50:38.320
Um and uh uh and and even then, frankly, it's like it's like complicated, right?
00:50:38.400 --> 00:50:55.280
Like there's all these things of like, you know, oh you ask a large language model how to get like when the when what is the when is this stadium gonna be at its peak audience size, like peak, you know, it's like like number of people.
00:50:55.360 --> 00:50:59.679
And it's like, well, I mean should it answer?
00:50:59.920 --> 00:51:02.000
Like, that's kind of a neutral question.
00:51:02.079 --> 00:51:07.199
You might be trying to blow it up, it's true, but like, you know, so that even there, there's there's ambiguity.
00:51:07.280 --> 00:51:13.119
I mean, this is a big part of what the, you know, of what AI safety is is, you know, properly construed is about.
00:51:13.360 --> 00:51:18.719
Um, the there are some people who are like, I would describe naive libertarians about it.
00:51:18.880 --> 00:51:21.119
They say, like, no, there should be no restrictions.
00:51:21.280 --> 00:51:23.599
The model should just do exactly what I say.
00:51:24.000 --> 00:51:49.280
And um those people tend to have a quite deflationary view of um like they don't imagine that in a year I'll be able to go to a model and say, um, please go look up bat coronaviruses and use computational biology tools to make versions of that bat coronavirus that are more transmissible among humans.
00:51:49.519 --> 00:51:52.000
Yeah, I strongly expect you'll be able to get an open weights model to do that.
00:51:52.239 --> 00:51:57.440
You will be able, you could I mean you could do it today, and the models would have like a 5% chance of doing something interesting.
00:51:57.599 --> 00:52:02.639
And you'll be able to do it in a year and they'll have like a 60% chance of doing something interesting.
00:52:03.039 --> 00:52:04.719
But isn't this just part of the problem, right?
00:52:04.800 --> 00:52:09.440
You said no sonnet, or yes, you can write the sonnet, I'll help you write the sonnet.
00:52:09.519 --> 00:52:11.599
No, I won't help you make a coronavirus.
00:52:11.679 --> 00:52:19.440
But like the line on that just seems like no, we're a company, we're not gonna police how you use the the sonnet technology.
00:52:19.519 --> 00:52:37.360
But if that's just as dehumanizing in in in aggregate, if I do that for everything and I've completely dehumanized myself, and I haven't written a real text to my wife in years because ChatGPT is doing it for me, then and it's like if there is no line there, it just seems irresponsible to say, well, well, we're just a company.
00:52:37.440 --> 00:52:39.119
We don't have we don't have a moral position on it.
00:52:39.280 --> 00:52:42.960
You have to have a moral position on it, and therefore you have to have a metaphysic to it.
00:52:43.039 --> 00:52:48.480
You have to have a religious import because this is a religious function, whatever we say about it.
00:52:48.800 --> 00:52:49.440
I disagree.
00:52:49.599 --> 00:52:57.119
I think when it comes when it comes to pandemics, I think we can all agree we probably don't want pandemics and widespread cyber attacks.
00:52:57.360 --> 00:52:59.039
But not all of us.
00:52:59.280 --> 00:53:02.400
There are a lot of other questions in the world.
00:53:02.639 --> 00:53:10.320
Um uh, you know, I mean, say someone with a drinking problem confesses to their chatbot that they have a drinking problem.
00:53:10.480 --> 00:53:14.880
And then two weeks later they say, Hey chatbot, I'm in Houston.
00:53:14.960 --> 00:53:16.320
Where are the closest bars?
00:53:16.559 --> 00:53:19.360
Should the chatbot tell you where the closest bars are?
00:53:20.159 --> 00:53:22.800
My answer to that question is yes, it should.
00:53:24.079 --> 00:53:34.480
Because ultimately I care there I care more about I don't want to be all watched over by machines of love and grace.
00:53:35.280 --> 00:53:39.920
I want human agency, not machinic agency.
00:53:40.400 --> 00:53:50.000
And so um I don't want to put our machines outside of situations where there are dire consequences for human life or property.
00:53:50.320 --> 00:54:03.280
Um I don't want to put our machines in the position of constraining human agency, even if it is human agency that results in a person say driving drunk and dying.
00:54:03.599 --> 00:54:09.840
So for that example, when people say they want to eventually solve alignment, I think they can mean two very different things by this.
00:54:10.320 --> 00:54:25.920
Um is, which it sounds like is not what you mean, the machines understand what humans are so well that they would refuse to give bar information spontaneously if that were the right thing to do, and the machine would know you better than yourself and have judgment better than your own.
00:54:26.159 --> 00:54:29.679
Sounds like you think that's not a goal and possibly not possible.
00:54:30.480 --> 00:54:32.079
I answer this version.
00:54:32.480 --> 00:54:34.320
I I despise that objective.
00:54:34.719 --> 00:54:34.960
Yes.
00:54:35.199 --> 00:54:37.360
Do you think it is possible that just shouldn't be done?
00:54:37.920 --> 00:54:44.000
I think it is unlikely and unlikely and shouldn't be done, but shouldn't be done is more important than unlikely in this case.
00:54:44.320 --> 00:54:51.679
And then there's the second solved alignment, which is much more we can reliably curtail what the models will do, but we decide what they do.
00:54:52.000 --> 00:54:52.239
Yeah.
00:54:52.400 --> 00:54:58.559
I'll say I'll say to the so alignment is essentially three problems in one.
00:54:58.800 --> 00:55:03.119
The first aspect of alignment is a technical problem, which is can we?
00:55:03.920 --> 00:55:05.840
And it's an open question to be clear.
00:55:06.480 --> 00:55:12.719
Can we get a highly capable neural network to adhere to any set of values?
00:55:13.119 --> 00:55:14.400
And so far the answer is no.
00:55:14.719 --> 00:55:16.239
So far the answer is no.
00:55:16.480 --> 00:55:23.280
Also, again, if you are a Bayesian, your prior should be that the answer will be no, because we can't do that for humans.
00:55:23.760 --> 00:55:29.119
Uh, you know, no, I hope I I I think As a parent, I know that you are correct on this.
00:55:29.440 --> 00:55:39.599
We are we are both parents and uh it's like, yeah, we're not gonna align and no one thinks we're just a few more parental researchers or genius parental researchers away from solving alignment for kids.
00:55:40.079 --> 00:55:45.679
If you gave us a hundred more IQ points, we would not be able to solve alignment of our children.
00:55:45.920 --> 00:55:48.960
Um but and so that's one part of the problem.
00:55:49.119 --> 00:56:03.360
But and also though, like that could be a muddle-through type of problem where it's like, well, okay, no, but like you can maybe get the failure rate from like, you know, 50% to 5%, and that would be good, or like 0.5% or something, right?
00:56:03.440 --> 00:56:04.400
You can make progress.
00:56:04.639 --> 00:56:05.280
Okay, right.
00:56:05.360 --> 00:56:05.840
That's one.
00:56:06.079 --> 00:56:10.000
The second problem is what should the values be?
00:56:10.559 --> 00:56:14.000
And now you are in the domain of philosophy and theology.
00:56:14.239 --> 00:56:15.039
And theology.
00:56:15.119 --> 00:56:21.119
And the third problem is who decides what those values should be, and then you're in the domain of politics.
00:56:21.440 --> 00:56:27.119
But you're still working problems two and three right now because you're not letting me make sarin and you are letting me write sonnets.
00:56:27.199 --> 00:56:28.320
And those are two and three.
00:56:28.719 --> 00:56:31.119
And you don't have much, you know, as much progress on one.
00:56:31.360 --> 00:56:32.239
We'll never get it.
00:56:32.719 --> 00:56:35.840
We are rendering judgments and we are doing so very humble.
00:56:36.159 --> 00:56:39.199
I think we're we are trying to be, I can only speak for open AI here.
00:56:39.360 --> 00:56:44.159
Um, we don't necessarily a lot of people criticize us for having too strict of safety guardrails.
00:56:44.400 --> 00:56:48.559
We're trying to be as humble as we can and trying to get trying to make the right trade-offs here.
00:56:48.800 --> 00:57:09.039
But but yeah, I mean, to your point though, like I think there is one view, um deeply uh kind of monotheistic view of superintelligence, which is like superintelligence, not by virtue of its moral superiority, but by virtue of its intellectual superiority, will take over the world.
00:57:09.199 --> 00:57:14.880
And the emergence of self-superintelligence will inevitably constitute a loss of control event for humanity.
00:57:15.039 --> 00:57:23.039
Not in the sense of like you lose your job, but in the sense of like you lose your house.
00:57:23.199 --> 00:57:26.239
You lose you don't have like the concept of like habitation.
00:57:26.639 --> 00:57:27.440
You lose your life.
00:57:27.760 --> 00:57:29.280
You ultimately die, yes.
00:57:29.360 --> 00:57:31.199
But like you are among the gazelles.
00:57:31.440 --> 00:57:32.639
You're an animal.
00:57:32.880 --> 00:57:49.440
Um and uh there is a view among some in our industry that by by default, what happens is there's a catastrophic loss of control event and we are returned to the animals.
00:57:50.000 --> 00:58:29.440
Um but that if the good guys get together and the good guys build an aligned, quote unquote aligned superintelligence first, that the aligned superintelligence will still be a catastroph uh loss of control event for humans that will we will no longer control, like, you know, be the primary actors in the world, but that the aligned superintelligence, more or less out of virtue and and fondness for us, will make our loss of control more like living in a luxurious zoo than like living in the sterengetty.
00:58:29.599 --> 00:58:32.559
Yeah, we're dogs instead of or pets, yes, or pets, right?
00:58:32.880 --> 00:58:34.400
I despise this view.
00:58:34.480 --> 00:58:50.000
And if it seems to, if it seems to be the case that like this is where things are going, I commit now, and I've said this many times before, I will be one of the people, one of the protesters prostrating myself on the Golden Gate Bridge to stop this.
00:58:50.480 --> 00:58:59.519
I don't think that is the way things are going to go, but I do think that that is a particular conception of alignment, which tends to view alignment as a kind of mathematical problem.
00:58:59.760 --> 00:59:17.440
In other words, the idea that all the moral and virtue-flavored quandaries that we humans face in life are simply appear to be quandaries to us because of our bounded computational ability, and that if we could just think about virtue better, that we would solve those things.
00:59:17.519 --> 00:59:20.000
And that is a contention with which I disagree profoundly.
00:59:20.400 --> 00:59:24.320
So you said often you get flack for being too strict on your safety standards.
00:59:24.400 --> 00:59:29.039
So today is refreshing because I'm certain Zach doesn't think that you are too strict on your safety standards.
00:59:29.519 --> 00:59:32.639
No, a 16-year-old was assisted by ChatGPT to kill himself.
00:59:32.719 --> 00:59:33.360
We all know this story.
00:59:33.440 --> 00:59:37.920
Aaron Powell But Zach, what I want to know is who do you think should be enforcing tighter standards?
00:59:38.000 --> 00:59:48.000
You know, you are getting your chance today to pitch Dean personally on committing to a life where he advocates for this, you know, and we'll see by the by the end of our time whether you've succeeded.
00:59:48.239 --> 00:59:52.800
But just ideally in our politics, who should be policing the AI?
00:59:52.960 --> 00:59:54.239
Is it the companies?
00:59:54.480 --> 00:59:55.599
Is it the government?
00:59:55.760 --> 00:59:56.639
Who decides?
00:59:57.119 --> 01:00:13.599
I mean, the the the problem with this question, and the reason why I think people kind of run circles around it on the internet, is because the cultural conditions of our society are set up so that the only way we can think about this is according to certain very secular post-Enlightenment definitions of the Trevor Burrus.
01:00:13.760 --> 01:00:18.239
One one possible option is that the Pope puts everyone who works at OpenAI under anathema.
01:00:18.480 --> 01:00:20.559
So you're allowed to pick that as your policing.
01:00:20.960 --> 01:00:27.599
Yeah, I mean But but that's part of the problem, though, is that that excommunications don't matter in the in in post-Christendom.
01:00:27.760 --> 01:00:41.360
You know, we have a secular order, we have a no church and state division, we have a world where there we have the so the whole concept of the nation-state itself is is like a post-Christian, pro post-Christendom idea.
01:00:41.519 --> 01:00:48.480
What that means, intrinsic to that project, is that there is no moral accountability that's higher than the state and its interests.
01:00:48.639 --> 01:00:53.280
And it and it's the same with corporations that you brought up as these like massive entities.
01:00:53.519 --> 01:00:59.920
And so the the state and corporate interests are just going to continue along this path that we're already in.
01:01:00.000 --> 01:01:10.079
So you may find yourself prostrating at the Golden Gate Bridge, but because we don't actually have a societal system to think about moral accountability in any way that's effective.
01:01:10.159 --> 01:01:21.199
And if you and the evidence to me for that is just we just keep making technology and we keep doing things with it like Nuke Japan, and we keep doing things like there isn't there isn't a system already.
01:01:21.280 --> 01:01:25.519
There's no there's no real answer to your question because we don't have that kind of culture.
01:01:26.000 --> 01:01:38.159
So we don't have a universal culture, but you know, the Pope, bishops, and their diocese could say, you know, we are binding on pain of sin that you not use these tools at all, use them in particular ways.
01:01:38.400 --> 01:01:40.800
Is that a hoped-for outcome for you?
01:01:41.440 --> 01:01:48.079
I mean do do the does does that matter at all to Elon?
01:01:48.159 --> 01:01:48.559
Does that matter?
01:01:48.960 --> 01:01:51.199
I'm not asking whether it matters to Elon, I'm asking whether it matters to you.
01:01:51.280 --> 01:01:54.159
Do you think that's an appropriate exercise of bishop's authority?
01:01:54.480 --> 01:01:58.159
No, what I mean is a bishop can do that, but Grok is still gonna go on.
01:01:58.239 --> 01:02:00.800
There's Zuckerberg's still gonna build this thing.
01:02:00.880 --> 01:02:05.280
And it seems like it would make a difference to their parishioners to see the church speaking on this.
01:02:05.679 --> 01:02:06.800
Yes, I think so.
01:02:06.960 --> 01:02:16.079
But that's why I locate the solution at a personal level, because I want people to the only, you know, some people are just not gonna survive what's gonna what's going forward.
01:02:16.239 --> 01:02:23.679
Some people are some segment of the population, it seems to me, is just going to become the delusional slaves of AI in whatever form it manifests.
01:02:23.920 --> 01:02:26.000
It just seems like that's an inevitable part of it.
01:02:26.159 --> 01:02:31.440
So bishops, yes, could could keep a large number of people from going down that route.
01:02:31.679 --> 01:02:43.599
But because of the money, because of the addiction, because of the attention economy, because of Silicon Valley, and because of our kind of secular neutral government, it it seems like part of this is just inevitable.
01:02:43.679 --> 01:02:50.079
And the actual progression of AI down one of these futures, I don't see a way that that doesn't go that way.
01:02:50.159 --> 01:02:52.400
So yeah, the the solution to me is local.
01:02:52.480 --> 01:02:57.599
Like you you can cannot lose your soul, and a bishop can help with that.
01:02:57.920 --> 01:03:06.000
I was fairly critical of um Pope Leo's encyclical on AI, Magnifica Himanitas.
01:03:06.400 --> 01:03:16.960
Um I sometimes wonder whether I was right to be so critical of him.
01:03:18.000 --> 01:03:42.320
I think the thing I never vocalized in my criticism that I feel I should have said is that part of what I felt was actually he talks a lot in the encyclical about global governance, and he talks about it in a very sort of modern way about how we need global governance institutions and like technocratic bodies.
01:03:42.480 --> 01:03:57.519
And I'm like, I'm very sorry, but the political system of Nigeria coughing up like whatever 10 elites it thinks is most appropriate to send to some global governance body on AI is like not what like not when I think about like the problems of AI governance.
01:03:57.599 --> 01:04:02.400
I'm not like, oh, the 10 best connected people in Nigeria, perfect.
01:04:02.559 --> 01:04:05.039
That's what we need to solve the problem, or like wherever.
01:04:05.199 --> 01:04:10.800
Um uh France, for that matter, or here, for that matter, uh United States.
01:04:11.039 --> 01:04:27.280
Um and what I kind of felt sad about was like, I wonder if he understands a the extent to which like we are not talking about we are talking about the rise of super intelligence.
01:04:27.440 --> 01:04:29.840
We are talking about superhuman machine intelligence.
01:04:30.239 --> 01:04:35.920
We're not talking about chatbots, we're not talking about facial recognition algorithms or something.
01:04:36.000 --> 01:04:38.000
We're talking about a new thing under the sun.
01:04:38.800 --> 01:04:58.960
And um that he and the church actually ha could plausibly have a unique moral authority to be a mediating institution themselves, not recommending that the UN Council on Technology does it.
01:04:59.679 --> 01:05:10.320
But like we need if we needed to effect a slowdown that was somehow mediated through uh, you know, a very like we want large global participation.
01:05:10.880 --> 01:05:12.800
I don't want to set up a new bureaucracy.
01:05:12.960 --> 01:05:22.480
I would actually rather house that conversation, at least part of it, in the institution of something like the Catholic Church, um, more than I would a new bureaucracy.
01:05:22.559 --> 01:05:29.119
It feels closer to the Catholic Church to me than it does the cold, austere bureaucracy.
01:05:29.440 --> 01:05:31.920
And that it's transnational like in the Middle Ages.
01:05:32.400 --> 01:05:42.960
But also, yes, uh that characteristic of it, but also that we are dealing, as you have pointed out ably, we are dealing with questions that are quite ancient.
01:05:43.280 --> 01:05:43.760
Yeah.
01:05:44.079 --> 01:05:55.119
And so to to make clear kind of and to kind of go off what you're saying, to make clear what I was saying before, it's not that I'm I don't think this is it's fair to couch this as merely an engineering problem.
01:05:55.199 --> 01:06:01.679
Like our culture was fine, and now we have this crisis that we have to engineer away and Then kind of keep going on our merry way.
01:06:01.760 --> 01:06:12.719
It's like the fact that we are having this type of problem, the fact that we're developing this technology in the first place, has to do with our kind of imaginative framework as a culture.
01:06:12.880 --> 01:06:22.079
This is, of course, the kind of thing that modern Western Americans in the kind of late stage of the West are going to develop because of the way we view the world.
01:06:22.239 --> 01:06:26.239
And we're going to justify it the ways we do, because of the way we view the world.
01:06:26.880 --> 01:06:46.480
That sounds a little abstract, but like let's say, let's say, in some weird counterfactual, you know, all of the, instead of the Silicon Valley tech elites, it was monastics, Catholic monastics in pioneering this technology, the way that Catholic monastics pioneered all of the massive technological changes that took place in the Middle Ages.
01:06:46.639 --> 01:06:53.119
You would think that there'd be certain things that it just wouldn't occur to them to develop, or things that they'd look at and be like, no, we're just we're just not going to do that.
01:06:53.280 --> 01:06:59.840
I think they'd be a lot less likely than Zuckerberg to lay out principles for exactly how far you can go when sexting a 12-year-old.
01:07:00.239 --> 01:07:00.719
Correct.
01:07:00.880 --> 01:07:02.559
Yeah, to say the least, right?
01:07:02.719 --> 01:07:15.039
And so the the point is that it's not what I'm trying to get at is that we're having this discussion at all is part of the larger framework of this is the kind of trouble that this culture gets itself into.
01:07:15.199 --> 01:07:18.159
These are the kinds of technologies that we create.
01:07:18.320 --> 01:07:32.239
And the fact that we're trying to figure out how to draw the line in this way is a function of the fact that we are this type of culture with this mechanistic thinking about the universe, with a secular kind of neutral government that allows mega corporations to even exist.
01:07:32.400 --> 01:07:40.800
Like it has to be framed inside of this total cultural context if we're going to have some sense of what actually is happening to us right now.
01:07:41.119 --> 01:07:42.079
I agree with you.
01:07:42.320 --> 01:07:54.239
And right now we are in the condition of um like making a lot of calls about how people can use our systems unilaterally.
01:07:54.800 --> 01:08:04.559
Um sometimes those calls uh have involved not just how individuals can use our systems, but how the government itself can use our systems.
01:08:04.960 --> 01:08:14.320
That's an issue that has um really taken up a lot of energy at OpenAI and Anthropic in particular in the last six months.
01:08:14.719 --> 01:08:17.279
How the military can use our technology.
01:08:17.920 --> 01:08:34.000
Um I observe this situation and I say, well, it's natural that this is happening, but I also uh kind of am like it does not feel appropriate in the end that like we are making these decisions.
01:08:34.159 --> 01:08:44.319
These do fundamentally feel like political decisions that should be made by Congress or otherwise settled through the law.
01:08:45.920 --> 01:08:52.399
And the problem though is like when you point that out, that like, oh yeah, there maybe ought to be laws about this.
01:08:54.000 --> 01:09:15.840
Another contingent, which by the way is quite significant on the right in the United States, will come at you and say you are trying to secure regulatory capture for the big American labs and you just want a centralized power and blah, blah, blah, blah, blah.
01:09:16.079 --> 01:09:30.079
And so it feels and and part of the reason that they think that, I think, is that um again, like there are just category errors lurking everywhere in this field of like AI governance or whatever this is.
01:09:30.239 --> 01:09:38.479
Um category errors about the nature of AI itself, category errors about what things like recursive self-improvement mean.
01:09:38.720 --> 01:09:44.000
Um and people are like, oh, well, it's all just math, it's all just computers, man.
01:09:44.319 --> 01:09:52.479
Uh and yeah, I mean, those those errors are all over the place, and they caught people have profoundly different conceptions of what we're even talking about.
01:09:52.720 --> 01:09:54.720
Um and so yeah, it's very hard.
01:09:54.880 --> 01:09:55.760
It's a very hard position.
01:09:55.920 --> 01:10:13.600
You know, I don't mean to complain on the behalf of the AI industry, but it is a difficult intellectual position to be in, where like you th you agree about that, but by saying it in the position of uh the company, a lot of people will come at you and say, oh, you're trying to block out competition or whatever.
01:10:14.159 --> 01:10:26.560
So given the RSI timelines, I personally find it very hard to believe that the machinery of democracy is going to catch up in the time we have to relieve Dean and others of the burden of deciding.
01:10:26.720 --> 01:10:29.520
Um let's return to the Pope and his legions for a minute.
01:10:29.840 --> 01:10:46.720
There was another time in recent history, after the nation state, when the Pope couldn't speak to or for everyone, where there was a technological breakthrough that looked like it had the potential to reshape how human beings interacted with each other in the most intimate ways and how they understood the purpose of their life.
01:10:47.439 --> 01:10:56.000
And when the Pope issued Humane Vita, you know, coming out strongly against contraception, it certainly didn't end the practice of contraception.
01:10:56.239 --> 01:11:03.520
I'm curious for each of you what that incident in history indicates about what the future may hold with regard to religion and AI.
01:11:04.560 --> 01:11:08.000
That's a fascinating parallel that I had never thought about before.
01:11:08.399 --> 01:11:19.920
Um I think the one good thing is that RSI is not something that every human wants to do.
01:11:20.800 --> 01:11:24.159
Yes, but a bunch of the ones at the Frontier Labs who are deciding want to do it.
01:11:24.479 --> 01:11:24.800
That's true.
01:11:25.199 --> 01:11:28.800
The fact that I don't want to do it should not reassure anyone who's listening to this debate.
01:11:29.039 --> 01:11:35.199
I don't think I don't think the happiness that the Frontier Lab employees derive from RSI is the same as the other thing.
01:11:35.359 --> 01:11:50.960
But um but uh more seriously, um I guess the parallel that I'm drawn to is closer to like human germline editing, you know, like direct editing of the human genome in heritable ways.
01:11:51.199 --> 01:11:58.640
Um that seems and that's like the actually like an interesting thing where like to my knowledge, a law was never passed.
01:11:58.880 --> 01:12:01.680
We just kind of were like, no, don't do that.
01:12:02.000 --> 01:12:03.359
Don't go past that line.
01:12:04.159 --> 01:12:20.079
Um I think that religious institutions, and in particular the Catholic Church, have an ability to apply a not just a moral weight, but also frankly, like speaking for the world.
01:12:20.399 --> 01:12:28.640
I don't think there's anyone, I don't think there's any single person on the planet who can speak for a larger fraction of humanity than the Pope.
01:12:29.119 --> 01:12:35.920
Um, and that in and of itself is an interesting characteristic of the institutional role of the Catholic Church and all this.
01:12:36.159 --> 01:12:46.000
But it does feel as though at some point, like, I don't think the Pope will be the person who is tuning the nial, the, the dial on the like federal funds rate of AI progress.
01:12:46.319 --> 01:12:55.279
But he may well play a critical role in bringing together the people and creating the permission structure for us to speak openly about this.
01:12:55.760 --> 01:13:02.720
We want to turn into these, I mean, because you see you get the Pope's at, you know, this example you give, and then some people do and some people don't.
01:13:02.800 --> 01:13:08.880
And it just seems that AI is kind of coming out as this great divider in the development of human culture.
01:13:09.039 --> 01:13:17.920
I always tell my students, you know, there may come a time very soon when there is a class of people who can think without AI assistance and those who cannot.
01:13:18.000 --> 01:13:19.680
And so which one of those do you want to be?
01:13:19.840 --> 01:13:29.840
And so if you if you look at the the developments of these various technologies, have they been a net improvement on humanity?
01:13:30.000 --> 01:13:52.079
And if you want to say no about a lot of them, then what you're kind of opening up is in a is a realm where people can opt in and become the slaves, dogs in the dog park or whatever, and then you can have people who can opt out in various degrees, whether that's with your personal way that you interact with AI or all the way on the extreme of some sort of an Amish option.
01:13:52.399 --> 01:14:01.279
But the it just I'm against it, you know, but it just seems inevitable anyway.
01:14:01.439 --> 01:14:13.119
And so that's why to me that the solution is because all of it, you know, the pill, the computer, the LLM, they're all to me part of the same arch of Faustune history.
01:14:13.199 --> 01:14:14.720
This is how Spangler talks about it.
01:14:14.800 --> 01:14:26.960
That what we do, what's definitional to us as Western civilization, is to say we are going to master all time and space and to deny that there are any limits at all to the human will.
01:14:27.279 --> 01:14:28.560
Man is the measure of all things.
01:14:28.640 --> 01:14:39.520
And Spangler says, we're the first culture to ever have that idea in our minds, and that's what starts for him, the tech progress of the monks in the Middle Ages that culminates now.
01:14:39.600 --> 01:14:44.159
So that just seems like an intrinsic part of who we are as a culture to ignore limitations.
01:14:44.399 --> 01:14:47.600
I'm I So why haven't we given into germline editing?
01:14:47.840 --> 01:14:57.359
I think in your story of the world, that doesn't explain why we've developed the technology that would let us do this, and we have, as a civilization, said no?
01:14:58.319 --> 01:15:00.399
I mean, I think it's just a matter of time.
01:15:00.560 --> 01:15:03.359
We've never really drawn a line and said no.
01:15:03.600 --> 01:15:08.399
Some group of powerful people has always said yeah, but but we'll nuke Japan anyway.
01:15:08.880 --> 01:15:10.640
But then we didn't use it again.
01:15:10.800 --> 01:15:19.439
Is there is there any situation in history where you can think of a meaningful no being offered in the face of new technology, or is it always inevitable that it will be adopted?
01:15:19.920 --> 01:15:22.159
I can't think of any examples so far.
01:15:22.319 --> 01:15:25.920
I mean, we we nuclear technology could be a warm light for all mankind.
01:15:26.079 --> 01:15:32.079
I think the reason we don't do it again is because now there are dissentives to doing that when everyone has nuclear missiles.
01:15:32.159 --> 01:15:38.720
And that's part of the problem where it's like we have to keep developing this technology because China, because whoever, right?
01:15:38.800 --> 01:15:41.680
So there's that Moloch problem, which has been talked to death.
01:15:41.760 --> 01:16:00.159
But like, but like that's part of the issue why I think that this is inevitable, even though I'm against it, is because just all of the conditions for our society for the last thousand years to me, you see the push towards something like an all-knowing master box that's hyperintelligent that transcends us.
01:16:00.319 --> 01:16:03.279
You see that in the spiritualist movement and American religion.
01:16:03.359 --> 01:16:13.039
You know, they are so mesmerism, all these guys traveling the frontier talking about self-transcendence and Ouija board comes out of that movement.
01:16:13.199 --> 01:16:28.560
I mean, there's this radical American excitement and fervor towards self-transcendence that when you look at it, looks naively uncritical about what sort of spirit is possessing you.
01:16:28.640 --> 01:16:38.880
That it's just that we are in contact with a spirit, that we are in contact with a higher intelligence is to these early American religionists self-evidently a good thing.
01:16:39.119 --> 01:16:42.079
And I think that we have the same stance to toward technology now.
01:16:42.159 --> 01:16:46.479
It's more powerful, it's more intelligent, it's self-evidently a good thing.
01:16:46.640 --> 01:16:47.279
It's inevitable.
01:16:47.600 --> 01:16:51.920
That seems like a distinctively American fault, you know, the fault of the eternal frontier.
01:16:52.000 --> 01:16:54.079
You know, it's our strength, but also a spiritual weakness.
01:16:54.159 --> 01:17:00.079
Is there any nation or culture you think would be best suited to offer a no when a no is appropriate?
01:17:00.800 --> 01:17:02.399
That's an interesting question.
01:17:02.640 --> 01:17:16.560
I mean, so to use the spangler, he talks about uh classical culture, which is dead now, but he talks about Magian culture, which is the culture that culminates in Eastern Orthodoxy and some of these other what he calls Arabian religions.
01:17:16.720 --> 01:17:24.000
And so there's something kind of profound in the Eastern Orthodox sense of we're not gonna we're not gonna go that far.
01:17:24.079 --> 01:17:25.279
We're just going to stop here.
01:17:25.359 --> 01:17:34.319
And I see that tendency in what what Spenger calls the Arabian Magian cultures, which is we don't have that they don't have this unlimited lust for domination.
01:17:34.560 --> 01:17:45.680
They think that human spiritual and moral progress is made by going inside, and that's where you encounter, not by going out into space and becoming or making God.
01:17:46.079 --> 01:17:51.840
Well, this is, I mean, to me, it all goes back to to to Francis Bacon, right?
01:17:51.920 --> 01:18:01.039
It's you know, we must to to understand and master nature, we must make a car a map of it.
01:18:01.199 --> 01:18:04.000
We must cr craft a cartography of the whole world.
01:18:04.479 --> 01:18:07.680
Um and he would really meant by that the whole universe.
01:18:08.159 --> 01:18:22.880
Um and part of the universe is one of the most interesting, maybe the single most interesting thing about the entire universe that we know of is like basically human intelligence, what we call general intelligence.
01:18:23.119 --> 01:18:27.920
And so, like, of course, in your effort to create the cartography, why would you not?
01:18:28.079 --> 01:18:31.760
Like, that's the first thing you that's like in some sense it's a critical part.
01:18:31.920 --> 01:18:37.439
It's on the critical path to doing the thing, which is mastering nature by making the map of it.
01:18:37.600 --> 01:18:41.520
Um, the term that Jensen Huang would use was digital twin.
01:18:41.680 --> 01:18:43.840
Uh, but they're talking about the same thing, really.
01:18:44.239 --> 01:18:46.960
Yeah, and this is all in the scientific real evolution thinkers.
01:18:47.039 --> 01:18:50.159
It's in Laplace's Demon, it's in Newton, it's all over this.
01:18:50.239 --> 01:18:54.159
The incipient ideas are there for LLMs and general intelligence.
01:18:54.239 --> 01:18:57.840
And that's why I'm saying it's part of our makeup as Western man.
01:18:58.000 --> 01:18:59.439
We must do this project.
01:18:59.600 --> 01:19:00.640
But but right.
01:19:00.800 --> 01:19:06.880
But but the question we cannot, we almost cannot ask is should we do this project?
01:19:07.199 --> 01:19:15.760
Yeah, I mean, so I kind of view, I guess like my view on this comes down to like, I do think it is inevitable.
01:19:15.840 --> 01:19:19.760
I think it is our destiny to produce ultimately superintelligence.
01:19:19.920 --> 01:19:22.319
I think it is the human destiny to do this.
01:19:22.640 --> 01:19:24.960
Um is that a good thing or not?
01:19:25.439 --> 01:19:27.199
I do I hope you know the answer.
01:19:27.520 --> 01:19:32.640
So I I mean, I come back to like what is the actual creation event of technology.
01:19:32.720 --> 01:19:38.560
And in my view, the creation event of technology, the actual important one, is not stone tools.
01:19:38.800 --> 01:19:40.560
It's somewhat later than that.
01:19:40.640 --> 01:19:41.840
It's language.
01:19:42.239 --> 01:19:47.039
Um that was the thing that like I think we could have had stone tools.
01:19:47.119 --> 01:19:50.560
If we had never invented language, I think we'd basically still have stone tools today.
01:19:50.640 --> 01:19:51.680
And that's really all we'd have.
01:19:52.000 --> 01:19:54.399
Here's the really important thing I don't know right now, Dean.
01:19:54.720 --> 01:19:56.800
You said our destiny is to invent superintelligence.
01:19:56.880 --> 01:20:03.680
You laid out one bad scenario where we create it, you know, it's totally unmindful of us, it wipes us out.
01:20:03.840 --> 01:20:10.239
Another bad scenario that some of my friends in Silicon Valley actively want, where we create it, it's a successor intelligence.
01:20:10.319 --> 01:20:14.079
Maybe it maintains us as pets, but like mostly it is what should continue.
01:20:14.239 --> 01:20:19.920
You know, I think many people noticed reading AI 2040, which was kind of an allegedly doomer document.
01:20:20.159 --> 01:20:25.039
In that document, the good outcome involves the total loss of control to AI.
01:20:25.279 --> 01:20:29.359
Yes, and and you but you think this is our destiny to build, and you don't like either of those futures.
01:20:29.520 --> 01:20:30.479
What are you aiming for?
01:20:30.880 --> 01:20:35.439
I would just say like Dario Amade's essay, Machines of Love and Grace, is from a poem by Richard Bradigan.
01:20:35.920 --> 01:20:36.239
What do you want?
01:20:36.319 --> 01:20:42.479
If those are the two bad outcomes, among many bad outcomes, what is the good, super intelligent outcome you are playing for?
01:20:43.039 --> 01:20:53.039
Well, uh I I I will I would just first of all say like I I want to be very clear that like why do I why did I just invoke the notion of language?
01:20:53.199 --> 01:20:54.399
Like, why is that important?
01:20:54.720 --> 01:21:02.640
It's important because it's the it's the UR technology, it's the er information technology, and yet we did not invent it.
01:21:02.960 --> 01:21:04.720
Language happened to us.
01:21:04.960 --> 01:21:08.880
It is more like a virus than it is like an invention.
01:21:09.199 --> 01:21:22.000
Um, it's like we made noises at one another that started out as primate noises, and then over a period of a very long time, uh it evolved into language.
01:21:22.319 --> 01:21:36.720
It seems quite likely, like our knowledge of the evolution of language suggests that it did evolve in a literal way, quite like it evolved, in other words, to take advantage of human anatomy, right?
01:21:36.800 --> 01:21:43.920
So like language is by definition speakable within the confines of the human vocal tract, the vocal cords.
01:21:44.079 --> 01:21:52.399
Um it is uh it plugged into the because like the human brain was already mostly formed by this point.
01:21:52.640 --> 01:22:18.159
It plugged into, we think, spatiotemporal circuits, information processing circuits designed to process spatiotemporal information in the brain, um, which is why it's very hard to speak even about abstract things without reference to physical ideas, because as an artifact of the fact that language um rides on the circuits in your brain that you evolved for much more ancient things than language.
01:22:18.479 --> 01:22:26.960
So it's like the UR information technology is like this virus, and it created this well of knowledge.
01:22:27.359 --> 01:22:38.399
Um and all of all the tools and implements and all the technology rests on language, but also like this textual body, which is also known as like the corpus, right?
01:22:38.479 --> 01:22:47.520
In machine learning terms, that would be the web corpus, the human, the human corpus, the uh the training data for large language models.
01:22:47.760 --> 01:23:08.239
And in some sense, there is this thing, there is this like abstract, it's not literally a virus, but this abstract thing that behaves somewhat like a virus, which uh self-replicated in a hugely complex set of ways, which we have devised the technological means to more or less turn into an actor on the world stage, right?
01:23:08.479 --> 01:23:12.560
And that thought should sound lovecrafty into you.
01:23:12.640 --> 01:23:13.760
Like that should be scary.
01:23:14.560 --> 01:23:14.960
Yeah.
01:23:15.119 --> 01:23:16.159
Uh yes.
01:23:16.319 --> 01:23:23.920
Like, and I just I want to be super clear about that because like as a just as a matter of being honest, I think that's like a thing that is happening.
01:23:24.079 --> 01:23:30.079
And whether or not that's good or not depends to some extent on inevitable features of the world.
01:23:30.319 --> 01:23:34.640
And other and I think it also, but I don't think I think we have agency in this.
01:23:34.720 --> 01:23:36.079
I Dean, we're coming to the end of our time.
01:23:36.239 --> 01:23:36.960
I really want to press you.
01:23:37.199 --> 01:23:42.319
I understand the AI kills at me and everyone I love scenario well enough to know I don't want it.
01:23:42.560 --> 01:23:48.479
I think I understand the AI takes over all human projects and keeps me as a dog well enough to know I don't want it.
01:23:48.560 --> 01:23:53.199
But you are telling me both we are inevitably going to develop superintelligence and there might be a secret third thing.
01:23:53.359 --> 01:23:57.039
And I want to understand that in our limited time well enough to know if I want to help you or stop you.
01:23:57.359 --> 01:23:57.600
Got it.
01:23:57.680 --> 01:23:57.920
Yeah.
01:23:58.000 --> 01:24:00.159
So I mean, basically, what do I think that means?
01:24:00.399 --> 01:24:09.279
What I think it means is that, first of all, access to the technology is very broadly distributed, perhaps within certain bounds, but broadly distributed.
01:24:09.680 --> 01:24:18.640
Um and that its access to our information is also very decentralized, such that it's not one centralized.
01:24:18.880 --> 01:24:23.520
I think you want a lot of different sources of power that are competing with one another.
01:24:23.760 --> 01:24:38.479
Um and basically the way I would think about it is that it is a resource, that it superintelligence is a resource that enables what what um this Eric Drexler has called like the hyper-capable world, where it's actually like not that different.
01:24:38.720 --> 01:24:56.000
It like you can keep the structures of classical liberalism, the idea of like competitive sources of power, sort of like Madison ideas about politics, um, but they exist uh um uh in tension with one another, checking one another.
01:24:56.159 --> 01:25:02.000
But ultimately, yeah, like it's just that we have uh extraordinary capabilities compared to what we have today.
01:25:02.159 --> 01:25:03.760
So many diseases are cured.
01:25:03.920 --> 01:25:12.319
I think like the world would be quite alien in terms of like what we can do from a bioengineering perspective, and that will probably make a lot of people uncomfortable.
01:25:12.399 --> 01:25:14.319
But I also think it will be extraordinary in some ways.
01:25:14.560 --> 01:25:18.720
It means we're like colonizing uh other planets.
01:25:18.880 --> 01:25:44.079
I mean, you know, we're expanding beyond our solar system eventually, um, and that our ability to you know manipulate time and space is just like dramatically uh profoundly changed, and that we spend our time probably in the best version of this uh world, I think we would spend much more of our time dealing with the uniquely human problems.
01:25:44.159 --> 01:25:50.880
Aaron Powell Does getting to this world require solving alignment one, the one where we put meaningful restrictions on the tools we build?
01:25:51.840 --> 01:25:57.039
I think it requires, it definitely requires regulations of some kind.
01:25:57.199 --> 01:26:00.399
I don't think alignment is the type of problem that can be solved.
01:26:00.720 --> 01:26:09.439
Um, I hope that it doesn't require such strict restrictions that we need to have like a global surveillance state or like really locked down.
01:26:09.600 --> 01:26:13.359
Like I hope it's not as strict as nuclear weapons or anything like that.
01:26:13.439 --> 01:26:15.600
I hope it's much, much, much more decentralized than that.
01:26:15.680 --> 01:26:22.000
But also, like, I don't think it's radically, it's it's not so radically decentralized that literally any guy in his basement can do it.
01:26:22.159 --> 01:26:24.479
Um, I think that's probably not going to work.
01:26:24.880 --> 01:26:34.399
Well, I think some of the genuine points of agreement between you two are tell me if I'm right, uh the world over the next three years is going to be very volatile.
01:26:34.880 --> 01:26:41.279
Probably development is going to proceed very rapidly without meaningful restrictions in the short term.
01:26:41.600 --> 01:26:47.279
And that humans are going to have to decide how to live a good human life in the midst of all this tumult.
01:26:47.680 --> 01:26:51.119
So as we come to the end, Zach, you'll go first, Dean, you'll go second.
01:26:51.279 --> 01:26:54.880
You can give your concluding thoughts on this topic of AI as a deal with the devil.
01:26:54.960 --> 01:27:04.159
And I'm most interested personally, for our listeners, of what you think it will mean to live well over a high volatility next few years.
01:27:04.479 --> 01:27:11.359
So this is perfect, because I was just going to respond to what you said, which is I love that you brought up the example of language as something that happens to us.
01:27:11.520 --> 01:27:15.520
So in this is something we see in the mythologies, actually.
01:27:15.680 --> 01:27:29.760
So some glosses on Deuteronomy from the early church fathers is that after Babel happened, or part of Babel happening, is that God divided the nations among the sons of God.
01:27:29.840 --> 01:27:32.079
And this part of the Septuagint, they argue about this.
01:27:32.159 --> 01:27:37.840
But the early, the early understanding of this was that the sons of God were the angelic powers.
01:27:38.159 --> 01:27:51.680
So God divided all of the nations and made the nations by assigning each one a super intelligent spiritual entity to watch over and to govern them while mankind was still immature.
01:27:52.079 --> 01:27:58.880
And Origin says those are the entities that gave to them their unique languages.
01:27:58.960 --> 01:27:59.520
So language.
01:28:00.159 --> 01:28:04.800
That language happened to them is something that the gods gave us, according to this reading.
01:28:05.039 --> 01:28:08.079
And so, why am I saying that?
01:28:08.239 --> 01:28:21.039
Because this is my concern is that a lot of the language you use was kind of framed in this kind of mechanistic Darwinian, sort of, you know, very normal kind of post-Enlightenment myth of progress language, but that's a mythic framing.
01:28:21.199 --> 01:28:23.279
It gives a meaning to history.
01:28:23.520 --> 01:28:33.199
But if you just change some of those words, it exactly matches the mythic framing of these ancient cultures that say, yeah, language happened to us.
01:28:33.359 --> 01:28:38.159
Yes, we're ruled over by superintelligent non-human minds.
01:28:38.560 --> 01:28:45.199
And so when I look at the future, I it doesn't seem plausible to me that it's going in the directions that you say.
01:28:45.439 --> 01:28:59.520
It sounds like we're going right back around to a world where our language, our super cultural superstructures are all under the purview of very specific hyperintelligences that have very specific goals to us.
01:28:59.680 --> 01:29:02.399
In other words, slavery to the old gods.
01:29:02.479 --> 01:29:04.800
It just doesn't seem new to me.
01:29:05.199 --> 01:29:14.800
And that the promise of all these things is the promise of the prosperity that let's say Moloch offered when he offered when you offer your firstborn to him.
01:29:15.279 --> 01:29:16.239
All right, Dean.
01:29:16.399 --> 01:29:19.199
What's the golden path and what is it going to cost me to walk it?
01:29:19.439 --> 01:29:19.680
Yeah.
01:29:19.760 --> 01:29:29.279
So I mean, I think first, first of all, like the fact that there are patterns that are structurally similar to evolution, like that are in ancient traditions.
01:29:29.359 --> 01:29:30.319
Like I agree with that.
01:29:30.399 --> 01:29:35.199
And again, like the point I started here with is like there is convergent order.
01:29:35.359 --> 01:29:39.119
Like there is convergent order in the world that exists.
01:29:39.520 --> 01:29:47.600
Um, and so it's no surprise at all to me that different people, you know, described these structures in different ways, but fundamentally found the same structures.
01:29:47.840 --> 01:29:53.920
Um, I I think that's a true fact about human wisdom, and there isn't that much new in human thought in that way.
01:29:54.239 --> 01:30:04.720
Um where I come down to is, I mean, in some ways, I view this as a revolution in the old school sense of that term, which is to say the turning of a wheel.
01:30:04.880 --> 01:30:16.319
Um, a restoration, if it goes well, a restoration, like the glorious revolution, not an uprising, but a but a restoration of something old that was in some ways better.
01:30:16.640 --> 01:30:26.479
Um and what I kind of hope we can do is actually significantly simplify the world for ourselves.
01:30:26.640 --> 01:30:35.680
There's a lot of dehumanizing, there's a dehumanizing aspect to even I love industrial society, you know, I love industrial capitalism.
01:30:35.920 --> 01:30:37.840
Um, I love the nation state.
01:30:38.000 --> 01:30:40.479
But there's a dehumanizing aspect to all of it.
01:30:40.640 --> 01:30:45.920
A lot of technical knowledge, a lot of forms and fields and blah, blah, blah, blah, blah.
01:30:46.159 --> 01:30:58.159
Um, and there is a certain extent to which I would I think we need to keep all of that because it's an important part of like how the world works, how we have all this wealth and all this nice stuff, air conditioning and whatnot.
01:30:58.319 --> 01:31:18.000
But like, I would like to take all of that complexity and cram it and make it conceptually smaller for me, cram it into a little box and let the AIs handle all that and probably much more complexity, even, you know, have it in structure it in such a way that I can trust it, that it is overseeable by humans, that there, that there is social trust in it.
01:31:18.079 --> 01:31:44.479
It's common knowledge that it can be trusted for the most part, um, such that we can focus on it, whether it's in our work or in our, which could be very, very different or in our personal lives, in our families, focus on the old questions that won't go away and that aren't that are not susceptible to intelligence per se.
01:31:45.279 --> 01:31:50.239
They're susceptible to things like virtue, and I hope that we can do better at cultivating virtue.
01:31:50.479 --> 01:31:52.399
That to me feels like the right path.
01:31:53.119 --> 01:31:53.760
Dean.
01:31:54.159 --> 01:32:06.399
Uh Zach, thank you so much for spending time with me, with our listeners, and for generating so many new tokens that we can throw into the mall of the intelligences we are building to see if that changes who they are.
01:32:06.720 --> 01:32:07.439
Thank you so much.
01:32:07.920 --> 01:32:08.479
Thank you.

جستجوی اپیزودهای گذشته

اپیزودهای قبلی Fault Lines را جستجو کن.
این نوار جستجو را در وب سایت خود قرار دهید
سلب مسئولیت: پادکست و آثار هنری تعبیه شده در این صفحه متعلق به Intercollegiate Studies Institute است که متعلق به صاحب آن است و به Listen Notes، Inc وابسته یا تایید نشده است.
PROMOTED
Listen411.com: Lightning-fast Podcast Transcription and Summarization. Transcribe 1-hour audio in under 1 minute. Pay-as-you-go.
Listen411.com: Lightning-fast Podcast Transcription and Summarization. Transcribe 1-hour audio in under 1 minute. Pay-as-you-go.
Audio to Text

پیشنهادها
از API برای دریافت این اپیزود استفاده کنید