1
00:00:00,070 -->
00:00:01,710
Here's a fun fact. This is crazy.
2
00:00:01,930 -->
00:00:04,140
So real GDP is not a quantity.
3
00:00:04,480 -->
00:00:10,840
As new goods get introduced,
they can cause the relationship between our
4
00:00:10,840 -->
00:00:14,560
wealth and our willingness to sacrifice
consumption for safety to go the other way,
5
00:00:15,290 -->
00:00:19,630
making us willing to sacrifice less
consumption for safety because the
6
00:00:19,630 -->
00:00:21,260
consumption tastes so damn good.
7
00:00:21,640 -->
00:00:26,130
This is sort of a pessimistic view,
but economic theory is primarily a
8
00:00:26,130 -->
00:00:27,330
destructive project.
9
00:00:27,740 -->
00:00:30,400
How do we detect the economic singularity?
10
00:00:31,640 -->
00:00:33,800
Okay, yeah, that's a big one.
11
00:00:34,660 -->
00:00:36,190
Hello, my name is Anson.
12
00:00:36,350 -->
00:00:40,640
I am a researcher at Epoch AI,
and today I have the pleasure of speaking
13
00:00:40,730 -->
00:00:44,610
with Phil Trammell. Phil is an economist at
Stanford University,
14
00:00:45,060 -->
00:00:51,720
and he has thought a lot about all sorts of
scenarios involving economic theory and AGI,
15
00:00:51,960 -->
00:00:55,920
and knows a lot about different kinds of
thought experiments relating to this topic.
16
00:00:56,150 -->
00:00:58,620
I thought it'd be really great to have you
on. Welcome, Phil.
17
00:00:59,200 -->
00:00:59,820
Thank you, Anson.
18
00:01:01,330 -->
00:01:03,040
Let's start with explosive growth.
19
00:01:04,580 -->
00:01:10,220
Most discussions about AI and explosive
growth so far have focused on one particular
20
00:01:10,220 -->
00:01:14,630
family of economic growth models,
which is the semi-endogenous family.
21
00:01:16,520 -->
00:01:19,890
What do you think about that? What are some
issues with how most people have been
22
00:01:19,890 -->
00:01:23,210
thinking about AI and explosive growth from
an economic perspective?
23
00:01:23,210 -->
00:01:26,710
I think the biggest issue is that not enough
people are thinking about it at all.
24
00:01:26,980 -->
00:01:30,350
There's been a lot of anchoring on a
particular semi-endogenous growth model,
25
00:01:30,350 -->
00:01:37,950
the Jones model. And the Jones model,
the constraint on growth,
26
00:01:38,220 -->
00:01:43,190
in particular on technology growth,
is just a lack of research inputs.
27
00:01:43,970 -->
00:01:46,570
You have researchers,
your R&D.
28
00:01:46,570 -->
00:01:50,460
They develop better technology and that that
speeds growth.
29
00:01:50,700 -->
00:01:55,860
The only reason why we don't have really,
really fast R&D already is because we only
30
00:01:55,860 -->
00:01:57,580
have so many people doing it.
31
00:01:57,580 -->
00:02:01,010
So if we automated it,
if we created lots of AIs,
32
00:02:01,010 -->
00:02:06,310
lots of robots doing R&D,
we would have really fast growth.
33
00:02:06,490 -->
00:02:08,750
One issue is that that might not be the whole
story,
34
00:02:08,780 -->
00:02:17,760
right? It could be that a so-called
Schumpeterian model is closer to correct in
35
00:02:17,760 -->
00:02:23,940
some contexts. This is a model in which the
main thing constraining growth is that every
36
00:02:23,940 -->
00:02:28,430
innovator needs a patent or a,
you know, a trade secret or something.
37
00:02:28,430 -->
00:02:35,130
They need a temporary monopoly on what
they're producing in order to justify the
38
00:02:35,130 -->
00:02:40,100
cost of developing the innovation.
39
00:02:40,880 -->
00:02:45,390
And so on those views,
growth has to proceed slowly enough.
40
00:02:47,200 -->
00:02:54,690
The sequence of new innovations has to come
sparsely enough for it to be worth doing any
41
00:02:54,690 -->
00:03:03,270
innovation at all. And yeah,
I mean, I think we see this potential worry
42
00:03:03,270 -->
00:03:10,100
in how the LLM market has unfolded where
because each one has so little moat.
43
00:03:11,020 -->
00:03:13,820
They keep pouring money into making better
and better models,
44
00:03:13,850 -->
00:03:20,840
but very quickly, you know,
someone makes an even better one and steals
45
00:03:20,930 -->
00:03:25,280
the lead and everyone quickly switches over
to the competitor.
46
00:03:26,600 -->
00:03:31,840
So there have been these worries about how
sustainable this is exactly. And in these
47
00:03:31,840 -->
00:03:38,760
models, R&D is, for simplicity,
typically presented as being just a function
48
00:03:38,760 -->
00:03:39,810
of capital expenditure.
49
00:03:40,720 -->
00:03:44,240
So if you ask, taking the model totally
literally,
50
00:03:44,690 -->
00:03:46,410
what happens if you automate R&D?
51
00:03:47,050 -->
00:03:50,500
Well, nothing because it's already all about
the lab equipment.
52
00:03:51,170 -->
00:03:52,280
That's going too extreme.
53
00:03:52,280 -->
00:03:56,180
Again, I think the Jones story is is closer
to right if you have to pick one.
54
00:03:57,560 -->
00:04:03,510
But there's been so much work anchored on the
Jones model, that I think probably taking a
55
00:04:03,510 -->
00:04:07,380
look at other approaches would be
informative.
56
00:04:07,930 -->
00:04:11,740
Given the Jones model,
or a model kind of in that class,
57
00:04:13,580 -->
00:04:16,170
I think we are sort of led astray by a few
things.
58
00:04:16,170 -->
00:04:19,050
But let's maybe say a bit about two in
particular.
59
00:04:19,050 -->
00:04:28,910
One is that in the Jones model,
there's not a hard limit on how
60
00:04:28,910 -->
00:04:31,330
much research can be parallelized at a given
time.
61
00:04:32,240 -->
00:04:37,390
So if you scale up the number of researchers
really fast,
62
00:04:38,640 -->
00:04:44,540
the rate of technological progress goes to
infinity with the research inputs.
63
00:04:46,410 -->
00:04:50,620
You know, maybe not one for one. Maybe the
rate of progress is like the square root of
64
00:04:50,620 -->
00:04:54,730
the inputs or something,
but still a sudden really arbitrarily big
65
00:04:55,370 -->
00:04:59,390
surge in the inputs means that you make
really fast technological progress.
66
00:05:01,450 -->
00:05:04,820
And I think that that just can't be right.
67
00:05:05,910 -->
00:05:11,100
If you think about an assembly line,
bringing in loads of engineers and management
68
00:05:11,100 -->
00:05:14,190
scientists or whatever to to speed up its
production.
69
00:05:17,450 -->
00:05:20,540
The more you stuff in,
the faster progress they'll make.
70
00:05:21,930 -->
00:05:24,820
But as the number of engineers goes to
infinity,
71
00:05:25,210 -->
00:05:28,770
the rate of progress that they can make in
five minutes,
72
00:05:29,510 -->
00:05:30,430
it just hits a ceiling.
73
00:05:31,590 -->
00:05:35,400
Right. So I guess like another example here
would be like, no matter how many GPT-3 that
74
00:05:35,440 -->
00:05:38,510
OpenAI is running, they can't just
arbitrarily push up the rate of algorithmic
75
00:05:38,510 -->
00:05:41,660
progress by running a trillion instances of
GPT-3.
76
00:05:41,960 -->
00:05:48,180
That's right. And importantly,
I mean, no LLM can single-handedly carry on
77
00:05:49,050 -->
00:05:50,600
AI research at all. Right.
78
00:05:51,120 -->
00:05:54,970
With GPT-3 alone, they can't really make any
progress,
79
00:05:54,970 -->
00:05:58,810
right? You need people and GPT-3,
or even just people.
80
00:05:59,840 -->
00:06:05,970
So I think it's actually like a little bit of
a different case, but it's another example of
81
00:06:05,970 -->
00:06:07,840
gross complementarity,
at least.
82
00:06:08,430 -->
00:06:12,300
Right. So it would be that even if we had
GPT-3 plus lots of humans,
83
00:06:12,460 -->
00:06:15,080
if we scaled that up arbitrarily,
we still wouldn't be able to just get
84
00:06:15,080 -->
00:06:16,340
arbitrary algorithmic progress.
85
00:06:16,340 -->
00:06:18,550
Yeah, yeah, that would be the tighter
analogy.
86
00:06:19,190 -->
00:06:24,500
I see. Okay. So should we just totally scrap
this Jones Law of motion then?
87
00:06:24,640 -->
00:06:28,600
Is this law of motion just fundamentally
going to be way too aggressive?
88
00:06:29,900 -->
00:06:35,000
No. So well, it's very simple.
89
00:06:35,000 -->
00:06:44,310
And it's, you know, for many purposes I think
it makes reasonable predictions and
90
00:06:45,850 -->
00:06:49,670
so we shouldn't always scrap it.
91
00:06:50,790 -->
00:06:56,590
And to the extent that it leads us astray,
it could lead us astray in either direction.
92
00:06:56,590 -->
00:07:01,640
And I think this is a common pattern i seem
to run into.
93
00:07:01,640 -->
00:07:06,170
When you think of a bottleneck that models
haven't been incorporating before,
94
00:07:06,170 -->
00:07:09,930
on the one hand, the lesson could be "Oh,
well,
95
00:07:09,930 -->
00:07:12,410
actually, growth won't be as explosive as we
thought,
96
00:07:12,410 -->
00:07:15,450
because once we relieve some other
bottleneck, we'll just slam into this new one
97
00:07:15,450 -->
00:07:17,640
we'd forgotten about". But on the other hand,
98
00:07:18,040 -->
00:07:23,430
to the extent that we've been constrained all
along by the bottleneck in question without
99
00:07:23,430 -->
00:07:30,910
realizing it then, you know,
if this is a new margin on which we could get
100
00:07:30,910 -->
00:07:37,100
more growth than we had been anticipating
because it could be relieved.
101
00:07:37,100 -->
00:07:43,270
It could be getting relieved more quickly
than before. So in this case,
102
00:07:43,370 -->
00:07:51,770
the reason why scaling up the number of
people and computers working on algorithmic
103
00:07:51,770 -->
00:07:58,390
progress, you know, just running GPT-3
wouldn't make progress go to infinity is the
104
00:07:58,710 -->
00:08:02,960
high latency that different human brains have
between each other,
105
00:08:03,220 -->
00:08:08,830
right? It's hard to efficiently divide a big
project among lots of different people who
106
00:08:08,830 -->
00:08:12,410
only have faint understandings of what,
you know,
107
00:08:14,510 -->
00:08:17,930
part they are playing in the big kind of,
you know,
108
00:08:17,930 -->
00:08:25,960
the big orchestration. And we've been getting
better over time at collaborating.
109
00:08:25,960 -->
00:08:30,600
And, you know, the internet allowed for
collaboration with people with complementary
110
00:08:30,600 -->
00:08:32,540
skills in very different parts of the world.
111
00:08:32,540 -->
00:08:35,510
And you know, whatever.
112
00:08:35,510 -->
00:08:39,040
I mean, obviously, if you go back far enough,
you know, the invention of language and
113
00:08:39,560 -->
00:08:44,100
writing and everything obviously lowered
latency in some sense.
114
00:08:46,220 -->
00:08:48,130
And that's been furthering growth all along.
115
00:08:50,120 -->
00:08:56,510
But the rate at which that bottleneck is
being relieved could rise pretty radically,
116
00:08:56,510 -->
00:09:05,030
I would think, in the event of you know,
neural nets,
117
00:09:05,050 -->
00:09:10,150
kind of virtual brains that are themselves
getting bigger and bigger.
118
00:09:10,150 -->
00:09:12,650
So instead of having one,
two, three, four employees,
119
00:09:12,650 -->
00:09:17,650
you have, you know, fully integrated minds
that that are,
120
00:09:18,230 -->
00:09:22,630
you know, two, three,
four times bigger. So,
121
00:09:22,900 -->
00:09:29,650
so assuming a sort of constant rate of
penalty for parallelization,
122
00:09:29,650 -->
00:09:35,670
you know, a constant sort of exponent on the
research inputs in the Jones model would be
123
00:09:35,670 -->
00:09:38,760
leading us astray in the sort of pessimistic
direction.
124
00:09:39,510 -->
00:09:43,010
If you want to think through the problem
carefully, we probably shouldn't be using the
125
00:09:43,010 -->
00:09:45,930
Jones model because it doesn't even work well
as a bound,
126
00:09:45,930 -->
00:09:49,860
right? It could be leading us astray in one
direction or the other. But if you want
127
00:09:49,860 -->
00:09:55,500
simplicity and if you're not going to be
extrapolating kind of too far out of sample,
128
00:09:57,910 -->
00:10:05,320
then it's still, you know,
kind of simple and insightful.
129
00:10:07,130 -->
00:10:10,560
Yeah. I guess the tricky thing for us is that
we're thinking about AI, and that almost
130
00:10:10,840 -->
00:10:15,860
necessarily entails us having to extrapolate
a lot of things pretty far out of bounds of
131
00:10:16,200 -->
00:10:21,660
the regimes that are being tested. Yeah. So
one question here that is also pertinent to
132
00:10:21,660 -->
00:10:26,730
models like Tom Davidson's full takeoff model
and the GATE model is that we might get
133
00:10:27,310 -->
00:10:31,310
declining returns to scale over time as we
make further and further technological
134
00:10:31,310 -->
00:10:35,720
progress. And I know that you've been
thinking a little bit about this.
135
00:10:35,720 -->
00:10:38,550
So how fast has this decline been?
136
00:10:38,550 -->
00:10:40,200
What do we know about this question?
137
00:10:40,800 -->
00:10:44,920
I think you're referring to this point that,
you know,
138
00:10:45,680 -->
00:10:47,100
ideas get harder to find.
139
00:10:47,330 -->
00:10:52,540
And the rate at which ideas are getting
harder to find might itself change over time.
140
00:10:53,180 -->
00:10:56,140
Right. So that's something else that the
Jones model doesn't feature.
141
00:10:56,140 -->
00:11:01,650
And I think that's a much smaller weakness
than this parallelizability point.
142
00:11:02,830 -->
00:11:09,210
But it's still a strong assumption and I
think probably an incorrect assumption.
143
00:11:10,470 -->
00:11:16,530
Yeah. Something I used to think was that in
the grand scheme of things,
144
00:11:17,000 -->
00:11:22,840
the rate at which ideas were getting harder
to find was itself growing kind of
145
00:11:24,610 -->
00:11:29,870
non-negligibly, and that you could see that
by comparing the,
146
00:11:32,590 -->
00:11:37,890
you know, the function from growth and
research inputs to growth in productivity
147
00:11:37,890 -->
00:11:40,500
outputs in the distant past to in the more
recent past.
148
00:11:41,550 -->
00:11:47,960
So there's this famous Cramer paper from 1993
that if you look over like the million years
149
00:11:47,960 -->
00:11:54,920
of the Malthusian past,
he estimates that basically ideas aren't
150
00:11:54,920 -->
00:11:59,630
getting harder to find at all. You know,
that every time you double the population,
151
00:12:00,190 -->
00:12:04,570
you double the rate at which they are coming
up with proportional productivity
152
00:12:04,570 -->
00:12:08,740
improvements. The data is really sketchy,
obviously,
153
00:12:09,500 -->
00:12:17,860
but I think even after accounting for that,
it seems pretty robust that this idea is
154
00:12:17,860 -->
00:12:23,240
getting harder to find effect does not appear
very strongly in the distant past,
155
00:12:24,010 -->
00:12:28,790
if anything like Cramer's data are to be
accepted.
156
00:12:30,820 -->
00:12:36,970
And by contrast, this well-known paper,
"Are ideas getting harder to find?" from 2020
157
00:12:37,350 -->
00:12:43,390
finds that over the last century you have to
be growing the research inputs kind of three
158
00:12:43,390 -->
00:12:48,330
times faster than the,
you know, your target rate of growth in
159
00:12:49,200 -->
00:12:54,050
productivity. So I thought "Okay,
well, so that's a big change".
160
00:12:54,050 -->
00:12:58,240
And if you project it forward,
then the rate at which ideas get harder to
161
00:12:58,240 -->
00:13:03,030
find will keep on rising until eventually we
just hit hit the ceiling of technological
162
00:13:03,030 -->
00:13:04,530
maturity or before that.
163
00:13:04,530 -->
00:13:10,000
Right. If there is some technology levels
that are feasible,
164
00:13:10,000 -->
00:13:13,510
but that we kind of just,
we never really get to because we slow down
165
00:13:13,510 -->
00:13:15,800
before we were anywhere close,
you know?
166
00:13:18,500 -->
00:13:19,690
I'm thinking about it more.
167
00:13:20,250 -->
00:13:22,680
I think that's probably not right.
168
00:13:23,450 -->
00:13:33,070
And the big difference between that analysis
of the distant
169
00:13:33,070 -->
00:13:36,860
past and the sort of analysis in the "Are
ideas getting harder to find"-
170
00:13:36,860 -->
00:13:44,880
paper, is that in the past,
they're looking at how quickly we can create
171
00:13:45,160 -->
00:13:48,300
more copies of the same object,
namely the human,
172
00:13:48,440 -->
00:13:55,370
the human body. And as I was saying before,
modern growth is mainly not about that,
173
00:13:55,710 -->
00:13:59,670
but about creating new kinds of goods and
higher quality goods.
174
00:14:01,350 -->
00:14:09,370
Another difference, of course,
is that modern growth is often driven by
175
00:14:09,650 -->
00:14:11,850
profit-seeking R&D, right.
176
00:14:13,070 -->
00:14:22,830
Which can be constrained by that creative
destruction issue I was getting at before.
177
00:14:23,500 -->
00:14:29,960
So the issue here would basically be if a few
hundred years ago,
178
00:14:29,960 -->
00:14:34,260
we come up with the idea of patents and we
have a sort of entrepreneurial culture take
179
00:14:34,260 -->
00:14:38,850
hold where people are trying to invent new
products,
180
00:14:38,850 -->
00:14:41,950
new production processes and everything and
capitalize on that.
181
00:14:42,510 -->
00:14:45,870
That's going to really boost growth at the
time,
182
00:14:45,870 -->
00:14:48,800
because it's something people could have been
doing all along that they weren't, and now
183
00:14:48,800 -->
00:14:55,500
they're doing it. But that process is just
capped by something different from the kind
184
00:14:55,500 -->
00:14:59,750
of old Malthusian, purely
resource-constrained thing driving growth,
185
00:14:59,750 -->
00:15:04,220
right. So you get this burst of extra growth
that starts picking up and the industrial
186
00:15:04,220 -->
00:15:10,250
revolution and so on. And that adds a lot.
187
00:15:11,740 -->
00:15:17,910
But if it itself starts slowing down for
whatever reason,
188
00:15:17,910 -->
00:15:20,720
that's just like unrelated to the rate at
which ideas are getting harder to find or
189
00:15:20,720 -->
00:15:27,530
whatever. Right? Then in a model which only
has room for ideas getting harder to find,
190
00:15:28,030 -->
00:15:30,390
you'll pick that up as ideas getting harder
to find.
191
00:15:31,360 -->
00:15:39,110
Okay, so there was a paper,
actually, by James Bessen and a co-author I'm
192
00:15:39,110 -->
00:15:42,870
forgetting just recently,
which makes this claim,
193
00:15:43,130 -->
00:15:46,610
actually, that ideas haven't even been
getting harder to find.
194
00:15:51,340 -->
00:15:54,150
Over the last century,
sort of properly understood,
195
00:15:55,740 -->
00:16:05,630
but that R&D has been contributing ever less
per per unit of
196
00:16:05,910 -->
00:16:09,410
of R&D, R&D expenditures or R&D effort.
197
00:16:10,200 -->
00:16:15,360
It's been contributing ever less to
productivity growth. Because for whatever
198
00:16:15,360 -->
00:16:22,280
reason, it's sort of unfolded in a direction
where new R&D efforts have ever more so been
199
00:16:22,280 -->
00:16:29,130
cannibalizing past ones which both
discourages investment to begin with.
200
00:16:29,130 -->
00:16:36,340
And of course, it itself means that you're
adding less to the you know.
201
00:16:39,620 -->
00:16:43,900
Yeah. If the big new thing is that you come
up with Amazon or something,
202
00:16:44,640 -->
00:16:46,980
but that makes you a multi-billionaire.
203
00:16:46,980 -->
00:16:49,900
And so it warrants like billions of dollars
in investments and how to get all the
204
00:16:49,900 -->
00:16:53,390
logistics right. But actually everyone
without you,
205
00:16:53,390 -->
00:16:57,000
everyone would just be shopping at Walmart
and the consumer surplus and everything would
206
00:16:57,000 -->
00:17:01,930
just be like a penny less,
then you're not actually increasing output
207
00:17:01,950 -->
00:17:05,200
very much. You're just sort of like
transferring it by winning the race a second
208
00:17:05,200 -->
00:17:08,100
faster. But both parties still run almost the
entire race.
209
00:17:09,000 -->
00:17:11,300
And that's been happening all along.
210
00:17:11,460 -->
00:17:13,340
But that kind of thing,
according to Besson,
211
00:17:14,780 -->
00:17:19,660
is happening more now. And so if you're just
looking at a sort of time series of R&D
212
00:17:19,660 -->
00:17:22,270
inputs to productivity growth over time,
you'll think "Oh,
213
00:17:22,270 -->
00:17:25,720
ideas are getting harder to find",
but that's not really what's going on. Now,
214
00:17:25,720 -->
00:17:30,710
I don't know if that's entirely right,
okay. But it's another example of how I think
215
00:17:30,710 -->
00:17:35,130
you're really comparing apples and oranges
when you look at modern growth,
216
00:17:35,180 -->
00:17:40,360
where it's all about ,
you know, kind of on the intensive margin.
217
00:17:40,770 -->
00:17:46,030
More and different kinds of things per person
than the old Malthusian kind of regime.
218
00:17:48,330 -->
00:17:54,290
And so I don't think you can infer that the
rate at which ideas are getting harder to
219
00:17:54,290 -->
00:17:55,180
find is itself rising.
220
00:17:56,230 -->
00:18:01,570
Right. So I guess the takeaway is we would
like to be able to answer this question,
221
00:18:01,570 -->
00:18:05,630
but then the data is not really comparable
between different regimes,
222
00:18:05,630 -->
00:18:08,220
both on the R&D inputs and the R&D output
side.
223
00:18:08,220 -->
00:18:12,830
There is also this sort of omitted variable
bias kind of issue that's going on,
224
00:18:12,830 -->
00:18:16,100
that kind of makes it not directly
comparable. Well,
225
00:18:16,100 -->
00:18:21,420
okay, that's kind of unfortunate because we
would like to know what these bottlenecks are
226
00:18:21,420 -->
00:18:23,830
for, say, hardware efficiency and software
efficiency.
227
00:18:24,450 -->
00:18:29,490
I guess it's just too hard to say. Okay
changing tacks a little bit.
228
00:18:30,050 -->
00:18:39,200
You've written a paper about how new products
and new varieties of goods is an important
229
00:18:39,200 -->
00:18:42,720
consideration when we're thinking about the
real impacts of economic growth.
230
00:18:44,480 -->
00:18:48,430
What exactly is missing from the current way
that we think about economic growth?
231
00:18:49,740 -->
00:18:58,940
Yeah, the problem is that we try to flatten
growth in the goods and services we
232
00:18:59,340 -->
00:19:06,810
enjoy into a single variable,
and put a number to how much that variable is
233
00:19:06,810 -->
00:19:13,500
growing. I'm not making a standard critique
of GDP here.
234
00:19:15,270 -->
00:19:18,780
I mean, it's somewhat standard,
but I'm not making the particular standard
235
00:19:18,780 -->
00:19:21,270
critique that it leaves out all kinds of
things,
236
00:19:21,270 -->
00:19:27,660
sources of value other than consumption,
like leisure or friendship or whatever.
237
00:19:28,610 -->
00:19:31,420
That's true. But what I'm saying is that even
when it just comes,
238
00:19:31,460 -->
00:19:34,060
even when it comes to consumption,
you know, the part of all that matters,
239
00:19:34,060 -->
00:19:38,480
that's just the knickknacks that we consume
in a very concrete way and enjoy.
240
00:19:41,420 -->
00:19:43,910
You just can't flatten things into this
single dimension.
241
00:19:45,440 -->
00:19:49,010
I think a really simple way to see that is to
just say,
242
00:19:49,010 -->
00:19:52,480
okay, let's say we only had one consumption
good:
243
00:19:53,430 -->
00:19:57,620
horse. Okay, so we're in the Golden Horde,
something.
244
00:19:57,620 -->
00:20:02,390
One of the empires that followed Genghis
Khan's when it when it split up.
245
00:20:03,270 -->
00:20:05,960
And, you know, they got a lot of use out of
their horses.
246
00:20:05,960 -->
00:20:08,740
They could ride them,
but they could also, you know,
247
00:20:08,740 -->
00:20:11,580
use their skin and their milk and everything.
248
00:20:12,360 -->
00:20:19,340
And let's say they could only consume horses.
The quality of the consumption basket that
249
00:20:19,340 -->
00:20:23,470
they got to enjoy. It rises as the number of
horses rises,
250
00:20:23,470 -->
00:20:27,740
like you'd rather have five horses than one,
but it only rises toward a ceiling,
251
00:20:27,740 -->
00:20:34,910
and that ceiling is below what a typical
modern middl-classs American gets to enjoy.
252
00:20:36,700 -->
00:20:44,260
And so whatever number you put to consumption
or GDP in the horse economy.
253
00:20:46,150 -->
00:20:52,450
If your index is going to be homogeneous of
degree one,
254
00:20:52,490 -->
00:20:56,350
if it's going to have the natural feature
that if you double all the stuff in it,
255
00:20:56,350 -->
00:20:59,700
then you double the number you assign to
consumption. So if you've got five horses,
256
00:21:00,150 -->
00:21:03,950
and then you go from that to ten horses,
then you've doubled consumption,
257
00:21:04,100 -->
00:21:08,640
right? Whatever number you put on how much
consumption five horses count for,
258
00:21:09,360 -->
00:21:13,530
as horses go to infinity,
consumption is going to rise to infinity.
259
00:21:13,530 -->
00:21:18,760
But the quality of the consumption basket is
still going to be below what's presumably the
260
00:21:18,760 -->
00:21:22,480
finite consumption basket that,
you know, the middle-class American gets.
261
00:21:23,650 -->
00:21:27,270
And the issue is, no matter how many loaves
of bread that I have in 1800.
262
00:21:27,790 -->
00:21:33,320
It doesn't matter because I would swap no
number of loaves of bread for my modern day
263
00:21:33,320 -->
00:21:33,800
laptop.
264
00:21:34,450 -->
00:21:38,130
Yeah, I mean, at least it's totally possible
to have that preference.
265
00:21:38,130 -->
00:21:40,300
Yeah, I think that's a common preference.
Yeah.
266
00:21:42,040 -->
00:21:42,190
Yeah.
267
00:21:42,190 -->
00:21:46,240
There was this previous paper that I think
you were involved in together with Leopold
268
00:21:46,240 -->
00:21:51,140
Aschenbrenner on whether or not we could try
to speed through the time of perils.
269
00:21:52,300 -->
00:21:56,420
So, relating existential risk and growth.
What does this imply for that?
270
00:21:57,070 -->
00:22:02,180
Yeah, this paper and some recent papers by
Chad Jones,
271
00:22:02,610 -->
00:22:06,920
they all are centered around this calculation
of how much consumption we might be willing
272
00:22:06,920 -->
00:22:11,070
to sacrifice for safety over time as we get
richer.
273
00:22:12,510 -->
00:22:18,930
And the assumption is that as as we get
richer,
274
00:22:19,890 -->
00:22:23,900
our marginal utility in consumption falls
because,
275
00:22:23,900 -->
00:22:31,820
you know, we've got these concave utility
functions. And we have more to lose if we die
276
00:22:32,690 -->
00:22:36,410
because we're better,
we're doing better and better. And so for
277
00:22:36,410 -->
00:22:42,320
both reasons, we'll be willing to sacrifice
more consumption for safety over time.
278
00:22:43,950 -->
00:22:48,690
And if that effect is strong enough,
it could mean that we're willing to tolerate
279
00:22:50,140 -->
00:22:54,680
really rapid AI development,
which could get out of hand and be disruptive
280
00:22:54,680 -->
00:22:56,470
or even existentially catastrophic.
281
00:22:57,800 -->
00:23:01,340
Right now we're willing to take that risk,
but in the future,
282
00:23:01,340 -->
00:23:05,200
we won't be, at least if we survive to a
future where we're rich enough to kind of
283
00:23:05,200 -->
00:23:14,350
want to make the tradeoff the other way. I
think that argument probably has a fair bit
284
00:23:14,350 -->
00:23:19,920
of truth to it. It's not just a theoretical
point that as we get richer,
285
00:23:20,070 -->
00:23:22,910
we get willing to sacrifice a lot of
consumption for safety.
286
00:23:24,620 -->
00:23:28,310
We see it in medicine being a luxury good.
287
00:23:28,980 -->
00:23:32,700
As people get richer and as societies get
richer,
288
00:23:32,860 -->
00:23:36,320
we spend not just more on medicine,
but a larger fraction of our incomes on
289
00:23:36,320 -->
00:23:41,530
medicine. On the other hand,
these trends can reverse.
290
00:23:41,610 -->
00:23:49,470
So, yoknow,ow health care spending is a
fraction of GDP actually has fallen a bit in
291
00:23:49,470 -->
00:23:50,960
the US in recent years.
292
00:23:50,960 -->
00:23:53,790
And in other advanced countries and,
you know,
293
00:23:56,290 -->
00:24:01,070
it's always very easy to predict a trend
correctly right before it turns around for
294
00:24:01,140 -->
00:24:07,500
one reason or another. But the deeper issue
with putting too much stock in this argument
295
00:24:08,390 -->
00:24:11,690
is that, as we were just saying,
in principle,
296
00:24:14,340 -->
00:24:22,870
as new new goods get introduced they can
cause the relationship between our wealth
297
00:24:22,950 -->
00:24:27,400
broadly construed and our willingness to
sacrifice consumption for safety to go the
298
00:24:27,400 -->
00:24:36,500
other way. New products can kind of start
making us willing to sacrifice less
299
00:24:36,500 -->
00:24:40,930
consumption for safety because the
consumption tastes so damn good,
300
00:24:41,880 -->
00:24:48,590
right. Now again, on balance historically,
I think this effect has tended not to win
301
00:24:48,590 -->
00:24:56,700
out. But if AI nudges the direction of
technological development so that,
302
00:24:58,130 -->
00:25:01,210
yeah, we're getting more copies of the old
things.
303
00:25:01,210 -->
00:25:07,150
And so we're kind of satiating in those,
but we're inventing these new things that are
304
00:25:07,150 -->
00:25:11,330
really desirable, you know,
because, they're like immortality pills or
305
00:25:11,330 -->
00:25:19,510
they're wires to the head that just give you
mind-blowing joy or something.
306
00:25:21,650 -->
00:25:27,600
Then, you know, it's totally consistent with
a reasonable-looking utility function to
307
00:25:27,600 -->
00:25:32,320
just, you know, be like the rat pushing the
heroin button or something and getting more
308
00:25:32,320 -->
00:25:37,290
expected utility out of a path that involves
more short-term pleasure,
309
00:25:37,290 -->
00:25:44,720
but a lot of it, and a higher chance of
disaster than than the safer path you know.
310
00:25:44,830 -->
00:25:49,050
At least for a non-negligible discount rate.
311
00:25:49,470 -->
00:25:50,630
I hope that doesn't happen.
312
00:25:50,630 -->
00:25:56,860
You know, I hope that growth is more in this
kind of satiating,
313
00:25:56,980 -->
00:26:02,890
safety-promoting direction in the near
future.
314
00:26:04,520 -->
00:26:08,520
Because I care about the distant future and
getting to it.
315
00:26:10,960 -->
00:26:15,290
But it's at least a logical possibility that
it goes the other way. And I think,
316
00:26:15,290 -->
00:26:18,740
you know, I mean, this is sort of related to
some of the stuff we were talking about
317
00:26:18,820 -->
00:26:22,500
before. But yeah, I think we can be led
astray,
318
00:26:22,500 -->
00:26:29,200
and I certainly have been led astray by,
by kind of doubling down too hard on a simple
319
00:26:29,310 -->
00:26:34,280
one-dimensional model of things in which
basically all the automation or AI can do is
320
00:26:34,760 -->
00:26:39,000
speed us up or slow us down on some
trajectory that's already set in stone.
321
00:26:41,400 -->
00:26:44,900
There's lots of different dimensions of
technological development and paths we could
322
00:26:44,900 -->
00:26:51,340
go down. And by speeding up one a bit more
than the others relative to how fast they've
323
00:26:51,340 -->
00:26:56,270
been proceeding relative to each other in the
past, a lot of our intuitions might break.
324
00:26:57,370 -->
00:26:58,670
Moving on to the GATE model.
325
00:27:00,820 -->
00:27:05,000
A core part of this model is the amount of
production that you get and it's is
326
00:27:05,000 -->
00:27:08,350
determined strongly by the fraction of tasks
that are automated.
327
00:27:09,300 -->
00:27:13,600
So there's this question,
though, about whether or not this framing
328
00:27:13,620 -->
00:27:17,070
even makes sense. So I'm curious what you
think about that. Does the fraction of tasks
329
00:27:17,350 -->
00:27:22,500
as the framing for these growth models hold
much water or doesn't it really make that
330
00:27:22,500 -->
00:27:22,990
much sense.
331
00:27:24,080 -->
00:27:31,200
Yeah. I think qualitatively they can be
pretty insightful.
332
00:27:31,200 -->
00:27:40,550
And I don't want to say the whole approach is
worthless or anything like that,
333
00:27:41,490 -->
00:27:44,980
but certainly I don't currently think they're
as useful as I used to think.
334
00:27:47,990 -->
00:27:54,820
For starters, I just think it's sort of
telling the very first task based model in a
335
00:27:54,820 -->
00:27:59,190
form anything like we know them today was
Zeria's in 1998,
336
00:28:00,280 -->
00:28:10,220
and it came out right when the O*NET task
database of US occupations was
337
00:28:10,220 -->
00:28:14,770
first put out, I think in 1997 by the Bureau
of Labor Statistics.
338
00:28:16,540 -->
00:28:20,510
So it's like, well, people collected all this
data thinking "Might as well,
339
00:28:20,510 -->
00:28:26,490
it could be useful for something." And people
come up with a model that makes use of this
340
00:28:26,490 -->
00:28:31,050
data. If we collected other data,
we probably would have come up with some
341
00:28:31,050 -->
00:28:39,940
other model. I don't think there's a deep
fact that
342
00:28:42,010 -->
00:28:47,890
work subdivides into these nuggets that we
call tasks and that we can sort of think
343
00:28:47,890 -->
00:28:51,220
about plowing through one by one and seeing
what happens to growth.
344
00:28:52,710 -->
00:29:02,290
A simple example of how that can go wrong was
explained to me by Pamela
345
00:29:02,290 -->
00:29:09,450
Mishkin at OpenAI. Who was one of the
co-authors on this paper that makes heavy use
346
00:29:09,450 -->
00:29:11,760
of a task based model:
"GPTs are GPTs".
347
00:29:14,510 -->
00:29:21,190
Daron Acemoglu wrote a paper kind of making
use of their estimates of automatability by
348
00:29:21,190 -->
00:29:28,590
LLMs, by task. And yeah,
it just seems this really rich sort of
349
00:29:28,590 -->
00:29:32,470
resource. And so I and some other people
where I'm working at Stanford,
350
00:29:32,470 -->
00:29:38,930
at least we were and I think we are still,
trying to follow up on that.
351
00:29:40,300 -->
00:29:45,170
And I talked to Pamela and she was like "Oh,
this whole framework doesn't really make much
352
00:29:45,170 -->
00:29:54,860
sense anyway". "What do you mean?" And she
sent me an email explaining it,
353
00:29:54,860 -->
00:29:59,870
and I didn't get it. So then we had a Zoom
call and this was her example.
354
00:30:01,350 -->
00:30:03,400
Let's say I've got two tasks.
355
00:30:04,720 -->
00:30:08,130
Write an email to this moron Phil.
356
00:30:08,440 -->
00:30:13,850
She didn't put it that way. To write an email
to him explaining the limitations of the
357
00:30:13,850 -->
00:30:23,140
task-based model. Task B is have a Zoom call
with Phil going through the limitations of
358
00:30:23,140 -->
00:30:24,740
the task-based model in more detail.
359
00:30:25,650 -->
00:30:30,230
If all she had to do was task A,
maybe an LLM could have done it better and
360
00:30:30,230 -->
00:30:38,060
faster. But given that task B is coming,
she felt and it certainly it could be the
361
00:30:38,060 -->
00:30:44,650
case, that if the LLM can't do task B,
have the Zoom call,
362
00:30:46,020 -->
00:30:52,920
it would take her more time to read and
internalize and remember the particular
363
00:30:52,920 -->
00:30:56,190
choices of terminology that the LLM would
produce in the email,
364
00:30:56,190 -->
00:30:59,440
so that we can discuss it on common terms
over the Zoom call.
365
00:31:00,130 -->
00:31:03,860
Right. Instead of having to keep going back
and saying, "Wait, what's Phil getting at?
366
00:31:03,860 -->
00:31:10,540
Oh yeah, the LLM used this term for
something." That would take more time.
367
00:31:10,540 -->
00:31:15,720
So given that task B is coming up,
the most efficient workflow is for her to be
368
00:31:15,720 -->
00:31:17,990
the one that writes the email,
right?
369
00:31:18,670 -->
00:31:27,860
So like an eval or a just a human-like
evaluation of a given
370
00:31:28,190 -->
00:31:35,130
task on whether an LLM can do the task would
predict that if someone's job consists of
371
00:31:35,130 -->
00:31:41,920
task A and task B and then we introduce an
LLM that you found could do task A,
372
00:31:42,910 -->
00:31:45,610
their productivity would double.
373
00:31:45,610 -->
00:31:49,490
Right. They could just do task B all day,
they'd get twice as much of it done. Maybe
374
00:31:49,490 -->
00:31:52,300
half of them would be fired. I mean it
depends on the effect or whatever. But
375
00:31:53,200 -->
00:31:55,400
actually it would have no effect at all in
this case.
376
00:31:55,400 -->
00:31:59,990
Right? Everyone would just keep on doing task
A.
377
00:32:02,350 -->
00:32:05,940
So yeah. So if these sorts of effects are
significant enough,
378
00:32:07,040 -->
00:32:16,700
I think that means that our projections from
task-based models could be wrong in at least
379
00:32:16,700 -->
00:32:22,130
three big ways. One is that the growth
impacts of automating technologies,
380
00:32:22,130 -->
00:32:31,980
including LLMs or AI in general,
could be delayed relative to what
381
00:32:33,340 -->
00:32:37,790
you naively would have thought. Because it's
got to automate A and B before you see any of
382
00:32:37,790 -->
00:32:41,560
the effects, right? Second bias is that when
they come,
383
00:32:41,560 -->
00:32:46,150
they'll be more sudden than you would have
thought. And the third,
384
00:32:46,790 -->
00:32:50,120
more sort osubtly,ly gets back to this
latency thing.
385
00:32:51,000 -->
00:32:54,440
This is what I'm thinking about all the time
now, so I have to work it into everything.
386
00:32:55,170 -->
00:33:01,500
The message of Pamela's example is that
there's a kind of increasing returns to scale
387
00:33:01,500 -->
00:33:08,000
and that it's more productive to have the
same mind doing tasks A and B.
388
00:33:09,000 -->
00:33:13,980
And that might outweigh the fact that if you
tried to outsource task A in isolation,
389
00:33:14,130 -->
00:33:21,360
this alternative system could way more
efficiently or quickly or something,
390
00:33:21,910 -->
00:33:25,870
perform task A. Despite that,
it's still, it's so beneficial.
391
00:33:27,520 -->
00:33:33,250
There's a kind of a positive spillover,
where doing task A makes the factor of
392
00:33:33,250 -->
00:33:37,300
production, in this case her body and brain,
more productive at task B,
393
00:33:37,300 -->
00:33:42,830
and that positive spillover is so big that it
totally displaces the use of the LLM at task
394
00:33:43,640 -->
00:33:50,950
A. Well, we can currently take advantage of
these sort of positive spillovers or these
395
00:33:52,700 -->
00:33:57,810
cases of learning by doing from concrete task
to concrete task.
396
00:33:58,210 -->
00:34:05,030
We can take advantage of them up to the size
of a given human's capacity for work.
397
00:34:07,750 -->
00:34:10,880
Some kinds of work maybe can be subdivided.
398
00:34:10,880 -->
00:34:14,290
Is there something you can just take out of
your day and sort of ask someone else to do?
399
00:34:18,470 -->
00:34:25,740
But not that much. And so what that's telling
us is that we have these spillovers where
400
00:34:25,740 -->
00:34:31,330
it's really helpful to have the same mind
doing all the different things. And,
401
00:34:34,190 -->
00:34:37,930
yeah, we can take advantage of the increasing
returns that this allows for, but only up to
402
00:34:37,930 -->
00:34:43,810
the size of what a single person can do in a
day or in a year or in a life.
403
00:34:47,590 -->
00:34:54,490
And so with just sort of bigger systems with
really big neural nets that can do a lot of
404
00:34:54,490 -->
00:34:59,570
things simultaneously and have really big
memories, that can use the learning they got
405
00:34:59,570 -->
00:35:06,550
from writing that email to Joe six years ago
on the other side of the world to inform the
406
00:35:06,550 -->
00:35:10,390
Zoom call that they have with Phil about
task-based models tomorrow.
407
00:35:13,440 -->
00:35:16,080
Yeah. You would expect to see even more
growth than you would predict from a naive
408
00:35:16,080 -->
00:35:19,490
task-based model where the best you can do is
just automate all the tasks one by one, and
409
00:35:19,490 -->
00:35:26,520
then kind of get a world in which sort of
everyone was still the same size.
410
00:35:27,150 -->
00:35:31,300
Yeah. Is this not a question,
though, of whether or not the tasks and
411
00:35:31,350 -->
00:35:34,030
owners happen to be at the right level of
abstraction?
412
00:35:35,390 -->
00:35:40,670
Couldn't I, say, make an argument about how,
for example,
413
00:35:40,670 -->
00:35:45,720
if we look at which occupations existed in
the past before the Industrial Revolution,
414
00:35:45,720 -->
00:35:48,480
and we compare that to the tasks of today,
indeed,
415
00:35:48,480 -->
00:35:52,590
like maybe a large fraction of those jobs,
using the precursor to O*NET,
416
00:35:52,590 -->
00:35:56,810
that wouldn't have been so bad that it's just
totally useless?
417
00:35:58,140 -->
00:36:03,620
I guess you're not saying that this is
totally useless, but then is this not just a
418
00:36:03,620 -->
00:36:05,650
question of O*NET being at the wrong level of
abstraction?
419
00:36:06,020 -->
00:36:08,270
I see what you're saying. So a few things.
420
00:36:08,360 -->
00:36:14,260
One is if we're talking about GATE in
particular, one thing it can do in kind of a
421
00:36:14,260 -->
00:36:19,630
best-case scenario is to be improved on over
time as data comes in.
422
00:36:20,590 -->
00:36:28,250
And so we could see like where it looks like
this is the function from model size to
423
00:36:29,040 -->
00:36:31,090
fraction of tasks automatable or whatever.
424
00:36:31,390 -->
00:36:34,990
Right. And if we're looking at the occupation
level and there's only like three,
425
00:36:35,150 -->
00:36:38,320
then it's just not going to be useful like
that.
426
00:36:39,260 -->
00:36:44,000
And you're going to have to turn to other
ways of indexing our progress,
427
00:36:45,970 -->
00:36:51,000
like SWE-bench or the task time length
growing or whatever.
428
00:36:51,000 -->
00:36:55,420
But you're not going to be able to use use
O*NET because the right level of granularity
429
00:36:55,550 -->
00:36:57,520
is just unreasonably large.
430
00:36:58,900 -->
00:37:03,370
On the other hand, I mean I don't think
there's one objective right level of
431
00:37:03,410 -->
00:37:08,480
granularity. Because for some purposes,
like with A and B,
432
00:37:08,540 -->
00:37:15,900
it could be that some people have questions
that need follow-up emails and that don't
433
00:37:15,900 -->
00:37:17,350
require follow-up Zoom calls.
434
00:37:18,380 -->
00:37:23,600
And you want to be able to capture the fact
that Pamela is going to be able to just ask
435
00:37:23,600 -->
00:37:25,560
the LLM to handle those.
436
00:37:28,940 -->
00:37:32,460
So I don't know, my lesson is more like
you've got to be careful when thinking about
437
00:37:32,460 -->
00:37:38,530
your application of a task-based model rather
than that there is a right task-based model,
438
00:37:38,530 -->
00:37:40,710
but it's just at a coarser level of
granularity or something.
439
00:37:41,860 -->
00:37:45,410
I think that's very much the kind of thing
that I was trying to get at.
440
00:37:45,520 -->
00:37:51,340
Which is like, if I'm trying to do this
coding task,
441
00:37:52,110 -->
00:37:59,590
maybe the description at O*NET-level would
look more like doing coding,
442
00:38:00,880 -->
00:38:02,110
doing data visualization.
443
00:38:02,780 -->
00:38:07,650
But then really the thing that we actually
care about is more like it's able to write
444
00:38:07,690 -->
00:38:11,520
this particular block of code in this Python
notebook,
445
00:38:11,690 -->
00:38:17,040
or it's able to write this Python script. And
what I meant by level of abstraction was
446
00:38:17,540 -->
00:38:21,290
maybe we need to actually step down to a
different thing that's more amenable to
447
00:38:21,290 -->
00:38:27,070
capturing that kind of automation. So it's
less of a problem of the framing of a
448
00:38:27,070 -->
00:38:29,790
fraction of tasks, but maybe it's more like,
you know,
449
00:38:29,970 -->
00:38:32,680
the particular data that we have,
the things that we're trying to use to
450
00:38:32,730 -->
00:38:35,290
measure these things,
are quickly becoming out of date. Maybe in
451
00:38:35,290 -->
00:38:39,000
the same way that like the precursor to O*NET
was out of date and they had to come up with
452
00:38:39,000 -->
00:38:39,530
a new thing.
453
00:38:40,610 -->
00:38:42,970
When you say "it quickly becoming out of
date",
454
00:38:42,970 -->
00:38:46,580
do you mean just because the nature of work
is changing fast,
455
00:38:46,580 -->
00:38:49,990
or do you mean that it's becoming out of date
because day to day,
456
00:38:50,570 -->
00:38:53,300
the list of tasks that you're doing is
different?
457
00:38:53,370 -->
00:38:56,340
Because you know, "code this particular
thing" is different from "code that
458
00:38:56,400 -->
00:38:58,050
particular thing".
459
00:38:58,130 -->
00:39:02,070
I think it's becoming out of date in the
sense that we are getting a better sense of
460
00:39:02,800 -->
00:39:05,520
what kinds of tasks AI is automating over
time.
461
00:39:06,020 -->
00:39:08,090
And maybe in the past,
we just didn't really have this
462
00:39:08,490 -->
00:39:14,100
understanding. Now we're realizing more and
more just how often these task descriptions
463
00:39:14,100 -->
00:39:17,950
might be compared to the actual test
automation progression.
464
00:39:19,070 -->
00:39:22,460
I see, I see. Yeah. Okay.
465
00:39:22,630 -->
00:39:32,340
So we should cut things up in a way that
corresponds more closely to how we think
466
00:39:32,510 -->
00:39:36,540
things will actually unfold? So that things
will be cleaner:
467
00:39:36,540 -->
00:39:40,180
where it can fully do A,
B and C before it can even do a little bit of
468
00:39:40,180 -->
00:39:40,540
D?
469
00:39:42,370 -->
00:39:45,050
Exactly. I think that would be the hope. But
then I think it's also kind of hard to do
470
00:39:45,680 -->
00:39:47,240
that. I'm not sure how I would cut things up.
471
00:39:47,920 -->
00:39:54,990
Yeah. No, that's a good thought. I haven't
thought about how we might categorize tasks
472
00:39:54,990 -->
00:39:58,880
in a way that's more amenable to tracking AI
automatability.
473
00:39:59,180 -->
00:40:03,930
Okay, so speaking of the GATE model,
would you say that the biggest problem with
474
00:40:03,930 -->
00:40:08,990
the GATE model or the greatest source of
uncertainty is coming from the AI R&D module?
475
00:40:08,990 -->
00:40:12,180
Or do you think it's actually not just that?
476
00:40:12,690 -->
00:40:14,370
Well, I can only speak for myself.
477
00:40:14,800 -->
00:40:21,820
I think the biggest source of uncertainty is
the assumption that there's some amount of
478
00:40:21,820 -->
00:40:29,650
compute where we can be confident that a
model trained with that much compute would be
479
00:40:29,650 -->
00:40:34,430
able to fully automate anything in
particular.
480
00:40:36,090 -->
00:40:44,910
And secondly, that we can interpolate between
here and there
481
00:40:46,020 -->
00:40:52,980
such that, with the kind of 20% of the
compute to the threshold to the finish line,
482
00:40:52,980 -->
00:40:56,060
we could automate whatever percent of the
tasks,
483
00:40:56,060 -->
00:40:59,500
right? This isn't an issue with GATE in
particular.
484
00:40:59,500 -->
00:41:02,000
It's inherited from Tom's takeoff
speeds-model,
485
00:41:02,150 -->
00:41:07,740
I think. But it would be really nice to have
a have a threshold like that,
486
00:41:07,900 -->
00:41:10,760
right? Where we could say,
"Okay, we know we'll get to AGI,
487
00:41:11,060 -->
00:41:16,760
to a system that can automate AI R&D or that
can do something else you're interested in".
488
00:41:16,760 -->
00:41:19,490
And then the interpolation just seems totally
made up. But I've,
489
00:41:20,190 -->
00:41:23,910
I've never I've never thought that whole
methodology was very well grounded.
490
00:41:24,810 -->
00:41:28,090
I think the interpolation in indeed is pretty
made up.
491
00:41:28,770 -->
00:41:30,720
I guess the issue is just like,
what do we do instead?
492
00:41:31,590 -->
00:41:33,790
Well, I'll tell you. I mean,
we can't do it now,
493
00:41:33,790 -->
00:41:42,100
right? But you know, I think it will start to
be the case that full-on O*NET tasks will be
494
00:41:42,100 -->
00:41:49,790
automated and full-on sub-classifications of
work time,
495
00:41:50,060 -->
00:41:53,810
certain types of eventually whole
occupations.
496
00:41:53,810 -->
00:42:02,580
But before that, you know categories from
time-use surveys or from timesheets,
497
00:42:02,580 -->
00:42:06,590
whatever. That they'll start to be
eliminated.
498
00:42:07,300 -->
00:42:12,250
And then we'll have the data to extrapolate
rather than interpolate,
499
00:42:12,980 -->
00:42:18,010
right. Instead of assuming a finish line and
then putting a point in the middle and then
500
00:42:18,010 -->
00:42:26,740
fitting a curve. We'll be able to say every
time we double the model size,
501
00:42:26,740 -->
00:42:32,340
we tend to cut the remaining net tasks in
half or whatever the relationship will look
502
00:42:32,340 -->
00:42:36,040
like. At the moment, there's that "GPTs are
GPTs"-paper.
503
00:42:36,260 -->
00:42:40,970
They just ask an LLM for its guesses about
what it could do.
504
00:42:42,940 -->
00:42:47,590
And then they have people try to judge it,
but they adjust the prompt that they give the
505
00:42:47,590 -->
00:42:49,350
LLM to kind of match the people.
506
00:42:49,900 -->
00:42:54,080
And then they extrapolate. In principle any,
you know what I mean?
507
00:42:54,080 -->
00:42:55,720
Like for anything, the people.
508
00:42:56,480 -->
00:43:02,370
Whatever, there's a whole long conversation.
But so even if you fully trust those
509
00:43:02,370 -->
00:43:06,850
estimates, that's sort of a single data
point, right? That's like what GPT-4 could
510
00:43:06,850 -->
00:43:10,970
automate. How partially it could automate
each O*NET task.
511
00:43:13,190 -->
00:43:16,160
People haven't done a follow-up one for
GPT-5,
512
00:43:17,030 -->
00:43:18,170
or for a reasoning model.
513
00:43:18,170 -->
00:43:25,450
They haven't recapitulated the whole
methodology of that paper.
514
00:43:28,480 -->
00:43:33,740
Yeah. I mean, like the Anthropic Economic
Index and other data on LLM usage that's
515
00:43:33,740 -->
00:43:42,790
starting to come out from the other providers
might let us sort of continuously start to do
516
00:43:42,790 -->
00:43:44,150
this sort of extrapolation.
517
00:43:46,260 -->
00:43:49,420
And, you know, they're not just anchoring on
O*NET tasks,
518
00:43:49,420 -->
00:43:52,600
right. Like Anthropic,
but they also have their own categorizations
519
00:43:52,600 -->
00:43:54,660
that might sort of cluster more naturally.
520
00:43:54,660 -->
00:44:00,030
They have this like endogenous clustering
thing where they apply some ML to try to
521
00:44:00,030 -->
00:44:02,240
figure out what people are using the models
for.
522
00:44:03,670 -->
00:44:08,670
And that's kind of maybe responding to your
point about how we can sort of create on the
523
00:44:08,670 -->
00:44:14,070
fly a task, a way of carving up work into
tasks that better corresponds to
524
00:44:16,370 -->
00:44:19,680
automatability. That approach will have its
flaws as well.
525
00:44:19,680 -->
00:44:23,110
But I think from at least a certain
perspective,
526
00:44:23,410 -->
00:44:30,890
it'll be more grounded than the sort of set
end point and then interpolate-approach that
527
00:44:30,890 -->
00:44:32,510
we've been having to make use of so far.
528
00:44:33,960 -->
00:44:36,390
But it's still early stages for that. And,
you know,
529
00:44:36,390 -->
00:44:42,970
yeah, I mean in a really fast takeoff
scenario then there won't be time,
530
00:44:42,970 -->
00:44:46,200
right? Because we'll move so quickly from
being able to automate almost nothing to
531
00:44:46,200 -->
00:44:51,600
being able to automate everything at once. So
maybe it's just intrinsically hard to
532
00:44:51,650 -->
00:44:56,010
predict. But yeah, I think probably not.
533
00:44:56,200 -->
00:45:02,190
I think probably we will be able to start
doing extrapolations that are a bit more
534
00:45:02,190 -->
00:45:05,910
empirical as time goes on and as we start
kind of developing a track record of
535
00:45:05,910 -->
00:45:08,230
automating tasks completely.
536
00:45:09,840 -->
00:45:17,870
So of course you can track progress and start
to extrapolate in this sort of more empirical
537
00:45:17,870 -->
00:45:22,560
way. Not just by looking at the dimension of
tasks,
538
00:45:22,600 -->
00:45:27,520
but by looking at task time length.
539
00:45:27,770 -->
00:45:33,490
Right. So that's METR have their famous study
of this.
540
00:45:33,530 -->
00:45:34,870
I think that's just great.
541
00:45:34,870 -->
00:45:44,060
That's amazing because it's the first real
example of of starting to do what we would
542
00:45:44,060 -->
00:45:45,570
have liked to be able to do all along.
543
00:45:48,960 -->
00:45:51,320
Yeah, track progress on something that,
in principle,
544
00:45:51,320 -->
00:46:00,200
would eventually get you to full automation
of at least some class of tasks.
545
00:46:02,640 -->
00:46:07,410
And we're far enough along already that we
can draw a line through the data that we have
546
00:46:07,450 -->
00:46:10,640
and sort of make an extrapolation.
547
00:46:10,720 -->
00:46:15,610
It's not like we just have 1 or 2 data points
right at the beginning, like maybe with
548
00:46:15,610 -->
00:46:24,950
O*NET. You might be able to,
you know, carve up the space of work or the
549
00:46:24,950 -->
00:46:28,200
space of R&D or the space of AI R&D in other
ways as well.
550
00:46:30,470 -->
00:46:38,800
I think Situational Awareness has been
criticized by some for
551
00:46:40,590 -->
00:46:47,120
being as hand-wavy as it is about the kind of
extrapolation by equivalent to human age,
552
00:46:47,160 -->
00:46:50,690
basically, right? As smart as a high schooler
or as smart as a college student.
553
00:46:52,700 -->
00:47:00,500
And I think that that framing is not
justified in that essay.
554
00:47:01,960 -->
00:47:08,130
But it could be, or something like it could
be.
555
00:47:08,660 -->
00:47:13,560
More experienced workers are paid a lot more
than more junior workers.
556
00:47:14,240 -->
00:47:17,600
If you know one thing about a person to
predict their wages.
557
00:47:17,780 -->
00:47:26,130
Right. Even more than you know,
education or race or gender or whatever,
558
00:47:26,320 -->
00:47:32,740
age explains so much. And what that's telling
us is that there's this,
559
00:47:32,740 -->
00:47:39,000
a lot of tacit knowledge that people accrue
through life.
560
00:47:39,000 -->
00:47:46,130
Well, okay, so maybe the way things will
unfold is that AI's first get good at
561
00:47:46,130 -->
00:47:50,520
learning the sorts of things you can learn
from books which they have access to no less
562
00:47:50,520 -->
00:47:54,670
than a college student. In fact,
they have better access in some sense.
563
00:47:56,650 -->
00:48:01,500
And so the people right out of college with
no kind of tacit experience will be first
564
00:48:01,500 -->
00:48:06,900
automated. And then as they start to get more
sample efficient and they're deployed for
565
00:48:06,900 -->
00:48:15,480
longer and in those kind of early-stage roles
they accumulate the data and that means that
566
00:48:15,480 -->
00:48:20,200
they can displace the people that had only
had three years of real world experience.
567
00:48:20,200 -->
00:48:23,550
And then you could see that creeping up.
Maybe. I'm not saying that's what's
568
00:48:23,550 -->
00:48:26,620
happening. You hear anecdotes,
of course, about young software engineers
569
00:48:26,620 -->
00:48:30,830
finding it harder to get jobs.
570
00:48:33,480 -->
00:48:38,210
But it could be that a story along the lines
sketched out in Situational Awareness comes
571
00:48:38,210 -->
00:48:42,750
to have an empirical grounding,
in which case we'll have this other way to
572
00:48:44,120 -->
00:48:48,280
extrapolate, right? And there might be others
as well that I haven't thought of.
573
00:48:48,750 -->
00:48:53,610
I guess it reminds me of like AI 2027,
where they're using this METR data for
574
00:48:54,740 -->
00:48:58,090
forecasting the timeline to a superhuman
coder. As far as I understand.
575
00:48:58,090 -->
00:49:01,620
And then instead of just doing this for a
superhuman coder,
576
00:49:01,770 -->
00:49:06,070
which I guess makes the most sense given what
kinds of tasks were included in the METR
577
00:49:06,070 -->
00:49:12,520
study, perhaps we could try to generalize
this somehow by getting more data or just
578
00:49:12,990 -->
00:49:17,150
generalizing the study to a broader fraction
of tasks in the economy.
579
00:49:18,000 -->
00:49:19,840
Yeah, but still looking at task length.
580
00:49:20,400 -->
00:49:27,050
Yeah. Right. Yeah. That would be kind of
expanding the scope of that particular
581
00:49:27,050 -->
00:49:28,950
dimension, the task length dimension.
582
00:49:29,630 -->
00:49:29,740
Right.
583
00:49:32,040 -->
00:49:35,250
How much should we update on economic theory
anyway?
584
00:49:37,220 -->
00:49:39,140
Yeah. Yeah, that's a profound question.
585
00:49:39,640 -->
00:49:44,290
Something I've had to wrestle with having
sort of specialized in it.
586
00:49:48,440 -->
00:49:50,340
Definitely less than I used to think.
587
00:49:53,560 -->
00:49:57,710
This is sort of a pessimistic view,
but a view I've somewhat come around to is
588
00:49:57,710 -->
00:50:01,990
that economic theory is primarily a
destructive project.
589
00:50:02,740 -->
00:50:05,640
And what I mean by that,
I mean, well.
590
00:50:06,240 -->
00:50:07,150
It doesn't sound promising.
591
00:50:07,220 -->
00:50:10,220
Well, yeah, but it's not so bad.
592
00:50:10,220 -->
00:50:19,670
What I mean is, if it's good for anything,
it's most useful for pointing out that
593
00:50:19,670 -->
00:50:25,270
intuitions that you might have had don't
actually hold in general.
594
00:50:26,740 -->
00:50:31,490
Okay. Sort of coming up with counterexamples
or sort of curious models in which something
595
00:50:31,490 -->
00:50:34,040
you thought was inevitable turns out not to
hold.
596
00:50:34,780 -->
00:50:37,110
And every now and then on further
investigation,
597
00:50:37,380 -->
00:50:44,870
hopefully empirical investigation,
you'll find that this theoretical curiosity
598
00:50:44,870 -->
00:50:48,740
is actually relevant. So it can sort of open
your mind.
599
00:50:54,480 -->
00:50:59,680
But it's a much more limited role than being
able to actually,
600
00:50:59,780 -->
00:51:01,860
deduce the truth from first principles,
right?
601
00:51:02,760 -->
00:51:06,000
You know, a classic example is like minimum
wages,
602
00:51:06,000 -->
00:51:15,090
right? You'll see people saying,
as an economist,
603
00:51:15,090 -->
00:51:22,850
I know X. As the price of something goes up,
people want people want less of it.
604
00:51:22,990 -->
00:51:27,010
And so minimum wage is going to cause
unemployment. That's not something,
605
00:51:27,010 -->
00:51:28,360
you know, as an economist.
606
00:51:28,360 -->
00:51:31,880
That's something you know as a shopper or
something. Everyone kind of has the intuition
607
00:51:31,880 -->
00:51:34,630
that typically when things get more
expensive,
608
00:51:35,110 -->
00:51:36,270
they'll be less demanded.
609
00:51:36,270 -->
00:51:37,750
You didn't need a PhD to learn that.
610
00:51:37,750 -->
00:51:44,470
If you learned anything in a PhD either it
was the the tools to empirically test whether
611
00:51:44,470 -->
00:51:49,860
that's true in some contexts,
the fact about whether it's true in some
612
00:51:49,860 -->
00:51:52,010
contexts because you did actually apply the
tools,
613
00:51:52,610 -->
00:51:56,170
or the theoretical insight that it's not true
by necessity.
614
00:51:56,890 -->
00:52:00,960
So it turns out that there are such things as
Giffen goods,
615
00:52:00,960 -->
00:52:05,710
or you can very commonly have
backward-sloping supply curves for labor.
616
00:52:05,710 -->
00:52:10,840
Because if someone gets paid more,
they might just work fewer hours,
617
00:52:10,840 -->
00:52:15,680
right. Sort of buy back their own leisure
with some of the money.
618
00:52:15,680 -->
00:52:19,520
And so labor markets can look all sorts of
ways from first principles.
619
00:52:21,500 -->
00:52:25,250
So this theory can shake you out of this kind
of intuition that you might have had walking
620
00:52:25,250 -->
00:52:28,290
into econ 101 or walking out of econ 101.
621
00:52:30,710 -->
00:52:33,620
And, you know, I think actually,
I mean, I think the empirical case on minimum
622
00:52:33,620 -->
00:52:35,620
wages is just sort of messy.
623
00:52:35,810 -->
00:52:38,960
In some cases they do probably cause some
unemployment,
624
00:52:38,960 -->
00:52:44,100
in other cases not. And it depends a lot on
magnitudes and all the rest of it.
625
00:52:44,100 -->
00:52:47,560
There's some people who triumphally say
"Economics is wrong.
626
00:52:47,560 -->
00:52:52,730
Minimum wages don't cause unemployment." I'm
not saying that. I'm just saying I think if
627
00:52:52,730 -->
00:52:56,920
economic theory has anything to add there,
it's destructive of the common-sense
628
00:52:56,920 -->
00:53:02,540
intuition we all have about prices and
quantities that we know from everyday life.
629
00:53:03,950 -->
00:53:10,880
And so generally when thinking about AI and
growth,
630
00:53:15,600 -->
00:53:20,980
there's some constructive points.
631
00:53:20,980 -->
00:53:26,380
The Jones model can, you know,
at least sort of to some extent,
632
00:53:26,380 -->
00:53:31,620
if you squint, explain the past and so it can
serve as a nice basis where you can swap out
633
00:53:31,670 -->
00:53:35,280
the Ls with Ks and see what full automation
would do to growth.
634
00:53:38,260 -->
00:53:44,970
But beyond that, I think it doesn't have that
much to add,
635
00:53:45,090 -->
00:53:48,880
even though I'm trying to add a little bit.
And if it does have something to add,
636
00:53:49,460 -->
00:53:56,940
it's by identifying ways in which the future
could be surprising.
637
00:53:56,940 -->
00:54:02,120
Which then will have to just kind of look
into more with whatever empirical tools we
638
00:54:02,120 -->
00:54:02,250
have.
639
00:54:03,410 -->
00:54:08,870
So I guess some of the ways in which you've
applied this is to say things like what is
640
00:54:08,870 -->
00:54:15,200
the impact of AGI on the probability of
explosive growth and on wages.
641
00:54:16,490 -->
00:54:21,010
And my understanding is that you thought that
it's kind of ambiguous just from a purely
642
00:54:21,010 -->
00:54:23,990
theoretical perspective. I'm curious if
that's right?
643
00:54:23,990 -->
00:54:27,610
And also whether you think you have a
different perspective if you take all of the
644
00:54:29,350 -->
00:54:31,070
empirical evidence into account?
645
00:54:31,070 -->
00:54:33,670
Like what's your overall view and what's your
purely
646
00:54:33,810 -->
00:54:37,450
theory-let's-try-to-find-the-counter-examples
kind of view?
647
00:54:38,370 -->
00:54:40,700
Yeah. Well it's all a little bit mixed
together.
648
00:54:40,700 -->
00:54:44,640
But one thing that I think definitely is in
this like "Aha,
649
00:54:44,780 -->
00:54:54,230
here's a theoretical curiosity point" is that
real GDP is such a bizarre chimera of a
650
00:54:54,230 -->
00:55:03,900
variable that you could have full automation
and really explosive growth in every
651
00:55:03,900 -->
00:55:11,330
intuitive sense of the term and yet real GDP
growth could go down.
652
00:55:13,110 -->
00:55:19,570
An example of why it might at least not go up
that much,
653
00:55:20,200 -->
00:55:27,620
which I think it probably won't all kind of
work out this way but I don't think this is
654
00:55:27,660 -->
00:55:34,630
crazy, is that you get this effect where
there's this common pattern you find where
655
00:55:37,560 -->
00:55:40,780
new goods, just as they're introduced,
have a really small GDP share.
656
00:55:41,240 -->
00:55:43,820
Because they have zero GDP share before
they're introduced,
657
00:55:43,820 -->
00:55:47,860
right. At first they're really expensive -
we're not very productive at making them.
658
00:55:48,710 -->
00:55:52,190
And as the price comes down,
as we get more productive,
659
00:55:54,630 -->
00:55:57,360
the price falls but the quantity rises
faster,
660
00:55:58,300 -->
00:56:02,550
right. So the elasticity of demand,
I always get this backwards,
661
00:56:02,550 -->
00:56:05,660
the elasticity of demand is greater than one.
662
00:56:06,280 -->
00:56:08,420
Yeah. Every time the price falls a little
bit,
663
00:56:08,420 -->
00:56:09,660
the quantity rises a lot.
664
00:56:11,250 -->
00:56:14,180
So the dollar value of the good rises.
665
00:56:15,740 -->
00:56:21,560
So the share is rising. And after a while it
goes the other way.
666
00:56:22,610 -->
00:56:28,580
Once the goods are really abundant,
at least relative to everything to everything
667
00:56:28,580 -->
00:56:38,430
else. Every time we have the price [go up],
the quantity only rises a
668
00:56:38,430 -->
00:56:41,590
little bit because we're basically satiated
in it. So you get this hump:
669
00:56:42,250 -->
00:56:46,470
new goods - small share;
goods that have been around for a medium
670
00:56:46,470 -->
00:56:49,380
length of time that we're mediumly productive
at -
671
00:56:49,380 -->
00:56:54,280
high share, they dominate GDP;
old goods like food -
672
00:56:54,280 -->
00:57:02,440
small share. So we're kind of continually
going through this hump.
673
00:57:02,680 -->
00:57:05,390
Everyone's familiar with Baumol's cost
disease.
674
00:57:05,540 -->
00:57:10,600
But the way it's usually presented is that AI
might have less of an effect on growth than
675
00:57:10,600 -->
00:57:13,920
you might have thought,
because we'll be bottlenecked by the few
676
00:57:13,920 -->
00:57:17,110
things that have not yet been automated that
you still need people for.
677
00:57:18,280 -->
00:57:24,110
And actually, you can have Baumol after after
full automation.
678
00:57:25,930 -->
00:57:31,480
Because, so remember the hump,
right?
679
00:57:32,930 -->
00:57:38,160
Real GDP growth at a given time is the
weighted average of the growth rates of all
680
00:57:38,160 -->
00:57:42,500
the goods where the weightings are the GDP
shares.
681
00:57:43,790 -->
00:57:50,820
The GDP shares will be dominated by the goods
that we're intermediately productive at in
682
00:57:50,820 -->
00:57:58,820
this view. Right. So let's say for every good
you have
683
00:58:00,530 -->
00:58:03,010
its own specific technology growth rate.
684
00:58:03,010 -->
00:58:10,810
Like how quickly it can be produced is some
arbitrary function of its current technology
685
00:58:10,810 -->
00:58:13,910
level. Okay. So it can be hyperbolic.
686
00:58:14,390 -->
00:58:17,290
You can have A dot equals A squared
something.
687
00:58:17,670 -->
00:58:22,630
So for every good, there is some finite date
by which we'll be able to produce infinite
688
00:58:22,940 -->
00:58:27,160
quantities of it in finite time. So it'll be
free. So GDP share will be zero.
689
00:58:28,180 -->
00:58:35,400
And we just go through these ever higher
index goods,
690
00:58:35,400 -->
00:58:37,820
ever more complex goods over time.
691
00:58:38,610 -->
00:58:45,680
And at any given time,
all of GDP are the goods that have a
692
00:58:46,140 -->
00:58:50,920
productivity level of five or whatever
happens to be in the middle as far as GDP
693
00:58:50,920 -->
00:58:58,800
shares go. So some effect like that can
produce something like a Baumol effect even
694
00:58:58,800 -->
00:59:03,210
after full automation. Yeah.
695
00:59:03,210 -->
00:59:08,220
I think it would be pretty weird if that kept
the absolute number low.
696
00:59:08,220 -->
00:59:11,250
Like anything as low as the current number
sort of indefinitely.
697
00:59:13,330 -->
00:59:23,220
But yeah, the idea that maybe it causes
measuredreal GDP growth to
698
00:59:23,870 -->
00:59:28,870
not be that high for a while when the world
is starting to look remarkably different.
699
00:59:32,840 -->
00:59:34,200
That doesn't seem crazy to me.
700
00:59:34,200 -->
00:59:39,340
And maybe it's worth knowing and just kind of
having as a scenario in your back pocket in
701
00:59:39,340 -->
00:59:43,810
case things start looking weird and anyone
says "What are you talking about?
702
00:59:43,810 -->
00:59:50,280
I don't see the numbers." I'm trying to be
cautious and all,
703
00:59:50,280 -->
00:59:54,380
butthat's an example of,
like, destructive economic theory.
704
00:59:55,150 -->
00:59:59,490
Do we have any quantitative sense of what the
hump looks like?
705
01:00:00,500 -->
01:00:01,830
Yeah, that's a good question.
706
01:00:01,950 -->
01:00:07,960
I mean, again, there's that Besson paper and
you could just do a bunch of case studies by
707
01:00:08,030 -->
01:00:15,000
good. Yeah, I should look into that more
quantitatively.
708
01:00:15,890 -->
01:00:19,770
Okay. And then kind of to go back to the
thing of what's your overall view then?
709
01:00:21,190 -->
01:00:24,420
Oh, I didn't get to the wages either. I
should say something about wages,
710
01:00:24,870 -->
01:00:28,860
but yeah, I don't know.
711
01:00:28,860 -->
01:00:30,230
My overall view on on what?
712
01:00:31,520 -->
01:00:34,880
The probability of explosive growth over the
next,
713
01:00:34,880 -->
01:00:36,430
say, five decades.
714
01:00:37,710 -->
01:00:43,920
I think, if if by explosive you mean like an
order of magnitude higher than now,
715
01:00:45,570 -->
01:00:48,360
that I think is more likely than not.
716
01:00:49,740 -->
01:00:59,030
I mean, with 25 years of 30% growth,
the world's very
717
01:00:59,030 -->
01:01:06,430
different. In fact, I was just punching in
the numbers on my phone on the way here.
718
01:01:08,800 -->
01:01:15,570
So 25 years of 30% growth is so different
that even our intuition that you can do 1.3
719
01:01:15,570 -->
01:01:19,650
to the 25, or you can do e to the 0.3 times
25 will be about the same.
720
01:01:20,030 -->
01:01:23,980
That's broken down. So 1.3 to the 25 is 700.
721
01:01:25,090 -->
01:01:27,710
The economy is 700 times bigger in some
sense.
722
01:01:29,960 -->
01:01:33,850
E to the 0.3 times 25 is 1800,
right?
723
01:01:33,850 -->
01:01:39,830
So just the additional compounding from
continuous growth at an annualized 30% a year
724
01:01:42,220 -->
01:01:47,220
more than doubles the number. Anyway,
so in a world that different,
725
01:01:49,690 -->
01:01:55,320
some funky effect where we're bottlenecked by
these new goods,
726
01:01:55,320 -->
01:01:58,400
which will be really advanced because they're
the kind they'll be like some weird
727
01:01:58,400 -->
01:02:08,220
nanobot-type thing that it takes a lot of
serial steps
728
01:02:08,370 -->
01:02:17,670
to to create. And we need it -
we can't make up for a lack of it with
729
01:02:18,710 -->
01:02:22,340
large quantities of all the goods that we've
been able to produce in the meantime.
730
01:02:22,340 -->
01:02:24,820
So we're kind of like bottlenecked by the
scarcity of this,
731
01:02:24,980 -->
01:02:30,270
of this new thing. But anyway some weird
effect like that or some natural resource
732
01:02:30,270 -->
01:02:39,190
constraint or some regulatory imposition,
like
733
01:02:40,640 -->
01:02:45,360
maybe we really do satiate and we start
wanting to just really make the world very
734
01:02:45,360 -->
01:02:52,110
safe and stable even if it means throttling
growth to only 30% a year or whatever.
735
01:02:52,110 -->
01:03:00,040
All of that roughly cancels out to me at the
moment,
736
01:03:00,040 -->
01:03:03,060
I think, with the counterarguments,
which are also valid.
737
01:03:03,060 -->
01:03:09,650
That, you know, due to international
competition people will continue to race with
738
01:03:09,650 -->
01:03:13,360
each other for military purposes,
even if they would prefer each to be safer.
739
01:03:15,590 -->
01:03:17,870
So that's where about where I land.
740
01:03:17,870 -->
01:03:24,050
I'm kind of even odds once you press too far
beyond order of magnitude.
741
01:03:24,590 -->
01:03:32,240
And how much does your answer change if we
don't consider the economic theory aspect of
742
01:03:32,240 -->
01:03:38,430
this? We don't consider those,
you know the hump of the productivity and,
743
01:03:38,510 -->
01:03:43,670
shares. Would your answer change much?
744
01:03:45,510 -->
01:03:49,880
Oh, sorry. I should have said before:
50 years is long enough that I do think we'll
745
01:03:49,880 -->
01:03:54,070
be able to develop robots and AI that can do
what we can do.
746
01:03:54,070 -->
01:03:57,970
But some of the uncertainty will also just be
that that doesn't work out.
747
01:03:58,010 -->
01:04:07,350
And then, yeah, it's a little hard to
separate it out.
748
01:04:07,800 -->
01:04:17,540
I mean, digging into the theory of what
chain-weighting is and so on has made me
749
01:04:19,170 -->
01:04:25,130
pretty viscerally feel like real GDP is a
much slipperier concept than than I ever used
750
01:04:25,130 -->
01:04:28,360
to think. I mean, here's a fun fact.
751
01:04:29,330 -->
01:04:38,870
This is crazy. So real GDP and lots of real
variables like inflation-adjusted variables,
752
01:04:38,870 -->
01:04:41,260
real capital or whatever,
let's say real GDP,
753
01:04:42,020 -->
01:04:46,040
is not a quantity. What do I mean?
754
01:04:48,330 -->
01:04:50,150
Yeah. It's not. Here's what I mean.
755
01:04:50,630 -->
01:04:53,380
Imagine a timeline of some economy.
756
01:04:53,640 -->
01:04:57,210
So, the US from 1950 to 2025,
75 years.
757
01:04:58,560 -->
01:05:05,830
Okay. And imagine an alternative timeline
with an alternative economy living it out
758
01:05:07,110 -->
01:05:11,210
that's exactly the same as the US in 1950,
at the beginning,
759
01:05:12,090 -->
01:05:17,290
in its own 1950, and exactly like the US in
2025,
760
01:05:17,290 -->
01:05:18,930
at the end in year 75.
761
01:05:20,170 -->
01:05:22,700
But in the middle things happened in a
different order.
762
01:05:25,310 -->
01:05:30,170
So the microwave was invented in 2006,
and the iPhone came out in 1971.
763
01:05:30,170 -->
01:05:34,560
And, you know, the distribution of wealth
changed hands,
764
01:05:34,960 -->
01:05:38,060
evolved in a different way. But at the end,
it's exactly the same.
765
01:05:38,060 -->
01:05:39,700
Everyone's got the same preferences.
766
01:05:42,070 -->
01:05:45,030
Exchanges the same goods and services for the
same dollar bills.
767
01:05:45,030 -->
01:05:51,300
Atom for atom. Everything unfolds exactly the
same in 2025 and in the 1950 on both
768
01:05:51,300 -->
01:05:53,830
timelines. Okay. Timeline A,
timeline B.
769
01:05:58,640 -->
01:06:05,140
Unless people have homothetic preferences,
meaning that the fraction of their income
770
01:06:05,140 -->
01:06:09,450
they spend on each good is constant,
no matter how rich they are.
771
01:06:10,220 -->
01:06:13,820
So no luxuries or inferior goods,
which is completely wrong,
772
01:06:13,870 -->
01:06:19,940
right? You don't spend the same fraction on
food when you're starving as when when you're
773
01:06:19,940 -->
01:06:23,020
richer. But unless people have homothetic
preferences,
774
01:06:23,560 -->
01:06:26,600
that are the exact same preferences across
the population,
775
01:06:27,740 -->
01:06:29,300
and totally stable over time.
776
01:06:31,280 -->
01:06:38,170
Unless those three conditions are met,
there is a timeline B on which real GDP
777
01:06:39,170 -->
01:06:43,720
growth chain-weighted across the years with
perfect measurement is any number.
778
01:06:47,990 -->
01:06:48,260
Okay.
779
01:06:49,800 -->
01:06:55,750
Isn't that crazy? I mean,
even the fact that there could be any
780
01:06:55,750 -->
01:06:58,470
variation means that,
to my mind, real GDP is not a quantity.
781
01:06:58,470 -->
01:07:01,450
Because it's like baking in the history. You
see what I'm saying?
782
01:07:01,450 -->
01:07:05,210
Like a yardstick should,
it shouldn't matter the order in which you
783
01:07:05,210 -->
01:07:08,350
measure things. It should order things in the
same way.
784
01:07:08,960 -->
01:07:15,620
But the order in which things happen can
change what share of GDP a given good was
785
01:07:15,620 -->
01:07:17,530
while it was growing quickly,
right?
786
01:07:17,530 -->
01:07:21,410
So let's say there's two of us and one of us
is going to be rich one year,
787
01:07:21,410 -->
01:07:24,770
and the other one is going to be rich the
other year. And the stuff that I like more,
788
01:07:24,770 -->
01:07:26,040
I'm going to bid up the price.
789
01:07:26,850 -->
01:07:29,390
I've got a lot of clones that have my
preferences and you've got a lot of clones.
790
01:07:29,730 -->
01:07:32,920
We bid up the price more of the things we
like when we're rich.
791
01:07:33,580 -->
01:07:40,340
The way things happen is that the things we
like are growing quickly in absolute units
792
01:07:41,180 -->
01:07:43,870
while we happen to have the money.
793
01:07:44,030 -->
01:07:48,790
So our preferences are kind of mostly
determining what GDP is.
794
01:07:51,610 -->
01:07:57,020
And the things you like are growing quickly
when you and your clones have the money.
795
01:08:00,040 -->
01:08:03,870
Real GDP is going to be higher across the two
years than if it's the other way,
796
01:08:04,130 -->
01:08:07,610
where the things I like grow when I'm poor
and vice versa..
797
01:08:08,030 -->
01:08:12,130
And it's that kind of effect that can mean
that you can scramble things up so that as
798
01:08:12,130 -->
01:08:14,740
long as people depart from perfect
homotheticity,
799
01:08:15,010 -->
01:08:18,810
constant preferences,
same across population,
800
01:08:19,190 -->
01:08:21,450
then real GDP can be any number.
801
01:08:21,770 -->
01:08:24,740
So maybe I've overinternalized this.
802
01:08:24,780 -->
01:08:30,180
But given that I've overinternalized this,
I sort of feel like I can't separate the
803
01:08:30,180 -->
01:08:32,820
theory from the overall opinion I think.
804
01:08:34,650 -->
01:08:37,880
Yeah. I guess it's like a funny way of
framing it though. I think I would still call
805
01:08:37,910 -->
01:08:41,100
it a quantity, but it's just it depends on
the history.
806
01:08:41,370 -->
01:08:48,010
Because, I mean you know,
in in physics we could say the work done is
807
01:08:48,010 -->
01:08:49,350
not a function of state.
808
01:08:49,350 -->
01:08:53,740
It's not just these are the things that you
have and it's only determined by that.
809
01:08:54,260 -->
01:08:57,280
It also depends on the path it took
historically.
810
01:08:57,900 -->
01:09:01,050
But then there are also other quantities that
are just dependent on the state. Like the
811
01:09:01,050 -->
01:09:02,150
internal energy or something.
812
01:09:03,970 -->
01:09:04,230
Yeah.
813
01:09:05,440 -->
01:09:07,700
I think that's sort of like a word choice,
though.
814
01:09:07,820 -->
01:09:11,360
I don't think that really makes a difference
to your claim. So,
815
01:09:11,360 -->
01:09:16,200
new topic. How do we detect the economic
singularity?
816
01:09:18,900 -->
01:09:21,060
Okay. Yeah, that's a big one.
817
01:09:21,170 -->
01:09:25,360
I don't have anything super original.
818
01:09:26,090 -->
01:09:28,970
But I have something a little bit original.
William Nordhaus,
819
01:09:28,990 -->
01:09:33,860
this Nobel Prize-winning economist,
has a paper from 2021 asking "Are we
820
01:09:33,860 -->
01:09:38,200
approaching an economic singularity?" And his
answer was no,
821
01:09:38,200 -->
01:09:45,340
because he observed that if we were,
we would see a bunch of macroeconomic
822
01:09:45,340 -->
01:09:48,710
variables moving in sort of predictable
directions.
823
01:09:48,710 -->
01:09:52,180
So we would see the capital share rising,
for instance.
824
01:09:52,280 -->
01:09:57,750
I think that's just the most robust one. If
we're approaching a world of full automation
825
01:09:57,750 -->
01:10:01,610
where we've got these robots and computers
that can do everything people can do,
826
01:10:03,810 -->
01:10:10,210
and there's just more of them then the share
of income paid to their owners has to be
827
01:10:11,130 -->
01:10:13,020
higher than the share paid to labor.
828
01:10:13,020 -->
01:10:16,710
And it's got to grow ever higher as their
relative quantity increases.
829
01:10:18,380 -->
01:10:19,630
But other examples like that.
830
01:10:19,630 -->
01:10:25,980
So he then plots these things and finds that
they're all kind of as flat as they ever
831
01:10:25,980 -->
01:10:28,790
were. So he says the singularity is not near.
832
01:10:30,570 -->
01:10:35,490
And methodologically,
I think that's a great insight.
833
01:10:36,980 -->
01:10:42,840
It can be. So the issue with it is that it
can be a really lagging indicator of whether
834
01:10:42,840 -->
01:10:47,370
basically this tells us is a singularity
underway.
835
01:10:48,770 -->
01:10:58,010
And it's sort of like being a weather
forecaster and just sort of looking out the
836
01:10:58,010 -->
01:11:06,100
window and saying whether it's raining.
837
01:11:06,240 -->
01:11:12,060
But I mean, if you start to see all of them
moving in the same direction a little bit,
838
01:11:12,480 -->
01:11:20,560
that should be an update. And it'll only be
super lagging on the most
839
01:11:22,070 -->
01:11:28,900
fully software intelligence explosion comes
first then it starts to impact the real
840
01:11:28,900 -->
01:11:35,540
economy very discreetly later sort of
scenario.
841
01:11:35,960 -->
01:11:38,820
Which I think is a possible scenario.
842
01:11:38,820 -->
01:11:46,990
But it's good to know that in other scenarios
you can extract some real information from
843
01:11:46,990 -->
01:11:49,410
the macroeconomic variables well in advance.
844
01:11:49,890 -->
01:11:52,130
So anyway, Nordhaus did that.
845
01:11:54,820 -->
01:12:01,720
One very small thing I've done is just get
the analyses up to date and expand on them a
846
01:12:01,720 -->
01:12:08,100
little bit, add a few kind of similar
variables to the list and find that they're
847
01:12:08,100 -->
01:12:11,110
all just as flat as they were when Nordhaus
measured them.
848
01:12:12,220 -->
01:12:13,290
I guess that's not that surprising.
849
01:12:13,610 -->
01:12:21,510
Which is not surprising,
but I can report that on Nordhaus's
850
01:12:21,510 -->
01:12:27,040
methodology we're not seeing any more
evidence than we did in 2021 or 2015 when he
851
01:12:27,040 -->
01:12:35,370
first wrote the paper actually. A sort of
subtler thing is that you can
852
01:12:37,150 -->
01:12:40,260
look at what's called a network-adjusted
capital share.
853
01:12:41,120 -->
01:12:44,330
The network-adjusted capital share is a
feature of a good.
854
01:12:45,060 -->
01:12:48,120
So the capital share is a feature of an
economy, right? What fraction of all income
855
01:12:48,120 -->
01:12:51,400
is received in exchange for the use of
capital.
856
01:12:51,970 -->
01:13:01,110
Network-adjusted capital share asks:
for every dollar of revenue spent on that
857
01:13:01,110 -->
01:13:04,550
good, if you trace it all the way back down
the supply chain,
858
01:13:05,310 -->
01:13:09,490
how much of it is paid out in exchange for
value added by capital,
859
01:13:10,010 -->
01:13:12,850
as opposed to value added by labor or taxes?
860
01:13:14,180 -->
01:13:15,370
What was the thing that we were tracing?
861
01:13:16,290 -->
01:13:17,260
A dollar of revenue.
862
01:13:18,520 -->
01:13:18,840
Okay.
863
01:13:19,330 -->
01:13:22,470
For example, Starbucks sells a cup of coffee.
864
01:13:23,640 -->
01:13:31,430
You give them $5. Some of it is paid to the
people working at Starbucks.
865
01:13:32,720 -->
01:13:36,530
So, you know, maybe $0.20 or let's say $1.
866
01:13:40,090 -->
01:13:49,770
Some of it is received as profits by the
people who own the physical infrastructure of
867
01:13:49,770 -->
01:13:53,630
Starbucks and the brand,
you know, the shareholders of Starbucks.
868
01:13:56,200 -->
01:14:03,460
So that's another dollar. And let's put aside
taxes and $3 are spent on intermediate
869
01:14:03,460 -->
01:14:06,310
inputs, say. I'm making up these numbers,
but you know.
870
01:14:06,310 -->
01:14:11,290
So on the coffee beans and the cups and the
electricity to keep the lights on and all of
871
01:14:12,140 -->
01:14:15,060
that. Now, as far as the Starbucks balance
sheet is concerned,
872
01:14:15,670 -->
01:14:16,900
those are capital expenses.
873
01:14:19,560 -->
01:14:24,560
But in reality, they're not all capital
expenses because we haven't traced them down
874
01:14:24,560 -->
01:14:29,410
the chain. You've got to ask of every dollar
they get in revenue for coffee beans:
875
01:14:30,440 -->
01:14:35,550
how much goes to the owners of the firm for
just owning the capital of the firm?
876
01:14:35,920 -->
01:14:39,240
How much goes to the people working at the
firm for their labor,
877
01:14:39,650 -->
01:14:44,960
and how much is spent on intermediate inputs
that that firm uses?
878
01:14:44,960 -->
01:14:48,370
If it's a farm, you know,
then the tractor or something,
879
01:14:48,370 -->
01:14:51,240
and then, okay, for every dollar they give
the tractor, you know,
880
01:14:51,240 -->
01:14:56,610
they give John Deere. How much is spent on
capital labor and intermediate inputs at John
881
01:14:56,610 -->
01:15:06,720
Deere. And so of that original $5 that you
gave,
882
01:15:07,160 -->
01:15:14,000
you can in principle ask,
okay, it's all ultimately going to capital or
883
01:15:14,000 -->
01:15:16,170
labor like all the way down the supply chain.
884
01:15:16,170 -->
01:15:20,100
I meanin some sense, it never ends,
right?
885
01:15:20,290 -->
01:15:24,320
You're cutting up the pennies ever smaller.
If you're going to try to do this like
886
01:15:24,320 -->
01:15:27,170
manually, you'd have to sort of stop after a
while,
887
01:15:27,660 -->
01:15:32,750
right? But you can say okay. Yeah.
888
01:15:32,750 -->
01:15:39,620
What's the capital kind of contribution in
some sense to this cup of coffee as opposed
889
01:15:39,620 -->
01:15:48,290
to the labor? So the reason this is relevant
is so one neat thing is that it sounds
890
01:15:48,290 -->
01:15:50,450
totally intractable to compute.
891
01:15:50,490 -->
01:15:54,050
There's this A.J. Jacobs book in which he...
892
01:15:54,930 -->
01:16:01,960
You know A.J. Jacobs? He writes these sort of
funky books in which he does some sort of
893
01:16:02,240 -->
01:16:05,920
stunt and writes about it.
894
01:16:06,450 -->
01:16:10,470
And in one of them, he tried to thank
everyone in the world for his cup of coffee.
895
01:16:11,570 -->
01:16:14,900
So at the beginning, he's thanking the
barista. But a few chapters in and he's,
896
01:16:14,900 -->
01:16:21,680
like, thanking the people who made the paint
for the truck that carried the whatever.
897
01:16:24,730 -->
01:16:29,620
I mean I haven't read the book,
but that's my understanding. So,
898
01:16:29,620 -->
01:16:37,380
yeah, it sounds intractable. It turns out the
US Bureau of Economic Analysis and the OECD
899
01:16:37,950 -->
01:16:43,430
similarly create these big input-output
tables every year where they actually try to
900
01:16:43,430 -->
01:16:48,610
figure out relatively coarsely,
but still at a level of granularity like 87
901
01:16:48,610 -->
01:16:56,340
industries or something I think it is,
how much of every dollar of revenue goes to,
902
01:16:56,920 -->
01:16:59,280
if we're like an average firm in that
industry,
903
01:17:00,440 -->
01:17:04,510
the immediate labour capital and then all the
intermediate inputs across all the other
904
01:17:04,510 -->
01:17:10,170
industries. So you get a big matrix and it
turns out that you can work out what the
905
01:17:10,170 -->
01:17:15,850
network-adjusted capital share is for any
item in the matrix with some matrix inversion
906
01:17:15,850 -->
01:17:21,130
and stuff. You kind of do that infinite sort
of tracing down the chain, and then you can
907
01:17:21,130 -->
01:17:24,690
plot what that is over time for any given
good.
908
01:17:26,560 -->
01:17:31,150
You might want to know if a good is
approaching having a network-adjusted capital
909
01:17:31,150 -->
01:17:38,070
share of one. For a few reasons.
910
01:17:38,160 -->
01:17:44,950
One is that if it's a kind of good that can
later drive the growth of everything else.
911
01:17:45,010 -->
01:17:48,710
If it's like semiconductors or whatever,
if we've like fully automated semiconductor
912
01:17:48,710 -->
01:17:52,000
production, we think that that will then
drive the intelligence explosion, which
913
01:17:52,000 -->
01:17:53,870
drives the automation of other things.
914
01:17:54,460 -->
01:17:57,640
This will actually be more of a leading
indicator, right?
915
01:17:57,690 -->
01:18:00,810
You'll actually be able to do some weather
forecasting instead of just looking out the
916
01:18:00,810 -->
01:18:05,230
window. But secondly,
you might just be interested in it because if
917
01:18:05,230 -->
01:18:08,740
a good if a good has a network-adjusted
capital share of one,
918
01:18:09,220 -->
01:18:15,430
everyone on earth could die and it would
still keep getting cranked out,
919
01:18:15,430 -->
01:18:21,910
right? And yeah, I mean,
that's the kind of thing that could have a
920
01:18:22,200 -->
01:18:23,630
big effect on the world.
921
01:18:25,320 -->
01:18:30,270
Having self-replicating in some sense in this
sort of grand supply chain sense.
922
01:18:30,320 -->
01:18:36,170
Having self-replicating drones would be
really important for military purposes,
923
01:18:36,170 -->
01:18:39,570
right? Or robocops or somethin.
924
01:18:39,750 -->
01:18:44,520
You know, if you don't no longer need the
support of your population to have have the
925
01:18:44,520 -->
01:18:47,760
equipment to suppress them,
maybe dictatorship will be more stable.
926
01:18:53,840 -->
01:19:03,120
Or if some omnicidal maniac wants to create
little killer machines that can
927
01:19:03,580 -->
01:19:08,100
self-replicate. That'll all.
928
01:19:08,720 -->
01:19:13,990
People can intervene to shut it down or
something, but at least you've crossed some
929
01:19:13,990 -->
01:19:18,020
threshold of it being more worrying when you
have a closed loop in which no one needs to
930
01:19:18,020 -->
01:19:22,340
actively intervene to keep the system going.
931
01:19:22,340 -->
01:19:25,460
Which then leaves its mark on the world.
932
01:19:29,220 -->
01:19:34,370
So anyway, I just thought that would be kind
of interesting, compute that and invert that
933
01:19:34,370 -->
01:19:38,730
matrix and look at that network-adjusted
capital share for different goods over time.
934
01:19:41,720 -->
01:19:49,470
And it turns out they're all basically as
flat as a board.
935
01:19:50,090 -->
01:19:54,300
Yeah. For semiconductors,
that's not at the industry level that's
936
01:19:54,300 -->
01:19:56,460
computed annually. It's only every five
years,
937
01:19:56,790 -->
01:19:59,900
unfortunately. So it's always really out of
date. But yeah,
938
01:19:59,900 -->
01:20:03,330
it's like 50-50. It's sort of always been
50-50.
939
01:20:08,160 -->
01:20:13,580
So there you go. So you got one more little
data point that maybe the singularity is not
940
01:20:13,630 -->
01:20:21,040
near, at least if it's going to be an
industrially intense singularity early on.
941
01:20:22,050 -->
01:20:27,640
I'm curious, if we kind of went back in time
here and we tried to apply a similar style of
942
01:20:27,640 -->
01:20:32,630
thinking to whether or not we could have
detected the Industrial Revolution happening.
943
01:20:32,740 -->
01:20:37,890
Suppose we went to like 1700 in France and we
tried to detect the Industrial revolution
944
01:20:38,160 -->
01:20:43,600
coming in advance with some kind of leading
indicator, what would people have done?
945
01:20:44,260 -->
01:20:45,100
How well would you have done?
946
01:20:46,020 -->
01:20:50,550
Yeah. I mean, in practice,
I think the people at the time would not have
947
01:20:50,550 -->
01:20:50,940
done well.
948
01:20:52,290 -->
01:20:52,360
Yeah.
949
01:20:53,060 -->
01:20:59,590
I mean, even if they'd kind of known to keep
an eye out for something like that.
950
01:21:01,560 -->
01:21:05,840
I do want to share a bit about the economists
of 1700s France,
951
01:21:05,840 -->
01:21:07,760
the Physiocrats they were called.
952
01:21:08,480 -->
01:21:11,200
They created the first of those input-output
tables,
953
01:21:11,990 -->
01:21:17,830
I think, as far as we know,
that the BEA now studiously collects every
954
01:21:18,490 -->
01:21:27,970
year or five years. And in some ways they had
some sort of free market thoughts which
955
01:21:28,340 -->
01:21:32,110
Adam Smith incorporated into Wealth of
Nations.
956
01:21:33,140 -->
01:21:38,780
But they also had this crazy idea,
at least a lot of them did,
957
01:21:38,780 -->
01:21:41,130
that the real wealth came from the earth.
958
01:21:41,130 -->
01:21:44,660
You know, it came from the land,
and everything else that people do,
959
01:21:45,000 -->
01:21:46,580
that's just some icing on the cake.
960
01:21:46,620 -->
01:21:49,220
That's rearranging the wood into a chair and
stuff and,
961
01:21:49,220 -->
01:21:52,500
yeah, you've got to do that down the line.
But to really get more wealth,
962
01:21:52,540 -->
01:21:56,740
you've got to speed up the beginning of the
pipeline.
963
01:21:57,130 -->
01:22:03,360
And so all these moves to the cities and all
those first glimmers of industrialization,
964
01:22:04,010 -->
01:22:07,340
they were all moving the wrong way. So
everyone had to get back to the countryside
965
01:22:07,920 -->
01:22:12,990
for economic growth -
it's like the worst possible advice you you
966
01:22:12,990 -->
01:22:20,410
can imagine! And maybe an early example of
economic theory gone awry.
967
01:22:22,680 -->
01:22:32,600
But anyway, I think the things to do would
have been first
968
01:22:32,600 -->
01:22:37,880
to look around for what's going on in other
countries.
969
01:22:38,010 -->
01:22:45,110
Sort of an obvious example,
but don't assume that what's happening in
970
01:22:45,110 -->
01:22:54,340
France is the best guide to what will be
happening in France in a few decades or in a
971
01:22:54,340 -->
01:23:03,290
century or something. Likewise,
I'm sort of realizing my own
972
01:23:04,640 -->
01:23:07,570
folly as I'm speaking,
but I was looking at the US for all these
973
01:23:08,080 -->
01:23:14,240
network-adjusted capital shares. But there
are other countries like Japan where there's
974
01:23:14,240 -->
01:23:17,430
a fair bit more automation,
at least in a lot of sectors.
975
01:23:18,190 -->
01:23:22,530
And because that demonstrates that
something's feasible, that means that even if
976
01:23:22,530 -->
01:23:26,030
for whatever reason it's not been implemented
here, it might come very quickly.
977
01:23:26,030 -->
01:23:29,730
As industrialization came over the English
Channel a bit later.
978
01:23:32,150 -->
01:23:35,730
One question that I have about explosive
growth is that a lot of the framing is
979
01:23:35,730 -->
01:23:39,440
usually around increasing the number of AI
researchers.
980
01:23:39,440 -->
01:23:43,490
But you also mentioned that there are these
kinds of other returns to scale.
981
01:23:44,800 -->
01:23:51,410
So the framing I want to take here is:
what exactly are the returns to tacit
982
01:23:51,410 -->
01:23:58,470
knowledge? So in particular,
suppose you could either 10x the human
983
01:23:58,470 -->
01:24:01,850
population on the one hand,
and on the other hand,
984
01:24:02,310 -->
01:24:05,760
you could allow all humans to share all of
their tacit knowledge with each other.
985
01:24:06,480 -->
01:24:09,550
So which do you think would have a bigger
effect on economic growth, and what would
986
01:24:09,550 -->
01:24:12,250
that look like? Is this like a growth effect
in the long run?
987
01:24:12,250 -->
01:24:15,690
If we maintain this,is there some kind of
level effect here?
988
01:24:16,400 -->
01:24:18,160
How do you expect this to work out?
989
01:24:19,080 -->
01:24:24,780
I expect that doubling the population would
have a bigger effect on on growth.
990
01:24:25,190 -->
01:24:32,040
I mean, again, mind you,
the increasing returns to scale that
991
01:24:32,230 -->
01:24:42,190
specialization could allow for or just better
coordination
992
01:24:42,190 -->
01:24:46,920
through one person's knowledge immediately
being accessed by another,
993
01:24:51,090 -->
01:24:57,490
that's something that we,
on some level have been improving on over
994
01:24:57,490 -->
01:25:00,190
time. I mean, not that quickly.
995
01:25:00,190 -->
01:25:09,800
But as I mentioned, I think the invention of
the internet and modern telecommunications
996
01:25:09,800 -->
01:25:19,560
and just all the academic infrastructure we
have for sharing papers,
997
01:25:19,880 -->
01:25:29,220
and even on the tacit knowledge front I just
think we've been getting better at
998
01:25:29,220 -->
01:25:30,400
communicating over time.
999
01:25:31,090 -->
01:25:39,670
And if that's had an effect,
it's been small enough that we've been able
1000
01:25:39,670 -->
01:25:46,780
to not be completely crazy when sort of
talking about the economy in really broad
1001
01:25:46,780 -->
01:25:54,460
brushstrokes. Even proposing that it's a
significant source of increasing returns to
1002
01:25:54,460 -->
01:26:04,370
scale such that with these big-brained AIs,
one 10x:ed AI is going to be
1003
01:26:04,530 -->
01:26:09,420
a lot more productive than ten smaller ones.
1004
01:26:11,550 -->
01:26:17,790
I think it's true. But the idea that this
would be such a large effect that it would
1005
01:26:17,790 -->
01:26:23,220
outweigh the raw replication effect – that
with ten times more people you'd have kind of
1006
01:26:23,390 -->
01:26:25,780
at baseline, a ten times larger economy.
1007
01:26:26,650 -->
01:26:30,450
And here the replication argument does work,
mind you. Because with ten times more people
1008
01:26:30,890 -->
01:26:36,160
with the same tastes,
they're going to make ten times as much
1009
01:26:36,160 -->
01:26:46,160
stuff. Yeah, I think I'd be very surprised if
this effect of kind of mind melding
1010
01:26:46,880 -->
01:26:53,580
was big enough to outweigh the full-on
elasticity of one "double the population,
1011
01:26:53,580 -->
01:26:56,740
double output" replication point.
1012
01:26:58,460 -->
01:27:01,140
But I don't know. I mean,
obviously we'd be really pushing the boundary
1013
01:27:01,140 -->
01:27:05,110
to develop AIs that could immediately share
all their tacit knowledge.
1014
01:27:05,820 -->
01:27:09,910
But actually there is another kind of way you
can shed some light on this,
1015
01:27:09,910 -->
01:27:13,400
which is just by looking at the returns to
working longer hours.
1016
01:27:18,530 -->
01:27:23,990
Something I've sort of been interested in is
why exactly is it that in some professions
1017
01:27:24,410 -->
01:27:27,480
you make so much more by working really long
hours?
1018
01:27:27,480 -->
01:27:29,060
You know, as a lawyer or whatever.
1019
01:27:31,030 -->
01:27:34,280
Often you'll have an early investment banker
or something, you'll have people working like
1020
01:27:34,280 -->
01:27:37,710
80-hour weeks. And that's really miserable.
1021
01:27:37,710 -->
01:27:39,920
And you think, "Well,
why can't you just get a normal job where
1022
01:27:39,920 -->
01:27:44,310
you're working like 40 hours a week but get
paid half as much?" But actually you get paid
1023
01:27:44,310 -->
01:27:47,600
less than half as much,
or the job is not available at all.
1024
01:27:48,740 -->
01:27:56,340
And I mean, sure, there are some genuine
workaholics who would want to work that much,
1025
01:27:56,340 -->
01:27:59,570
but I think there are people for whom it sort
of feels suboptimal.
1026
01:27:59,570 -->
01:28:07,670
And the reason why that's what we end up
doing it's got to be because of this sort of
1027
01:28:07,670 -->
01:28:09,240
increasing returns to scale thing.
1028
01:28:09,510 -->
01:28:15,920
Where it's more productive to have this
breadth of tasks associated with a project
1029
01:28:16,320 -->
01:28:17,540
located in a single brain.
1030
01:28:17,540 -->
01:28:19,420
If you can't all get it into a single person,
1031
01:28:19,420 -->
01:28:28,350
then you make do with as few as you can so
that you can have the integration of the
1032
01:28:28,350 -->
01:28:37,890
disparate parts of the project. So that's
kind of like sharing tacit knowledge with one
1033
01:28:37,890 -->
01:28:42,300
other person. It's just yourself for another
workweek crammed into the same workweek.
1034
01:28:44,250 -->
01:28:51,310
And yeah the CRS's view,
the constant returns to scale view,
1035
01:28:51,310 -->
01:28:57,370
would say that working 80 hours gets you
twice the salary or less if you're kind of
1036
01:28:57,370 -->
01:28:59,640
overworked or something.
1037
01:29:01,080 -->
01:29:05,550
And the increasing returns to scale view says
that it gets you more than twice the salary.
1038
01:29:06,810 -->
01:29:11,840
And it does. But it doesn't get you more than
four times the salary.
1039
01:29:12,940 -->
01:29:16,340
See what I'm saying? So,
the primary effect is,
1040
01:29:16,800 -->
01:29:18,960
I think, just the fact that you're working
more hours.
1041
01:29:21,120 -->
01:29:22,350
Is that the case, though?
1042
01:29:22,480 -->
01:29:25,740
I guess, in this case I wonder how comparable
it is.
1043
01:29:27,840 -->
01:29:31,660
If we're really sharing the tacit knowledge
across the entire economy,
1044
01:29:31,660 -->
01:29:35,290
of all the possible agents in this economy.
1045
01:29:35,290 -->
01:29:41,980
And it feels potentially a lot bigger than
just sharing with one person and stuffing it
1046
01:29:41,980 -->
01:29:42,520
within the same week.
1047
01:29:42,650 -->
01:29:50,280
It's a good point. It's a good point. I would
wonder how much everyone needs everyone
1048
01:29:50,280 -->
01:29:54,860
else's tacit knowledge,
right? Because to a large extent,
1049
01:29:55,240 -->
01:30:00,160
we're just doing jobs that require
independent knowledge bases.
1050
01:30:03,530 -->
01:30:06,110
But I can't rule it out.
1051
01:30:06,590 -->
01:30:12,480
Maybe someone knows something that rules it
out, but I definitely take the point that it
1052
01:30:12,480 -->
01:30:17,340
might be a bit bigger than the investment
banker-returns to scale.
1053
01:30:17,340 -->
01:30:18,960
But it's a lower bound.
1054
01:30:19,180 -->
01:30:22,980
Intuitively, for me, it's something close to
an upper bound. But yeah,
1055
01:30:23,620 -->
01:30:26,480
I don't have more to say about that.
1056
01:30:26,480 -->
01:30:30,970
Which is going to be more valuable if we
consider the thought experiment of two half
1057
01:30:31,060 -->
01:30:34,530
Jupiter brains versus a Jupiter brain,
which one would be more valuable?
1058
01:30:35,880 -->
01:30:45,750
So, we all know that having twice as many
people leads to
1059
01:30:47,790 -->
01:30:52,390
more than twice as much output through
agglomeration.
1060
01:30:52,620 -->
01:30:57,980
You know, people cluster in cities nowadays
almost entirely because of each other.
1061
01:30:57,980 -->
01:31:03,800
Not because of some natural resource,
some port or something that the city happens
1062
01:31:03,800 -->
01:31:13,660
to be next to. And so everyone gets
1063
01:31:13,660 -->
01:31:17,730
to go to their own favorite kind of barber
and eat at their own favorite kind of
1064
01:31:17,730 -->
01:31:24,240
restaurant. Whereas if we were all just
living in towns of 100 people,
1065
01:31:24,700 -->
01:31:27,940
we'd all have to kind of make do with
something generic.
1066
01:31:31,640 -->
01:31:36,250
On current margins, economists tend to think
that this effect is pretty small,
1067
01:31:37,050 -->
01:31:44,650
right? That you get benefits from
specialization up to maybe the size of a
1068
01:31:44,650 -->
01:31:47,430
small city or something or a small country.
1069
01:31:48,870 -->
01:31:55,210
But beyond that, it's basically constant
returns to scale.
1070
01:31:57,130 -->
01:32:04,970
One argument for this is that the gains from
trade between pretty similar countries with
1071
01:32:04,970 -->
01:32:10,720
similar natural resource endowments and so on
are estimated to be pretty small.
1072
01:32:11,380 -->
01:32:18,390
It's sort of funny, economists are
associated,
1073
01:32:18,450 -->
01:32:27,820
I think, with the idea that tariffs are
really horrible and that there's all these
1074
01:32:27,820 -->
01:32:31,900
gains from trade. And implicitly that's an
argument about increasing returns to scale.
1075
01:32:31,900 -->
01:32:35,090
Because if there was constant returns to
scale, then why bother trading between
1076
01:32:35,090 -->
01:32:41,020
countries? Each country can just sort of chop
the land in half and each country produces
1077
01:32:41,020 -->
01:32:46,650
half as much on its own. And that is in fact
what is typically estimated.
1078
01:32:46,810 -->
01:32:51,370
The gains from trade between the US and
Canada or something,
1079
01:32:51,370 -->
01:32:53,520
it's maybe a few percent of GDP.
1080
01:32:53,710 -->
01:32:58,630
Which is a lot in the grand scheme,
but it's not that much.
1081
01:32:58,630 -->
01:33:06,660
And so all the recent sort of fretting about
tariffs and stuff.
1082
01:33:06,660 -->
01:33:13,230
There can be large short-run losses if you've
built up a trade network and then you
1083
01:33:13,230 -->
01:33:17,220
suddenly sever it, because the supply chain
has to be reorganized.
1084
01:33:17,240 -->
01:33:21,010
But in the long run, standard models,
standard estimates predict that the effect
1085
01:33:21,010 -->
01:33:22,270
should be small. Okay.
1086
01:33:25,130 -->
01:33:34,700
On the other hand, I think in the long run
the effects could be pretty large after all.
1087
01:33:34,700 -->
01:33:35,090
Interesting.
1088
01:33:35,090 -->
01:33:43,060
My sort of pet theory of this would be that a
large part of what's happening,
1089
01:33:44,690 -->
01:33:49,310
as we've developed more advanced technology,
is that we make better use of our latent
1090
01:33:49,310 -->
01:33:50,920
capacity for specialization.
1091
01:33:52,920 -->
01:34:00,430
So as academic fields have gotten more
specialized,
1092
01:34:00,710 -->
01:34:05,520
they've been making use of the fact that you
had more academics to fill all these
1093
01:34:05,520 -->
01:34:10,860
specializations. One thing we could have done
is just keep on calling everyone a natural
1094
01:34:10,860 -->
01:34:14,740
philosopher or whatever. Like basically all
the academics were back in the Middle Ages,
1095
01:34:15,260 -->
01:34:16,600
or theologian or lawyers.
1096
01:34:16,600 -->
01:34:23,910
But they had like four kinds of academics.
And then just have lots of people
1097
01:34:26,620 -->
01:34:35,440
largely duplicating each other's work. And if
there'd been this big influx in academics and
1098
01:34:35,440 -->
01:34:40,870
people hadn't yet come up with,
you know, molecular biology and all the rest
1099
01:34:40,870 -->
01:34:44,970
of it, then as they did so you'd be getting
all of these gains.
1100
01:34:45,570 -->
01:34:50,440
Which on some level you could attribute to
the development of molecular biology,
1101
01:34:50,440 -->
01:34:54,670
but on some level you've got to attribute it
to the fact that there's more people now to
1102
01:34:55,450 -->
01:34:58,120
take up this kind of further specialisation.
1103
01:34:59,310 -->
01:35:08,010
And likewise, if you suddenly moved a bunch
of people from small towns into a big city.
1104
01:35:08,380 -->
01:35:12,860
And you just had a bunch of
middle-of-the-road barbers and diners and so
1105
01:35:12,860 -->
01:35:18,800
on that hadn't yet specialised,
there would be these gains up to the degree
1106
01:35:18,800 -->
01:35:24,130
of specialisation that the size of the city
allowed for. But it wouldn't all be realised
1107
01:35:24,130 -->
01:35:33,160
at once. So, if a view like that is right,
1108
01:35:34,890 -->
01:35:38,490
I think it kind of helps to explain what
would otherwise be a bit suspicious.
1109
01:35:38,490 -->
01:35:48,110
Which is that, when moving from full
individual-level autarky where everyone's
1110
01:35:48,110 -->
01:35:50,640
self-sufficient. It's all,
you know, everyone has their own little house
1111
01:35:50,640 -->
01:35:51,730
on the prairie, right?
1112
01:35:52,040 -->
01:36:01,770
There's no gains from trade at all. And
moving from that to a modern
1113
01:36:01,800 -->
01:36:05,040
sort of small city or small country.
1114
01:36:06,310 -->
01:36:09,810
Assuming it's not got oil. That's that's a
big case where where there are gains from
1115
01:36:09,810 -->
01:36:16,670
trade. I should have said. But that's that's
clearly not about specialization.
1116
01:36:16,670 -->
01:36:19,310
That's just about a necessary natural
resource.
1117
01:36:20,010 -->
01:36:24,210
But yeah, you get all these gains from trade
up to that size and then it just sort of
1118
01:36:24,210 -->
01:36:27,050
suddenly levels off. And that's just a fact
of nature.
1119
01:36:27,910 -->
01:36:33,340
It seems a little suspicious to me that,
in the whole space of ways to arrange matter
1120
01:36:33,340 -->
01:36:39,410
and energy to produce things of value,
you get strong benefits from specialization
1121
01:36:39,570 -->
01:36:46,840
up to the size of a small modern city in 2025
and then no gains at all after that?
1122
01:36:47,130 -->
01:36:54,650
I would guess that with enough time,
if we just stagnated at current population
1123
01:36:54,650 -->
01:37:04,510
sizes, we would develop more and more
specializations and we would continue to
1124
01:37:04,510 -->
01:37:08,920
urbanize globally and not just increase the
fraction of people in cities but the fraction
1125
01:37:08,920 -->
01:37:10,430
of people in the biggest cities.
1126
01:37:11,150 -->
01:37:16,920
And in time we would extract ever more
benefits of specialization.
1127
01:37:16,920 -->
01:37:20,060
But just what has happened over the last,
I don't know, century or something, is that
1128
01:37:20,060 -->
01:37:23,920
populations have grown really quickly and
urbanization has grown really quickly.
1129
01:37:24,480 -->
01:37:27,370
And that has sort of created this big
overhang,
1130
01:37:27,370 -->
01:37:32,300
if you like, of potential specialization to
to be exploited.
1131
01:37:32,880 -->
01:37:37,600
That's kind of one big thought experiment.
Here's another thought experiment.
1132
01:37:39,200 -->
01:37:47,740
It seems to me that the welfare capacity of a
being,
1133
01:37:48,640 -->
01:37:54,720
of a brain, probably tends to grow
superlinearly in the size of the brain.
1134
01:37:56,650 -->
01:38:02,130
So in the EA [Effective Altruism] community,
in the animal welfare community,
1135
01:38:03,250 -->
01:38:08,060
there's a pretty strong inclination that it
goes the other way.
1136
01:38:08,060 -->
01:38:15,150
And a lot of research that seems to support
the conclusion that it goes the other way.
1137
01:38:21,390 -->
01:38:27,410
People will, like Rethink Priorities when
coming up with their animal welfare weights,
1138
01:38:29,210 -->
01:38:31,870
say there are two things that matter.
1139
01:38:31,870 -->
01:38:36,890
There's how intensely a given creature can
feel pleasure or pain,
1140
01:38:36,890 -->
01:38:38,460
like on a utilitarian perspective.
1141
01:38:38,460 -->
01:38:45,050
How intensely it can feel pleasure and pain,
and its list of capacities for valenced
1142
01:38:45,050 -->
01:38:48,370
experience. Can it feel depression? Can it
feel anxiety? Can it feel elation?
1143
01:38:49,450 -->
01:38:55,000
And they find that small creatures can check
off a surprisingly large fraction of the
1144
01:38:55,000 -->
01:38:58,540
boxes that large ones can,
including ourselves.
1145
01:38:59,410 -->
01:39:03,920
And the intensity on some level doesn't seem
to be that much smaller.
1146
01:39:04,050 -->
01:39:07,950
You know, little creatures can squeal like
they're just about,
1147
01:39:08,300 -->
01:39:12,840
asintensely feeling pain or pleasure as we
can.
1148
01:39:14,570 -->
01:39:19,380
And so they think, "Okay,
well, so all this extra gray matter that
1149
01:39:19,420 -->
01:39:27,010
we've got isn't adding much to our welfare
capacity." And I think this is neglecting a
1150
01:39:27,610 -->
01:39:35,880
dimension of welfare capacity that is
analogous to the size of a population.
1151
01:39:36,340 -->
01:39:42,620
Which I call the "size" of the experience. So
my thought experiment is
1152
01:39:46,710 -->
01:39:50,570
if you imagine a split-brain case.
1153
01:39:50,640 -->
01:39:54,770
You know, these split-brain patients that on
some level seem to have two separate streams
1154
01:39:54,770 -->
01:39:59,900
of experience. If someone's submerged in an
ice bath,
1155
01:39:59,900 -->
01:40:04,720
say. So, they're experiencing some pain all
over their body. And then you cut their
1156
01:40:04,720 -->
01:40:07,840
corpus callosum. Now suddenly you have two
streams of experience,
1157
01:40:07,920 -->
01:40:10,620
each of which is only feeling cold on half of
a body.
1158
01:40:12,370 -->
01:40:20,340
On the Rethink Priorities-type view,
you've basically doubled the amount of pain
1159
01:40:20,340 -->
01:40:24,030
in the world because you now have two beings
that check off the same list and have the
1160
01:40:24,030 -->
01:40:27,370
same intensity. But I would say that's crazy.
1161
01:40:27,370 -->
01:40:31,180
That you've just by one snip,
you've doubled the amount of pain in this
1162
01:40:31,180 -->
01:40:35,030
bathtub. No. You've probably left it about
the same,
1163
01:40:35,030 -->
01:40:38,980
or if anything, diminished it a little bit to
the extent that there are sort of
1164
01:40:38,980 -->
01:40:42,820
psychologically sophisticated kinds of
suffering that can only arise when you have
1165
01:40:42,820 -->
01:40:50,470
the hemispheres communicating. And likewise I
just think it's a bit absurd to think that
1166
01:40:50,470 -->
01:40:55,810
there's this intense non-monotonicity where
if you disaggregated my neurons into lots of
1167
01:40:55,810 -->
01:40:58,240
small mouse brains and then even smaller fly
brains,
1168
01:40:58,680 -->
01:41:03,460
that the sum of the welfare capacities across
the whole set of creatures would rise and
1169
01:41:03,460 -->
01:41:05,770
rise and rise until suddenly it was dust.
1170
01:41:05,770 -->
01:41:07,550
And then, it's down to zero.
1171
01:41:08,070 -->
01:41:17,850
I think it's probably more integration,
more complexity means more capacity
1172
01:41:17,850 -->
01:41:22,730
for value itself. Okay,
so I've got more thought experiments along
1173
01:41:22,730 -->
01:41:28,110
those lines. So, I've got my economics
thought and I've got this brain thought.
1174
01:41:28,940 -->
01:41:31,840
So what does this all add up to?
1175
01:41:34,890 -->
01:41:39,940
And yeah, what we were talking about before,
the parallelizability thought.
1176
01:41:39,970 -->
01:41:44,210
About how, at least eventually on some
margin,
1177
01:41:44,210 -->
01:41:46,920
latency could be a really strong bottleneck.
1178
01:41:46,920 -->
01:41:54,710
And so you'd get way more out of one big chip
than two small chips or one automated
1179
01:41:54,710 -->
01:41:59,010
researcher that in some sense has a double
brain rather than two people with normal
1180
01:41:59,070 -->
01:42:08,950
brain side by side. All of it seems to add up
to the possibility that in a sort
1181
01:42:08,950 -->
01:42:17,630
of radical future where we're turning Jupiter
into some giant computer
1182
01:42:20,410 -->
01:42:24,370
two half Jupiter brains could could be a lot
less valuable than one whole Jupiter brain.
1183
01:42:26,220 -->
01:42:29,020
I don't think this kind of extends all the
way up.
1184
01:42:29,150 -->
01:42:32,150
Apparently the galaxy is like 100,000 light
years across.
1185
01:42:32,470 -->
01:42:38,390
So unless we're engaged in this very long,
slow dance where we're all playing little
1186
01:42:38,390 -->
01:42:41,870
parts that can't communicate with each other
for a very long time,
1187
01:42:43,290 -->
01:42:46,700
you probably at some point get plain old
constant returns to scale.
1188
01:42:46,740 -->
01:42:53,470
Because you just kind of can't communicate
efficiently over far enough distances.
1189
01:42:54,340 -->
01:42:59,540
But but you'd have to get pretty big for the
speed of light to start being an issue here.
1190
01:42:59,890 -->
01:43:04,380
And if you get increasing returns to scale
for a long way,
1191
01:43:07,170 -->
01:43:11,290
I think that has some potential implications.
1192
01:43:11,540 -->
01:43:19,840
One of them is that we should perhaps expect
a more peaceful future than we might have
1193
01:43:19,840 -->
01:43:25,000
anticipated, if you believe the liberal peace
hypothesis that countries that benefit from
1194
01:43:25,300 -->
01:43:28,700
being able to trade with each other are less
likely to go to war.
1195
01:43:29,520 -->
01:43:33,490
Well, the gains from trade are just going to
rise over time as we make use of our capacity
1196
01:43:33,490 -->
01:43:38,420
for specialization. At least if we don't grow
into space or something more quickly than we
1197
01:43:38,420 -->
01:43:40,410
can absorb that capacity.
1198
01:43:40,440 -->
01:43:44,000
So if the increasing returns to scale thing
actually starts showing up more,
1199
01:43:46,130 -->
01:43:51,500
then maybe that's kind of good news on the
peace front. Another implication maybe is
1200
01:43:51,500 -->
01:43:56,000
that if you're thinking of trying to invest
for a really,
1201
01:43:56,000 -->
01:44:01,720
really, really, really long time to turn
matter and energy into hedonium or whatever.
1202
01:44:02,610 -->
01:44:06,740
Then you should be more risk tolerant than
you otherwise would have been. Because it's
1203
01:44:06,740 -->
01:44:12,350
better to own one whole Jupiter brain than
two half Jupiter brains.
1204
01:44:12,350 -->
01:44:18,390
I guess this is true prudentially as well,
if you're just trying to be a kind of
1205
01:44:18,460 -->
01:44:24,700
hedonist, like a selfishly hedonist futurist
and you want to turn yourself into a Jupiter
1206
01:44:24,700 -->
01:44:27,730
brain. Then you want to increase the chance
you get the whole.
1207
01:44:27,730 -->
01:44:32,870
You care about getting the whole one more
than like a, 50-50 chance of getting half.
1208
01:44:34,150 -->
01:44:36,000
I don't know all the places this could go.
1209
01:44:36,000 -->
01:44:38,120
It's obviously opening a massive can of
worms,
1210
01:44:38,170 -->
01:44:40,130
but I think it's sort of fun to think about.
1211
01:44:41,400 -->
01:44:46,050
Yeah. On the peace thing,
do you think this is a core explanation for
1212
01:44:46,050 -->
01:44:49,760
why we've seen fewer and fewer individual
nation states over time?
1213
01:44:50,030 -->
01:44:56,110
If we compare say, 1800 to today,
is the reason actually just returns to scale?
1214
01:44:56,490 -->
01:44:58,510
And it's all just like flowing through this.
1215
01:45:00,160 -->
01:45:05,910
Yeah. I've wondered a little bit about that.
I don't think it necessarily tells us how big
1216
01:45:05,910 -->
01:45:07,400
the returns to scale are.
1217
01:45:07,440 -->
01:45:11,900
Because, you know, as long as it's true that
to some extent we all benefit from being able
1218
01:45:11,900 -->
01:45:21,450
to coordinate better. Then there's going to
be some pressure toward having larger states.
1219
01:45:24,380 -->
01:45:31,170
And with enough time,
just like over the course of evolution even
1220
01:45:32,370 -->
01:45:37,250
traits with very small fitness advantages can
eventually become dominant.
1221
01:45:40,490 -->
01:45:46,360
So, yeah, I don't think it tells us anything
quantitatively.
1222
01:45:46,360 -->
01:45:53,160
But states are kind of effective mechanisms
for coordinating groups of people.
1223
01:45:53,160 -->
01:45:55,700
And they've been getting bigger over time.
1224
01:45:56,080 -->
01:45:58,770
And this is an example of something I was
saying before.
1225
01:45:58,860 -->
01:46:03,510
That like this sort of latency bottleneck or
whatever you want to call it,
1226
01:46:03,580 -->
01:46:07,260
coordination bottleneck,
has been around all along.
1227
01:46:07,710 -->
01:46:11,210
It doesn't feature in most of our growth
models,
1228
01:46:11,210 -->
01:46:12,840
really. Certainly not the Jones.
1229
01:46:14,210 -->
01:46:17,200
And it's been getting relieved over time.
1230
01:46:17,200 -->
01:46:19,480
And so to some extent,
we've got to attribute the growth that we've
1231
01:46:19,500 -->
01:46:23,160
had so far to the ongoing relief of this
bottleneck.
1232
01:46:24,730 -->
01:46:34,110
Which means that if in the event of full
automation and
1233
01:46:34,950 -->
01:46:40,500
literal or almost literal galaxy brains that
constraint's getting relieved more quickly,
1234
01:46:41,250 -->
01:46:44,830
that would be a kind of new dimension on
which growth could proceed faster than
1235
01:46:44,830 -->
01:46:45,060
before.
1236
01:46:46,930 -->
01:46:51,920
So this is not necessarily Phil's theory for
why world government is going to exist if you
1237
01:46:51,920 -->
01:46:54,450
just go with the long-run equilibrium?
1238
01:46:55,740 -->
01:46:59,310
Oh, no. Well, it's an argument for thinking
that that eventually a world government is
1239
01:46:59,310 -->
01:47:02,790
more likely than you would have thought
otherwise.
1240
01:47:02,820 -->
01:47:09,470
Sure. But whatever form that takes,
I mean, it could be that countries just
1241
01:47:09,490 -->
01:47:15,120
collaborate more over time or something or
trade more with each other over time, and the
1242
01:47:15,120 -->
01:47:21,040
regulations sync up and it ends up kind of
functionally looking like a world government.
1243
01:47:21,040 -->
01:47:26,710
I don't know, this prediction would have
seemed a lot more sensible a few decades ago
1244
01:47:26,710 -->
01:47:28,430
or something, or maybe 1990.
1245
01:47:30,200 -->
01:47:34,750
And since then, I think for the first time in
history basically,
1246
01:47:34,750 -->
01:47:38,220
we've seen an increase in the number of
countries in the world, right?
1247
01:47:38,610 -->
01:47:42,780
Not a huge increase, but Czechoslovakia split
up and,
1248
01:47:43,040 -->
01:47:47,190
you know, various countries have split up and
they haven't merged.
1249
01:47:48,620 -->
01:47:49,380
Like all of history?
1250
01:47:51,660 -->
01:47:56,090
Maybe it's overstating it,
but actually I was reading something a while
1251
01:47:56,090 -->
01:48:02,460
back thatthis past 35 years,
basically has been unique on record.
1252
01:48:02,840 -->
01:48:05,720
We have no idea like this average size of
tribes,
1253
01:48:07,100 -->
01:48:12,090
but yeah, it's been anomalous as far as the
records go,
1254
01:48:12,090 -->
01:48:20,670
I think. And you know,
there are more wars now than there were
1255
01:48:22,240 -->
01:48:23,780
ten years ago or whatever.
1256
01:48:25,510 -->
01:48:27,580
So, you never know what the future holds.
1257
01:48:27,620 -->
01:48:34,250
But this is just one argument on the on the
pile for the thought that,
1258
01:48:34,250 -->
01:48:39,190
in the long run if the more efficient
arrangement wins out,
1259
01:48:39,190 -->
01:48:44,510
it'll probably involve kind of more
integration and fewer states.
1260
01:48:45,330 -->
01:48:50,030
Because the degree of increasing returns to
scale will itself rise over time as we make
1261
01:48:50,050 -->
01:48:51,820
better use of our latent capacity.
1262
01:48:53,430 -->
01:48:56,610
Okay. I think this is a good place to end.
1263
01:48:57,620 -->
01:49:00,040
Thank you for joining us on the podcast,
Phil.
1264
01:49:00,770 -->
01:49:01,400
Thank you, Anson.
1265
01:49:02,480 -->
01:49:05,090
And thank you all for tuning in to Epoch AI's
podcast.
1266
01:49:05,460 -->
01:49:07,910
We look forward to you joining us for future
episodes.
