WEBVTT
1
00:00:17.620 --> 00:00:18.420
Hey Ben.
2
00:00:18.420 --> 00:00:19.340
Hey Matt.
3
00:00:19.340 --> 00:00:24.870
It has been almost a month to the day that you and I have sat in front of a microphone and tried to talk.
4
00:00:24.870 --> 00:00:24.940
Mm-hmm.
5
00:00:24.940 --> 00:00:36.110
And at least I'm sat in front of a microphone now and not actually recording on my earbuds in my wife's bedroom, childhood bedroom. So back in the right continent.
6
00:00:36.110 --> 00:00:36.720
Right. Right.
7
00:00:36.720 --> 00:01:01.160
Although I do have a funny story on that. I set myself up for a work recording the other day and, you know, for an internal like chat kind of video casting, and I had my pompous microphone in shot obviously because, you know, I'm one of those kind of podcasty people. And of course, I forgot to actually select it in the dropdown. So it was using my terrible earbuds as the recording. And so that's going to go out internally and I'm going to have the absolute...
8
00:01:01.160 --> 00:01:07.380
Mickey taken out of me by being this, you know, hey, I thought we got somebody who was like media savvy, but no.
9
00:01:07.380 --> 00:01:27.260
No I'm not. But that's not what we're going to talk about today. We don't even really know what we're going to talk about today, but something happened to me this morning. It was the first time. That was fun. I was intrigued by a super cool new exploit that has been discovered. It exploits [pretty strong]. It's like...
10
00:01:27.260 --> 00:01:28.100
No.
11
00:01:28.100 --> 00:01:49.320
Do you know, I can't remember the dude's name, unfortunately. XOR, E-A-X, E-A-X, E-A-X is their handle, his GitHub handle. He did SandSifter, which was like a comprehensive search through the entire space of all possible x86 opcodes looking for undocumented opcodes, which was fun. He did M/o/Vfuscator,
12
00:01:49.320 --> 00:02:18.200
which was, hey, every instruction... the MOV instruction is Turing complete on its own with certain caveats, and he wrote a compiler that compiles to just MOV instructions, which is fun. And then most recently he's discovered this cool facility there. If you have ring zero access, like, you know, if you're like the lowest level access, you might just think, hey, yeah, I've got root, what more do I need to know? There are layers below that, that like the firmware of the system runs at.
13
00:02:18.200 --> 00:02:47.160
and other things that you're not supposed to be able to see about your own computer that you own under your desk. And he's found a way to get into it by misconfiguring the RAM system in some particular way, in a really cunning way. Anyway. I was like, this sounds cool. I'm going to... While I'm in the shower, I gave a quick query to Claude saying, can you just summarize this for me? I want to learn about it a little bit. Give me some pointers. And then I had my shower and I came back out and I looked. And for the very first time, I had a massive alert saying, no, you can't do this.
14
00:02:47.160 --> 00:02:50.760
You are clearly some evil hacker.
15
00:02:50.760 --> 00:02:54.810
We will not allow you to use our AI to hack things. You're a bad person.
16
00:02:54.810 --> 00:02:55.140
I hope.
17
00:02:55.140 --> 00:02:59.460
I don't think that's exactly what it said. I'm sorry if anyone from Anthropic is listening, but it was something like that.
18
00:02:59.460 --> 00:02:59.460
Mm-hmm.
19
00:02:59.460 --> 00:02:59.660
Yeah, you're paraphrasing a little bit.
20
00:02:59.660 --> 00:03:14.120
And it gave me the option to use, you know, like a watered down model, which I did. And it was perfectly good for summarizing a document, which I then went off and read anyway. So, you know. It was fine, but it made me think. Are we entering a new age where we're being gatekept by things that...
21
00:03:14.120 --> 00:03:17.500
we don't have any control about.
22
00:03:17.500 --> 00:03:24.240
Mm-hmm. Yeah. You know, this is, it's sort of like a new problem and also not a new problem.
23
00:03:24.240 --> 00:03:25.520
Right.
24
00:03:25.520 --> 00:03:30.680
You know, I had a similar experience this week where a friend of mine was...
25
00:03:30.680 --> 00:03:30.680
Oh.
26
00:03:30.680 --> 00:03:35.520
locked out of their AWS account for reasons that are kind of opaque to them.
27
00:03:35.520 --> 00:03:35.560
That...
28
00:03:35.560 --> 00:03:35.560
And...
29
00:03:35.560 --> 00:03:43.360
is a nightmare situation for me because yeah, that goes down and then you're like, well, how do I... who do I call?
30
00:03:43.360 --> 00:03:43.580
Right.
31
00:03:43.580 --> 00:03:53.360
Right, right. Right. And I mean, we have this sort of digital dependence problem in many facets. Like, I'm going to give you a hypothetical scenario here.
32
00:03:53.360 --> 00:03:53.650
Mm-hmm.
33
00:03:53.650 --> 00:04:03.460
Your choices are... You can continue to own and use a mobile device, a mobile phone. Or not.
34
00:04:03.460 --> 00:04:11.080
But if you continue to use a mobile device, then you need to cut off one of your fingers. It can be any finger.
35
00:04:11.080 --> 00:04:17.770
Whichever one you want, you can use whatever method you want to cut it off, but you need to cut off a finger.
36
00:04:17.770 --> 00:04:17.970
What?
37
00:04:17.970 --> 00:04:19.520
What? This was not on my bingo card.
38
00:04:19.520 --> 00:04:19.710
Yeah.
39
00:04:19.710 --> 00:04:20.100
Right! Okay.
40
00:04:20.100 --> 00:04:25.120
What do you choose? You lose the left pinky. Like that's the choice that 99% of people are making.
41
00:04:25.120 --> 00:04:25.920
There's some...
42
00:04:25.920 --> 00:04:27.180
It's like, yeah, I kind of need a phone.
43
00:04:27.180 --> 00:04:34.900
Yeah, I... I don't like the way this analogy is going, but I understand where... I think I understand your point. Yes, I get it. Yeah. Okay.
44
00:04:34.900 --> 00:04:40.800
Yeah, it's this dependence. And it's like, it does... it's not even good enough to have the device. You need...
45
00:04:40.800 --> 00:04:40.800
Right.
46
00:04:40.800 --> 00:04:44.920
Like the network. Right. You need the connectivity.
47
00:04:44.920 --> 00:04:49.040
Yeah. You know, teleport me back to 1970 with the top of the line phone and it's useless.
48
00:04:49.040 --> 00:04:49.240
Right.
49
00:04:49.240 --> 00:04:49.900
It's an interesting...
50
00:04:49.900 --> 00:04:52.620
Right.
51
00:04:52.620 --> 00:04:57.160
yeah, device, but it doesn't really... yeah.
52
00:04:57.160 --> 00:05:06.200
So we have this digital dependency, right? And it comes in many forms. Like AI models are the most recent form of it, you know, cloud services.
53
00:05:06.200 --> 00:05:07.020
Right.
54
00:05:07.020 --> 00:05:13.980
All of these, like, you know, your Google account, right, with your email and everything else. And you've got probably a lot of identity tied up in that.
55
00:05:13.980 --> 00:05:14.390
Yeah.
56
00:05:14.390 --> 00:05:15.240
Oh my gosh.
57
00:05:15.240 --> 00:05:32.110
Shit, I have a story about that, right? Like, my vanity domain godbolt.org is a Google Apps for Your Domain, when that was a thing. That was like a thing that if you were a Google fanboy like I was, and I was an early adopter of everything, you could say, hey, I would like to bring my own domain to a Gmail account effectively.
58
00:05:32.110 --> 00:05:32.640
Mm-hmm.
59
00:05:32.640 --> 00:05:42.720
Subsequently, when I worked at Google, I discovered that that is not how it works. There's a massive internal hack to make that work. And then it sort of became Google Apps Enterprise-y type thing.
60
00:05:42.720 --> 00:05:55.620
But unfortunately, those who early adopted, like I did, are stuck in this... it's this hinterland between it's not really an enterprise account and it's also not a Gmail account. So I can't like do half the things in my house that are internet connected with my godbolt.org account.
61
00:05:55.620 --> 00:06:06.080
And so I want to just move. And I can't just move because so many things are tied into my login identity as matt@godbolt.org that goes through like the Google login process, that now I can't... I'm stuck.
62
00:06:06.080 --> 00:06:09.520
So I'm like... what do I do? But yeah, sorry, I interrupted.
63
00:06:09.520 --> 00:06:09.800
Yeah.
64
00:06:09.800 --> 00:06:16.140
You were in the middle of the flow of talking about other digital dependencies and...
65
00:06:16.140 --> 00:06:32.080
Yeah. No, well, I mean, that's another great example, right? And so like, I don't think it's realistic to try to eliminate all of these things. But to the extent that you can be...
66
00:06:32.080 --> 00:06:47.940
more resilient to some other entity just unilaterally deciding maybe, in a way where they think you're acting in your best interest, like my friend in his AWS account, maybe.
67
00:06:47.940 --> 00:06:48.420
Right.
68
00:06:48.420 --> 00:06:50.890
Maybe it was. Or not.
69
00:06:50.890 --> 00:06:51.040
Yeah.
70
00:06:51.040 --> 00:06:57.740
It's sort of like, you don't know, and you don't really have a way to know, and you don't have a way to like validate what it is that they're doing and be like, actually that...
71
00:06:57.740 --> 00:06:58.280
I know.
72
00:06:58.280 --> 00:07:08.690
It's not how that works. Like, to the extent that you can limit that liability, it sometimes makes a lot of sense. And so like, you know.
73
00:07:08.690 --> 00:07:08.980
Right.
74
00:07:08.980 --> 00:07:23.990
I feel like open source was like a great example of this, where you know I certainly remember early on in my career the pain of having to deal with, you know, third-party software vendors, and you would try to understand how their software worked.
75
00:07:23.990 --> 00:07:35.680
And they would give you answers that were clearly wrong. And then, you know, you maybe tried to dig into it a little bit. And then, you know, I had the experience very early on in my career of like, you know.
76
00:07:35.680 --> 00:07:50.980
I don't even think that it was any more complicated than like reading through like the scripts they had sent us. But they were like, don't modify those. Right. And I'm like, okay, I'm not modifying them.
77
00:07:50.980 --> 00:07:58.340
I'm just reading them. Like I'm trying to understand like this thing that you sent me that we paid a bunch of money for that doesn't seem to work. And like, God forbid, if I had actually tried to like reverse engineer any of their software, right?
78
00:07:58.340 --> 00:08:02.830
I can't imagine what their reaction would have been to that. Like maybe some kind of lawsuit.
79
00:08:02.830 --> 00:08:03.180
Yeah.
80
00:08:03.180 --> 00:08:13.980
But... like you're stuck in this place where you're like, you know. I don't understand how this works. I'm not empowered to change it. And yet I am dependent on it.
81
00:08:13.980 --> 00:08:14.980
Right?
82
00:08:14.980 --> 00:08:43.320
Empowered is the word, the active word. That's really interesting. Yeah. It occurred to me that like another thing, it was like early Google search engine optimization once Google became the monopoly. Like, you know, you had this thing where an algorithm would change and people's would drop off of the listings, and again it's like, but they couldn't choose that. That's not something they could choose to be in or out. They were like dependent on something, but not because they chose it. Which is even... but yeah, the empowerment is an interesting
83
00:08:43.320 --> 00:08:57.600
aspect of... I do wonder, yeah, so what can we do about this, Ben? Well, I mean, I'm kind of stuck, right? I mean, what... I'll have my own internet search, I'll just, you know, my own data center, my own... yeah, what. Yeah.
84
00:08:57.600 --> 00:09:04.880
Right. Mesh network internet for everybody. That's the solution. Is we're all just going to get some powerful antennas and put them on the top of our houses and mesh all of the houses together.
85
00:09:04.880 --> 00:09:06.760
Don't tell me. Yeah, mesh.
86
00:09:06.760 --> 00:09:07.420
And that's how the internet's going to work.
87
00:09:07.420 --> 00:09:12.720
It's going to end up in some crypto scam though, isn't it? At some point, like some...
88
00:09:12.720 --> 00:09:13.080
Yeah. At some point, yeah, there's like, you know.
89
00:09:13.080 --> 00:09:17.220
Anything distributed nowadays ends up in some... Well, there's a... we can make a chain, some kind of...
90
00:09:17.220 --> 00:09:19.190
Right. There's a coin. You got to buy the coin.
91
00:09:19.190 --> 00:09:19.740
Hey there.
92
00:09:19.740 --> 00:09:20.850
And we're issuing the coins.
93
00:09:20.850 --> 00:09:21.640
You got AI coin.
94
00:09:21.640 --> 00:09:23.210
And who gets the money from these coins?
95
00:09:23.210 --> 00:09:23.410
Ugh.
96
00:09:23.410 --> 00:09:23.800
Yes, exactly.
97
00:09:23.800 --> 00:09:24.390
Oh, token coin.
98
00:09:24.390 --> 00:09:24.590
Yeah.
99
00:09:24.590 --> 00:09:26.360
I'm sure it's been done. I'm sure...
100
00:09:26.360 --> 00:09:26.540
a billion times at this point.
101
00:09:26.540 --> 00:09:28.340
It has already been done.
102
00:09:28.340 --> 00:09:33.060
The terrible idea that launched a thousand slide decks.
103
00:09:33.060 --> 00:09:35.400
Yeah.
104
00:09:35.400 --> 00:09:49.880
So, you know, like... I think it sort of depends on how far down the sort of, you know, crazy tinfoil hat, you know, internet libertarian, you know, route that you want to go.
105
00:09:49.880 --> 00:10:10.490
And I say that recognizing that like a lot of the things that people, you know, from like the Electronic Frontier Foundation and, you know, some elements of the open source community, you know, 20, 30 years ago... a lot of things that they were saying, you know, with panic in their voices would come to pass, have come to pass, and nobody seems to care.
106
00:10:10.490 --> 00:10:11.180
Right.
107
00:10:11.180 --> 00:10:17.800
Like the amount of spying that like all the devices that you pay for do on you is like off the charts.
108
00:10:17.800 --> 00:10:36.700
Somehow that's okay when... I mean, the thing is, there is... so I've always thought of it as there is some kind of implied social contract between me and... I'm going to pick on Google again just because I'm a bit of a fanboy for it, and I feel like having been there I've got some standing in there, some...
109
00:10:36.700 --> 00:10:55.180
And I've always felt that like, yes, it serves ads, yes, it reads my emails or does something with them, and I don't mind that because I get a decent email service that I don't have to pay for and it's integrated with everything. And like the slightly spooky face recognition is great when, for example, you are preparing a family
110
00:10:55.180 --> 00:10:55.180
Mm-hmm.
111
00:10:55.180 --> 00:11:12.100
board. And I'm like, hey, let's just find pictures of my father-in-law. Here's hundreds of pictures of Tony. And like, oh, that's great, right? That's kind of an amazing thing. But yeah, I do realize that I've given up something for that, but I've probably given up more than I think.
112
00:11:12.100 --> 00:11:22.200
than I think about. And I've probably given up more than I would if someone sat me down and said, well, what about these things? And I was like, oh, yeah, I hadn't really thought. Oh, that's a good point. And, you know, but I'm getting something back
113
00:11:22.200 --> 00:11:24.700
out of this.
114
00:11:24.700 --> 00:11:26.220
Mm-hmm.
115
00:11:26.220 --> 00:11:43.980
And like some of the things, though, I'm not getting something back out of it. So like, you know, automatic number plate recognition stuff, the Flock camera stuff that's going on around, you know, like there's a lot of things where it's like, yeah, that's... I'm uneasy because it is not... ironically, it's not directly a company making money. I mean, it is obviously, but it's like...
116
00:11:43.980 --> 00:12:02.380
Law enforcement, a government thing that is for the benefit of the people has decided that they would like this information and then they would be happy to pay someone to put cameras up to do all this kind of stuff. And that feels like, no, that's a step too far somehow. Now, I don't know where the boundary is there. I mean, it feels obviously... but one I'm sort of choosing.
117
00:12:02.380 --> 00:12:15.120
or at least choosing... not choosing, not choosing to. But then... I get so much benefit, and how will I get the same benefit without, you know, putting together a whole bunch of open source software, which is maybe where you're going with this.
118
00:12:15.120 --> 00:12:19.160
As you mentioned, the mesh network of like internet stuff.
119
00:12:19.160 --> 00:12:29.340
Well... Yeah, I mean, this is kind of what I'm saying, is it sort of depends on how far down the rabbit hole you want to go. Right. And I think.
120
00:12:29.340 --> 00:12:40.740
For me personally, first recognizing that 90... nine... I'm just going to put two nines in that, percent of people are not going to want to go down the rabbit hole at all.
121
00:12:40.740 --> 00:13:02.760
Right. Like they're going to want to do the minimum amount that they need to do to keep themselves safe and secure from like scammers and phishers and, you know, people like that. And you know, some of them don't even really care that much about it, from some of the things I've seen. But you know, that's like the minimum amount that they're going to want to put into this, and the rest of it, they're like perfectly happy to be completely dependent.
122
00:13:02.760 --> 00:13:10.380
and exposed in like a data sharing type sense. Like, I'm very happy to trade my data for free email. Right? Like, it's fine.
123
00:13:10.380 --> 00:13:11.500
Yeah. That's, I guess, both. Yeah, yeah.
124
00:13:11.500 --> 00:13:22.800
Right. Like they're very happy to trade their data for just about anything. And in fact, many of them... What was the... Was it Snow Crash that had the gargoyles in it?
125
00:13:22.800 --> 00:13:23.660
Do you know what I'm talking about?
126
00:13:23.660 --> 00:13:24.300
I...
127
00:13:24.300 --> 00:13:25.220
Did you read Snow Crash?
128
00:13:25.220 --> 00:13:27.300
I did read Snow Crash, and now I'm just trying to think about...
129
00:13:27.300 --> 00:13:29.380
I think it was, I think it was Snow Crash.
130
00:13:29.380 --> 00:13:29.380
I don't remember gargoyles. Was it?
131
00:13:29.380 --> 00:13:41.020
that had basically like there was a whole profession where you just walked around with a bunch of like data gathering equipment strapped to your body.
132
00:13:41.020 --> 00:13:41.240
Ah.
133
00:13:41.240 --> 00:13:46.690
right like cameras and like infrared and like, you know, microphones and all these things
134
00:13:46.690 --> 00:13:47.240
Right. Does ring a bell.
135
00:13:47.240 --> 00:13:52.620
And like that was your job. You just walk... it was like you're like a little mini like Google Maps car, but you were just a person.
136
00:13:52.620 --> 00:13:54.000
Put a person doing that, yeah.
137
00:13:54.000 --> 00:14:04.300
Right. And that's what you did for, you know, for your livelihood. And they referred to these people as gargoyles, right? And I think like the great trick, it's like, you know.
138
00:14:04.300 --> 00:14:17.880
So much of what is being built is like 90s cyberpunk dystopia. Right? We're just literally using those books as like a user manual for like how to build a society.
139
00:14:17.880 --> 00:14:19.020
Yeah. Oh. Yeah.
140
00:14:19.020 --> 00:14:25.070
And I feel like one of the great tricks of social media is basically like turning everyone into gargoyles for free.
141
00:14:25.070 --> 00:14:25.320
Now I'm sad.
142
00:14:25.320 --> 00:14:32.600
Like you don't even have to pay them. Right. Like you just walk around and you say like, hey, you should post this to your Instagram, and then people take pictures of everything around them all the time.
143
00:14:32.600 --> 00:14:32.600
Yeah.
144
00:14:32.600 --> 00:14:41.160
and just post it with, you know, geotags on it to social media, and then it's like, all right, cool. We've done it. We've gathered all the data that is.
145
00:14:41.160 --> 00:14:42.140
That was... I remember like,
146
00:14:42.140 --> 00:14:43.480
We didn't even have to pay for it.
147
00:14:43.480 --> 00:14:59.410
the like the surveillance things that like found all of the photographs of like a monument somewhere in Paris or something like that, we're able to reconstruct a full high detailed 3D model from like just thousands of tourist photographs of it, for example, that they've posted to their Instagram. So yeah, we're in that world, aren't we? Yeah, gosh.
148
00:14:59.410 --> 00:14:59.900
There you go.
149
00:14:59.900 --> 00:15:13.180
Yeah. So my point is that 99% of the people, maybe 95, I don't know, some very high percentage of the people out there are not going to be interested in going down this rabbit hole at all.
150
00:15:13.180 --> 00:15:25.000
But... I think it is important. Personally, I think it is important to be able to defend the ability to do it if you want. Because A, it just might make you sleep better at night.
151
00:15:25.000 --> 00:15:35.140
Like, I know a lot of anxious people. I know a lot of paranoid people. And they're just like, that's who they are. And they just want to know that these things are going to be safe and secure. And then they reduce their dependency as much as possible.
152
00:15:35.140 --> 00:15:35.480
Right, fair.
153
00:15:35.480 --> 00:15:37.100
And they just want to be able to do that.
154
00:15:37.100 --> 00:15:48.680
And I think that you should. And also, there are lots of situations in which... you know... people who... maybe...
155
00:15:48.680 --> 00:16:01.180
in situations where they can be easily exploited by governments, by other people, need to be able to keep themselves safe and secure. And having the tools available to them to do that
156
00:16:01.180 --> 00:16:17.840
is important. Right? And you know, I don't know that I can go in and enumerate all of those situations, but I think we could certainly imagine lots of situations in which that might be not only something that someone would want to do, but be, you know, reasonably morally justified in wanting to do it.
157
00:16:17.840 --> 00:16:25.020
And so, for all of those reasons, I think it's important to be able to do this. But it is also really hard.
158
00:16:25.020 --> 00:16:31.960
Right. I mean, so like... Going back to the beginning of this, or not nearly the beginning, as you mentioned your friend's AWS account. I mean, so...
159
00:16:31.960 --> 00:16:31.960
Right.
160
00:16:31.960 --> 00:16:44.900
If they can't recover that account, do they go and buy servers and rack them? Or, you know, is there a happy end to this story, by the way, I should say?
161
00:16:44.900 --> 00:16:50.900
Right. I mean, immediately what you probably do is hope that you did everything with infrastructure as code and then try to port it all over to...
162
00:16:50.900 --> 00:16:54.400
you know, Google Cloud or... Azure, I guess. I don't know.
163
00:16:54.400 --> 00:16:55.590
Let's just switch it one...
164
00:16:55.590 --> 00:16:58.200
one problem for the same category.
165
00:16:58.200 --> 00:17:00.060
Yeah, you know. One feudal lord to another feudal lord, right?
166
00:17:00.060 --> 00:17:00.640
Feudal lord.
167
00:17:00.640 --> 00:17:02.280
Yeah. Like, okay.
168
00:17:02.280 --> 00:17:03.680
Jeff Bezos, feudal lord.
169
00:17:03.680 --> 00:17:03.740
Yeah.
170
00:17:03.740 --> 00:17:05.560
You heard it here first.
171
00:17:05.560 --> 00:17:10.940
So, you know, like that's like on day one, that's probably as good as it gets.
172
00:17:10.940 --> 00:17:11.400
Yeah.
173
00:17:11.400 --> 00:17:28.600
But I think this is why, like, you know, a lot of times, like especially in our industry, people will wind up building their own data centers, not because they strictly have to, but because, you know, when things go wrong or when they have problems or when they want to try to do things that are maybe not standard.
174
00:17:28.600 --> 00:17:37.000
They want the level of control to be able to do it, right? Like they want to be able to say like, yeah, we can put whatever hardware in these machines we want. We can, you know.
175
00:17:37.000 --> 00:17:37.200
Mm-hmm.
176
00:17:37.200 --> 00:17:48.430
configure them however we want. We can use whatever network technology that we want. We're not limited by what a third party vendor is willing to offer to us.
177
00:17:48.430 --> 00:17:48.840
Right.
178
00:17:48.840 --> 00:17:49.660
Yeah. Yeah.
179
00:17:49.660 --> 00:17:54.700
Obviously that is out of the reach of... most typical people.
180
00:17:54.700 --> 00:17:55.820
Your average.
181
00:17:55.820 --> 00:18:05.560
But I will say, I think one thing that is overlooked... you know, kind of like not that expensive
182
00:18:05.560 --> 00:18:15.220
way to get yourself a lot of digital independence is just get a cheap computer and put it on your network and leave it on all the time.
183
00:18:15.220 --> 00:18:16.960
Right.
184
00:18:16.960 --> 00:18:22.320
Great. And, you know, if you want to take that up a level, like get some remote access for it, right?
185
00:18:22.320 --> 00:18:24.850
Tailscale, something something, to make it a very... yeah, exactly, yeah.
186
00:18:24.850 --> 00:18:25.120
So you can SSH into it.
187
00:18:25.120 --> 00:18:36.100
Yeah, yeah. Most of those types of computers that you can buy use very low power, so you don't really need to worry too much about how much power they're drawing.
188
00:18:36.100 --> 00:18:41.870
They're quite... powerful in terms of like the compute that they can, you know, supply.
189
00:18:41.870 --> 00:18:42.440
Yep.
190
00:18:42.440 --> 00:18:52.310
And you will probably find that most of the things that you would want to run in some other service provider, you could run on those types of machines today.
191
00:18:52.310 --> 00:18:52.700
Yep.
192
00:18:52.700 --> 00:18:58.820
And in a weird sort of like everything is backwards and I don't understand how anything works anymore...
193
00:18:58.820 --> 00:19:06.420
The absolute trick that you can pull is, you know, use the LLM of your choice, which you are absolutely dependent on these days, for
194
00:19:06.420 --> 00:19:06.860
At least.
195
00:19:06.860 --> 00:19:14.280
to build you the software that you need to run on that computer to replace whatever it is that you're dependent on the third-party services to do.
196
00:19:14.280 --> 00:19:14.560
Put a pin in that.
197
00:19:14.560 --> 00:19:23.490
right, so if you have some like, you know, awesome note-taking app that depends on some cloud service and you're like, wow, if this ever disappeared I'd be totally screwed
198
00:19:23.490 --> 00:19:24.480
Right. Interesting.
199
00:19:24.480 --> 00:19:32.240
And you want to go down the rabbit hole, which again, 95% of people don't. Something that you could do is you could build your own.
200
00:19:32.240 --> 00:19:32.620
Yeah.
201
00:19:32.620 --> 00:19:43.060
and run it on this computer that runs in your house. And maybe, you know, have some backups that, you know, write to a USB drive or, you know, go back to the cloud and encrypt them and upload them.
202
00:19:43.060 --> 00:19:47.020
Yeah, I was going to say, that's one thing where like the cloud maybe isn't holding you to ransom unless...
203
00:19:47.020 --> 00:19:47.020
That's another way.
204
00:19:47.020 --> 00:19:54.420
Two things happen. One, you need your backup. And two, the cloud provider decides that they're not going to let you.
205
00:19:54.420 --> 00:19:55.260
Right. Right.
206
00:19:55.260 --> 00:20:00.900
So that seems like a dual uncorrelated failure or whatever we used to say.
207
00:20:00.900 --> 00:20:11.420
So, yeah, that is an interesting call to action. Now I wonder. And we have touched on this before because I'm getting a slight deja vu feeling here. But I wonder, it is...
208
00:20:11.420 --> 00:20:11.500
Right.
209
00:20:11.500 --> 00:20:26.900
The fact that we have been so lucky for so long, like 20, 30 years in the sort of software profession that we're in, where essentially we've always been able to just, inverted commas, quotes, just build it ourself.
210
00:20:26.900 --> 00:20:34.740
And with open source, that has been free. Like, is that spoiling us? Because like if I worked in the manufacturing industry and I made grommets
211
00:20:34.740 --> 00:21:12.380
on a machine, and then the vendor that made the machine said, no, you can't use this machine anymore, that's a real problem that many grommet manufacturers may have always had, and they might have to switch to another provider at short notice because they've gone out of business and now they can't get the parts to repair them. So there's a little bit of like a, maybe, you know, other people have always had to deal with this dependence. It's like, unless you want to become a core... your core competency is building lathes to make grommets, as well as the grommets themselves, then you've always had to outsource some significant part of your endeavor. Whereas you and I have been always like, oh, an email client, yeah, I can't be bothered to write an email client, but I...
212
00:21:12.380 --> 00:21:22.660
could in principle. I feel like I could. Is it... are we just spoiled? And then does an LLM now move us back to where we were to start with, you know?
213
00:21:22.660 --> 00:21:32.760
Yeah, I think a lot of the difference there is that once you've bought the grommets, they're yours. Right. The company who built the grommets never shows up and says, your license to these grommets has expired.
214
00:21:32.760 --> 00:21:34.060
And we're going to go into your...
215
00:21:34.060 --> 00:21:34.060
Or the lathe.
216
00:21:34.060 --> 00:21:37.580
Maybe in that instance for the grommet manufacturer. But yeah, either way, both of those things are true.
217
00:21:37.580 --> 00:21:37.780
Yeah.
218
00:21:37.780 --> 00:21:37.980
Right.
219
00:21:37.980 --> 00:21:43.460
Like it is possible to own the thing and it's yours.
220
00:21:43.460 --> 00:21:44.590
It's really interesting. I mean, obviously, again,
221
00:21:44.590 --> 00:21:44.800
right?
222
00:21:44.800 --> 00:21:50.180
And it's kind of like servers, right? Like you buy a server. You own the server.
223
00:21:50.180 --> 00:21:50.920
Yeah.
224
00:21:50.920 --> 00:21:56.840
Modulo maybe some weird things that happen when you ask LLMs about how those chips work.
225
00:21:56.840 --> 00:21:57.300
Ah. Ha ha!
226
00:21:57.300 --> 00:22:01.880
You know, you can do pretty much whatever you want with them.
227
00:22:01.880 --> 00:22:02.700
Yeah.
228
00:22:02.700 --> 00:22:13.100
And like, you know, you want to screw them up and break them to experiment, go ahead and do that, right? Like there will be no ramifications other than the fact that you lost however much money you spent on the server.
229
00:22:13.100 --> 00:22:13.500
Yeah.
230
00:22:13.500 --> 00:22:21.680
Right? So I think that is sort of like the fundamental difference. And some of it gets back to like software as a service type models where it's like, you know.
231
00:22:21.680 --> 00:22:25.880
You never quite own. And, you know, like, you know, video games these days, right?
232
00:22:25.880 --> 00:22:37.020
You know, obviously our hobbies are, my hobbies are old computers, and I've got like buckets and buckets of cartridges and CDs and DVDs, even, that I can put into various old machines and they work fine, but like.
233
00:22:37.020 --> 00:22:54.800
But nowadays, if you want to play a game for like PS4, PS5, it's like you don't really own it in any meaningful way. It exists as bits on the disc, maybe it doesn't even exist on a hard disk anymore, it just exists in the cloud, which means that it goes away.
234
00:22:54.800 --> 00:22:55.120
Yeah.
235
00:22:55.120 --> 00:22:59.020
when you don't own it anymore. You're renting. You've got a license to own something, which is like
236
00:22:59.020 --> 00:22:59.670
Right, right.
237
00:22:59.670 --> 00:23:00.240
we've gone... Yeah.
238
00:23:00.240 --> 00:23:00.840
Yeah.
239
00:23:00.840 --> 00:23:11.230
I mean, you buy a book and you read the small print in the front copy about, you know, you may not, you know, give it away or whatever, or there's some level of restrictions for what you can use it for.
240
00:23:11.230 --> 00:23:12.680
and reproduce it. You know?
241
00:23:12.680 --> 00:23:24.120
But like, that seems like... those seem very reasonable compared to end user license agreements in software or whatever else. Yeah, I don't know. How did we get here? How do we let this happen?
242
00:23:24.120 --> 00:23:27.680
Well, the 90s cyberpunk set the model, and oh, we're just going to do all these things.
243
00:23:27.680 --> 00:23:27.960
Yeah.
244
00:23:27.960 --> 00:23:28.680
I mean, I don't know.
245
00:23:28.680 --> 00:23:31.360
I'm being a little tongue-in-cheek there.
246
00:23:31.360 --> 00:23:31.740
Is this Neal Stephenson's fault?
247
00:23:31.740 --> 00:23:32.040
Is that what you're saying?
248
00:23:32.040 --> 00:23:33.720
Yeah, exactly. Right.
249
00:23:33.720 --> 00:23:35.280
Oh man.
250
00:23:35.280 --> 00:23:36.580
William Gibson can be blamed.
251
00:23:36.580 --> 00:23:39.400
William Gibson. Oh, yeah, yeah, yeah.
252
00:23:39.400 --> 00:23:40.660
Is that Neuromancer as well?
253
00:23:40.660 --> 00:23:44.300
Yeah, yeah, Neuromancer, Mona Lisa Overdrive, and...
254
00:23:44.300 --> 00:23:44.670
Thank you.
255
00:23:44.670 --> 00:23:46.120
Okay, what was the middle one?
256
00:23:46.120 --> 00:23:48.600
My nerd credentials are definitely going down here.
257
00:23:48.600 --> 00:23:50.720
I don't think I've ever read Neuromancer.
258
00:23:50.720 --> 00:23:53.560
like Counter Zero, Count Zero.
259
00:23:53.560 --> 00:23:55.210
Count.
260
00:23:55.210 --> 00:23:56.830
It ends with Zero.
261
00:23:56.830 --> 00:23:57.640
Count Binface.
262
00:23:57.640 --> 00:23:59.260
Count Binface.
263
00:23:59.260 --> 00:24:02.040
Who sadly lost today at Clacton-on-Sea. I love my countrymen.
264
00:24:02.040 --> 00:24:03.900
Oh, did he? Oh man.
265
00:24:03.900 --> 00:24:05.500
Yeah, it didn't work out. 26% of the vote though.
266
00:24:05.500 --> 00:24:05.800
That's pretty good.
267
00:24:05.800 --> 00:24:07.460
It's not bad.
268
00:24:07.460 --> 00:24:12.810
I mean, the other guy, the guy who resigned and who won't be named here, did win.
269
00:24:12.810 --> 00:24:13.280
Yes.
270
00:24:13.280 --> 00:24:14.220
Nigel Binface.
271
00:24:14.220 --> 00:24:17.120
Yeah, no. Don't besmirch the Binface.
272
00:24:17.120 --> 00:24:26.530
Use that one. Yeah, I mean, it's... Yeah, you've got to love the Brits for not taking things too seriously, but still.
273
00:24:26.530 --> 00:24:26.750
yeah
274
00:24:26.750 --> 00:24:27.650
You know, I still...
275
00:24:27.650 --> 00:24:28.100
I mean...
276
00:24:28.100 --> 00:24:45.240
someone pointed out that like the people of Clacton, the place where... you know, this is real for them. So we're all treating it as like a bit of fun, but you know, and it is making some political points, but like, yeah, it's a thing. Anyway, that was not on my list of things to talk about with you today.
277
00:24:45.240 --> 00:24:50.500
But you gave me an opportunity to talk about it. So, Count... what was he saying? Count Basie?
278
00:24:50.500 --> 00:24:51.720
Count Chocula.
279
00:24:51.720 --> 00:24:53.440
No, Count... Count Zero.
280
00:24:53.440 --> 00:24:57.100
I think it's...
281
00:24:57.100 --> 00:25:10.140
The books, that's right.
282
00:25:10.140 --> 00:25:10.300
I think it's like counters. I forget what the middle book is called.
283
00:25:10.300 --> 00:25:10.300
I love to count.
284
00:25:10.300 --> 00:25:10.320
But all of those, I mean, you know, every...
285
00:25:10.320 --> 00:25:10.320
Okay, I think...
286
00:25:10.320 --> 00:25:19.110
What is it? Who's your favorite vampire? The one from Sesame Street. He doesn't count. Oh, I assure you that he does.
287
00:25:19.110 --> 00:25:22.000
That's all right. That's brilliant. I don't know how we're going to top that now.
288
00:25:22.000 --> 00:25:26.260
We've peaked. We've peaked. 20 minutes into this conversation, we peaked.
289
00:25:26.260 --> 00:25:26.560
yeah
290
00:25:26.560 --> 00:25:32.860
Oh yeah, no, we're talking about 90s cyberpunk and the dystopian future that we live in.
291
00:25:32.860 --> 00:25:39.100
Thanks. Yeah, so you were saying that the 90s... this is the blueprint. So what's coming in the future? Well, actually, no. Is there like, so.
292
00:25:39.100 --> 00:25:56.080
I think... we've talked a lot about how folks feel about LLMs. And I think, you know, you and I have both been on various rollercoaster-like ride journeys with this is great, to this is terrible, to this is I'm going to lose my job, to now I'm the most powerful programmer, to now this is what a mess I've got, and everything in between, right?
293
00:25:56.080 --> 00:26:03.790
And that's not even taking into account all of the moral and ethical aspects, which are also super important but I have sort of compartmentalized.
294
00:26:03.790 --> 00:26:04.400
okay
295
00:26:04.400 --> 00:26:14.140
But... you know, not being able to run an LLM. If, for example, tomorrow Anthropic decides that I've been a bad boy.
296
00:26:14.140 --> 00:26:24.470
or my company decided that, you know, they're not liked by Anthropic or whomever else they're providing, or whoever, you know, whatever. That's a concern.
297
00:26:24.470 --> 00:26:24.940
Yeah.
298
00:26:24.940 --> 00:26:33.600
So you mentioned about running your own stuff. Can we do that with LLMs? Do I just need a warehouse-sized computer to be able to run something worthwhile?
299
00:26:33.600 --> 00:26:46.180
Yeah, you know, I think it's arguable that the end state of all information is to be free.
300
00:26:46.180 --> 00:26:57.310
Right. You know, it's like all of these models of like copy protection where you have like a private key that only three people in the world have. It's like someone is going to figure that out. Like, ask Sony.
301
00:26:57.310 --> 00:26:57.560
Yeah.
302
00:26:57.560 --> 00:27:02.000
right, like someone is going to figure that out.
303
00:27:02.000 --> 00:27:05.450
This is not going to work.
304
00:27:05.450 --> 00:27:05.720
Yeah.
305
00:27:05.720 --> 00:27:18.980
Right. And, you know, the model weights and LLMs are just too valuable for people not to try to figure out what they are. Right. And especially when you consider the fact that there is no.
306
00:27:18.980 --> 00:27:27.860
like world government entity that can, you know, enforce any sort of protection for any of this information in any sort of like.
307
00:27:27.860 --> 00:27:28.760
Right.
308
00:27:28.760 --> 00:27:30.660
global way.
309
00:27:30.660 --> 00:27:39.310
Are model weights different though, in the same way? They're more like a compiled program. Right. If you, you know, if you imagine I give you the EXE
310
00:27:39.310 --> 00:27:39.640
Sure.
311
00:27:39.640 --> 00:27:53.520
of my code but not the source code to it. I give you the EXE and you can go away and use that program as much as you like, but you can't really do anything more with it. You can't extend it or build on it or anything like that because it's kind of like already been compiled. It's very difficult to reverse engineer.
312
00:27:53.520 --> 00:27:53.520
Yeah.
313
00:27:53.520 --> 00:28:08.940
And I feel like the weights, or you know even the open weights that you can go and get on the internet, have that flavor. It's like, sure. Here's the result of compiling, but you know, we won't, you can't buy the compiler because it's like 45 data centers full of machines that ran for like two years.
314
00:28:08.940 --> 00:28:37.540
to compile the program. And now here's the program. So, you know, the fact that the program is dressed up to me as a service is, you know, problematic. But even if they handed me the weights, I guess that's what I asked you, in fairness, but like, I'm just now thinking ahead, right? You know, you could give me the, you know, 24 terabyte disc that contains the weights to the best model and I could somehow cobble together running it, but I'd be stuck with that until the next time somebody hands me the 48 terabytes of the next model.
315
00:28:37.540 --> 00:28:38.020
Sure.
316
00:28:38.020 --> 00:28:45.880
It feels like. I don't know. I've always balked at the term open weights because it's like, open? Yes, I guess. I suppose so, but...
317
00:28:45.880 --> 00:28:46.980
Right.
318
00:28:46.980 --> 00:28:50.600
Yeah, sorry. Anyway, I cut you off. Where were you going with this?
319
00:28:50.600 --> 00:29:05.000
well no, I mean, you know, that's a fair point and that is definitely, you know, an avenue that we could explore here, is like how open are open source models, if all you're getting, if you're, yeah.
320
00:29:05.000 --> 00:29:05.950
What does it mean? I mean, because like, yeah, open source. I suppose that's it. I compare it to open source, and open source is like, hey, here's the source code. Now that might as well be alien.
321
00:29:05.950 --> 00:29:13.860
yeah
322
00:29:13.860 --> 00:29:20.780
to most people on the planet, but in principle, you can look at it and go, this was meant for us to build on and continue, and in fact the people who gave me this or gave me access to this, they are using this to continue and get better.
323
00:29:20.780 --> 00:29:22.630
and the next version of the product will be built.
324
00:29:22.630 --> 00:29:22.780
Mm-hmm.
325
00:29:22.780 --> 00:29:28.520
And it's most likely, you know, obviously there are edge cases to these. Whereas the weights feels like so derived.
326
00:29:28.520 --> 00:29:37.940
It's not really the cool and interesting bit of it from the point of view of continuing and being a builder and not being locked into this brain forevermore.
327
00:29:37.940 --> 00:29:38.340
Yeah.
328
00:29:38.340 --> 00:29:43.520
I wonder what... but then the resources are so, so massive that I can't imagine.
329
00:29:43.520 --> 00:29:44.420
Yeah.
330
00:29:44.420 --> 00:29:53.980
that you could do much with it. But at least you could, right? So in principle... And have you tried this, by the way? Have you experimented with local models or anything?
331
00:29:53.980 --> 00:30:08.060
Running local models? No, it's been on my list of things. It's like... there's going to be a tipping point probably in the next 18 to 24 months for me where it's like, part of my day job is probably going to be figuring this out.
332
00:30:08.060 --> 00:30:15.800
And so like, I'm kind of just waiting for that to happen because then it'll be like more of a focus, right?
333
00:30:15.800 --> 00:30:15.940
Woof!
334
00:30:15.940 --> 00:30:21.040
You know what I mean? Like, it's not just like something I'm monkeying around with. It's like, oh no, I need to make this work for real.
335
00:30:21.040 --> 00:30:22.420
You actually have to understand it.
336
00:30:22.420 --> 00:30:24.100
To be able to do it, yeah.
337
00:30:24.100 --> 00:30:26.360
Yeah. You know, and like, the world will be completely different in 24 months.
338
00:30:26.360 --> 00:30:26.700
What?
339
00:30:26.700 --> 00:30:31.560
Technologies will be different. Everything will be different. So I might as well figure it out for real when I need to figure it out for real.
340
00:30:31.560 --> 00:30:48.040
Yeah. Yeah, it's interesting. I suppose... I mean, we're, sorry, excuse the dog. We're meandering all over here, but like, you know, like even getting the equipment to run these things on, obviously one can find GPUs around or, you know, in the house I have a few around, but they're not really up to scratch for this kind of stuff.
341
00:30:48.040 --> 00:30:48.230
Mm-hmm.
342
00:30:48.230 --> 00:31:05.300
And then, you know, like I know from my own day job that sourcing, you know, top end GPUs and the RAM that goes with them is incredibly difficult right now, and I wonder if that's going to change. But one can only hope that there are some advances in the way that we...
343
00:31:05.300 --> 00:31:05.730
right
344
00:31:05.730 --> 00:31:11.220
evaluate, not evaluate, to make better how we run inference or whatever
345
00:31:11.220 --> 00:31:12.400
Sure, yeah
346
00:31:12.400 --> 00:31:26.600
on hardware, because GPUs are designed well for graphics. I still feel sad for rows and rows of graphic processing units in racks that have never had a video display plugged into them ever.
347
00:31:26.600 --> 00:31:26.700
Right.
348
00:31:26.700 --> 00:31:37.590
They've never generated a frame of a game, which is the point of a GPU, in my opinion. But, you know, it makes me sad. It's like battery-farmed hens, you know. It's like, oh, God, it's not right. Be free. Do what you're supposed to do.
349
00:31:37.590 --> 00:31:38.060
Yes.
350
00:31:38.060 --> 00:31:40.460
I need some free range GPUs, please.
351
00:31:40.460 --> 00:31:44.390
Yeah, well, I've got a drawer of... I've got a Voodoo 2 in a drawer somewhere.
352
00:31:44.390 --> 00:31:44.640
I wouldn't.
353
00:31:44.640 --> 00:31:45.680
I think that'll help, right?
354
00:31:45.680 --> 00:31:49.140
We feed them a steady diet of Doom Eternal and...
355
00:31:49.140 --> 00:31:55.320
Oh. That's too new for my GPUs, more like VGA adapters with a bit.
356
00:31:55.320 --> 00:31:56.440
Yeah. Uh-huh.
357
00:31:56.440 --> 00:31:57.220
Yeah, anyway.
358
00:31:57.220 --> 00:32:11.110
Yeah, no, there's, I mean, well, and this is kind of what I'm saying about the, you know, the information wants to be free argument on models is like, yes, it does take a lot of resources to train them and generate those weights.
359
00:32:11.110 --> 00:32:12.020
Right.
360
00:32:12.020 --> 00:32:23.450
But once you've done it... like, they're not going to disappear. And imagine a world where there is an open source
361
00:32:23.450 --> 00:32:29.460
agentic coding model that is better than, say, Fable 5. Right. Like.
362
00:32:29.460 --> 00:32:31.020
Right.
363
00:32:31.020 --> 00:32:45.700
That is probably going to happen at some point. Right? Like. Do you maybe have some situations in which you want to use a better one? Sure. But if you're just trying to like vibe code the notes app to run on your personal server, to like, it was like.
364
00:32:45.700 --> 00:32:46.020
You don't really.
365
00:32:46.020 --> 00:32:48.900
That model will be more than sufficient, right?
366
00:32:48.900 --> 00:33:01.220
All right. And I can see a world in which that kind of open... yeah. The thing that I'm interested in, and now this is going to be part two of a massive thing, is that like, you know, after the base training of a model, like where you've just fed
367
00:33:01.220 --> 00:33:15.940
some subset of all human knowledge in through however you've acquired it, which is its own massive asterisk and footnote and ethical dilemma, but you've got some amount of knowledge, you train a neural net on it, and then there are...
368
00:33:15.940 --> 00:33:35.000
so much post-training that happens to like make it valuable, make it aligned, make it safe. And then tuning after that to make it, you know, like, hey, don't write code that's terrible. And that's the bit I'd be interested in. I'd like to be able to go like, hey, I don't want to have to explain in text over and over again to a model that...
369
00:33:35.000 --> 00:33:38.830
Don't write comments like this, for God's sake, stop it.
370
00:33:38.830 --> 00:33:39.360
Mm-hmm.
371
00:33:39.360 --> 00:34:05.140
because you know you and I, I'm sure, have done that a million times. But I want to be able to say like, why didn't we just tune it? Can I personalize the weights so that it doesn't do that in the first place rather than me having to exhort it every single time to do it the way I want? And that feels like the last steps of an open model where you're like, okay, that's tractable. Here's the 20 terabyte intermediate model. You can add layers of tuning onto it. And then like, maybe that is something I could do. Maybe I'm just beyond, and you know, yeah.
372
00:34:05.140 --> 00:34:08.160
I have to admit I don't know nearly enough about how.
373
00:34:08.160 --> 00:34:09.580
how that process works now.
374
00:34:09.580 --> 00:34:12.120
how that process works to know whether that's a feasible thing or not.
375
00:34:12.120 --> 00:34:17.620
No, but it seems like I can imagine open companies starting that process, or you know maybe maybe this stuff exists on Hugging Face.
376
00:34:17.620 --> 00:34:17.920
Sure.
377
00:34:17.920 --> 00:34:24.220
Maybe this is the kind of thing that, you know, I would, if I spent a bit more time looking into it, I would discover it can actually be done.
378
00:34:24.220 --> 00:34:26.000
Yeah.
379
00:34:26.000 --> 00:34:34.840
But now it's interesting. And yeah, just so, and the irony here that the sort of leader in open weight, as far as I can tell, is the Chinese government.
380
00:34:34.840 --> 00:34:35.200
Right.
381
00:34:35.200 --> 00:34:35.560
Yeah.
382
00:34:35.560 --> 00:34:36.890
That seems like, wow.
383
00:34:36.890 --> 00:34:37.200
Yep.
384
00:34:37.200 --> 00:34:41.980
That was not on my bingo card for this year, is like champions of openness and freedom.
385
00:34:41.980 --> 00:34:44.300
Yeah.
386
00:34:44.300 --> 00:34:47.200
You know, I'm sure you can find a, as you say, a 90s sci-fi novel that...
387
00:34:47.200 --> 00:34:50.680
We need an AI company that's open. What would we call it?
388
00:34:50.680 --> 00:34:51.880
No.
389
00:34:51.880 --> 00:34:52.830
I don't.
390
00:34:52.830 --> 00:34:53.140
No.
391
00:34:53.140 --> 00:34:55.440
I just can't think of a good name.
392
00:34:55.440 --> 00:34:56.680
No, there's no... I don't think you can pick one.
393
00:34:56.680 --> 00:34:59.160
I wonder if the domain, I'll just check some domains and see if anything's available.
394
00:34:59.160 --> 00:35:00.560
Yeah, see whether you can find one.
395
00:35:00.560 --> 00:35:01.340
Yeah.
396
00:35:01.340 --> 00:35:02.380
Yeah.
397
00:35:02.380 --> 00:35:07.600
But yeah, like it's entirely possible that in the limit, the only thing that matters is GPUs, power and cooling.
398
00:35:07.600 --> 00:35:07.940
Anyway.
399
00:35:07.940 --> 00:35:19.440
Right. Because like. Models will get good enough. Their information will leak out into the world as all information does. And you will reach a point where.
400
00:35:19.440 --> 00:35:33.200
you know, you don't necessarily have to be beholden to a company to use their models as a service. You can go and either purchase or maybe just acquire for free because they're just freely available.
401
00:35:33.200 --> 00:35:33.280
Mm-hmm.
402
00:35:33.280 --> 00:35:39.080
a model that you can run on your own hardware and then you can purchase that hardware and then you own that hardware.
403
00:35:39.080 --> 00:35:39.420
Yeah.
404
00:35:39.420 --> 00:35:48.340
And no one can take it away from you. And then you could have, you know, it's like that server that's running in your house maybe it becomes a little bit more beefy and uses more power.
405
00:35:48.340 --> 00:35:51.200
Right.
406
00:35:51.200 --> 00:35:51.800
but it is yours and no one can turn it off for you other than
407
00:35:51.800 --> 00:35:55.500
But it's yours, right? And no one can turn it off and no one can tell you not to use it. And, you know.
408
00:35:55.500 --> 00:35:59.320
And you're using solar panels so that the electricity company can't cut you off.
409
00:35:59.320 --> 00:36:00.650
Right. Further and further down the rabbit hole.
410
00:36:00.650 --> 00:36:01.280
Down the rabbit hole, yeah.
411
00:36:01.280 --> 00:36:03.020
And you got the mesh internet that you're not connected.
412
00:36:03.020 --> 00:36:03.220
That's it.
413
00:36:03.220 --> 00:36:04.600
You don't have an internet service provider.
414
00:36:04.600 --> 00:36:05.360
Except...
415
00:36:05.360 --> 00:36:08.580
mesh internet that only works when I'm not inside the tinfoil wrapped house, where like, yeah, no, it is.
416
00:36:08.580 --> 00:36:11.460
Yes, exactly. Right.
417
00:36:11.460 --> 00:36:23.500
Gosh. Wow. Well, friend, we've gone round and we've covered an awful lot of things that I wasn't expecting to, but you and I haven't spoken for ages, so this is not too surprising.
418
00:36:23.500 --> 00:36:25.190
that rambling and in some ways but I think perhaps
419
00:36:25.190 --> 00:36:27.140
It's been... It's been a little bit.
420
00:36:27.140 --> 00:36:35.740
in the tinfoil wrapped house, with our own utility providing systems in place, is where we should leave this, with our own intelligent.
421
00:36:35.740 --> 00:36:35.850
Mm-hmm.
422
00:36:35.850 --> 00:36:36.180
Yes.
423
00:36:36.180 --> 00:36:48.200
Well, what I would say is about the power and cooling thing is that at least there is an existence proof of something which doesn't require that much cooling because I'm using it right now to talk with you. And I don't mean my mouth.
424
00:36:48.200 --> 00:36:56.920
I mean, but although that is the primary way in which I think, to be fair, as you've probably realized, but like, so it has to be possible to have something which has at least human intelligence that runs on, what,
425
00:36:56.920 --> 00:36:57.960
80 watts.
426
00:36:57.960 --> 00:37:10.300
100 watts, right? You know, something like that. Maybe I'm not as fast as Claude is, but like somewhere in there there's a scaling thing, so I'm sure we can make it happen along the way. And that's the hope obviously, is somebody will come up with some cool 3D
427
00:37:10.300 --> 00:37:10.300
Yeah, right.
428
00:37:10.300 --> 00:37:10.300
Yeah.
429
00:37:10.300 --> 00:37:17.000
silicon hardware thing that is vastly more efficient at computing whatever it is, but
430
00:37:17.000 --> 00:37:21.130
Right. Or just fusion power. That works too. That'd be awesome.
431
00:37:21.130 --> 00:37:22.160
Fusion.
432
00:37:22.160 --> 00:37:28.100
Yeah. We still got a lot of exhaust heat to put somewhere, which doesn't seem like the best thing to have knocking around, you know.
433
00:37:28.100 --> 00:37:28.800
Could be true. Yeah.
434
00:37:28.800 --> 00:37:32.180
So. Yeah. Anyway, with that, on that bombshell... Yeah.
435
00:37:32.180 --> 00:37:36.280
on our fusion powered tinfoil wrapped home.
436
00:37:36.280 --> 00:37:36.280
Right.
437
00:37:36.280 --> 00:37:38.960
I will see you next time, friend.
438
00:37:38.960 --> 00:37:40.960
Next time.
1
00:00:17.620 --> 00:00:18.420
Hey Ben.
2
00:00:18.420 --> 00:00:19.340
Hey Matt.
3
00:00:19.340 --> 00:00:24.870
It has been almost a month to the day that you and I have sat in front of a microphone and tried to talk.
4
00:00:24.870 --> 00:00:24.940
Mm-hmm.
5
00:00:24.940 --> 00:00:36.110
And at least I'm sat in front of a microphone now and not actually recording on my earbuds in my wife's bedroom, childhood bedroom. So back in the right continent.
6
00:00:36.110 --> 00:00:36.720
Right. Right.
7
00:00:36.720 --> 00:01:01.160
Although I do have a funny story on that. I set myself up for a work recording the other day and, you know, for an internal like chat kind of video casting, and I had my pompous microphone in shot obviously because, you know, I'm one of those kind of podcasty people. And of course, I forgot to actually select it in the dropdown. So it was using my terrible earbuds as the recording. And so that's going to go out internally and I'm going to have the absolute...
8
00:01:01.160 --> 00:01:07.380
Mickey taken out of me by being this, you know, hey, I thought we got somebody who was like media savvy, but no.
9
00:01:07.380 --> 00:01:27.260
No I'm not. But that's not what we're going to talk about today. We don't even really know what we're going to talk about today, but something happened to me this morning. It was the first time. That was fun. I was intrigued by a super cool new exploit that has been discovered. It exploits [pretty strong]. It's like...
10
00:01:27.260 --> 00:01:28.100
No.
11
00:01:28.100 --> 00:01:49.320
Do you know, I can't remember the dude's name, unfortunately. XOR, E-A-X, E-A-X, E-A-X is their handle, his GitHub handle. He did SandSifter, which was like a comprehensive search through the entire space of all possible x86 opcodes looking for undocumented opcodes, which was fun. He did M/o/Vfuscator,
12
00:01:49.320 --> 00:02:18.200
which was, hey, every instruction... the MOV instruction is Turing complete on its own with certain caveats, and he wrote a compiler that compiles to just MOV instructions, which is fun. And then most recently he's discovered this cool facility there. If you have ring zero access, like, you know, if you're like the lowest level access, you might just think, hey, yeah, I've got root, what more do I need to know? There are layers below that, that like the firmware of the system runs at.
13
00:02:18.200 --> 00:02:47.160
and other things that you're not supposed to be able to see about your own computer that you own under your desk. And he's found a way to get into it by misconfiguring the RAM system in some particular way, in a really cunning way. Anyway. I was like, this sounds cool. I'm going to... While I'm in the shower, I gave a quick query to Claude saying, can you just summarize this for me? I want to learn about it a little bit. Give me some pointers. And then I had my shower and I came back out and I looked. And for the very first time, I had a massive alert saying, no, you can't do this.
14
00:02:47.160 --> 00:02:50.760
You are clearly some evil hacker.
15
00:02:50.760 --> 00:02:54.810
We will not allow you to use our AI to hack things. You're a bad person.
16
00:02:54.810 --> 00:02:55.140
I hope.
17
00:02:55.140 --> 00:02:59.460
I don't think that's exactly what it said. I'm sorry if anyone from Anthropic is listening, but it was something like that.
18
00:02:59.460 --> 00:02:59.460
Mm-hmm.
19
00:02:59.460 --> 00:02:59.660
Yeah, you're paraphrasing a little bit.
20
00:02:59.660 --> 00:03:14.120
And it gave me the option to use, you know, like a watered down model, which I did. And it was perfectly good for summarizing a document, which I then went off and read anyway. So, you know. It was fine, but it made me think. Are we entering a new age where we're being gatekept by things that...
21
00:03:14.120 --> 00:03:17.500
we don't have any control about.
22
00:03:17.500 --> 00:03:24.240
Mm-hmm. Yeah. You know, this is, it's sort of like a new problem and also not a new problem.
23
00:03:24.240 --> 00:03:25.520
Right.
24
00:03:25.520 --> 00:03:30.680
You know, I had a similar experience this week where a friend of mine was...
25
00:03:30.680 --> 00:03:30.680
Oh.
26
00:03:30.680 --> 00:03:35.520
locked out of their AWS account for reasons that are kind of opaque to them.
27
00:03:35.520 --> 00:03:35.560
That...
28
00:03:35.560 --> 00:03:35.560
And...
29
00:03:35.560 --> 00:03:43.360
is a nightmare situation for me because yeah, that goes down and then you're like, well, how do I... who do I call?
30
00:03:43.360 --> 00:03:43.580
Right.
31
00:03:43.580 --> 00:03:53.360
Right, right. Right. And I mean, we have this sort of digital dependence problem in many facets. Like, I'm going to give you a hypothetical scenario here.
32
00:03:53.360 --> 00:03:53.650
Mm-hmm.
33
00:03:53.650 --> 00:04:03.460
Your choices are... You can continue to own and use a mobile device, a mobile phone. Or not.
34
00:04:03.460 --> 00:04:11.080
But if you continue to use a mobile device, then you need to cut off one of your fingers. It can be any finger.
35
00:04:11.080 --> 00:04:17.770
Whichever one you want, you can use whatever method you want to cut it off, but you need to cut off a finger.
36
00:04:17.770 --> 00:04:17.970
What?
37
00:04:17.970 --> 00:04:19.520
What? This was not on my bingo card.
38
00:04:19.520 --> 00:04:19.710
Yeah.
39
00:04:19.710 --> 00:04:20.100
Right! Okay.
40
00:04:20.100 --> 00:04:25.120
What do you choose? You lose the left pinky. Like that's the choice that 99% of people are making.
41
00:04:25.120 --> 00:04:25.920
There's some...
42
00:04:25.920 --> 00:04:27.180
It's like, yeah, I kind of need a phone.
43
00:04:27.180 --> 00:04:34.900
Yeah, I... I don't like the way this analogy is going, but I understand where... I think I understand your point. Yes, I get it. Yeah. Okay.
44
00:04:34.900 --> 00:04:40.800
Yeah, it's this dependence. And it's like, it does... it's not even good enough to have the device. You need...
45
00:04:40.800 --> 00:04:40.800
Right.
46
00:04:40.800 --> 00:04:44.920
Like the network. Right. You need the connectivity.
47
00:04:44.920 --> 00:04:49.040
Yeah. You know, teleport me back to 1970 with the top of the line phone and it's useless.
48
00:04:49.040 --> 00:04:49.240
Right.
49
00:04:49.240 --> 00:04:49.900
It's an interesting...
50
00:04:49.900 --> 00:04:52.620
Right.
51
00:04:52.620 --> 00:04:57.160
yeah, device, but it doesn't really... yeah.
52
00:04:57.160 --> 00:05:06.200
So we have this digital dependency, right? And it comes in many forms. Like AI models are the most recent form of it, you know, cloud services.
53
00:05:06.200 --> 00:05:07.020
Right.
54
00:05:07.020 --> 00:05:13.980
All of these, like, you know, your Google account, right, with your email and everything else. And you've got probably a lot of identity tied up in that.
55
00:05:13.980 --> 00:05:14.390
Yeah.
56
00:05:14.390 --> 00:05:15.240
Oh my gosh.
57
00:05:15.240 --> 00:05:32.110
Shit, I have a story about that, right? Like, my vanity domain godbolt.org is a Google Apps for Your Domain, when that was a thing. That was like a thing that if you were a Google fanboy like I was, and I was an early adopter of everything, you could say, hey, I would like to bring my own domain to a Gmail account effectively.
58
00:05:32.110 --> 00:05:32.640
Mm-hmm.
59
00:05:32.640 --> 00:05:42.720
Subsequently, when I worked at Google, I discovered that that is not how it works. There's a massive internal hack to make that work. And then it sort of became Google Apps Enterprise-y type thing.
60
00:05:42.720 --> 00:05:55.620
But unfortunately, those who early adopted, like I did, are stuck in this... it's this hinterland between it's not really an enterprise account and it's also not a Gmail account. So I can't like do half the things in my house that are internet connected with my godbolt.org account.
61
00:05:55.620 --> 00:06:06.080
And so I want to just move. And I can't just move because so many things are tied into my login identity as matt@godbolt.org that goes through like the Google login process, that now I can't... I'm stuck.
62
00:06:06.080 --> 00:06:09.520
So I'm like... what do I do? But yeah, sorry, I interrupted.
63
00:06:09.520 --> 00:06:09.800
Yeah.
64
00:06:09.800 --> 00:06:16.140
You were in the middle of the flow of talking about other digital dependencies and...
65
00:06:16.140 --> 00:06:32.080
Yeah. No, well, I mean, that's another great example, right? And so like, I don't think it's realistic to try to eliminate all of these things. But to the extent that you can be...
66
00:06:32.080 --> 00:06:47.940
more resilient to some other entity just unilaterally deciding maybe, in a way where they think you're acting in your best interest, like my friend in his AWS account, maybe.
67
00:06:47.940 --> 00:06:48.420
Right.
68
00:06:48.420 --> 00:06:50.890
Maybe it was. Or not.
69
00:06:50.890 --> 00:06:51.040
Yeah.
70
00:06:51.040 --> 00:06:57.740
It's sort of like, you don't know, and you don't really have a way to know, and you don't have a way to like validate what it is that they're doing and be like, actually that...
71
00:06:57.740 --> 00:06:58.280
I know.
72
00:06:58.280 --> 00:07:08.690
It's not how that works. Like, to the extent that you can limit that liability, it sometimes makes a lot of sense. And so like, you know.
73
00:07:08.690 --> 00:07:08.980
Right.
74
00:07:08.980 --> 00:07:23.990
I feel like open source was like a great example of this, where you know I certainly remember early on in my career the pain of having to deal with, you know, third-party software vendors, and you would try to understand how their software worked.
75
00:07:23.990 --> 00:07:35.680
And they would give you answers that were clearly wrong. And then, you know, you maybe tried to dig into it a little bit. And then, you know, I had the experience very early on in my career of like, you know.
76
00:07:35.680 --> 00:07:50.980
I don't even think that it was any more complicated than like reading through like the scripts they had sent us. But they were like, don't modify those. Right. And I'm like, okay, I'm not modifying them.
77
00:07:50.980 --> 00:07:58.340
I'm just reading them. Like I'm trying to understand like this thing that you sent me that we paid a bunch of money for that doesn't seem to work. And like, God forbid, if I had actually tried to like reverse engineer any of their software, right?
78
00:07:58.340 --> 00:08:02.830
I can't imagine what their reaction would have been to that. Like maybe some kind of lawsuit.
79
00:08:02.830 --> 00:08:03.180
Yeah.
80
00:08:03.180 --> 00:08:13.980
But... like you're stuck in this place where you're like, you know. I don't understand how this works. I'm not empowered to change it. And yet I am dependent on it.
81
00:08:13.980 --> 00:08:14.980
Right?
82
00:08:14.980 --> 00:08:43.320
Empowered is the word, the active word. That's really interesting. Yeah. It occurred to me that like another thing, it was like early Google search engine optimization once Google became the monopoly. Like, you know, you had this thing where an algorithm would change and people's would drop off of the listings, and again it's like, but they couldn't choose that. That's not something they could choose to be in or out. They were like dependent on something, but not because they chose it. Which is even... but yeah, the empowerment is an interesting
83
00:08:43.320 --> 00:08:57.600
aspect of... I do wonder, yeah, so what can we do about this, Ben? Well, I mean, I'm kind of stuck, right? I mean, what... I'll have my own internet search, I'll just, you know, my own data center, my own... yeah, what. Yeah.
84
00:08:57.600 --> 00:09:04.880
Right. Mesh network internet for everybody. That's the solution. Is we're all just going to get some powerful antennas and put them on the top of our houses and mesh all of the houses together.
85
00:09:04.880 --> 00:09:06.760
Don't tell me. Yeah, mesh.
86
00:09:06.760 --> 00:09:07.420
And that's how the internet's going to work.
87
00:09:07.420 --> 00:09:12.720
It's going to end up in some crypto scam though, isn't it? At some point, like some...
88
00:09:12.720 --> 00:09:13.080
Yeah. At some point, yeah, there's like, you know.
89
00:09:13.080 --> 00:09:17.220
Anything distributed nowadays ends up in some... Well, there's a... we can make a chain, some kind of...
90
00:09:17.220 --> 00:09:19.190
Right. There's a coin. You got to buy the coin.
91
00:09:19.190 --> 00:09:19.740
Hey there.
92
00:09:19.740 --> 00:09:20.850
And we're issuing the coins.
93
00:09:20.850 --> 00:09:21.640
You got AI coin.
94
00:09:21.640 --> 00:09:23.210
And who gets the money from these coins?
95
00:09:23.210 --> 00:09:23.410
Ugh.
96
00:09:23.410 --> 00:09:23.800
Yes, exactly.
97
00:09:23.800 --> 00:09:24.390
Oh, token coin.
98
00:09:24.390 --> 00:09:24.590
Yeah.
99
00:09:24.590 --> 00:09:26.360
I'm sure it's been done. I'm sure...
100
00:09:26.360 --> 00:09:26.540
a billion times at this point.
101
00:09:26.540 --> 00:09:28.340
It has already been done.
102
00:09:28.340 --> 00:09:33.060
The terrible idea that launched a thousand slide decks.
103
00:09:33.060 --> 00:09:35.400
Yeah.
104
00:09:35.400 --> 00:09:49.880
So, you know, like... I think it sort of depends on how far down the sort of, you know, crazy tinfoil hat, you know, internet libertarian, you know, route that you want to go.
105
00:09:49.880 --> 00:10:10.490
And I say that recognizing that like a lot of the things that people, you know, from like the Electronic Frontier Foundation and, you know, some elements of the open source community, you know, 20, 30 years ago... a lot of things that they were saying, you know, with panic in their voices would come to pass, have come to pass, and nobody seems to care.
106
00:10:10.490 --> 00:10:11.180
Right.
107
00:10:11.180 --> 00:10:17.800
Like the amount of spying that like all the devices that you pay for do on you is like off the charts.
108
00:10:17.800 --> 00:10:36.700
Somehow that's okay when... I mean, the thing is, there is... so I've always thought of it as there is some kind of implied social contract between me and... I'm going to pick on Google again just because I'm a bit of a fanboy for it, and I feel like having been there I've got some standing in there, some...
109
00:10:36.700 --> 00:10:55.180
And I've always felt that like, yes, it serves ads, yes, it reads my emails or does something with them, and I don't mind that because I get a decent email service that I don't have to pay for and it's integrated with everything. And like the slightly spooky face recognition is great when, for example, you are preparing a family
110
00:10:55.180 --> 00:10:55.180
Mm-hmm.
111
00:10:55.180 --> 00:11:12.100
board. And I'm like, hey, let's just find pictures of my father-in-law. Here's hundreds of pictures of Tony. And like, oh, that's great, right? That's kind of an amazing thing. But yeah, I do realize that I've given up something for that, but I've probably given up more than I think.
112
00:11:12.100 --> 00:11:22.200
than I think about. And I've probably given up more than I would if someone sat me down and said, well, what about these things? And I was like, oh, yeah, I hadn't really thought. Oh, that's a good point. And, you know, but I'm getting something back
113
00:11:22.200 --> 00:11:24.700
out of this.
114
00:11:24.700 --> 00:11:26.220
Mm-hmm.
115
00:11:26.220 --> 00:11:43.980
And like some of the things, though, I'm not getting something back out of it. So like, you know, automatic number plate recognition stuff, the Flock camera stuff that's going on around, you know, like there's a lot of things where it's like, yeah, that's... I'm uneasy because it is not... ironically, it's not directly a company making money. I mean, it is obviously, but it's like...
116
00:11:43.980 --> 00:12:02.380
Law enforcement, a government thing that is for the benefit of the people has decided that they would like this information and then they would be happy to pay someone to put cameras up to do all this kind of stuff. And that feels like, no, that's a step too far somehow. Now, I don't know where the boundary is there. I mean, it feels obviously... but one I'm sort of choosing.
117
00:12:02.380 --> 00:12:15.120
or at least choosing... not choosing, not choosing to. But then... I get so much benefit, and how will I get the same benefit without, you know, putting together a whole bunch of open source software, which is maybe where you're going with this.
118
00:12:15.120 --> 00:12:19.160
As you mentioned, the mesh network of like internet stuff.
119
00:12:19.160 --> 00:12:29.340
Well... Yeah, I mean, this is kind of what I'm saying, is it sort of depends on how far down the rabbit hole you want to go. Right. And I think.
120
00:12:29.340 --> 00:12:40.740
For me personally, first recognizing that 90... nine... I'm just going to put two nines in that, percent of people are not going to want to go down the rabbit hole at all.
121
00:12:40.740 --> 00:13:02.760
Right. Like they're going to want to do the minimum amount that they need to do to keep themselves safe and secure from like scammers and phishers and, you know, people like that. And you know, some of them don't even really care that much about it, from some of the things I've seen. But you know, that's like the minimum amount that they're going to want to put into this, and the rest of it, they're like perfectly happy to be completely dependent.
122
00:13:02.760 --> 00:13:10.380
and exposed in like a data sharing type sense. Like, I'm very happy to trade my data for free email. Right? Like, it's fine.
123
00:13:10.380 --> 00:13:11.500
Yeah. That's, I guess, both. Yeah, yeah.
124
00:13:11.500 --> 00:13:22.800
Right. Like they're very happy to trade their data for just about anything. And in fact, many of them... What was the... Was it Snow Crash that had the gargoyles in it?
125
00:13:22.800 --> 00:13:23.660
Do you know what I'm talking about?
126
00:13:23.660 --> 00:13:24.300
I...
127
00:13:24.300 --> 00:13:25.220
Did you read Snow Crash?
128
00:13:25.220 --> 00:13:27.300
I did read Snow Crash, and now I'm just trying to think about...
129
00:13:27.300 --> 00:13:29.380
I think it was, I think it was Snow Crash.
130
00:13:29.380 --> 00:13:29.380
I don't remember gargoyles. Was it?
131
00:13:29.380 --> 00:13:41.020
that had basically like there was a whole profession where you just walked around with a bunch of like data gathering equipment strapped to your body.
132
00:13:41.020 --> 00:13:41.240
Ah.
133
00:13:41.240 --> 00:13:46.690
right like cameras and like infrared and like, you know, microphones and all these things
134
00:13:46.690 --> 00:13:47.240
Right. Does ring a bell.
135
00:13:47.240 --> 00:13:52.620
And like that was your job. You just walk... it was like you're like a little mini like Google Maps car, but you were just a person.
136
00:13:52.620 --> 00:13:54.000
Put a person doing that, yeah.
137
00:13:54.000 --> 00:14:04.300
Right. And that's what you did for, you know, for your livelihood. And they referred to these people as gargoyles, right? And I think like the great trick, it's like, you know.
138
00:14:04.300 --> 00:14:17.880
So much of what is being built is like 90s cyberpunk dystopia. Right? We're just literally using those books as like a user manual for like how to build a society.
139
00:14:17.880 --> 00:14:19.020
Yeah. Oh. Yeah.
140
00:14:19.020 --> 00:14:25.070
And I feel like one of the great tricks of social media is basically like turning everyone into gargoyles for free.
141
00:14:25.070 --> 00:14:25.320
Now I'm sad.
142
00:14:25.320 --> 00:14:32.600
Like you don't even have to pay them. Right. Like you just walk around and you say like, hey, you should post this to your Instagram, and then people take pictures of everything around them all the time.
143
00:14:32.600 --> 00:14:32.600
Yeah.
144
00:14:32.600 --> 00:14:41.160
and just post it with, you know, geotags on it to social media, and then it's like, all right, cool. We've done it. We've gathered all the data that is.
145
00:14:41.160 --> 00:14:42.140
That was... I remember like,
146
00:14:42.140 --> 00:14:43.480
We didn't even have to pay for it.
147
00:14:43.480 --> 00:14:59.410
the like the surveillance things that like found all of the photographs of like a monument somewhere in Paris or something like that, we're able to reconstruct a full high detailed 3D model from like just thousands of tourist photographs of it, for example, that they've posted to their Instagram. So yeah, we're in that world, aren't we? Yeah, gosh.
148
00:14:59.410 --> 00:14:59.900
There you go.
149
00:14:59.900 --> 00:15:13.180
Yeah. So my point is that 99% of the people, maybe 95, I don't know, some very high percentage of the people out there are not going to be interested in going down this rabbit hole at all.
150
00:15:13.180 --> 00:15:25.000
But... I think it is important. Personally, I think it is important to be able to defend the ability to do it if you want. Because A, it just might make you sleep better at night.
151
00:15:25.000 --> 00:15:35.140
Like, I know a lot of anxious people. I know a lot of paranoid people. And they're just like, that's who they are. And they just want to know that these things are going to be safe and secure. And then they reduce their dependency as much as possible.
152
00:15:35.140 --> 00:15:35.480
Right, fair.
153
00:15:35.480 --> 00:15:37.100
And they just want to be able to do that.
154
00:15:37.100 --> 00:15:48.680
And I think that you should. And also, there are lots of situations in which... you know... people who... maybe...
155
00:15:48.680 --> 00:16:01.180
in situations where they can be easily exploited by governments, by other people, need to be able to keep themselves safe and secure. And having the tools available to them to do that
156
00:16:01.180 --> 00:16:17.840
is important. Right? And you know, I don't know that I can go in and enumerate all of those situations, but I think we could certainly imagine lots of situations in which that might be not only something that someone would want to do, but be, you know, reasonably morally justified in wanting to do it.
157
00:16:17.840 --> 00:16:25.020
And so, for all of those reasons, I think it's important to be able to do this. But it is also really hard.
158
00:16:25.020 --> 00:16:31.960
Right. I mean, so like... Going back to the beginning of this, or not nearly the beginning, as you mentioned your friend's AWS account. I mean, so...
159
00:16:31.960 --> 00:16:31.960
Right.
160
00:16:31.960 --> 00:16:44.900
If they can't recover that account, do they go and buy servers and rack them? Or, you know, is there a happy end to this story, by the way, I should say?
161
00:16:44.900 --> 00:16:50.900
Right. I mean, immediately what you probably do is hope that you did everything with infrastructure as code and then try to port it all over to...
162
00:16:50.900 --> 00:16:54.400
you know, Google Cloud or... Azure, I guess. I don't know.
163
00:16:54.400 --> 00:16:55.590
Let's just switch it one...
164
00:16:55.590 --> 00:16:58.200
one problem for the same category.
165
00:16:58.200 --> 00:17:00.060
Yeah, you know. One feudal lord to another feudal lord, right?
166
00:17:00.060 --> 00:17:00.640
Feudal lord.
167
00:17:00.640 --> 00:17:02.280
Yeah. Like, okay.
168
00:17:02.280 --> 00:17:03.680
Jeff Bezos, feudal lord.
169
00:17:03.680 --> 00:17:03.740
Yeah.
170
00:17:03.740 --> 00:17:05.560
You heard it here first.
171
00:17:05.560 --> 00:17:10.940
So, you know, like that's like on day one, that's probably as good as it gets.
172
00:17:10.940 --> 00:17:11.400
Yeah.
173
00:17:11.400 --> 00:17:28.600
But I think this is why, like, you know, a lot of times, like especially in our industry, people will wind up building their own data centers, not because they strictly have to, but because, you know, when things go wrong or when they have problems or when they want to try to do things that are maybe not standard.
174
00:17:28.600 --> 00:17:37.000
They want the level of control to be able to do it, right? Like they want to be able to say like, yeah, we can put whatever hardware in these machines we want. We can, you know.
175
00:17:37.000 --> 00:17:37.200
Mm-hmm.
176
00:17:37.200 --> 00:17:48.430
configure them however we want. We can use whatever network technology that we want. We're not limited by what a third party vendor is willing to offer to us.
177
00:17:48.430 --> 00:17:48.840
Right.
178
00:17:48.840 --> 00:17:49.660
Yeah. Yeah.
179
00:17:49.660 --> 00:17:54.700
Obviously that is out of the reach of... most typical people.
180
00:17:54.700 --> 00:17:55.820
Your average.
181
00:17:55.820 --> 00:18:05.560
But I will say, I think one thing that is overlooked... you know, kind of like not that expensive
182
00:18:05.560 --> 00:18:15.220
way to get yourself a lot of digital independence is just get a cheap computer and put it on your network and leave it on all the time.
183
00:18:15.220 --> 00:18:16.960
Right.
184
00:18:16.960 --> 00:18:22.320
Great. And, you know, if you want to take that up a level, like get some remote access for it, right?
185
00:18:22.320 --> 00:18:24.850
Tailscale, something something, to make it a very... yeah, exactly, yeah.
186
00:18:24.850 --> 00:18:25.120
So you can SSH into it.
187
00:18:25.120 --> 00:18:36.100
Yeah, yeah. Most of those types of computers that you can buy use very low power, so you don't really need to worry too much about how much power they're drawing.
188
00:18:36.100 --> 00:18:41.870
They're quite... powerful in terms of like the compute that they can, you know, supply.
189
00:18:41.870 --> 00:18:42.440
Yep.
190
00:18:42.440 --> 00:18:52.310
And you will probably find that most of the things that you would want to run in some other service provider, you could run on those types of machines today.
191
00:18:52.310 --> 00:18:52.700
Yep.
192
00:18:52.700 --> 00:18:58.820
And in a weird sort of like everything is backwards and I don't understand how anything works anymore...
193
00:18:58.820 --> 00:19:06.420
The absolute trick that you can pull is, you know, use the LLM of your choice, which you are absolutely dependent on these days, for
194
00:19:06.420 --> 00:19:06.860
At least.
195
00:19:06.860 --> 00:19:14.280
to build you the software that you need to run on that computer to replace whatever it is that you're dependent on the third-party services to do.
196
00:19:14.280 --> 00:19:14.560
Put a pin in that.
197
00:19:14.560 --> 00:19:23.490
right, so if you have some like, you know, awesome note-taking app that depends on some cloud service and you're like, wow, if this ever disappeared I'd be totally screwed
198
00:19:23.490 --> 00:19:24.480
Right. Interesting.
199
00:19:24.480 --> 00:19:32.240
And you want to go down the rabbit hole, which again, 95% of people don't. Something that you could do is you could build your own.
200
00:19:32.240 --> 00:19:32.620
Yeah.
201
00:19:32.620 --> 00:19:43.060
and run it on this computer that runs in your house. And maybe, you know, have some backups that, you know, write to a USB drive or, you know, go back to the cloud and encrypt them and upload them.
202
00:19:43.060 --> 00:19:47.020
Yeah, I was going to say, that's one thing where like the cloud maybe isn't holding you to ransom unless...
203
00:19:47.020 --> 00:19:47.020
That's another way.
204
00:19:47.020 --> 00:19:54.420
Two things happen. One, you need your backup. And two, the cloud provider decides that they're not going to let you.
205
00:19:54.420 --> 00:19:55.260
Right. Right.
206
00:19:55.260 --> 00:20:00.900
So that seems like a dual uncorrelated failure or whatever we used to say.
207
00:20:00.900 --> 00:20:11.420
So, yeah, that is an interesting call to action. Now I wonder. And we have touched on this before because I'm getting a slight deja vu feeling here. But I wonder, it is...
208
00:20:11.420 --> 00:20:11.500
Right.
209
00:20:11.500 --> 00:20:26.900
The fact that we have been so lucky for so long, like 20, 30 years in the sort of software profession that we're in, where essentially we've always been able to just, inverted commas, quotes, just build it ourself.
210
00:20:26.900 --> 00:20:34.740
And with open source, that has been free. Like, is that spoiling us? Because like if I worked in the manufacturing industry and I made grommets
211
00:20:34.740 --> 00:21:12.380
on a machine, and then the vendor that made the machine said, no, you can't use this machine anymore, that's a real problem that many grommet manufacturers may have always had, and they might have to switch to another provider at short notice because they've gone out of business and now they can't get the parts to repair them. So there's a little bit of like a, maybe, you know, other people have always had to deal with this dependence. It's like, unless you want to become a core... your core competency is building lathes to make grommets, as well as the grommets themselves, then you've always had to outsource some significant part of your endeavor. Whereas you and I have been always like, oh, an email client, yeah, I can't be bothered to write an email client, but I...
212
00:21:12.380 --> 00:21:22.660
could in principle. I feel like I could. Is it... are we just spoiled? And then does an LLM now move us back to where we were to start with, you know?
213
00:21:22.660 --> 00:21:32.760
Yeah, I think a lot of the difference there is that once you've bought the grommets, they're yours. Right. The company who built the grommets never shows up and says, your license to these grommets has expired.
214
00:21:32.760 --> 00:21:34.060
And we're going to go into your...
215
00:21:34.060 --> 00:21:34.060
Or the lathe.
216
00:21:34.060 --> 00:21:37.580
Maybe in that instance for the grommet manufacturer. But yeah, either way, both of those things are true.
217
00:21:37.580 --> 00:21:37.780
Yeah.
218
00:21:37.780 --> 00:21:37.980
Right.
219
00:21:37.980 --> 00:21:43.460
Like it is possible to own the thing and it's yours.
220
00:21:43.460 --> 00:21:44.590
It's really interesting. I mean, obviously, again,
221
00:21:44.590 --> 00:21:44.800
right?
222
00:21:44.800 --> 00:21:50.180
And it's kind of like servers, right? Like you buy a server. You own the server.
223
00:21:50.180 --> 00:21:50.920
Yeah.
224
00:21:50.920 --> 00:21:56.840
Modulo maybe some weird things that happen when you ask LLMs about how those chips work.
225
00:21:56.840 --> 00:21:57.300
Ah. Ha ha!
226
00:21:57.300 --> 00:22:01.880
You know, you can do pretty much whatever you want with them.
227
00:22:01.880 --> 00:22:02.700
Yeah.
228
00:22:02.700 --> 00:22:13.100
And like, you know, you want to screw them up and break them to experiment, go ahead and do that, right? Like there will be no ramifications other than the fact that you lost however much money you spent on the server.
229
00:22:13.100 --> 00:22:13.500
Yeah.
230
00:22:13.500 --> 00:22:21.680
Right? So I think that is sort of like the fundamental difference. And some of it gets back to like software as a service type models where it's like, you know.
231
00:22:21.680 --> 00:22:25.880
You never quite own. And, you know, like, you know, video games these days, right?
232
00:22:25.880 --> 00:22:37.020
You know, obviously our hobbies are, my hobbies are old computers, and I've got like buckets and buckets of cartridges and CDs and DVDs, even, that I can put into various old machines and they work fine, but like.
233
00:22:37.020 --> 00:22:54.800
But nowadays, if you want to play a game for like PS4, PS5, it's like you don't really own it in any meaningful way. It exists as bits on the disc, maybe it doesn't even exist on a hard disk anymore, it just exists in the cloud, which means that it goes away.
234
00:22:54.800 --> 00:22:55.120
Yeah.
235
00:22:55.120 --> 00:22:59.020
when you don't own it anymore. You're renting. You've got a license to own something, which is like
236
00:22:59.020 --> 00:22:59.670
Right, right.
237
00:22:59.670 --> 00:23:00.240
we've gone... Yeah.
238
00:23:00.240 --> 00:23:00.840
Yeah.
239
00:23:00.840 --> 00:23:11.230
I mean, you buy a book and you read the small print in the front copy about, you know, you may not, you know, give it away or whatever, or there's some level of restrictions for what you can use it for.
240
00:23:11.230 --> 00:23:12.680
and reproduce it. You know?
241
00:23:12.680 --> 00:23:24.120
But like, that seems like... those seem very reasonable compared to end user license agreements in software or whatever else. Yeah, I don't know. How did we get here? How do we let this happen?
242
00:23:24.120 --> 00:23:27.680
Well, the 90s cyberpunk set the model, and oh, we're just going to do all these things.
243
00:23:27.680 --> 00:23:27.960
Yeah.
244
00:23:27.960 --> 00:23:28.680
I mean, I don't know.
245
00:23:28.680 --> 00:23:31.360
I'm being a little tongue-in-cheek there.
246
00:23:31.360 --> 00:23:31.740
Is this Neal Stephenson's fault?
247
00:23:31.740 --> 00:23:32.040
Is that what you're saying?
248
00:23:32.040 --> 00:23:33.720
Yeah, exactly. Right.
249
00:23:33.720 --> 00:23:35.280
Oh man.
250
00:23:35.280 --> 00:23:36.580
William Gibson can be blamed.
251
00:23:36.580 --> 00:23:39.400
William Gibson. Oh, yeah, yeah, yeah.
252
00:23:39.400 --> 00:23:40.660
Is that Neuromancer as well?
253
00:23:40.660 --> 00:23:44.300
Yeah, yeah, Neuromancer, Mona Lisa Overdrive, and...
254
00:23:44.300 --> 00:23:44.670
Thank you.
255
00:23:44.670 --> 00:23:46.120
Okay, what was the middle one?
256
00:23:46.120 --> 00:23:48.600
My nerd credentials are definitely going down here.
257
00:23:48.600 --> 00:23:50.720
I don't think I've ever read Neuromancer.
258
00:23:50.720 --> 00:23:53.560
like Counter Zero, Count Zero.
259
00:23:53.560 --> 00:23:55.210
Count.
260
00:23:55.210 --> 00:23:56.830
It ends with Zero.
261
00:23:56.830 --> 00:23:57.640
Count Binface.
262
00:23:57.640 --> 00:23:59.260
Count Binface.
263
00:23:59.260 --> 00:24:02.040
Who sadly lost today at Clacton-on-Sea. I love my countrymen.
264
00:24:02.040 --> 00:24:03.900
Oh, did he? Oh man.
265
00:24:03.900 --> 00:24:05.500
Yeah, it didn't work out. 26% of the vote though.
266
00:24:05.500 --> 00:24:05.800
That's pretty good.
267
00:24:05.800 --> 00:24:07.460
It's not bad.
268
00:24:07.460 --> 00:24:12.810
I mean, the other guy, the guy who resigned and who won't be named here, did win.
269
00:24:12.810 --> 00:24:13.280
Yes.
270
00:24:13.280 --> 00:24:14.220
Nigel Binface.
271
00:24:14.220 --> 00:24:17.120
Yeah, no. Don't besmirch the Binface.
272
00:24:17.120 --> 00:24:26.530
Use that one. Yeah, I mean, it's... Yeah, you've got to love the Brits for not taking things too seriously, but still.
273
00:24:26.530 --> 00:24:26.750
yeah
274
00:24:26.750 --> 00:24:27.650
You know, I still...
275
00:24:27.650 --> 00:24:28.100
I mean...
276
00:24:28.100 --> 00:24:45.240
someone pointed out that like the people of Clacton, the place where... you know, this is real for them. So we're all treating it as like a bit of fun, but you know, and it is making some political points, but like, yeah, it's a thing. Anyway, that was not on my list of things to talk about with you today.
277
00:24:45.240 --> 00:24:50.500
But you gave me an opportunity to talk about it. So, Count... what was he saying? Count Basie?
278
00:24:50.500 --> 00:24:51.720
Count Chocula.
279
00:24:51.720 --> 00:24:53.440
No, Count... Count Zero.
280
00:24:53.440 --> 00:24:57.100
I think it's...
281
00:24:57.100 --> 00:25:10.140
The books, that's right.
282
00:25:10.140 --> 00:25:10.300
I think it's like counters. I forget what the middle book is called.
283
00:25:10.300 --> 00:25:10.300
I love to count.
284
00:25:10.300 --> 00:25:10.320
But all of those, I mean, you know, every...
285
00:25:10.320 --> 00:25:10.320
Okay, I think...
286
00:25:10.320 --> 00:25:19.110
What is it? Who's your favorite vampire? The one from Sesame Street. He doesn't count. Oh, I assure you that he does.
287
00:25:19.110 --> 00:25:22.000
That's all right. That's brilliant. I don't know how we're going to top that now.
288
00:25:22.000 --> 00:25:26.260
We've peaked. We've peaked. 20 minutes into this conversation, we peaked.
289
00:25:26.260 --> 00:25:26.560
yeah
290
00:25:26.560 --> 00:25:32.860
Oh yeah, no, we're talking about 90s cyberpunk and the dystopian future that we live in.
291
00:25:32.860 --> 00:25:39.100
Thanks. Yeah, so you were saying that the 90s... this is the blueprint. So what's coming in the future? Well, actually, no. Is there like, so.
292
00:25:39.100 --> 00:25:56.080
I think... we've talked a lot about how folks feel about LLMs. And I think, you know, you and I have both been on various rollercoaster-like ride journeys with this is great, to this is terrible, to this is I'm going to lose my job, to now I'm the most powerful programmer, to now this is what a mess I've got, and everything in between, right?
293
00:25:56.080 --> 00:26:03.790
And that's not even taking into account all of the moral and ethical aspects, which are also super important but I have sort of compartmentalized.
294
00:26:03.790 --> 00:26:04.400
okay
295
00:26:04.400 --> 00:26:14.140
But... you know, not being able to run an LLM. If, for example, tomorrow Anthropic decides that I've been a bad boy.
296
00:26:14.140 --> 00:26:24.470
or my company decided that, you know, they're not liked by Anthropic or whomever else they're providing, or whoever, you know, whatever. That's a concern.
297
00:26:24.470 --> 00:26:24.940
Yeah.
298
00:26:24.940 --> 00:26:33.600
So you mentioned about running your own stuff. Can we do that with LLMs? Do I just need a warehouse-sized computer to be able to run something worthwhile?
299
00:26:33.600 --> 00:26:46.180
Yeah, you know, I think it's arguable that the end state of all information is to be free.
300
00:26:46.180 --> 00:26:57.310
Right. You know, it's like all of these models of like copy protection where you have like a private key that only three people in the world have. It's like someone is going to figure that out. Like, ask Sony.
301
00:26:57.310 --> 00:26:57.560
Yeah.
302
00:26:57.560 --> 00:27:02.000
right, like someone is going to figure that out.
303
00:27:02.000 --> 00:27:05.450
This is not going to work.
304
00:27:05.450 --> 00:27:05.720
Yeah.
305
00:27:05.720 --> 00:27:18.980
Right. And, you know, the model weights and LLMs are just too valuable for people not to try to figure out what they are. Right. And especially when you consider the fact that there is no.
306
00:27:18.980 --> 00:27:27.860
like world government entity that can, you know, enforce any sort of protection for any of this information in any sort of like.
307
00:27:27.860 --> 00:27:28.760
Right.
308
00:27:28.760 --> 00:27:30.660
global way.
309
00:27:30.660 --> 00:27:39.310
Are model weights different though, in the same way? They're more like a compiled program. Right. If you, you know, if you imagine I give you the EXE
310
00:27:39.310 --> 00:27:39.640
Sure.
311
00:27:39.640 --> 00:27:53.520
of my code but not the source code to it. I give you the EXE and you can go away and use that program as much as you like, but you can't really do anything more with it. You can't extend it or build on it or anything like that because it's kind of like already been compiled. It's very difficult to reverse engineer.
312
00:27:53.520 --> 00:27:53.520
Yeah.
313
00:27:53.520 --> 00:28:08.940
And I feel like the weights, or you know even the open weights that you can go and get on the internet, have that flavor. It's like, sure. Here's the result of compiling, but you know, we won't, you can't buy the compiler because it's like 45 data centers full of machines that ran for like two years.
314
00:28:08.940 --> 00:28:37.540
to compile the program. And now here's the program. So, you know, the fact that the program is dressed up to me as a service is, you know, problematic. But even if they handed me the weights, I guess that's what I asked you, in fairness, but like, I'm just now thinking ahead, right? You know, you could give me the, you know, 24 terabyte disc that contains the weights to the best model and I could somehow cobble together running it, but I'd be stuck with that until the next time somebody hands me the 48 terabytes of the next model.
315
00:28:37.540 --> 00:28:38.020
Sure.
316
00:28:38.020 --> 00:28:45.880
It feels like. I don't know. I've always balked at the term open weights because it's like, open? Yes, I guess. I suppose so, but...
317
00:28:45.880 --> 00:28:46.980
Right.
318
00:28:46.980 --> 00:28:50.600
Yeah, sorry. Anyway, I cut you off. Where were you going with this?
319
00:28:50.600 --> 00:29:05.000
well no, I mean, you know, that's a fair point and that is definitely, you know, an avenue that we could explore here, is like how open are open source models, if all you're getting, if you're, yeah.
320
00:29:05.000 --> 00:29:05.950
What does it mean? I mean, because like, yeah, open source. I suppose that's it. I compare it to open source, and open source is like, hey, here's the source code. Now that might as well be alien.
321
00:29:05.950 --> 00:29:13.860
yeah
322
00:29:13.860 --> 00:29:20.780
to most people on the planet, but in principle, you can look at it and go, this was meant for us to build on and continue, and in fact the people who gave me this or gave me access to this, they are using this to continue and get better.
323
00:29:20.780 --> 00:29:22.630
and the next version of the product will be built.
324
00:29:22.630 --> 00:29:22.780
Mm-hmm.
325
00:29:22.780 --> 00:29:28.520
And it's most likely, you know, obviously there are edge cases to these. Whereas the weights feels like so derived.
326
00:29:28.520 --> 00:29:37.940
It's not really the cool and interesting bit of it from the point of view of continuing and being a builder and not being locked into this brain forevermore.
327
00:29:37.940 --> 00:29:38.340
Yeah.
328
00:29:38.340 --> 00:29:43.520
I wonder what... but then the resources are so, so massive that I can't imagine.
329
00:29:43.520 --> 00:29:44.420
Yeah.
330
00:29:44.420 --> 00:29:53.980
that you could do much with it. But at least you could, right? So in principle... And have you tried this, by the way? Have you experimented with local models or anything?
331
00:29:53.980 --> 00:30:08.060
Running local models? No, it's been on my list of things. It's like... there's going to be a tipping point probably in the next 18 to 24 months for me where it's like, part of my day job is probably going to be figuring this out.
332
00:30:08.060 --> 00:30:15.800
And so like, I'm kind of just waiting for that to happen because then it'll be like more of a focus, right?
333
00:30:15.800 --> 00:30:15.940
Woof!
334
00:30:15.940 --> 00:30:21.040
You know what I mean? Like, it's not just like something I'm monkeying around with. It's like, oh no, I need to make this work for real.
335
00:30:21.040 --> 00:30:22.420
You actually have to understand it.
336
00:30:22.420 --> 00:30:24.100
To be able to do it, yeah.
337
00:30:24.100 --> 00:30:26.360
Yeah. You know, and like, the world will be completely different in 24 months.
338
00:30:26.360 --> 00:30:26.700
What?
339
00:30:26.700 --> 00:30:31.560
Technologies will be different. Everything will be different. So I might as well figure it out for real when I need to figure it out for real.
340
00:30:31.560 --> 00:30:48.040
Yeah. Yeah, it's interesting. I suppose... I mean, we're, sorry, excuse the dog. We're meandering all over here, but like, you know, like even getting the equipment to run these things on, obviously one can find GPUs around or, you know, in the house I have a few around, but they're not really up to scratch for this kind of stuff.
341
00:30:48.040 --> 00:30:48.230
Mm-hmm.
342
00:30:48.230 --> 00:31:05.300
And then, you know, like I know from my own day job that sourcing, you know, top end GPUs and the RAM that goes with them is incredibly difficult right now, and I wonder if that's going to change. But one can only hope that there are some advances in the way that we...
343
00:31:05.300 --> 00:31:05.730
right
344
00:31:05.730 --> 00:31:11.220
evaluate, not evaluate, to make better how we run inference or whatever
345
00:31:11.220 --> 00:31:12.400
Sure, yeah
346
00:31:12.400 --> 00:31:26.600
on hardware, because GPUs are designed well for graphics. I still feel sad for rows and rows of graphic processing units in racks that have never had a video display plugged into them ever.
347
00:31:26.600 --> 00:31:26.700
Right.
348
00:31:26.700 --> 00:31:37.590
They've never generated a frame of a game, which is the point of a GPU, in my opinion. But, you know, it makes me sad. It's like battery-farmed hens, you know. It's like, oh, God, it's not right. Be free. Do what you're supposed to do.
349
00:31:37.590 --> 00:31:38.060
Yes.
350
00:31:38.060 --> 00:31:40.460
I need some free range GPUs, please.
351
00:31:40.460 --> 00:31:44.390
Yeah, well, I've got a drawer of... I've got a Voodoo 2 in a drawer somewhere.
352
00:31:44.390 --> 00:31:44.640
I wouldn't.
353
00:31:44.640 --> 00:31:45.680
I think that'll help, right?
354
00:31:45.680 --> 00:31:49.140
We feed them a steady diet of Doom Eternal and...
355
00:31:49.140 --> 00:31:55.320
Oh. That's too new for my GPUs, more like VGA adapters with a bit.
356
00:31:55.320 --> 00:31:56.440
Yeah. Uh-huh.
357
00:31:56.440 --> 00:31:57.220
Yeah, anyway.
358
00:31:57.220 --> 00:32:11.110
Yeah, no, there's, I mean, well, and this is kind of what I'm saying about the, you know, the information wants to be free argument on models is like, yes, it does take a lot of resources to train them and generate those weights.
359
00:32:11.110 --> 00:32:12.020
Right.
360
00:32:12.020 --> 00:32:23.450
But once you've done it... like, they're not going to disappear. And imagine a world where there is an open source
361
00:32:23.450 --> 00:32:29.460
agentic coding model that is better than, say, Fable 5. Right. Like.
362
00:32:29.460 --> 00:32:31.020
Right.
363
00:32:31.020 --> 00:32:45.700
That is probably going to happen at some point. Right? Like. Do you maybe have some situations in which you want to use a better one? Sure. But if you're just trying to like vibe code the notes app to run on your personal server, to like, it was like.
364
00:32:45.700 --> 00:32:46.020
You don't really.
365
00:32:46.020 --> 00:32:48.900
That model will be more than sufficient, right?
366
00:32:48.900 --> 00:33:01.220
All right. And I can see a world in which that kind of open... yeah. The thing that I'm interested in, and now this is going to be part two of a massive thing, is that like, you know, after the base training of a model, like where you've just fed
367
00:33:01.220 --> 00:33:15.940
some subset of all human knowledge in through however you've acquired it, which is its own massive asterisk and footnote and ethical dilemma, but you've got some amount of knowledge, you train a neural net on it, and then there are...
368
00:33:15.940 --> 00:33:35.000
so much post-training that happens to like make it valuable, make it aligned, make it safe. And then tuning after that to make it, you know, like, hey, don't write code that's terrible. And that's the bit I'd be interested in. I'd like to be able to go like, hey, I don't want to have to explain in text over and over again to a model that...
369
00:33:35.000 --> 00:33:38.830
Don't write comments like this, for God's sake, stop it.
370
00:33:38.830 --> 00:33:39.360
Mm-hmm.
371
00:33:39.360 --> 00:34:05.140
because you know you and I, I'm sure, have done that a million times. But I want to be able to say like, why didn't we just tune it? Can I personalize the weights so that it doesn't do that in the first place rather than me having to exhort it every single time to do it the way I want? And that feels like the last steps of an open model where you're like, okay, that's tractable. Here's the 20 terabyte intermediate model. You can add layers of tuning onto it. And then like, maybe that is something I could do. Maybe I'm just beyond, and you know, yeah.
372
00:34:05.140 --> 00:34:08.160
I have to admit I don't know nearly enough about how.
373
00:34:08.160 --> 00:34:09.580
how that process works now.
374
00:34:09.580 --> 00:34:12.120
how that process works to know whether that's a feasible thing or not.
375
00:34:12.120 --> 00:34:17.620
No, but it seems like I can imagine open companies starting that process, or you know maybe maybe this stuff exists on Hugging Face.
376
00:34:17.620 --> 00:34:17.920
Sure.
377
00:34:17.920 --> 00:34:24.220
Maybe this is the kind of thing that, you know, I would, if I spent a bit more time looking into it, I would discover it can actually be done.
378
00:34:24.220 --> 00:34:26.000
Yeah.
379
00:34:26.000 --> 00:34:34.840
But now it's interesting. And yeah, just so, and the irony here that the sort of leader in open weight, as far as I can tell, is the Chinese government.
380
00:34:34.840 --> 00:34:35.200
Right.
381
00:34:35.200 --> 00:34:35.560
Yeah.
382
00:34:35.560 --> 00:34:36.890
That seems like, wow.
383
00:34:36.890 --> 00:34:37.200
Yep.
384
00:34:37.200 --> 00:34:41.980
That was not on my bingo card for this year, is like champions of openness and freedom.
385
00:34:41.980 --> 00:34:44.300
Yeah.
386
00:34:44.300 --> 00:34:47.200
You know, I'm sure you can find a, as you say, a 90s sci-fi novel that...
387
00:34:47.200 --> 00:34:50.680
We need an AI company that's open. What would we call it?
388
00:34:50.680 --> 00:34:51.880
No.
389
00:34:51.880 --> 00:34:52.830
I don't.
390
00:34:52.830 --> 00:34:53.140
No.
391
00:34:53.140 --> 00:34:55.440
I just can't think of a good name.
392
00:34:55.440 --> 00:34:56.680
No, there's no... I don't think you can pick one.
393
00:34:56.680 --> 00:34:59.160
I wonder if the domain, I'll just check some domains and see if anything's available.
394
00:34:59.160 --> 00:35:00.560
Yeah, see whether you can find one.
395
00:35:00.560 --> 00:35:01.340
Yeah.
396
00:35:01.340 --> 00:35:02.380
Yeah.
397
00:35:02.380 --> 00:35:07.600
But yeah, like it's entirely possible that in the limit, the only thing that matters is GPUs, power and cooling.
398
00:35:07.600 --> 00:35:07.940
Anyway.
399
00:35:07.940 --> 00:35:19.440
Right. Because like. Models will get good enough. Their information will leak out into the world as all information does. And you will reach a point where.
400
00:35:19.440 --> 00:35:33.200
you know, you don't necessarily have to be beholden to a company to use their models as a service. You can go and either purchase or maybe just acquire for free because they're just freely available.
401
00:35:33.200 --> 00:35:33.280
Mm-hmm.
402
00:35:33.280 --> 00:35:39.080
a model that you can run on your own hardware and then you can purchase that hardware and then you own that hardware.
403
00:35:39.080 --> 00:35:39.420
Yeah.
404
00:35:39.420 --> 00:35:48.340
And no one can take it away from you. And then you could have, you know, it's like that server that's running in your house maybe it becomes a little bit more beefy and uses more power.
405
00:35:48.340 --> 00:35:51.200
Right.
406
00:35:51.200 --> 00:35:51.800
but it is yours and no one can turn it off for you other than
407
00:35:51.800 --> 00:35:55.500
But it's yours, right? And no one can turn it off and no one can tell you not to use it. And, you know.
408
00:35:55.500 --> 00:35:59.320
And you're using solar panels so that the electricity company can't cut you off.
409
00:35:59.320 --> 00:36:00.650
Right. Further and further down the rabbit hole.
410
00:36:00.650 --> 00:36:01.280
Down the rabbit hole, yeah.
411
00:36:01.280 --> 00:36:03.020
And you got the mesh internet that you're not connected.
412
00:36:03.020 --> 00:36:03.220
That's it.
413
00:36:03.220 --> 00:36:04.600
You don't have an internet service provider.
414
00:36:04.600 --> 00:36:05.360
Except...
415
00:36:05.360 --> 00:36:08.580
mesh internet that only works when I'm not inside the tinfoil wrapped house, where like, yeah, no, it is.
416
00:36:08.580 --> 00:36:11.460
Yes, exactly. Right.
417
00:36:11.460 --> 00:36:23.500
Gosh. Wow. Well, friend, we've gone round and we've covered an awful lot of things that I wasn't expecting to, but you and I haven't spoken for ages, so this is not too surprising.
418
00:36:23.500 --> 00:36:25.190
that rambling and in some ways but I think perhaps
419
00:36:25.190 --> 00:36:27.140
It's been... It's been a little bit.
420
00:36:27.140 --> 00:36:35.740
in the tinfoil wrapped house, with our own utility providing systems in place, is where we should leave this, with our own intelligent.
421
00:36:35.740 --> 00:36:35.850
Mm-hmm.
422
00:36:35.850 --> 00:36:36.180
Yes.
423
00:36:36.180 --> 00:36:48.200
Well, what I would say is about the power and cooling thing is that at least there is an existence proof of something which doesn't require that much cooling because I'm using it right now to talk with you. And I don't mean my mouth.
424
00:36:48.200 --> 00:36:56.920
I mean, but although that is the primary way in which I think, to be fair, as you've probably realized, but like, so it has to be possible to have something which has at least human intelligence that runs on, what,
425
00:36:56.920 --> 00:36:57.960
80 watts.
426
00:36:57.960 --> 00:37:10.300
100 watts, right? You know, something like that. Maybe I'm not as fast as Claude is, but like somewhere in there there's a scaling thing, so I'm sure we can make it happen along the way. And that's the hope obviously, is somebody will come up with some cool 3D
427
00:37:10.300 --> 00:37:10.300
Yeah, right.
428
00:37:10.300 --> 00:37:10.300
Yeah.
429
00:37:10.300 --> 00:37:17.000
silicon hardware thing that is vastly more efficient at computing whatever it is, but
430
00:37:17.000 --> 00:37:21.130
Right. Or just fusion power. That works too. That'd be awesome.
431
00:37:21.130 --> 00:37:22.160
Fusion.
432
00:37:22.160 --> 00:37:28.100
Yeah. We still got a lot of exhaust heat to put somewhere, which doesn't seem like the best thing to have knocking around, you know.
433
00:37:28.100 --> 00:37:28.800
Could be true. Yeah.
434
00:37:28.800 --> 00:37:32.180
So. Yeah. Anyway, with that, on that bombshell... Yeah.
435
00:37:32.180 --> 00:37:36.280
on our fusion powered tinfoil wrapped home.
436
00:37:36.280 --> 00:37:36.280
Right.
437
00:37:36.280 --> 00:37:38.960
I will see you next time, friend.
438
00:37:38.960 --> 00:37:40.960
Next time.
