We Built the Internet. Here's Why AI Can't Replace You.

WEBVTT
00:00:00.080 --> 00:00:01.840
So welcome to Computers, Coffee, and Beer.
00:00:02.080 --> 00:00:05.759
I'm Keith here in Conversation as always with Julian Verligay.
00:00:06.080 --> 00:00:09.519
And today we're going to be talking about whether you should learn to code.
00:00:09.839 --> 00:00:11.439
TLDR, the answer is yes.
00:00:11.519 --> 00:00:12.400
You still need to learn to code.
00:00:12.640 --> 00:00:14.640
We get there's these coding genies, they're amazing.
00:00:14.800 --> 00:00:17.120
You're still not going to be able to use them that well if you don't know how to code.
00:00:17.280 --> 00:00:21.280
And then how to, uh, because both of us had kind of weird on-ramps into computing.
00:00:21.440 --> 00:00:23.760
And uh we think that there's something to learn from that.
00:00:24.000 --> 00:00:27.199
So, first of all, how how did how did you learn how to code?
00:00:27.519 --> 00:00:32.960
Yeah, I have a relatively concrete like first experience with writing code memory, which because it was like when I was a little kid.
00:00:33.119 --> 00:00:34.799
So I grew up in a Marine Corps household.
00:00:34.880 --> 00:00:37.359
My dad was a career officer in the US Marine Corps.
00:00:37.439 --> 00:00:45.679
Uh, and so it's a matter of public record that in 1983, when he brought home an Apple IIe, uh that device cost 3,000 US dollars.
00:00:45.759 --> 00:00:48.880
And those are 3,0194 US dollars, right?
00:00:48.960 --> 00:00:56.719
Um, and also, since he's a public servant, I got curious about this, is also a matter of public record that his salary that year was$30,000 US dollars.
00:00:56.880 --> 00:00:58.399
So this is a meaningful investment.
00:00:58.479 --> 00:01:03.280
This was like a big, this was probably my parents probably had fights about this that I wasn't in the room for, right?
00:01:03.679 --> 00:01:07.840
And, you know, what's the Apple IIe for in 1983 if you're an artillery officer, right?
00:01:07.920 --> 00:01:09.040
Which is what he did.
00:01:10.079 --> 00:01:11.599
Not much, to be totally honest with you.
00:01:11.680 --> 00:01:16.480
Like there's a bunch of pencil and paper calculation you do, like, you know, to figure out where the shell's gonna land or whatever.
00:01:16.560 --> 00:01:18.959
And he wrote little basic programs for that stuff.
00:01:19.200 --> 00:01:21.439
But you know, it's this big thrumming piece of hardware.
00:01:21.519 --> 00:01:23.439
It's not, you know, we haven't had don't have laptops yet.
00:01:23.519 --> 00:01:27.359
The Apple IIC, the luggable Apple II, uh had not been developed yet.
00:01:27.599 --> 00:01:29.439
Um so he had this thing.
00:01:29.519 --> 00:01:32.000
And in hindsight, like he must have just sort of thought computers were cool.
00:01:32.079 --> 00:01:32.560
You know what I mean?
00:01:32.640 --> 00:01:36.079
Like in hindsight, I'd probably come by this interest in computers a little bit, honestly.
00:01:36.319 --> 00:01:37.920
But I was six years old.
00:01:38.000 --> 00:01:46.719
Um and we, like, you know, like a lot of other people, I guess, explored this device with like the books that it came with, right?
00:01:46.799 --> 00:01:56.719
So it came with sort of a little book that was like, you know, a manual about like like electronic level stuff and to some extent the 6502 and like what the memory map stuff on Apple IIe was.
00:01:56.799 --> 00:01:59.200
I was in no position to understand that, neither was my dad.
00:01:59.359 --> 00:02:01.519
But there was also this book about AppleSoft Basic.
00:02:01.840 --> 00:02:11.120
So it came with this programming language, burned in a ROM, called BASIC Beginner's All-Purpose Symbolic Instruction Code, which is one of those tortured acronyms from middle uh 20th century computing.
00:02:11.360 --> 00:02:26.319
And of course, like this was supposed to be uh a software language that was so easy that mere civilians could be able, you know, would be able to write it because uh there weren't these complicated control structures and complicated notions of modularity and all these other things mucking things up.
00:02:26.479 --> 00:02:30.240
There were just little lines of code and you could jump around the lines of code, and that was that.
00:02:30.400 --> 00:02:36.319
Um go to And so I exactly, yeah, go to and there were for loops and so there's a little bit more than that.
00:02:36.400 --> 00:02:40.879
But yeah, very close to sort of not a ton of resources there for structured programming.
00:02:41.120 --> 00:02:43.039
And there were also like enthusiast magazines, right?
00:02:43.120 --> 00:02:53.599
So there were places, there were these magazines like Byte and whatnot, where you could buy them at the newsstand, take them home in your sweaty little hands, uh, right, and and open up their glossy print.
00:02:53.759 --> 00:02:56.319
And in their pages, there'd be these programs that were supposed to do neat things.
00:02:56.400 --> 00:02:58.240
There'd be like a game or something you could play.
00:02:58.400 --> 00:02:59.919
Um, we never got one of those to work.
00:03:00.000 --> 00:03:07.039
Like we'd sort of like type it all in and just like never get it to the point where we didn't make a mistake somewhere and and not be able to run the game.
00:03:07.199 --> 00:03:10.639
Um, but it was like an experience trying to get a computer to do something.
00:03:10.719 --> 00:03:13.120
Um, and it made a kind of big impression on me.
00:03:13.280 --> 00:03:19.120
I think I didn't I don't I don't know if this is really the time that the bugging that the coding bug took for me, right?
00:03:19.199 --> 00:03:26.080
Like I didn't sort of immediately go out and seek other mentors and find other resources and learn more and like consume things at the library or something like that.
00:03:26.240 --> 00:03:27.599
Again, I was six.
00:03:27.919 --> 00:03:29.360
But it was a very memorable impression.
00:03:29.439 --> 00:03:32.080
It was kind of like a cute father-son memory as well.
00:03:32.240 --> 00:03:34.800
I think the the bug bit me for real in high school.
00:03:34.879 --> 00:03:46.479
I think that was when I uh had exposure to um to a computer actually at at my high school in Portsmouth, Rhode Island, um, PHS, Go Patriots, which had a PDP-11 still.
00:03:46.639 --> 00:03:51.599
It had a PDP-11 that was actually running RSTS, which was like DEX operating system for the PDP-11.
00:03:51.759 --> 00:03:54.879
Uh, it was also one of the early machines that ran Unix, but it wasn't running Unix there.
00:03:54.960 --> 00:03:58.159
And they had little PT100 terminals set up in the computer lab.
00:03:58.319 --> 00:04:05.039
And I could write little Pascal programs uh using a bizarre editor that I now know was Tico, one of the predecessors of Emacs.
00:04:05.360 --> 00:04:08.960
One of these bizarre editors where it's like all crazy keystrokes and stuff.
00:04:09.199 --> 00:04:14.000
Um, and that was kind of when it really the when the pill really started to dissolve into my bloodstream, I think.
00:04:14.080 --> 00:04:15.439
But how about yourself, Julian?
00:04:15.759 --> 00:04:17.839
So it's actually funny, we never talked about this.
00:04:18.160 --> 00:04:21.199
Like my story is very, very similar to yours.
00:04:21.360 --> 00:04:23.519
Uh almost identical.
00:04:23.759 --> 00:04:48.240
So uh my parents bought an Amstrad back in the days, and it came with a book, and in this book there was BASIC uh and how to program in Basic with little games at the end of the book that I tried to copy on my computer and never quite got them to run, but actually forced me to debug what was going on, and that's when I started to make sense of what the program was doing.
00:04:48.319 --> 00:04:54.319
But I was a little bit older, I wasn't six years old, I was I would say seven or eight or something like that, but still very young.
00:04:54.959 --> 00:05:02.000
And but same thing back then what I was really trying to do was to get that thing to play a video game for me.
00:05:02.319 --> 00:05:05.279
So I wasn't really interested in the code whatsoever.
00:05:05.519 --> 00:05:11.839
And then in high school I learned Pascal when I was like yeah, 14, something like that.
00:05:12.000 --> 00:05:17.519
And then that's when I really got more interested in true software and and things got more serious.
00:05:17.839 --> 00:05:22.399
But I would say, and I think that's probably different from you, so then I learned C.
00:05:23.519 --> 00:05:24.639
But I would say the revelation.
00:05:25.199 --> 00:05:27.199
Same so far, believe it or not, but yeah, I keep going.
00:05:27.439 --> 00:05:33.759
So um the revelation for me was uh scheme and functional programming.
00:05:33.920 --> 00:05:44.399
That was an eye-opening moment because I stayed in the realm of uh, you know, the classic programming languages, more more mainstream things, up until the age of 20.
00:05:44.560 --> 00:05:48.000
So I got exposed to functional programming really late.
00:05:48.160 --> 00:05:53.759
Um and then and then OPAML, that was another another one that was an another eye-opener for me.
00:05:54.000 --> 00:05:58.639
Um, but I would say what where I really caught the bug was with Pascal.
00:05:58.959 --> 00:06:00.319
This is the prescription so far, right?
00:06:00.399 --> 00:06:04.480
Start when you're six, learn learn basic to try to you know copy somebody else's program.
00:06:04.800 --> 00:06:10.480
Yeah, but I mean that's a bad prescription because today, how I mean to be honest, I don't even know what basic is.
00:06:10.879 --> 00:06:13.920
I would have never done that in today's world.
00:06:14.079 --> 00:06:14.639
Never.
00:06:14.959 --> 00:06:22.319
Because all I wanted to do was play video games, and today you can just play video games on pretty much anything you can get your hands on.
00:06:23.120 --> 00:06:25.360
Anything with a screen has games in it.
00:06:25.680 --> 00:06:40.959
So I don't think I I I'm wondering if it would be a good idea to build a device that you can give a six-year-old that only has you know the same things we had, so that they they actually have to write down the programs to get the.
00:06:46.720 --> 00:06:48.160
Maybe that's what we should do.
00:06:48.240 --> 00:06:53.680
I don't know, but I think this is not gonna fly in today's world, right?
00:06:53.759 --> 00:06:56.319
Like where video games are everywhere.
00:06:56.480 --> 00:06:59.839
Um, and so so I I don't think that's a good starting point.
00:07:00.079 --> 00:07:01.360
And then so what's your take?
00:07:01.519 --> 00:07:04.240
So have you ever been exposed to functional programming?
00:07:04.319 --> 00:07:06.800
Or when how when did that happen?
00:07:07.199 --> 00:07:10.480
In 1998, I spent one year in Oregon.
00:07:10.560 --> 00:07:12.720
And I was at a place called Oregon Graduate Institute.
00:07:12.959 --> 00:07:17.759
Oregon Graduate Institute is this bizarre grad school that used to exist, it doesn't exist anymore.
00:07:17.920 --> 00:07:27.040
Um, for that Nike and Intel dreamed up, because those are the two major sort of consumers of graduate degree employees in in Oregon.
00:07:27.279 --> 00:07:34.000
And so it has an exercise science department, an exercise physiology department for Nike, and a computer science department for Intel and nothing else.
00:07:34.160 --> 00:07:36.879
And it grants PhDs, Oregon Graduate Institute, right?
00:07:37.040 --> 00:07:40.720
So I went there and there was um not that big a CS department.
00:07:40.879 --> 00:07:43.680
One chunk of the CS department was busy doing some system stuff.
00:07:43.759 --> 00:07:50.079
There's a project called Quasar by Dylan McNamy that was oriented around like doing quality of service stuff in in operating systems.
00:07:50.160 --> 00:07:52.639
So it was like hacking up the Linux kernel so you could play video.
00:07:52.800 --> 00:08:00.639
By the way, the the mid-1990s to late 1990s, it's still arguably a research topic how you like reliably play video on a personal computer.
00:08:00.959 --> 00:08:07.360
It's still sort of a little bit like, yeah, it works a little bit, but the sound and the video get chunked up and you like drop frames and whatever.
00:08:07.680 --> 00:08:10.639
So making something reliable there was like still kind of a research thing.
00:08:10.879 --> 00:08:15.920
I got started porting uh Linux to a machine called the i960 until it had this process.
00:08:16.399 --> 00:08:19.120
When the FFM peg came out, do you know?
00:08:19.519 --> 00:08:21.279
Oh no, I don't off the top of my head.
00:08:21.360 --> 00:08:21.600
Do you?
00:08:21.839 --> 00:08:22.800
Because um, no.
00:08:22.959 --> 00:08:27.600
I it was written by Mid Ach, right, who's one of my favorite uh programmers.
00:08:27.680 --> 00:08:28.959
But anyway, keep keep on going.
00:08:29.199 --> 00:08:33.679
It was one of those crazy things where it's like so many things had to happen before YouTube could have happened, you know what I mean?
00:08:33.840 --> 00:08:36.879
One of them was just like this had to be commoditized that you can just play video, right?
00:08:37.120 --> 00:08:38.559
It hadn't happened yet anyway.
00:08:38.799 --> 00:08:45.120
So we were uh I I was doing a research project where I was porting Linux kernel to this new CPU called the I-960.
00:08:45.200 --> 00:08:48.559
The I960 is like a weird risk processor that Intel dreamed up on.
00:08:48.879 --> 00:08:50.320
Not super important right now.
00:08:50.559 --> 00:08:54.960
But across the my cubicle barrier was a bunch of people working on GHC.
00:08:55.039 --> 00:08:59.279
Like by by sheer coincidence, OGI had a bunch of Haskell people in it.
00:08:59.519 --> 00:09:02.159
And Oregon, by the way, still has a bunch of Haskell people in it.
00:09:02.559 --> 00:09:12.320
A lot of the people that were in that office with me ended up working on like security enhanced L4 and like other kinds of, you know, formal methods y type stuff that's that's sort of downstream of the Haskell mindframe.
00:09:12.480 --> 00:09:22.000
Um, so you know, I I was busy kind of being one of these, you know, dirty, grubby, let's write a million miles of C code and debug it later, kind of people that I was at this time in my life.
00:09:22.240 --> 00:09:28.320
And then there are these other people right next to me having this very different experience where it seemed like once they got their program to compile, it worked.
00:09:28.480 --> 00:09:30.240
And I was like, well, tell me more about this.
00:09:30.559 --> 00:09:30.720
Yeah.
00:09:30.879 --> 00:09:33.440
To ask them how much memory it's allocated.
00:09:36.080 --> 00:09:39.200
So actually, for early stories, just a small parenthesis.
00:09:39.360 --> 00:09:50.879
So while while I was uh at Facebook, you know, um I was one of the few uh writing functional code back in the days when Facebook was still small, like 2010, something like that.
00:09:51.039 --> 00:09:54.720
And so all the interviews for functional programmers were sent to me.
00:09:54.960 --> 00:09:58.799
And so that meant all the Haskell developers were sent to me.
00:09:59.120 --> 00:10:16.080
And um, so you so I I've seen a lot of funny behavior with with uh with uh developers, because some of them basically they pretended to know to know Haskell, but it was just a show off, and really they didn't know that much.
00:10:16.240 --> 00:10:20.879
So some of them would ask, um, is it okay if I finish it if I write this in Haskell?
00:10:21.039 --> 00:10:22.240
I said, sure, go ahead.
00:10:22.399 --> 00:10:24.559
And then they'll go back yourself out, buddy.
00:10:24.720 --> 00:10:26.720
Well, uh become what you wish for.
00:10:26.960 --> 00:10:31.200
Then you'll switch to by you know so you you know I used to do this this go ahead.
00:10:31.759 --> 00:10:37.120
So you had the people who are uh faking it, like they pretend they know Haskell, but really they don't.
00:10:37.360 --> 00:10:40.799
And then there are those who actually know Haskell and who did a good job.
00:10:40.960 --> 00:10:47.759
But then at the end, my favorite question was uh was always like, so you ask them first, you know, what's the what's the complexity?
00:10:48.000 --> 00:10:49.279
They do a really good job.
00:10:49.759 --> 00:10:53.679
And then you go like, what's the complexity in space?
00:10:54.399 --> 00:11:00.799
And then you watch them, you know, sweat and go like, uh, how much does this actually allocate?
00:11:00.960 --> 00:11:01.519
Do you know?
00:11:01.679 --> 00:11:02.159
Right, right.
00:11:02.320 --> 00:11:14.559
And and so that was so I mean one big caveat with Haskell is well, which you don't have Widow Campbell, so that's my rant against Haskell, and Haskell people are gonna hate me for it, but that's okay.
00:11:14.639 --> 00:11:15.759
I can take the hate.
00:11:16.559 --> 00:11:27.360
The the one thing that you know, when you whenever you say the program just works, yes, but you don't know if it's going to terminate or how much memory it's going to allocate.
00:11:27.840 --> 00:11:31.120
If you don't care about these two things, then fine.
00:11:31.279 --> 00:11:33.519
You know, whenever it compiles, it actually works.
00:11:33.679 --> 00:11:38.720
The main difference is that you can actually debug O'Campbell, like you debug C.
00:11:38.879 --> 00:11:39.600
You know, there's no problem.
00:11:39.919 --> 00:11:41.759
Okay, well, let's uh this is what Hashtag is.
00:11:41.919 --> 00:11:43.360
This is a rabbit shell that's worth exploring.
00:11:43.759 --> 00:11:44.080
Yeah, yeah.
00:11:44.240 --> 00:11:45.519
So those are rabbit holes worth exploring.
00:11:45.600 --> 00:11:46.480
Is that just laziness?
00:11:46.559 --> 00:11:48.240
Do you think it's like the fact that Haskell's lazy?
00:11:48.399 --> 00:11:49.120
It uh that doesn't matter.
00:11:49.440 --> 00:11:50.240
Definitely laziness.
00:11:50.559 --> 00:11:51.840
Laziness by default.
00:11:52.240 --> 00:11:54.480
The fact that there's laziness that's totally fine.
00:11:54.639 --> 00:11:55.919
You can have laziness.
00:11:56.159 --> 00:11:59.600
The the big problem is that laziness is the default.
00:12:00.000 --> 00:12:20.000
And so they they did that because they're obsessed with expressiveness of the language, and from the perspective of uh theoretical computer scientists, expressiveness is de defined as how many lambda terms actually normalize.
00:12:20.320 --> 00:12:30.159
And in that world, well, the the evaluation strategy that makes the most term normalize is the lazy evaluation strategy called by me.
00:12:30.879 --> 00:12:36.240
So that's that's what it boils down to, but it's a completely impractical choice.
00:12:36.639 --> 00:12:42.639
And so my favorite rant about this is actually uh the one from uh Greg Morriset.
00:12:42.720 --> 00:12:45.120
So I I I spoke about him in another episode.
00:12:45.200 --> 00:12:50.240
He's one of my favorite guys, and so he, you know, he was ranting about Haskell.
00:12:50.320 --> 00:12:56.399
He's saying, so the Haskell people they're obsessed with having everything in the type, right?
00:12:56.720 --> 00:13:06.240
So they don't want you to be able to have a function write to a file or log something, because you know, it has to be part of the type.
00:13:06.320 --> 00:13:11.360
It has to be a monad that shows that, whoa, whoa, this function could write to a file.
00:13:11.440 --> 00:13:14.159
That we really need this in the type, right?
00:13:14.559 --> 00:13:19.200
And so these are people who are really obsessed with having all the information in the type.
00:13:20.240 --> 00:13:26.159
But the one thing they don't seem to care about is is this value actually a value?
00:13:26.960 --> 00:13:29.360
They don't care about that, that's totally fine.
00:13:29.519 --> 00:13:32.000
And when you think about it, it doesn't make any sense.
00:13:32.480 --> 00:13:37.679
If laziness was not by default, then you would see it in the type.
00:13:37.919 --> 00:13:45.120
You would see in the type that it's a lazy thing, which means that when you try to force it, maybe it won't terminate.
00:13:45.440 --> 00:13:48.639
You know, something could happen, an exception could be thrown, whatever.
00:13:48.720 --> 00:13:53.919
So the type would be more precise if laziness was not the default, right?
00:13:54.480 --> 00:13:58.960
So those are just the kind of like terms that that might be a little bit more democratically accessible these days.
00:13:59.039 --> 00:14:04.720
It's as if sort of everything were a future or a promise or something, uh in as if like that were the default thing.
00:14:05.039 --> 00:14:05.600
Yes, exactly.
00:14:07.039 --> 00:14:08.000
Yeah, that's right.
00:14:08.159 --> 00:14:15.759
So yeah, it's a so so for the for those who come from JavaScript, it's as if every single value in your language was a promise.
00:14:16.399 --> 00:14:23.919
And then whatever you want to do with it, it's whatever whatever you compute, whenever you compute anything, you create a promise.
00:14:24.159 --> 00:14:30.720
So try to imagine what is actually going down when you execute that thing and how much you actually allocate.
00:14:30.960 --> 00:14:35.919
Um and the way this thing is actually computed is a graph reduction algorithm.
00:14:36.080 --> 00:14:37.600
So it doesn't use a stack.
00:14:37.759 --> 00:14:45.360
So you know, we like to think about program executing line by line, and then that's how we like to debug, right?
00:14:45.440 --> 00:14:50.799
We will print before, print after, or use GDB if you don't use printf debugging.
00:14:50.960 --> 00:14:53.600
You know, that's you're not supposed to, but whatever.
00:14:53.840 --> 00:14:55.039
We're old school.
00:14:55.279 --> 00:15:05.840
Um, and and you're not able to do that because the whole thing is a graph reduction algorithm that could, you know, evaluate things in an order that you you you you don't understand.
00:15:06.159 --> 00:15:09.600
And so for me, that's absolutely it.
00:15:09.759 --> 00:15:17.279
What what was the the one big bad design decision that Haskell made was laziness by default.
00:15:17.679 --> 00:15:28.480
I I'd had this experience in X98 where it seemed like there were these like people who were way smarter than me and were saying stuff about category theory I didn't understand at all, having this amazing programming experience writing Haskell.
00:15:28.639 --> 00:15:33.039
I started messing around with it a little bit, and it's sort of in a toy context, it does that, right?
00:15:33.120 --> 00:15:39.919
Like if you're taking a physics textbook and you're trying to make a little physics package that like embodies the stuff in the textbook, you write these little Haskell expressions.
00:15:40.000 --> 00:15:45.919
They're kind of like taking a little tech from the physics textbook and making it Haskell, and it like kind of works, you know, and you're like, wow, this is kind of wild.
00:15:46.000 --> 00:15:48.720
This is really different than my C life or whatever, right?
00:15:49.039 --> 00:15:52.879
And um, yeah, but nobody wants C life, you know?
00:15:53.679 --> 00:15:56.559
Yeah, well, uh that's another like we'll get we'll get to the C part of things.
00:15:56.879 --> 00:16:00.559
I mean, when you're at the bottom of the barrel, you know, anything better.
00:16:00.720 --> 00:16:08.960
But to be clear, like I always identified as a C programmer first and a C programmer by like sort of professional necessity or whatever, but hang on for a second.
00:16:09.279 --> 00:16:18.000
Um we uh when I was at VMware, we were working on this system called V probes, and it was basically a passive instrumentation, kind of machine-level instrumentation system, right?
00:16:18.080 --> 00:16:21.519
So the idea is you've got this, like you're running Windows or something in a VM.
00:16:21.759 --> 00:16:28.000
You want to be able to say like how often did I do this kind of system call in Windows and you know which processes are running and blah, blah, blah.
00:16:28.159 --> 00:16:32.399
And we basically built this little framework that could like safely instrument Windows underneath it.
00:16:32.480 --> 00:16:33.600
We called it v probes.
00:16:33.759 --> 00:16:35.519
We needed it to be like a little bit programmable.
00:16:35.600 --> 00:16:38.159
We need it to be sort of uh not Turing complete, right?
00:16:38.240 --> 00:16:43.519
Because we didn't want you to run an infinite loop in the middle of you know an interrupt handler in Windows or something and break your VM.
00:16:43.679 --> 00:16:45.120
We want to be reasonably safe.
00:16:45.360 --> 00:16:58.399
So we did this kind of thing that in hindsight, I didn't know uh about VPF and these other systems that sort of build this, but it's like very similar to a Berkeley packet filter and these other systems that have similar constraints, where there's this little tiny VM, it can like run your little bytecodes.
00:16:58.559 --> 00:17:07.039
Um, you know, you only have so many ticks of the of the virtual machine clock before we time you out and so on, and then a few primitives that make it easy to like gather stats and a couple of other things, right?
00:17:07.119 --> 00:17:09.440
So we built this thing, we need to be able to program it a little bit.
00:17:09.519 --> 00:17:18.079
And the way that we decided to program it a little bit was going to be this like subset of C that we were that I was calling Emmet at the time, um which is my first son's name, by the way.
00:17:18.240 --> 00:17:22.960
And the compiler for Emmet needed to be like needed to basically include the type system of C.
00:17:23.039 --> 00:17:28.480
We wanted to be able to pound include like kernel header files, for instance, so that we know what a proc structure looked like and things like that.
00:17:28.799 --> 00:17:32.400
And but then the actual instrumentation code had to be something else, right?
00:17:32.640 --> 00:17:37.200
And so it was kind of like this mild, you know, good-sized compiler project in a way, right?
00:17:37.599 --> 00:17:45.039
And uh I decided to do it with like Parsec and Haskell and all these like other like night, you know, power tools that that community has for language processing.
00:17:45.200 --> 00:17:47.039
And we got to work and shipped it and stuff.
00:17:47.279 --> 00:17:54.559
And then as soon as I handed that uh that project off, as soon as I like, you know, uh, I think pretty soon after that is when I started at Facebook, actually.
00:17:54.640 --> 00:18:00.319
Um, the team, you know, when I caught up for a coffee with the guy who took it over, he's just like, yeah, we just rewrote it all on OCaml, like first thing.
00:18:00.640 --> 00:18:02.400
We just and it was OCaml, by the way.
00:18:02.480 --> 00:18:04.079
It was like it's about you'd get a kick out of this.
00:18:04.160 --> 00:18:09.759
Uh so you guys you guys won one, at least like in in my limited footprint of attempts to ship Haskell.
00:18:10.000 --> 00:18:12.880
Yeah, it was more or less just like nobody could figure out what it was doing, right?
00:18:12.960 --> 00:18:15.599
Nobody could figure out like why it was so slow or why whatever was going on.
00:18:15.839 --> 00:18:16.160
Exactly.
00:18:16.319 --> 00:18:17.119
So that's the thing.
00:18:17.279 --> 00:18:29.200
So I think it's Haskell tends to produce correct code, and there are a few people who can write efficient Haskell code, but they really know the compiler inside out.
00:18:29.440 --> 00:18:36.000
And I think others who don't who write efficient Haskell code are those who are cheating.
00:18:36.079 --> 00:18:38.559
They basically write everything in a monad.
00:18:39.119 --> 00:18:46.799
And then if you do that, you're writing uh you know procedural programming using you know a functional language, so then I really don't see the point.
00:18:47.839 --> 00:18:52.000
But if you don't do that, you really need to know how the compiler is going to work things out.
00:18:52.480 --> 00:18:54.000
And it's incredibly brittle, right?
00:18:54.240 --> 00:18:59.039
Because you know, there could be a compiler optimization that kicks in, sorry.
00:18:59.440 --> 00:19:09.680
And if it does kick in, uh it's going to um in place modify a data structure and it's going to be very fast and very efficient.
00:19:10.079 --> 00:19:20.720
But now, you know, for some reason you change something that seemed completely unrelated, and this optimization optimization doesn't kick in anymore, and all of a sudden you're allocating a gate of memory, right?
00:19:21.039 --> 00:19:25.440
So it's memory that will be collected, don't get me wrong, because there's a garbage collector.
00:19:25.519 --> 00:19:28.480
But so that's my experience with Haskell.
00:19:28.640 --> 00:19:34.720
It's uh on small programs, it feels magic because it works and uh the output is good.
00:19:35.039 --> 00:19:47.279
But when you're trying to ship something professionally, managing the performance and predictably, like you know, in in a way that you know that uh is just always gonna work.
00:19:47.359 --> 00:19:52.720
Um it's reasoning about the performance in Haskell is is a nightmare, is basically what it boils down to.
00:19:52.880 --> 00:19:59.759
And it's really because you don't know um how much it's going to allocate and why.
00:20:00.160 --> 00:20:08.559
I think one of the reasons that this is still not like popularly well understood, by the way, Julian, is like unless you are implementing novel lock-free data structures, like just don't do this.
00:20:08.720 --> 00:20:09.519
Does it make any sense?
00:20:09.599 --> 00:20:22.400
Like, like unless you have unless you're building your own kind of you know, multiprocessor immutable structure, um, or you're implementing a garbage collector or some some other kind of exotic systems thing, um, I'm not sure you need to know this stuff.
00:20:22.640 --> 00:20:27.759
And understanding this stuff is an interaction of both like compilers, like a compiler level abstraction, right?
00:20:27.839 --> 00:20:34.079
Like release release semantics and acquire semantics is like a language level abstraction that hides real diversity in different CPUs.
00:20:34.160 --> 00:20:37.759
Like like ARM has a different memory model, actually, than x86.
00:20:38.160 --> 00:20:42.240
X86 does some of this stuff for you kind of automatically in a way that's hard to reason about.
00:20:42.319 --> 00:20:51.119
And it means if you just kind of write this code in a sloppy way at C or C and it runs on an x86, it might not work on other processors, even though you've got a perfectly good C compiler for that other processor.
00:20:51.359 --> 00:20:57.759
So I mean this is really kind of like I I understand like your frustration, and I also understand why people are like, what are you talking about, Grandpa?
00:20:58.079 --> 00:21:01.440
I think you should you should know these things if you want to write efficient code.
00:21:01.599 --> 00:21:14.079
And so I think I I tend to agree, of course, by the way, and I think we we did skip over an obvious thing in 2026 that we probably should not skip over if we're talking about uh how to learn to code, which is should you even learn to code?
00:21:14.160 --> 00:21:14.319
Right?
00:21:14.480 --> 00:21:16.720
We've we've got you know we've got these genies.
00:21:17.200 --> 00:21:24.240
You you say, hey, make me a CRM, don't make any mistakes, write lots of unit tests, go, and it can do it.
00:21:24.400 --> 00:21:32.240
So why do I need to know all this crazy stuff about where the curly prices go and when I want to acquire semantics and release semantics and what a load is and what a store is.
00:21:32.720 --> 00:21:55.359
Yeah, so I mean, if we are operating under the assumption that uh an AI is going to be able to program better than any human being on any given task, then learning how to program becomes purely basically intellectual curiosity, right?
00:21:55.839 --> 00:21:59.279
And and that's true, but
00:22:00.079 --> 00:22:02.880
I don't know that this is gonna happen anytime soon.
00:22:03.200 --> 00:22:06.720
So I go back and forth on this, you know.
00:22:06.799 --> 00:22:09.759
On Monday I think, oh my god, AI is amazing.
00:22:10.000 --> 00:22:12.400
And then on Tuesday I'm like, wow, I can't believe you got that.
00:22:12.559 --> 00:22:13.680
This is so wrong.
00:22:14.000 --> 00:22:18.079
So it's um so yeah, I don't know.
00:22:18.240 --> 00:22:23.599
I don't know where we're gonna land on this, but I don't think it's going to work out.
00:22:23.759 --> 00:22:26.559
I don't think uh so if I and and I've been wrong.
00:22:27.039 --> 00:22:37.039
This particular exotic corner of computing that we're dwelling on, which is like kind of lock-free data structures and whatever, this is like has been one of my kind of Turing tests for for coding genies for a long time.
00:22:37.119 --> 00:22:46.559
Is like, write me a lock-free hash table that will accept any key and any value, you know, a template a templated C23 lock key hash table, lock-free hash table.
00:22:46.720 --> 00:22:48.960
Um, don't make any mistakes, right?
00:22:49.119 --> 00:22:57.680
And it does seem like that's gotten better over time, but it's also sort of clear that it's sort of debugging it, you know, in a sequential, like it's sort of thinking about a sequential execution model.
00:22:57.759 --> 00:23:08.000
It's hard to describe exactly, but like it hasn't internalized the message of like Morris Rulia, he's the art of multiprocessor programming, or like other people that sort of have thought about this stuff in a in a deep way for a long time.
00:23:08.079 --> 00:23:09.279
It just doesn't have that model.
00:23:09.440 --> 00:23:12.880
But it could be like that, that's one RL data set away from having that ability, right?
00:23:12.960 --> 00:23:19.839
It could be that like, you know, they they hire me to write 20 of those things or something, and like now it works and that's all over.
00:23:20.240 --> 00:23:32.240
Well, the other thing is whenever you're going to you know test an AI like that, you have to remember that it has read all the code of a gazillion lock-free hashtags, right?
00:23:32.559 --> 00:23:38.559
And so, and it it understands all the comments and and so that that is a bit of a problem, right?
00:23:38.799 --> 00:23:53.519
Where you have somebody, let's say you have somebody, a smart human being, who has no understanding about computers, doesn't understand how the code runs, or doesn't have a good mental model about how the code runs, right?
00:23:53.759 --> 00:24:05.359
Uh but that person has, you know, read and and memorized all the lock-free hash tables that there's ever been implemented with all the comments and what they say.
00:24:06.000 --> 00:24:06.160
Right.
00:24:06.480 --> 00:24:10.160
Don't you think that this person would be able to bluff you in a test like that?
00:24:10.400 --> 00:24:10.720
Probably.
00:24:10.960 --> 00:24:11.200
Sure.
00:24:11.359 --> 00:24:11.440
Right?
00:24:11.759 --> 00:24:12.319
Sure, sure.
00:24:12.640 --> 00:24:18.000
So and and that's very worrisome because the hash table is not the interesting thing, right?
00:24:18.079 --> 00:24:22.720
What you want to see is what happens when you ask something it has never seen, right?
00:24:22.880 --> 00:24:25.119
And how how does it do then, right?
00:24:25.599 --> 00:24:27.920
So I I don't know, man.
00:24:28.079 --> 00:24:30.160
I I keep on going back and forth.
00:24:30.319 --> 00:24:30.559
Yeah.
00:24:30.799 --> 00:24:37.119
I think AIs are fascinating, but I right now I don't see it.
00:24:37.279 --> 00:24:41.839
And I don't see it- I mean, I'm tempted to draw an analogy with the Haskell compiler in some ways, right?
00:24:41.920 --> 00:24:53.680
Which is like I, you know, it's nice when I'm able to get Haskell to work the first time on a toy problem, and especially if it's like a domain I'm not that familiar with, and I just want to get it done with, and and I'm happy that it works.
00:24:53.920 --> 00:25:02.559
Um, but if I care about resource utilization, I need a robust model of like what computers are and how they do what they do to even know what to ask for.
00:25:02.640 --> 00:25:18.480
And I think that like one of the things that I've been that I've found most surprising about these models so far, and I think it's still true with Fable, I think it's still true with GPT-5.6, is it seems like they're almost more like amplifiers for expert programmers than they are, you know, floor raisers for people who don't know how to program.
00:25:18.799 --> 00:25:22.319
And though I I don't know exactly what it is that went into expert programmers.
00:25:22.400 --> 00:25:26.240
It makes them so much more effective at like prompting these things and getting good behavior out of them over time.
00:25:26.559 --> 00:25:26.799
I know.
00:25:27.039 --> 00:25:29.599
Some of it's knowing how to code and knowing how computers work, though.
00:25:30.000 --> 00:25:35.200
Because you know, whenever you're writing code, a lot of it is boring and repetitive, right?
00:25:35.440 --> 00:25:38.559
And the the interesting part is actually really small.
00:25:38.799 --> 00:25:44.240
And so what these LLMs do is they get all the boring part away, but done.
00:25:44.400 --> 00:25:55.279
And then the f you you read, you you understand the few mistakes they made, and and then the places where you actually need to do something else, either you tell them or you do it yourself, right?
00:25:55.440 --> 00:26:02.000
And so that really what what your job is now is to make design decisions and review code carefully.
00:26:02.160 --> 00:26:12.480
Um, except in some places where whenever I'm writing something, which is rare, very rare, but that is very dense and very complex.
00:26:12.960 --> 00:26:15.920
Like there's 20 lines of code I really need to get right.
00:26:16.000 --> 00:26:17.839
Then I usually don't use an LLM.
00:26:18.000 --> 00:26:23.200
But other than that, it's basically saving you a ton of typing.
00:26:23.359 --> 00:26:37.279
The problem is I've had 25, well, it depends how you count, but 20 years of professional experience and 30 years of experience writing those programs and building that intuition.
00:26:37.920 --> 00:26:42.000
And so now imagine you don't have that intuition yet.
00:26:42.720 --> 00:26:45.839
It's because how how do you build an intuition, right?
00:26:45.920 --> 00:27:00.000
You write code, that code ends up uh biting you in the ass, coming back with a vengeance, and then you wake up at two in the morning, you know, because of a silly mistake you made, and then you learn.
00:27:00.079 --> 00:27:11.440
And then the next time you come close to something like that, I'm gonna add tests for this, I'm gonna do this, and because you know you've got those scars, and those scars cost you, right?
00:27:11.920 --> 00:27:14.799
Now, imagine you were 20.
00:27:15.039 --> 00:27:18.319
I mean, I I'm trying to imagine myself when I was 20.
00:27:18.480 --> 00:27:21.680
And so one thing you need to know about me is I'm incredibly lazy.
00:27:21.839 --> 00:27:24.960
And that's why I got into programming, because computers do things.
00:27:25.440 --> 00:27:26.960
That's like exactly.
00:27:27.119 --> 00:27:30.319
Yeah, computer people are sort of too lazy for mathematics or whatever.
00:27:30.799 --> 00:27:31.920
Exactly, yeah, exactly.
00:27:32.079 --> 00:27:36.640
And and so I know exactly why I would do 20 years old with an LM.
00:27:36.799 --> 00:27:49.680
I would try to get it to write pretty much every program that I want to work, and you know, frenetically um you know, shake it until it produces something that that seems like it's working.
00:27:49.920 --> 00:27:58.000
And then once it's not working, I would go back to the LM and say and shake it frenetically, saying, well, look, this thing has a burn.
00:27:58.079 --> 00:28:00.960
You need to, you know, you need to fix this, right?
00:28:01.119 --> 00:28:09.839
And and because I'm lazy, I think it would take me a very, very long time to come to the realization that, oh my God, this is not gonna work.
00:28:10.079 --> 00:28:13.440
I need to actually understand what's going on and write it myself.
00:28:13.599 --> 00:28:16.000
But I would have probably wasted a ton of time.
00:28:16.240 --> 00:28:27.279
And frankly, for the level of quality that was asked for me at university, I I would have probably gotten away with just LLM LLMizing everything, you know?
00:28:27.839 --> 00:28:31.680
And it would have been perfect, but I wasn't looking for the perfect grade.
00:28:31.759 --> 00:28:33.759
So it's like, oh, it's gonna be good enough.
00:28:33.839 --> 00:28:35.359
Um I'm lazy enough.
00:28:35.440 --> 00:28:37.279
So that's probably what I would have done.
00:28:37.519 --> 00:28:40.480
So, how do we prevent that?
00:28:40.799 --> 00:28:45.119
Because we're not going to go back to a world where kids are not gonna have LLMs.
00:28:45.279 --> 00:28:46.240
They're gonna have LLMs.
00:28:46.319 --> 00:28:58.160
Well, so I have a theory about this, and I want to run this past you a little bit too, actually, because we skipped over a couple a few steps in like my adolescent computer upbringing and yours too, because both of us apparently went through a C phase, right?
00:28:58.400 --> 00:29:05.440
So uh when I was trying to do Pascal stuff in in uh as a junior in high school, because I needed to take the AP test and I was excited about computers or whatever.
00:29:05.680 --> 00:29:08.240
This is a period in my life where I didn't have a computer at home.
00:29:08.400 --> 00:29:11.759
Like the Apple IIe had been sold, we were between computers.
00:29:11.920 --> 00:29:15.759
Um, there was the PDP 11 at school, but I didn't like actually have unfettered access to that.
00:29:15.839 --> 00:29:21.119
I had like, you know, a computer lab at lunch if I brought my lunch into the to where the VT100s were.
00:29:21.279 --> 00:29:26.319
Um, so I actually wrote a lot of code, pencil and paper, at this time in my life.
00:29:26.400 --> 00:29:28.240
I've wrote I had like little books about C.
00:29:28.480 --> 00:29:30.000
I'd read them, you'd flip them around.
00:29:30.079 --> 00:29:31.759
There's little 3D objects that we used to have.
00:29:31.839 --> 00:29:32.640
They're called books.
00:29:32.720 --> 00:29:35.440
They were like thin pieces of paper stacked up in a Z-axis.
00:29:35.680 --> 00:29:36.400
Never heard of them.
00:29:36.640 --> 00:29:37.759
Um yeah.
00:29:38.160 --> 00:29:40.079
And uh and used to learn a program this way.
00:29:40.240 --> 00:29:47.359
And so I'd have my little book about C and be like, oh, okay, cool, and not really understand pointers or whatever, but be starting to get it a little bit as it had weirder things with arrays in it.
00:29:47.519 --> 00:29:52.079
And I'd write these little programs out longhand in like a notebook, and then I'd have my chance to key them in, right?
00:29:52.160 --> 00:29:57.759
And you'd get to the computer and like, you know, try to get them to go and it wouldn't compile, but then the next day I'd be able to compile and so on.
00:29:58.000 --> 00:30:09.839
Um, but I think one of the things that uh that that ingrained in my brain that is really hard to get any other way that I know of anyway, is like a mental is the the sort of robust mental model of executing the machine, right?
00:30:09.920 --> 00:30:14.880
The robust mental model of kind of like line by line, okay, first it does this, then it does that, then it does that, then it does that.
00:30:15.039 --> 00:30:19.039
Um because like I know I can't just like interactively be like, well, does this work?
00:30:19.200 --> 00:30:19.680
Well, does this work?
00:30:19.839 --> 00:30:20.400
Well, does this work?
00:30:20.559 --> 00:30:21.759
Keep slopping things at it.
00:30:21.920 --> 00:30:28.960
And so a little bit like how um, you know, I I have a my son is a rising sophomore in college, right?
00:30:29.119 --> 00:30:33.599
And every college in the world has this crisis now of like, how do we do writing assignments?
00:30:33.839 --> 00:30:44.079
And one answer is because basically, like you can, you know, slop your way through darn near anything and like write a way better than undergraduate level essay about almost anything with a one-line prompt now.
00:30:44.240 --> 00:30:47.519
I mean, yeah, there's like cheating detectors and stuff, but it's a whole arms race.
00:30:47.839 --> 00:30:57.039
The uh and one answer is just to like give people pencil and paper and a notebook and sit them down and say, like, you're gonna write this out longhand and then we're gonna read it and grade it the old-fashioned way.
00:30:57.200 --> 00:31:04.000
And I think that's a might have a place if you really care about learning how computers work and learning how to code well these days.
00:31:04.240 --> 00:31:10.799
Um, and I think they're I think you should, but that also is like an insanely monastic thing to say out loud now.
00:31:10.880 --> 00:31:11.519
Does that make any sense?
00:31:11.599 --> 00:31:14.960
Like Yeah, but I mean I I remember I had a taste of that.
00:31:15.119 --> 00:31:19.279
So I had access to a computer in high school.
00:31:19.519 --> 00:31:22.480
So I was able to program directly on a laptop.
00:31:22.720 --> 00:31:28.480
Uh, but um the exams at university were done on pen and pa with pen and paper.
00:31:28.799 --> 00:31:33.839
And uh ton of um uh work, homework at university was pen and paper.
00:31:34.240 --> 00:31:39.279
And so I wrote an insane amount of code using pen and paper as well.
00:31:40.240 --> 00:31:42.000
And that's how actually you were graded.
00:31:42.720 --> 00:31:49.039
So you actually had to be pretty good at writing, you know, code with pen and paper because that that's what was graded, you know.
00:31:49.359 --> 00:31:58.079
The the project um being graded directly, project that you would write with a computer, that came in my fourth or fifth year.
00:31:58.240 --> 00:32:01.599
So almost all my studies I was I was using pen and paper.
00:32:01.759 --> 00:32:10.720
And I'm with you that this was um a very useful experience because you you have to have a mental model of what's going on in the machine.
00:32:11.599 --> 00:32:16.000
But I don't know that uh somebody using Emacs doesn't have that model.
00:32:16.240 --> 00:32:18.480
I think the pen and paper is just a pain in the ass.
00:32:18.880 --> 00:32:33.279
I think uh it's but uh the LLM is a step further because the LLM Yeah, you can just not understand what's going on at all and have you know a level of understanding.
00:32:33.519 --> 00:32:35.039
So let's let's put it this way.
00:32:35.440 --> 00:32:50.880
I think somebody who learned how to program directly with computer with Emacs, running Emacs instead of a pen and paper, I don't know that this person doesn't understand as well the C model than somebody who used pen and paper, right?
00:32:51.440 --> 00:32:58.079
Um so I don't know that we have to go as far as back to pen and paper, because back then it was just for logistics.
00:32:58.240 --> 00:33:08.799
But what we should definitely do is use computers that don't give you access to LMs or the internet or anything, and then you have to actually, you know, read the docs, you know.
00:33:09.039 --> 00:33:09.279
Yeah, yeah.
00:33:09.519 --> 00:33:11.039
And that was that was a big one, right?
00:33:11.119 --> 00:33:13.680
Like man, you know, you were reading docs.
00:33:13.920 --> 00:33:17.680
Yeah, I remember like yeah, getting good at using the documentation tools was a was a whole thing.
00:33:18.000 --> 00:33:22.160
I was told you my crazy, like uh my crazy keyboard attestation idea before.
00:33:22.480 --> 00:33:24.720
Have I been over been down this rabbit hole with you?
00:33:25.039 --> 00:33:25.359
No.
00:33:25.839 --> 00:33:29.839
So, you know, like you you've got these manufacturers like Apple who are like making keyboards, right?
00:33:30.319 --> 00:33:36.880
You could have them um imagine, you know, they've got they've got this magic button that you touch with your fingerprints already, right?
00:33:36.960 --> 00:33:38.079
So they've got that one magic button.
00:33:38.160 --> 00:33:39.680
Imagine there's a new magic button.
00:33:40.079 --> 00:33:50.559
And what the magic button does is like sign a digest of the keystrokes from the last 30 seconds with Apple's private key, or with the with a key that's on the keyboard anyway.
00:33:50.799 --> 00:33:57.200
Um this would be the basis of a technology that would let you say a human wrote these characters, right?
00:33:57.279 --> 00:34:03.839
This would be the basis of a technology that lets you say like this got typed by a person instead of by an LLM.
00:34:04.559 --> 00:34:10.159
Um and I think there's analogs with like cameras and stuff too, if you're worried about like AI propaganda and things like that, right?
00:34:10.239 --> 00:34:18.480
Can imagine having a doing a similar thing with like, hey, a photon sensor actually captured this from a real physical light cone modulo encryption working, right?
00:34:18.719 --> 00:34:24.800
So why couldn't an LLM use the same key to generate?
00:34:25.280 --> 00:34:25.920
It's a secret.
00:34:26.480 --> 00:34:31.039
Because Apple doesn't give their keys to OpenAI and doesn't give their keys to Anthropic.
00:34:31.119 --> 00:34:35.199
They put them in a you know secure enclave on your keyboard that they ship.
00:34:35.360 --> 00:34:37.119
The same way your phone has these things.
00:34:38.400 --> 00:34:38.719
Okay.
00:34:38.960 --> 00:34:44.960
And this is like a the this is like a weird you know standard that I that I think could be you could imagine proposing.
00:34:45.119 --> 00:34:46.880
It takes like a little bit of coordination, right?
00:34:46.960 --> 00:34:55.519
Because like once you have a keyboard that can produce this, you need something that can kind of consume it, and you need to like ship it around somehow, and you need to like decide how the little JSON blobs are laid out and blah, blah, blah.
00:34:55.679 --> 00:35:04.159
But you can imagine a world where I can basically have a Google Docs, except only humans you know, hitting the springs on keyboards are able to populate the characters in the Google Doc.
00:35:04.480 --> 00:35:08.320
Um, or at the very least, I can go back and see which keys I hit, right?
00:35:08.480 --> 00:35:16.800
And if I suspiciously pasted in, you know, 5,000 characters that have lots of M dashes in it or whatever, you might say, huh, this looks suspicious, right?
00:35:17.199 --> 00:35:21.039
Um this is like kind of a crazy, like that's a good idea.
00:35:21.280 --> 00:35:22.880
And I we're gonna want this eventually, right?
00:35:23.119 --> 00:35:33.119
We're gonna want to be able to say, like, hey, this was you know, not only am I did I just sort of run some pangram defeating adversarial thing over AI slot output.
00:35:33.199 --> 00:35:34.639
This actually wasn't AI slop output.
00:35:34.719 --> 00:35:38.800
This actually came out of my my keyboard and my my joints.
00:35:39.280 --> 00:35:40.719
Um I want this now, right?
00:35:40.800 --> 00:35:42.639
Like I wish I wish I could prove this to people sometimes.
00:35:42.719 --> 00:35:47.840
Like I wish I could like the emails that I actually write by hand because I'm like, I want you to pay attention to this, because I paid attention to it.
00:35:48.000 --> 00:35:53.920
Like it would be lovely if I could like stick some hard to fake thing on the side, being like, Keith actually wrote this.
00:35:54.320 --> 00:36:07.119
I'm sort of building a little um a list of the things that we're losing, I guess, from like the the era where like you sort of you know would sit in Vim raw dogging characters, like or Emacs, if you're, you know, as your preference may be.
00:36:07.360 --> 00:36:09.519
Um the the end of that world.
00:36:09.599 --> 00:36:16.559
And I think like because as a programmer, you're always trying to do something, you know, you're you're trying to do something new, right?
00:36:16.639 --> 00:36:18.960
Because if you're just doing what you did before, you just run that program.
00:36:19.039 --> 00:36:19.920
You wrote it already.
00:36:20.079 --> 00:36:22.719
Um so you're always kind of trying to get the computer to do something new.
00:36:22.880 --> 00:36:26.719
That sensation of like, I want this very simple thing to work.
00:36:26.800 --> 00:36:27.760
Like, why doesn't it work?
00:36:27.840 --> 00:36:31.039
Why am I not smart enough to get the machine to do what I'm trying to get it to do?
00:36:31.199 --> 00:36:33.119
That sensation sort of never goes away.
00:36:33.360 --> 00:36:42.480
And I think one of the things that took me almost like 25 years or something, you know, part of my entire professional career as a programmer to realize is like I got good at this somewhere along the way.
00:36:42.559 --> 00:36:43.280
Does that make any sense?
00:36:43.599 --> 00:36:57.599
Like, there was never a moment where I like felt like, oh, now I'm an expert, because I always imagine that experts were just like that, you know, John Carmack or someone would just sit there and like his mind would like reach tendril-wise out into the CPU and like deposit the right bytes on disk or whatever.
00:36:57.760 --> 00:36:58.159
Yeah, exactly.
00:36:58.320 --> 00:36:59.360
Just deposit like the correction.
00:37:01.679 --> 00:37:02.079
All the time.
00:37:02.320 --> 00:37:03.440
Yeah, yeah, exactly.
00:37:03.679 --> 00:37:08.960
And um I mean, you know, like it turns out he's mortal too, and so is every human programmer.
00:37:09.039 --> 00:37:15.119
And you actually spend your whole life kind of just sitting there being like, why am I not smart enough to get this freaking box to do this stupid thing I'm trying to get it to do?
00:37:15.280 --> 00:37:25.679
It turns out that like in year 20, it's a more complicated, more ambitious thing that you're trying to get it to do, um, and probably closer to sort of some margin of what's been done before than than when you're just getting started.
00:37:25.840 --> 00:37:28.000
But you never actually feel that great at it.
00:37:28.079 --> 00:37:28.800
Does that make any sense?
00:37:28.880 --> 00:37:31.519
Like you never sort of like, oh man, I really nailed this whole programming thing now.
00:37:31.599 --> 00:37:33.199
I don't need to get any better at this.
00:37:33.360 --> 00:37:40.079
Um and that is uh, I don't know whether that remains in this world with like the coding genies.
00:37:40.159 --> 00:37:41.760
I think like well, I think there's another thing.
00:37:42.079 --> 00:37:45.440
The thing about the the margin of like you still need to do the hard part a little bit, right?
00:37:45.519 --> 00:37:49.599
You still need to sort of like boil it down to some kernel of like, okay, this is the tricky part.
00:37:49.840 --> 00:37:51.760
Um, and that still involves some intellectual effort.
00:37:51.840 --> 00:37:52.159
But go ahead.
00:37:52.719 --> 00:37:58.079
So so yeah, I think that there are a couple of other things that uh I've I've witnessed with pro programmers.
00:37:58.320 --> 00:38:13.760
So the first one is because programmers are constantly um you know, have constantly to face the fact that they were wrong, because the machine has shown them that what they what they had in mind was not true, was not working.
00:38:14.480 --> 00:38:16.480
It's it teaches them humidity.
00:38:16.639 --> 00:38:20.480
So there is there's two kinds of programmers.
00:38:20.639 --> 00:38:40.960
There are those who to whom it teaches humidity, and they're like um, you know, when whenever they approach another problem, they're like um, well, I might actually be wrong about this, because you know, I was absolutely sure that this function I wrote 10 minutes ago was correct, and yet here we go again, it's incorrect.
00:38:41.119 --> 00:38:56.480
And and so when you realize how sloppy your brain is, normally, you know, it makes you question, you know, how how opinionated you should be about something, because you know that your brain can be wrong and can be sloppy about complex things, right?
00:38:56.719 --> 00:39:01.840
And so most programmers I know that actually translates in their everyday life, right?
00:39:02.000 --> 00:39:05.599
They've programmed for a long time, they know that they can be wrong.
00:39:05.760 --> 00:39:19.119
So whenever you ask them to think about something complex, whether it's related to programming or not, they will um they will have a tendency to think, well, maybe I'm wrong about this, and you know, be cautious, right?
00:39:19.280 --> 00:39:26.480
But there's another class of programmers, and I would say they're a minority, but they exist, who approach the problem in a different way.
00:39:26.639 --> 00:39:35.199
So the way they do things is they write their program, and as they write their program, they're absolutely certain they're right, right?
00:39:35.360 --> 00:39:36.079
It's it's correct.
00:39:36.239 --> 00:39:36.960
Of course it's correct.
00:39:37.119 --> 00:39:37.679
Oh, yeah, yeah, okay.
00:39:38.079 --> 00:39:42.559
Until the machine says, uh-uh, uh-uh-uh, not working.
00:39:42.800 --> 00:39:55.760
And then they get used to the fact that basically the way they operate is of course I'm right, I'm 100% right, until something irrefutably proves that I'm wrong.
00:39:56.079 --> 00:39:56.400
Right?
00:39:56.960 --> 00:40:06.480
And the problem is if you operate this way in pretty much anything other than maths and computer science, basically you're going nowhere, right?
00:40:06.559 --> 00:40:11.840
Like if you're talking about design and you say, I'm absolutely sure the button has to be blue.
00:40:12.000 --> 00:40:25.599
And and until somebody proves to me that this button doesn't have to be damn blue, I'm I'm going to assume I'm right, then that leads to, you know, a kind of toxic behavior.
00:40:25.840 --> 00:40:44.719
So I think I would say overall, spending your days having a machine telling you you're wrong is healthy and it leads to healthy behaviors, you know, outside of, you know, in in the rest of your of your life, um, outside of a few exceptions.
00:40:44.960 --> 00:40:50.239
And yes, I think that's part of what uh LLMs are probably going to change.
00:40:50.719 --> 00:41:09.360
Also when it comes to maths, because maths is a is a lot like that too, where you have to be you have to be precise, you know, and and part of mathematics was of course the interesting part was you had to be inspired, you had to have ideas and how to connect things and how to model things.
00:41:09.599 --> 00:41:18.639
But there was the grunt work of actually writing down the proofs and when you had to be meticulous enough to make sure that you know you were not making any mistake, right?
00:41:19.199 --> 00:41:31.920
And now that part is probably going away because you'll probably give it to an AI and say, well, use this and that theorem, go go prove this, and it's probably gonna work, right?
00:41:32.079 --> 00:41:35.519
So that's probably going away too.
00:41:36.239 --> 00:41:37.039
Yeah, it's wild.
00:41:37.199 --> 00:41:45.599
We had a uh, you know, at Pelobot we had a conference of sort of working mathematicians a couple months ago, just like a three-day kind of discussion about AI and math.
00:41:45.760 --> 00:41:48.480
And it felt like sort of 18 months ago for programmers.
00:41:48.639 --> 00:41:55.519
It felt like sort of the, you know, the way that the programmers were when it was first becoming clear that this was sort of important, that was going to change the way that we operated.
00:41:55.760 --> 00:42:06.159
And the the whole set of reactions kind of ran the gamut from like, you know, people who are in an academic department where they model the profession of mathematics as the production of new theorems.
00:42:06.320 --> 00:42:10.480
And if there are machines that do this, like why do we still have math departments and how do I get funding, right?
00:42:10.639 --> 00:42:24.159
Versus people who are all the way on the other side who are like, hey, I got into this because I was curious about the the universe and curious about these intellectual questions, and think about all the amazing, you know, chunks of the idea tree we're going to be able to unlock once we, once we're able to answer some of these long-standing questions.
00:42:24.320 --> 00:42:26.800
Um, and I think they're they're kind of both right in a certain way.
00:42:26.880 --> 00:42:33.199
I think it's probably true that that mathematics ends up getting reframed if you ever did frame it as just sort of the construction of new theorems.
00:42:33.280 --> 00:42:46.559
I think it's they're they're having a little bit of an analog to what we're talking about with coding agents too, which is like you need to know what to build and you know what makes sense to build and what's interesting to build and know what actually sort of is useful and what connects you know to the real world in a grounded way.
00:42:46.800 --> 00:42:47.840
And they don't know that yet.
00:42:47.920 --> 00:42:50.079
And I think there's a similar thing, oddly enough, with mathematics.
00:42:50.239 --> 00:42:55.920
You think of mathematics as being this kind of pure crystalline structure in the space of ideas, and there's a perspective where it is, I guess.
00:42:56.079 --> 00:42:59.119
But the theorems that we find interesting are very, very sparse, right?
00:42:59.199 --> 00:43:07.840
Like the parts of mathematics that we actually care to explore are really, really sparse in that sort of vast galaxy of kind of little trivial changes on things we already know.
00:43:08.079 --> 00:43:14.400
Um, and you still need somebody to sort of say, like, no, you know, Riemann's data function, this is a really interesting object.
00:43:14.480 --> 00:43:17.760
Or yeah, but you could also have an LLM say that too, right?
00:43:17.840 --> 00:43:25.119
You can have an LM say maybe an LLM or something else, an AI that says this is actually a really interesting figure.
00:43:25.280 --> 00:43:29.760
Um If I was 20 years old today, well, 20 maybe, maybe a bit younger than that.
00:43:29.920 --> 00:43:36.960
Let's say when I started serious programming seriously, let's say 14 years old, I would be a vibe coder today.
00:43:37.119 --> 00:43:40.320
I would be a vibe coder because you know, I was lazy.
00:43:40.480 --> 00:43:43.840
I was very, very lazy, and I still am.
00:43:44.079 --> 00:43:46.000
And that's what got me into programming.
00:43:46.159 --> 00:43:48.159
Like I wanted computers to do things for me.
00:43:48.320 --> 00:43:57.840
So when I was 14 years old, I would have become a vibe coder, which means I would have frenetically asked, prompt the you know, the computer to do things for me.
00:43:58.000 --> 00:43:58.159
Yep.
00:43:58.559 --> 00:44:05.039
And so just so that the old Is following what I call vibe coding, and I think smarter people have defined this before me.
00:44:05.199 --> 00:44:15.039
But what I mean by vibe coding is you write a prompt, it generates a program, and you never actually look at this program or try to understand what it does.
00:44:15.199 --> 00:44:20.320
The only thing you do is you run this thing and you see if the behavior is what you want, right?
00:44:20.800 --> 00:44:28.239
So for example, you write, write me, you know, um an app that tracks my categories, right?
00:44:28.320 --> 00:44:43.119
And so then the LLM produces something, you run the app, and then you say, oh no, I want the button to be there, not there, and you you keep on prompting, and basically the LLM keeps on iterating on the on the program that you're trying to build.
00:44:43.280 --> 00:44:45.519
That's vibe coding in my book.
00:44:46.000 --> 00:45:00.320
And I think back in those early days, given that the kind of things I was trying to do was simple, I would have probably never gotten out of the LLM of the vibe coding jail, if you will.
00:45:00.639 --> 00:45:09.599
I would have kept on, you know, and so one of the challenges for educators is going to be that.
00:45:09.679 --> 00:45:14.559
It's going to be you you now have a new generation that's coming.
00:45:14.719 --> 00:45:21.920
They need to actually understand things, whether it's in mathematics or you know, or computer science.
00:45:22.159 --> 00:45:30.880
And how do we and so I like your idea of um the keyboard that actually signs the fact that keys were pressed.
00:45:31.119 --> 00:45:35.599
That's actually pretty neat because now you actually have to press it with your fingers.
00:45:35.760 --> 00:45:48.639
Of course, there will be people who cheat and who will have an LLM that another computer that produces the code and they then they they copy that code um with with their little fingers.
00:45:48.880 --> 00:45:50.400
But at least they're writing something.
00:45:50.800 --> 00:46:06.559
I mean if there's a if there's a secure enclave, if like you if you kind of imagine that hardware encryption works, or if you kind of imagine that there's a tamper resistant store that like can sign stuff that I can slap on a keyboard, which you know there this there are hardware products based on this today, like HKMs or whatever.
00:46:06.719 --> 00:46:11.039
Um I think you can actually do this in a way that's like not easy to cheat on, right?
00:46:11.199 --> 00:46:13.440
Maybe we should make up a maybe we should make a hardware go fund me.
00:46:13.599 --> 00:46:14.719
Maybe this should be the theory.
00:46:14.960 --> 00:46:26.320
Well, the the only thing, the only way to cheat at this point is to have a second computer that is connected to an LM and that produces the code or produces the text, and then you have to copy the text.
00:46:26.800 --> 00:46:32.480
Well, so remember that second computer would need to either know the secret that's in the hardware key or robotically produce it, right?
00:46:32.559 --> 00:46:34.079
So so there's the analog hole, right?
00:46:34.159 --> 00:46:36.800
Like I could have robot hands type on this thing, right?
00:46:36.880 --> 00:46:39.599
And then the robot hands really would look like they they typed it.
00:46:39.840 --> 00:46:42.000
But that's a pretty expensive analog hole, right?
00:46:42.400 --> 00:46:43.360
Exactly, exactly.
00:46:43.519 --> 00:46:46.079
So you can still cheat, yeah, but it's expensive.
00:46:46.159 --> 00:46:52.639
It's just like I can put an iPhone camera with a screen that's playing an AI slob video or whatever, and then like you'd have a you know, iPhone camera.
00:46:52.960 --> 00:47:04.159
Well, I mean the other thing you could probably do is hijack the signal that sends it because you know the hardware that signs the keys is listening to the device.
00:47:04.559 --> 00:47:05.840
And so you can probably hijack the key.
00:47:06.000 --> 00:47:07.519
No, no, the hardware that it's on device.
00:47:07.599 --> 00:47:08.480
You buy the you buy the keyboard.
00:47:09.119 --> 00:47:09.679
No, I get it.
00:47:09.760 --> 00:47:10.159
I get it.
00:47:10.320 --> 00:47:10.559
Sorry.
00:47:10.800 --> 00:47:14.320
You buy the keyboard with with there's a little thing on the keyboard.
00:47:14.400 --> 00:47:14.639
Yeah.
00:47:14.800 --> 00:47:22.159
And so every time I press a key, there's a signal that leaves the key that goes to the thing that's signs, right?
00:47:22.880 --> 00:47:24.880
And so all you have to do is hijack the key.
00:47:26.719 --> 00:47:31.519
So if I have a J tag or something and I electronically mess with the switches, yeah, that's true.
00:47:31.679 --> 00:47:37.360
You could make this sort of switch array of the keyboard, uh, you could you could instrument that and and cheat that way.
00:47:37.440 --> 00:47:37.920
That's true.
00:47:38.159 --> 00:47:43.119
But yeah, yeah, I mean you're talking about sort of like attacker hardware at that point or whatever, right?
00:47:43.280 --> 00:47:45.119
Um and the market for that's always gonna be smaller.
00:47:45.760 --> 00:47:57.360
It's still much slower because you could easily add something that says that if the if the keys are coming in too quickly, then it's not it's faster than you can.
00:47:57.760 --> 00:48:03.760
Yeah, if it's if it's more than like the actions per minute of like a championship World of Warcraft player, then like you don't believe it, right?
00:48:03.920 --> 00:48:04.239
Yeah.
00:48:04.559 --> 00:48:05.199
Yeah, exactly.
00:48:05.599 --> 00:48:07.360
I really think there's sort of something here, by the way.
00:48:07.440 --> 00:48:09.519
Uh this would be this would be fun to try to start doing things.
00:48:09.679 --> 00:48:15.679
And then then you could like make web browsers that have tech century fields that like require that you use one of this thing, and then blah, blah, blah.
00:48:15.920 --> 00:48:18.639
Um, I I really think there's sort of something interesting here.
00:48:18.800 --> 00:48:23.840
Um, it's tricky because you have to, you know, you have to coordinate lots of parts of the software stack, right?
00:48:23.920 --> 00:48:26.960
This is not a problem that we've had to worry about until right now, basically.
00:48:27.119 --> 00:48:31.119
Um, and so the software stack kind of assumes that like input devices are input devices.
00:48:31.280 --> 00:48:36.800
But this also comes up with uh like people with all the sort of countermeasures on websites for for bots, right?
00:48:36.880 --> 00:48:42.239
Like you could imagine a website that like wants a mouse that's like this and and wants a keyboard that's like this.
00:48:42.400 --> 00:48:55.519
And if you know there is not a key attesting that like some rollerball moved the XY wheels on a mouse the right way, then I refuse to sort of take this like clawed computer use, you know, trying to scrape my website thing.
00:48:55.599 --> 00:48:56.960
Like that could work, you know.
00:48:57.199 --> 00:49:09.360
Um that that idea of like how we get back to the econom like AI right now has sort of broken the economics of the internet that like you and I grew up with and and it hasn't reacted yet.
00:49:09.519 --> 00:49:10.719
Like we're not at equilibrium yet.
00:49:10.800 --> 00:49:15.360
So there still isn't internet out there where there's lots of like free content that people put up there.
00:49:15.440 --> 00:49:18.480
But like the economic motivations for doing that aren't really there anymore.
00:49:18.639 --> 00:49:26.239
If I actually like have a bunch of like amazing technical content, unless that is also marketing for something I'm selling, there's not a lot of reason for me to put that up for free anymore.
00:49:26.400 --> 00:49:29.920
It used to be that like, okay, Google will like help me to sell advertising against it.
00:49:30.079 --> 00:49:38.880
Yes, that's true in concept, but what's actually gonna happen now is I'm just like making the products of anthropic and open AI and whoever else much stronger, and I have no way to participate in that.
00:49:39.119 --> 00:49:51.119
And so, you know, the the thin end of this wedge is that the arms race of people trying to prevent scrapers and trying to prevent sort of uh people from mechanistically accessing the data on their website is getting ratcheted up a little bit.
00:49:51.199 --> 00:49:55.760
So maybe those are the people who end up applying the pressure for for this to show up in input devices.
00:49:56.639 --> 00:50:07.199
Yeah, I I mean the the bigger problem that I'm actually worried about, I'm actually worried about this, is what's going to happen to the next generation of coders.
00:50:07.360 --> 00:50:09.840
So, okay, there's a future where we don't need that.
00:50:10.079 --> 00:50:10.719
All right, fine.
00:50:10.880 --> 00:50:12.639
And then you shouldn't be learning how to do that.
00:50:12.800 --> 00:50:13.760
Do you try to believe that by the way?
00:50:13.920 --> 00:50:14.960
Like seriously, like tough.
00:50:15.360 --> 00:50:16.400
I don't believe it at all.
00:50:16.559 --> 00:50:16.719
Okay.
00:50:16.880 --> 00:50:17.599
I don't believe it at all.
00:50:17.840 --> 00:50:20.880
I kind of don't either, and I don't know why exactly, but I but I'm with you.
00:50:20.960 --> 00:50:21.679
So keep going.
00:50:22.239 --> 00:50:43.360
So but the thing is, what plans to doubt in my mind every single time is when I talk to super smart people, you know, who work on top, you know, in one of these top companies who I'm friends with and who just are completely convinced that yeah, it's a done deal.
00:50:43.599 --> 00:50:54.719
You know, like top, top, top engineers in those top companies building those frontier models, they all tell me, yeah, yeah, I mean in two years, soft engineering dumb, you know.
00:50:54.880 --> 00:51:28.960
That's every time stops me in my tracks, and I'm like, whoa, like that and and if that person thinks that, and that's not you know, that's not somebody who is you know super enthusiastic about everything and uh who believes in the in and in every threat the internet you know has produced, this is somebody who's actually critical of a lot of things, so that always stops in my tracks, but at the same time, the dissonance with my experience with those LLMs is just so big every single time.
00:51:29.280 --> 00:51:33.440
So here's uh uh here's a way I see it.
00:51:34.480 --> 00:51:46.639
If we had an LLM where you could say, design me a piece of uh engineering, a piece of software, you know, that is actually really hard to design.
00:51:47.360 --> 00:51:58.960
And it doesn't have to be full-fledged, doesn't have to be the full thing, but you know, something that a graduate student would design and write and get right if you you gave them enough time, right?
00:51:59.199 --> 00:52:08.079
So let's say you told an LLM, hey, design me a database that has this and that property, right?
00:52:08.400 --> 00:52:20.719
And if it was actually able to produce something that works, then I would say, oh shit, you know, you know, time's up, it's it's happening, right?
00:52:20.960 --> 00:52:30.719
And or or a database of something else, you know, could be what would your threshold so dumb dumb uh it so happens I'm in the middle of such a project, by the way, and it's literally a database.
00:52:30.960 --> 00:52:36.400
Uh like what's your threshold for how one-shot-ified that needs to be, I guess, right?
00:52:36.480 --> 00:52:39.199
Because like it's still very much a project, to be clear.
00:52:39.280 --> 00:52:41.039
Like my experience so far is it's still very much like that.
00:52:41.360 --> 00:52:42.239
That's a that's the big thing.
00:52:42.480 --> 00:52:45.440
For me, it would have to be a one-shot, yeah, yeah, yeah.
00:52:45.519 --> 00:52:49.119
And it would have to be different enough from something that exists.
00:52:49.280 --> 00:52:52.880
So let's say here's my here's my my take.
00:52:53.280 --> 00:53:08.239
It would have to be one shot, it would have to be different enough for from something something that exists that you know it's it's uh it clearly it was not by copying, you know, another program or translating another program.
00:53:08.880 --> 00:53:17.519
And it would have to be uh sophisticated enough that you know the complexity uh you know is non-trivial.
00:53:17.920 --> 00:53:25.519
So to give you an idea, Postgres from Postgres from 30 25 years ago would be enough.
00:53:26.239 --> 00:53:54.320
You know, if if you were to say, write me an implementation of SQL that had this and that property, and that does this, this, this, and that, and you test this and that property, and it actually able to do the whole thing by itself, that means it was able to design the whole thing and was able to test the whole thing, and it was because so far the things that I see are things that don't have one of those three things, right?
00:53:54.559 --> 00:53:58.719
So either it's there's a gazillion of those on the internet.
00:53:58.960 --> 00:54:01.599
So you say, all right, write me a lock-free hash table.
00:54:01.679 --> 00:54:04.400
Well, how many lock-free hash tables are there on the internet?
00:54:04.800 --> 00:54:05.519
A lot of them.
00:54:05.840 --> 00:54:16.320
So if you read them all and if you, you know, you you you will be able to put something together without understanding what makes a hash table, a lock-free hash table, or a lock-free hash table.
00:54:16.639 --> 00:54:21.920
The the number two thing is, you know, it has to be by itself.
00:54:22.320 --> 00:54:35.360
Because if there's a human who's how hand holding and saying, oh no, do this and then don't do that, then okay, but then the human has you know help with the design decisions and who really made the design at this point, right?
00:54:35.599 --> 00:54:36.239
Right, right.
00:54:36.639 --> 00:54:41.760
And the third thing is that it has to be big enough, right?
00:54:41.840 --> 00:54:49.679
It cannot be something small where you keep on, you know, brute forcing the thing, you'll eventually get to something that works.
00:54:49.920 --> 00:55:00.800
So, you know, something the bar for me would be hey, write me a clone of Unix that has this and that property that no other Unix system ever had.
00:55:01.119 --> 00:55:01.599
Interesting.
00:55:01.760 --> 00:55:03.519
And um actually make it work.
00:55:03.760 --> 00:55:04.079
Yeah, yeah.
00:55:04.159 --> 00:55:06.800
Yeah, it boots on a PC and it like yeah.
00:55:07.280 --> 00:55:16.000
It boots on a PC, and if you can do that, and it doesn't have to be at the level of performance of Linux, it could be like Minix or something like that.
00:55:16.159 --> 00:55:27.679
You know, I I don't expect the full professional thing, but but if you can actually write that, that's the moment where I say, oh shit, shit, you know, we're fucked.
00:55:27.920 --> 00:55:29.679
Um that's it.
00:55:29.760 --> 00:55:32.159
Like this is my drop.
00:55:32.559 --> 00:55:58.000
If I can try and like close the circle here a little bit about learning to code too, like a pretty influential book in my journey here was discovering uh the Lions Book, freshman year in college, which is um there was a University of New South Wales professor in the 70s uh named Lions, and he basically produced this little book that was the source listing of Unix version 7, so like 1977 era Linux, or Unix rather, which was like 10,000 lines.
00:55:58.400 --> 00:56:02.880
Very, very comprehensible, like you can understand this program, it's not that crazy.
00:56:03.039 --> 00:56:11.920
And then uh a bunch of text that's about the same size, that sort of has cross-references to uh to lines and page numbers in the source code that explains how it all works.
00:56:12.079 --> 00:56:16.320
And if you kind of like read this short, this kind of like novella, it's it's 75 pages or something.
00:56:16.480 --> 00:56:19.440
If you read this novella, at the end of it, you kind of understand how Unix is.
00:56:19.599 --> 00:56:25.199
You understand like what the abstractions are, you understand how the machine, you know, how those abstractions map to machine facilities and so on.
00:56:25.280 --> 00:56:31.519
And it's an archaic machine and it's like there's nuance and it, you know, all the multiprocessor stuff we talked about that weren't multiprocessors yet.
00:56:31.599 --> 00:56:34.079
There's lots of stuff that's very uniprocessor and blah, blah, blah.
00:56:34.239 --> 00:56:36.239
There's still lots of stuff of work to do on Unix.
00:56:36.400 --> 00:56:50.960
Um, but that actually inspired me freshman year to uh to take the operating system class uh that Brown University had at the time, which at the time was was um based on a bunch of kind of ideas and operating system research that were that were au courant in the late 90s, right?
00:56:51.039 --> 00:56:57.679
So it was a microkernel, it was object-oriented, it was in C, uh, lots of other things about it that were kind of hard.
00:56:58.079 --> 00:57:06.480
And I took the class, failed the class, took the class again, passed the class, uh, and then wanted to rebuild the project for it.
00:57:06.559 --> 00:57:17.920
And so me and three other uh folks, like Michael Castell, Dave Powell, and Jason Lango, built Winix, which is like our attempt to bring the Lions book into the early 2000s, right?
00:57:18.000 --> 00:57:22.000
It was like our attempt to bring, you know, a 10,000 line or so comprehensible.
00:57:22.159 --> 00:57:26.400
You can get your head around in a couple of weeks, maybe a couple months of study, Unix implementation of the modern era.
00:57:26.639 --> 00:57:30.239
As far as I know, this is still the operating systems lab course at Brown University.
00:57:30.320 --> 00:57:33.679
Um, Tom Deppner, the professor, like actually wrote a wrote a textbook about it.
00:57:33.920 --> 00:57:40.400
And, you know, the property that this one had was basically like tractability, it was just sort of like make all the decisions in like a relatively simple way.
00:57:40.480 --> 00:57:52.239
But it's still, there's still a file system, and the file system still has you know permission bits on it, and there still are, you know, um, you know, there's still virtual memory, there's still fork, you still like care about inefficient implementation of fork and all these things.
00:57:52.400 --> 00:57:56.239
And there's a lot of nuance there that like I think is is pretty rich and pretty satisfying.
00:57:56.480 --> 00:57:58.320
And you're right, that'd be fascinating to see.
00:57:58.480 --> 00:58:04.159
And and while we're and while we're on this, by the way, you mentioned before about like sort of the life cycle of programs.
00:58:04.320 --> 00:58:10.239
It's a it's a fact about the software industry that the vast majority of economic activity and software is in maintenance, right?
00:58:10.320 --> 00:58:19.519
The vast majority of sort of where the money goes, yeah, whether you're in sort of like a fang company or opening out or whatever, yeah, um, is not you know, and that's usually a big problem.
00:58:19.599 --> 00:58:19.760
Yeah.
00:58:20.000 --> 00:58:25.119
Because the the people who are in charge of the company, they need those maintenance people.
00:58:25.280 --> 00:58:31.360
And what they don't realize is those maintenance people are very good at maintaining the system in place.
00:58:31.679 --> 00:58:36.320
They're very bad, horrible people to ask, should we should we do this new thing?
00:58:36.480 --> 00:58:36.960
Right, right.
00:58:37.119 --> 00:58:43.039
And that's how big companies become basically, you know, frozen.
00:58:43.119 --> 00:58:50.079
Like nothing new happens because every time there's a new idea, the boss thinks, Well, I'm gonna ask my guys what they think about it.
00:58:50.239 --> 00:58:52.320
Well, I can tell you what you guys think about it.
00:58:52.400 --> 00:58:56.639
And I mean, some of the teams that you were involved in, I've seen them evolve.
00:58:56.800 --> 00:58:57.119
Yeah, yeah.
00:58:57.280 --> 00:59:02.880
They've seen go from you were the new kids on the block, you know, full of new ideas and dah, dah, dah.
00:59:03.039 --> 00:59:12.320
And you come 10 years later, and whenever you ask that same team, well, shall we do you're not able to finish the shall we do this with a single thing?
00:59:12.559 --> 00:59:13.440
No, nothing's possible.
00:59:13.679 --> 00:59:15.760
No, no, go away.
00:59:15.920 --> 00:59:18.320
Go away, leave it, leave the room now.
00:59:19.599 --> 00:59:23.119
And so and so that's a problem for companies, right?
00:59:23.360 --> 00:59:31.519
Because they need people to rely on that that they trust, because of course what matters to them is a system that is actually running, right?
00:59:31.679 --> 00:59:33.280
And that's how they make the money.
00:59:33.440 --> 00:59:36.880
So they don't want to upset the existing ecosystem.
00:59:37.039 --> 00:59:46.480
So a natural inclination of the leaders of a company like that is going to be like, well, let's ask the people who are actually running my stuff what they think about this new thing.
00:59:46.719 --> 00:59:52.800
And what they don't realize is that the people who are maintaining things, they love that mindset.
00:59:52.960 --> 00:59:55.519
They want to keep on maintaining whatever they're maintaining.
00:59:55.760 --> 00:59:59.920
If you come in, come in with something too drastic, too novel, they don't like it.
01:00:00.079 --> 01:00:05.440
And it's not because they're evil spirited or you know, they're bad people, none of that.
01:00:05.599 --> 01:00:10.000
It's just that it's not in their temperament to like new things.
01:00:10.239 --> 01:00:14.480
Can I try and like get through a couple of takeaways here that are hopefully useful?
01:00:14.880 --> 01:00:16.400
So, for one thing, right?
01:00:17.519 --> 01:00:18.320
Learn to code.
01:00:18.480 --> 01:00:19.840
Like that was one of the preliminaries.
01:00:19.920 --> 01:00:23.920
It's like the there's the people are asking the question is knowing how to code a depreciating asset?
01:00:24.079 --> 01:00:25.280
I think it's an appreciating asset.
01:00:25.360 --> 01:00:31.039
It seems like it's providing more leverage, not less, over time as these agents kind of take over some of the grittier work.
01:00:31.280 --> 01:00:32.639
Um, how do you learn to code?
01:00:32.960 --> 01:00:34.480
You learn to code by coding, right?
01:00:34.639 --> 01:00:42.880
By writing a lot of it, including probably writing a lot of it without the aid of LLMs, which I have to admit seems to require a superhuman amount of discipline.
01:00:43.119 --> 01:00:44.800
I don't have a perfect solution to that right now.
01:00:45.199 --> 01:00:50.320
You know, we've we've talked a little about long hand and magical keyboards that that force you to do it, maybe.
01:00:50.400 --> 01:00:52.719
Um maybe other pedagogical tools are part of it.
01:00:52.960 --> 01:00:54.719
And you also need to read the code.
01:00:54.880 --> 01:01:10.719
Like I think neither of us think that the era of vibe coding, at least for ambitious software projects, maybe you can vibe code your CRM or your recipe book or your you know, your exercise diary, things that have been done a million times before, but doing something new at the boundaries of software, which is what Red New Software is for, right?
01:01:10.800 --> 01:01:13.039
If you're not doing something new, just use the old software.
01:01:13.199 --> 01:01:16.639
Um if you're not if you're doing something new at the boundary of software, you still have to read it.
01:01:16.719 --> 01:01:25.840
And the more you know about how computers work and the more you know about how code works, the more expert you're gonna be at reading it and uh at exercising editorial control over what these agents are doing.
01:01:26.079 --> 01:01:28.400
What are some other big takeaways that were missing here?
01:01:28.719 --> 01:01:29.760
It's just fun.
01:01:30.000 --> 01:01:32.079
You know, it's funny because it's fun.
01:01:32.400 --> 01:01:38.800
You know, it's like, you know, it's as if somebody was telling you, you know, why should I read poetry?
01:01:39.039 --> 01:01:41.760
Well, because, you know, it's fucking awesome.
01:01:41.920 --> 01:01:47.760
You know, that's that's uh that's the other thing with science and computers and maths and all that stuff.
01:01:47.920 --> 01:01:51.039
It's fucking awesome, you know, an LLM can do it.
01:01:51.119 --> 01:02:05.920
Yeah, an LM can do it, fine, but I mean, it's true that now we're professional software developers, so we think like professional software developers, which is like at this level you're expected to write things that are new, that's true.
01:02:06.400 --> 01:02:14.000
But remember, for a long time we were not professional developers, and we're just writing things that were not new.
01:02:14.159 --> 01:02:21.599
Like, I've written a MADOC, and you know, Jason would probably laugh at my implementation, you know?
01:02:21.920 --> 01:02:28.000
Well, but I guarantee you Jason started off by just writing, you know, by writing a free list or something, by just writing some malloc that you'd yeah.
01:02:28.400 --> 01:02:28.880
Exactly.
01:02:29.119 --> 01:02:30.960
And so I think that's the other thing.
01:02:31.119 --> 01:02:34.000
Like, you know, it's just awesome.
01:02:34.320 --> 01:02:47.199
You know, understanding mathematics, understanding computers, understanding science is awesome, and it will, you know, open your mind to so many other things that you know that's the missing takeaway for me.
01:02:47.360 --> 01:02:49.920
Like, yeah, it's just part of a birthright as humans, right?
01:02:50.000 --> 01:02:51.519
It's just a part of intellectual culture now.
01:02:51.760 --> 01:02:53.920
It's like this is a thing you can do, and it's rad.
01:02:54.880 --> 01:02:56.320
It's rad, exactly.
01:02:56.480 --> 01:03:03.280
And you'll understand things in ways that you know will change the way you see the world and how you think about things.
01:03:03.440 --> 01:03:14.159
So, for example, in mathematics, what's really good when you when you're good at maths, which is not my case, but what maths it brought to me is the ability to formalize, right?
01:03:14.320 --> 01:03:28.800
So you have a problem, and I still do it today, you know, like whenever a problem gets a little bit complic complicated, even you know, for mundane things, I still find myself going like, okay, x plus two equal y.
01:03:29.199 --> 01:03:44.000
And I'm basically formalizing whatever, you know, in that particular situation, whatever, you know, the the problem, how it should be formalized, and then I apply the rules of mathematics to solve the problem, even in mundane situations, right?
01:03:44.239 --> 01:03:50.159
And for software, I think, and I I think I've said that before, but I'll say it again.
01:03:50.480 --> 01:03:56.639
What is the difference between, and that's why so many people are good at Excel and not so good at programming?
01:03:56.880 --> 01:04:10.400
The difference for me between a programmer and on and a non-programmer is that the programmer is able to simulate the the machine, what the machine is doing, especially what's happening in memory in their mind, right?
01:04:10.559 --> 01:04:19.280
And so that the for loop is really, or the non-trivial while loop is really where where for me the difference happens, right?
01:04:19.440 --> 01:04:24.480
Where you take somebody who knows Excel, they never have to think about state.
01:04:24.800 --> 01:04:30.400
They just have this thing is equal to that thing, which is equal to that thing, and then I do plus this and plus that.
01:04:30.639 --> 01:04:45.920
The moment you have to maintain a state in your mind, that's the moment you become a programmer in my book, which is ironic because I like functional programming, and in functional programming, you're not supposed to reason in terms of states, but you know, I'm gonna leave the irony at that.
01:04:46.239 --> 01:04:48.480
Uh but you know what?
01:04:48.639 --> 01:04:52.000
This ability to simulate things will help you.
01:04:52.239 --> 01:04:55.039
There are many, many cases where you'll have to do that.
01:04:55.199 --> 01:04:59.039
You know, in your life, um, you're running, you know, anything.
01:04:59.119 --> 01:05:01.840
You're running, you're trying to make a decision, you know.
01:05:02.079 --> 01:05:03.599
Shall I take this job?
01:05:03.840 --> 01:05:06.000
You know, and you're running a simulation.
01:05:06.079 --> 01:05:10.000
You're like, well, if I take this job, I have to go and work at this place.
01:05:10.159 --> 01:05:12.239
Is this place close enough to where I live?
01:05:12.480 --> 01:05:15.679
And then if I do this, then this will happen and that will happen.
01:05:16.159 --> 01:05:23.440
You're running a simulation, you're maintaining a state in your mind, and that's just one example, but there'll be many others.
01:05:23.679 --> 01:05:42.960
And so even if you know the AI takes over the world and programming is completely useless in terms of you know getting dollars out of it and making a living out of it, the fact that you still went through that mental exercise has a ton of value in my book.
01:05:43.280 --> 01:05:44.079
Yeah, yeah.
01:05:44.639 --> 01:05:45.199
NC.
01:05:45.760 --> 01:05:46.159
Awesome.
01:05:46.320 --> 01:05:46.639
Yeah, yeah.
01:05:47.360 --> 01:05:51.119
Old Man Yells a cloud, eat your Wheaties, learn the code, kids.
01:05:51.280 --> 01:05:51.440
Okay.
01:05:52.239 --> 01:05:52.639
Exactly.
01:05:52.880 --> 01:05:55.679
Learn to code, it's also it's learn to code, it's awesome.
01:05:55.920 --> 01:05:59.679
We recently had a guy start at a pebble bed, a young man named Bology Dagupati.
01:06:00.079 --> 01:06:02.000
Who's a grandmaster of chess?
01:06:02.239 --> 01:06:06.079
And he's he's very young, he's 20 or early 20s anyway.
01:06:06.320 --> 01:06:09.920
And uh he's only lived in a world where computers are dominant at chess, right?
01:06:10.000 --> 01:06:14.960
He's only lived in a world where this is an activity humans still do, but computers are dominant at it.
01:06:15.280 --> 01:06:19.440
And the way that they use computers is non-trivial, right?
01:06:19.519 --> 01:06:21.280
They still rely on these tools, right?
01:06:21.360 --> 01:06:24.559
These tools help make them better at playing other humans.
01:06:24.880 --> 01:06:27.360
Um, and it's interesting how they do that.
01:06:27.440 --> 01:06:49.679
And I think like that's a thing that still is waiting to be discovered and figured out is like, okay, you still need to learn how to do all the things that programmers do, but you also have these weird little kind of superhuman assistants, and and can we use them to develop our intuition and and strengthen faster than writing things in notebooks and banging our heads against the keyboards the way Julian and I did, because that's what worked for us.
01:06:49.920 --> 01:06:51.760
Um that's interesting.
01:06:52.000 --> 01:06:53.519
Um But there's a difference.
01:06:53.679 --> 01:06:59.280
The the the big difference is that chess is against an opponent, uh programming is against yourself.
01:06:59.519 --> 01:07:01.199
Yeah, chess is an entertainment activity, right?
01:07:01.280 --> 01:07:02.559
We do it, we always did it for the other.
01:07:02.719 --> 01:07:03.920
Well, and and there's an opponent.
01:07:04.079 --> 01:07:05.280
There's an opponent, right?
01:07:05.360 --> 01:07:05.519
Yeah.
01:07:05.679 --> 01:07:11.280
And so what's different with programming is that it's against the battle is against yourself.
01:07:11.519 --> 01:07:16.239
You have an idea in mind, and you're almost certainly going to get it wrong.
01:07:16.480 --> 01:07:21.039
And you think you're right, but your reasoning is incorrect, incomplete.
01:07:21.360 --> 01:07:36.480
And that's where the value of letting that go through, you know, and letting that play out, and you know, pay the price for that bad reasoning down the line, that's invaluable, right?
01:07:36.719 --> 01:07:42.800
So, which is a little bit different than your pl when you're playing against another another uh an opponent, right?
01:07:43.280 --> 01:07:46.320
Then you only need to be better than your opponent.
01:07:46.480 --> 01:07:48.800
You don't need to be right in absolute terms.
01:07:48.960 --> 01:07:56.079
So you're not really playing against yourself, you're playing against somebody else, which changes, I think, the dynamic.
01:07:56.559 --> 01:08:02.480
Yeah, I mean, but if we viewed, if we try to view programming as like a solitaire game or something, I'm not sure it's so intrinsically different, right?
01:08:02.639 --> 01:08:08.639
You're um when you're bad at chess, you lose a lot and you learn from that losing experience and you you know improve.
01:08:08.880 --> 01:08:17.840
And um when you're very good at it, you start to think of yourself as as playing against yourself in the sense of like, you know, it might not be that I'm that I'm facing a bunch of opponents that are way stronger than me.
01:08:18.880 --> 01:08:30.720
But that's kind of my point, which is that if you are playing chess and you're playing a ton of chess against humans, then your mistakes you will have to own that or own them at some point, right?
01:08:30.880 --> 01:08:40.720
And so that is the nice feedback loop, which is like, well, if you suck at chess, you're gonna lose tournaments and you're gonna try to improve, you know, a chess because you don't want to lose, right?
01:08:40.960 --> 01:08:47.520
And the problem with with with programming, because it's solitary, solitary, am I pronouncing this right?
01:08:47.600 --> 01:08:48.000
I don't know.
01:08:48.159 --> 01:09:01.279
Yeah, um you don't have that experience, meaning when something goes wrong, you you don't have that feedback loop that tells you this this was wrong, you shouldn't do things this way.
01:09:01.439 --> 01:09:22.000
There and and so that's why I think it's gonna be much harder for programming or for maths, you know, or anything that doesn't require another human being, as long as you can compete compete with an another human being or compete in a place where you cannot cheat, where you cannot use the computer, you're safe.
01:09:22.239 --> 01:09:22.800
Yep, yep.
01:09:23.039 --> 01:09:25.359
You will improve because you have that feedback loop.
01:09:25.520 --> 01:09:31.840
You will remember the times where you lost, it will be painful, and you'll remember to do better next time.
01:09:32.000 --> 01:09:39.840
But if you don't have that feedback loop that tells you you are wrong and it's gonna cost you, it's gonna be tough.
01:09:40.319 --> 01:09:42.640
Thank you for listening to another episode of Computers, Coffee and Beer.
01:09:42.720 --> 01:09:44.640
We had the computers, we had the coffee, we had the beer.
01:09:44.800 --> 01:09:47.760
If I may crudely summarize, should you learn to code?
01:09:47.840 --> 01:09:48.640
Oh my god, yes.
01:09:48.800 --> 01:09:49.119
Absolutely.
01:09:49.279 --> 01:09:51.039
First of all, because it's wonderful and you're gonna love it.
01:09:51.199 --> 01:09:55.600
Secondly, because it's actually gonna make you better at your software engineering job, which isn't going anywhere.
01:09:55.920 --> 01:09:57.199
How do you go about doing it?
01:09:57.359 --> 01:09:58.800
To some extent, the old-fashioned way.
01:09:58.880 --> 01:10:02.079
Yeah, there's lots of things, there's lots of crutches out there that make it hard to do it the old-fashioned way.
01:10:02.239 --> 01:10:05.920
Figure out how to tie your hand, tie yourself to the mast and learn it anyway.
01:10:06.000 --> 01:10:21.279
Um, and you're still gonna need to read a lot of code, almost even in a world where it's only computers writing the code, the fact that you know how programs work and know how computers work is gonna make you way more overpowered as a user of these coding genies than uh than people who are just vibe coding in today's parlance.
01:10:21.760 --> 01:10:25.279
Um, Julian, thank you so much, and thank you for listening.
01:10:25.359 --> 01:10:29.039
And we look forward to talking to you on another episode of Computers, Coffee and Beer.
01:10:29.279 --> 01:10:31.840
Um, remember to smash like and subscribe.
01:10:32.239 --> 01:10:33.439
Uh, and have a good one.
01:10:33.680 --> 01:10:34.640
Thanks, Keith.