WEBVTT
00:00:00.000 --> 00:00:11.000
Alright, welcome to another episode of Runtime Arguments, the podcast where we talk about all kinds of cool, techy… Programming… things.
00:00:11.000 --> 00:00:13.000
Well, we, we.
00:00:13.000 --> 00:00:19.000
I'm Jim McQuillan, and uh, my partner here is Wolf. Say hello, Wolf.
00:00:19.000 --> 00:00:20.000
Hey everybody.
00:00:21.000 --> 00:00:37.000
So, uh, yeah, this is our, uh, episode 30, our 31st episode. And today, uh, we're going to talk about what to do when you have more problem than you have computer.
00:00:37.000 --> 00:00:52.000
We're gonna get into lots of cool things, uh, multi-processing, threading, uh, I don't know what all. Wolf's got all kinds of, all kinds of tricks up his sleeve, and we're gonna learn some of those today, so I'm kinda looking forward to that. So, how was your week?
00:00:52.000 --> 00:00:53.000
Wolf?
00:00:53.000 --> 00:01:08.000
Oh my god, so annoying! But, you know, in a nice way. I live in a rental and have for a very long time. And I have the greatest landlord of all time. He is a really terrific guy.
00:01:08.000 --> 00:01:18.000
And, uh, they went out of their way to replace the flooring over the major part of the house, and that meant.
00:01:18.000 --> 00:01:33.000
furniture going into a pod in the front yard and the family sort of splitting up into various locations. So my wife and me in a hotel with one of the three dogs.
00:01:33.000 --> 00:01:43.000
Uh, one of the boys living with grandma, with two of the dogs, and one of the boys here in the house, not to leave his room. So…
00:01:44.000 --> 00:01:46.000
He stuck it out, huh?
00:01:44.000 --> 00:01:54.000
They said… he did, he did. So they, they said this would be 2 days, and I said, okay, it'll probably be 5, I'll reserve a hotel room for 4.
00:01:55.000 --> 00:02:01.000
And we came back today and they were still doing a little bit of work. So I think I was right.
00:01:59.000 --> 00:02:04.000
And today was the fourth day? Is that right?
00:02:04.000 --> 00:02:13.000
It was like I spent Sunday night and this is Thursday as we speak, so Wednesday night was our last night there.
00:02:08.000 --> 00:02:10.000
Yah.
00:02:10.000 --> 00:02:11.000
Okay.
00:02:13.000 --> 00:02:16.000
So it's, it's, it's good to be back home, isn't.
00:02:16.000 --> 00:02:27.000
It is, and it looks good, it's gorgeous. I mean, of course, Jennifer is not as happy as me, because she sees things and notices things.
00:02:27.000 --> 00:02:31.000
So she can't be as easily satisfied as I am.
00:02:28.000 --> 00:02:37.000
And… And us… us guys, we just… I don't know what it is, we just don't pay attention, do we?
00:02:37.000 --> 00:02:40.000
We just, we walk in, we're happy.
00:02:37.000 --> 00:02:43.000
You know, in this way, ignorance, ignorance kind of is bliss.
00:02:43.000 --> 00:02:44.000
Yeah.
00:02:44.000 --> 00:02:46.000
Just be happy, right?
00:02:45.000 --> 00:02:47.000
How about you? How was your week?
00:02:46.000 --> 00:02:59.000
Uh, busy this week. I just got back from vacation. I spent, uh, 9 days, uh, gone. Uh, my buddy Ron and I, we drove up to see Scotty in Winnipeg.
00:03:00.000 --> 00:03:13.000
Which, uh, you know, we live in the Detroit area, so that's, uh, round trip, that's 2,500 miles in the car, and, uh, you know what? I enjoy every minute of it.
00:03:13.000 --> 00:03:20.000
I, I, I love the, I love the adventure. I love the drive. And we, we, uh, we had a good time. So, but I'm back.
00:03:18.000 --> 00:03:33.000
You're lucky it's summertime, because in the winter in Canada, you can often get pulled over by a snowplow. I don't know if you've noticed the color of the lights on the various vehicles, but they're yellow on…
00:03:27.000 --> 00:03:31.000
Or run over. Yeah.
00:03:31.000 --> 00:03:34.000
Yeah, you get run over.
00:03:33.000 --> 00:03:39.000
police vehicles and they're red and blue or something like that on snow plows.
00:03:37.000 --> 00:03:39.000
Yeah, I don't know.
00:03:39.000 --> 00:03:50.000
I don't know. But it was, it was a good time, but of course, uh, being gone for 9 days, it put me behind, uh, in my normal work, so this week was catch-up time.
00:03:50.000 --> 00:03:58.000
And, uh, so that's what I did. Right up until, like, 2 minutes before we turned on the microphones, I was busy working.
00:03:58.000 --> 00:04:04.000
Uh, but we're here now and we're gonna talk about some cool stuff. Uh, let's see, do we have feedback?
00:04:04.000 --> 00:04:22.000
Um, last episode, we talked about Wi-Fi, and uh, you know… uh, uh, we've got lots of friends on the podcast, and one of them is, is my friend Marlon, our friend, uh, he works at Meta, he's a brilliant, uh, engineer there, and, um.
00:04:22.000 --> 00:04:28.000
Uh, he listens and, uh, you know, we talked about, uh, different, uh, uh.
00:04:28.000 --> 00:04:44.000
levels of Wi-Fi, uh, different, uh, you know, 802.11 something. And I mentioned that the whole thing that drove me to that conversation was the fact that in my house, I, the signal doesn't reach everywhere I would like it to reach.
00:04:44.000 --> 00:04:58.000
And we talked about mesh networks, and various ways I can fix this, and one of the things I was thinking about doing is just getting a better access point that maybe does Wi-Fi 7, which is maybe a stronger signal.
00:04:58.000 --> 00:05:13.000
Marlin says, just move my access point. Move it to a more central location in the house and I'll be fine, because placement is probably the most important thing when it comes to Wi-Fi signal throughout the building.
00:05:13.000 --> 00:05:16.000
And, um, uh, yeah.
00:05:13.000 --> 00:05:21.000
Can I actually use a phrase? Is it location, location, location?
00:05:17.000 --> 00:05:19.000
Yes.
00:05:20.000 --> 00:05:38.000
That's exactly what it is, like it always is, right? They answer to everything. And the location of my access point is not a good location. Unfortunately, the way my house is laid out, moving it, drilling through walls, and drilling through floors.
00:05:38.000 --> 00:05:53.000
Um, it presents a challenge, so I'm gonna… I don't know, maybe over the next month or so, I'm gonna try to figure this thing out. I suspect, uh, I… I can get, uh, much better coverage without spending any money, and that… that… that appeals to me.
00:05:53.000 --> 00:06:12.000
Uh, but thanks, Marlon, for that advice. Uh, it's… it's kind of obvious, um… you know, I, I, the solutions I look for are usually on Amazon, uh, not on, uh, just common sense. But, uh, so yeah, that was, uh, that's the feedback we got from the last episode.
00:06:12.000 --> 00:06:22.000
Um, and of course, we love feedback, we encourage it, send us feedback, uh, you can send it to, uh, feedback at runtimearguments.fm.
00:06:22.000 --> 00:06:39.000
Uh, of course, we'll, uh, we'll repeat that again at the end, and we have show notes, and we have a website, we'll… we have all that contact information there, so you can, um… Tell us what you'd like to hear. Tell us what we said wrong. Sometimes tell us what we said right.
00:06:39.000 --> 00:06:50.000
uh, hopefully today, um, we've got some ideas that will, uh, cause people to tell us what they think. So, hopefully. Anyway.
00:06:49.000 --> 00:06:51.000
You, you know.
00:06:51.000 --> 00:06:55.000
Over the course of the last week, something really good did happen.
00:06:56.000 --> 00:06:58.000
Uh, what's that?
00:06:56.000 --> 00:07:01.000
And that is, we took my little open source idea.
00:07:01.000 --> 00:07:03.000
And.
00:07:02.000 --> 00:07:04.000
Right.
00:07:03.000 --> 00:07:13.000
for a DAP multiplexer, and, um, and we did a big demo of it, um, that is now recorded and available on YouTube.
00:07:14.000 --> 00:07:41.000
Yeah, you gotta go to mug.org, or go to the… mug.org is where you'll see the announcement. If you go to YouTube and just search for mug.org, you'll see Wolf's, um, talk on… uh, uh, DAP mux, which was really, really cool for, um, controlling a debugger and seeing variables and… all the kinds of things you want to do when you're debugging. Uh, so check that out.
00:07:41.000 --> 00:07:52.000
Um, you know, when we do these episodes, we, uh, we put together a little… sometimes it's an outline, sometimes it's, uh, it's never a script.
00:07:52.000 --> 00:07:59.000
But, uh, we sort of jot down the ideas of what we're going to talk about. So we did that with this episode.
00:07:59.000 --> 00:08:11.000
create a great outline, discussing all the things we're gonna discuss. And, uh, a couple of days ago, Wolf and I had a conversation about it, and it was kind of funny, because we… we got into a heated, uh, discussion.
00:08:11.000 --> 00:08:17.000
about one of the things. We'll get more into it later. But, you know.
00:08:17.000 --> 00:08:26.000
I had a realization, and this is true about a lot of things between Wolf and myself, we come at things from a whole different direction.
00:08:26.000 --> 00:08:27.000
Umm.
00:08:27.000 --> 00:08:43.000
Wolf, Wolf is a genius software developer and he thinks, I don't know what he thinks, like I haven't figured it out, but it's what's going on in his brain is different from what's going on in mine.
00:08:43.000 --> 00:08:54.000
I'll tell you what it was. We're going to cover it more later, but it's the idea of multiprocessing and clustering.
00:08:54.000 --> 00:09:09.000
And in Wolf's mind, they're exactly the same thing. And, and I understand from his point of view, yeah, they are, the more I think about it. But from my point of view, I think in terms of what's it going to take to implement this thing at the hardware level.
00:09:09.000 --> 00:09:18.000
Because I'm kind of a full solution provider, and I have to think about plugging things together. And that's where I think.
00:09:14.000 --> 00:09:15.000
Plus.
00:09:16.000 --> 00:09:26.000
And you sign the… you sign the checks. When… when you need 5 machines to do it, you have to sign a check that says 5 machines!
00:09:20.000 --> 00:09:21.000
Yes.
00:09:22.000 --> 00:09:24.000
You're right.
00:09:25.000 --> 00:09:36.000
Right. Right, right. So I'm thinking about it from a whole different standpoint, and to me, uh, multiprocessing and clustering are entirely different.
00:09:36.000 --> 00:09:48.000
And to Wolf, they're the same thing. It's just how we look at things. Um, and when I realized that, it's like, oh, that's what was going on the other day when we had that conversation.
00:09:48.000 --> 00:10:02.000
And really, like I said, that explains lots of our conversations. We just come at it from different points of view, and maybe that's what makes this podcast interesting, assuming it is. Um, but anyway…
00:10:00.000 --> 00:10:04.000
I mean, that's half the title.
00:10:03.000 --> 00:10:07.000
Yeah, right. Yeah.
00:10:07.000 --> 00:10:24.000
Yeah, I guess it's the purpose of the podcast, right? We have different ways of looking at things and sometimes it goes smoothly and sometimes not so smoothly, but this is going to be a great episode. I think I'm ready to move on. Are you?
00:10:24.000 --> 00:10:27.000
Uh, I think I'm ready.
00:10:25.000 --> 00:10:27.000
You ready to go?
00:10:27.000 --> 00:10:30.000
All right, let's get into it. Wolf, go ahead.
00:10:30.000 --> 00:10:50.000
Um, so… There's lots of ways to talk about this topic. We could be super… pedantic, and… I don't think that's valuable. What I think is valuable is getting to the essence of the problem.
00:10:50.000 --> 00:10:55.000
Um, and the problem is when, uh.
00:10:55.000 --> 00:11:00.000
You're solving some… Workload, some problem.
00:11:01.000 --> 00:11:09.000
Usually, you're at one of the extremes. Usually, you've got more computer than you have code.
00:11:09.000 --> 00:11:19.000
Um, for instance, uh, my Mac has… I don't know. What? How many cores does my Mac have? Does it, does it say up here if I pull it down?
00:11:17.000 --> 00:11:21.000
Oh, 20, I, I, a bunch.
00:11:21.000 --> 00:11:23.000
Yeah, a bunch of cores.
00:11:21.000 --> 00:11:24.000
I can't keep track of them anymore.
00:11:23.000 --> 00:11:44.000
like, maybe 18 or something, and 40 GPU cores, something like that, and uh… a regular, like, an NVIDIA GPU card has, you know, in the tens of thousands of cores, or something. They're all simple, but… but it's got so many. So… Almost all the time.
00:11:44.000 --> 00:12:04.000
You have one core doing your job. And… some huge number of cores not doing your job. If you're lucky, they're doing some other job that is also important to you, but a lot of times, all that hardware is just a waste.
00:12:04.000 --> 00:12:16.000
Or, you're in the opposite situation. You have a problem that is so big that you didn't bring enough computer to solve it.
00:12:17.000 --> 00:12:22.000
And it just seems like you're so rarely in the middle. So the question is.
00:12:22.000 --> 00:12:28.000
How do you balance this? And so I want to talk about.
00:12:28.000 --> 00:12:41.000
Essentially, this whole episode is… Um, when getting… Two hunks of code, both of them aimed at the same goal.
00:12:41.000 --> 00:12:49.000
to run at the same time. At least… From your point of view, it looks like they're running at the same time.
00:12:49.000 --> 00:12:54.000
Now, there's constraints. There's gonna be some reason.
00:12:54.000 --> 00:13:06.000
that they can or can't actually run at the same time. There's something that you don't have enough of. That's the whole reason why this is a problem that needs addressing.
00:13:06.000 --> 00:13:13.000
Maybe you don't have enough, um… Uh, CPU.
00:13:13.000 --> 00:13:27.000
your problem is that, you know, you… remember the old days, when we just had one CPU in our computer? And if you wanted to solve a problem that.
00:13:23.000 --> 00:13:24.000
Oh yeah.
00:13:27.000 --> 00:13:40.000
you know, would easily benefit from 10 CPUs or something. Instead of running it on 10 CPUs, what you did is you ran it, but it took 10 times as long, or worse.
00:13:40.000 --> 00:13:41.000
Umm.
00:13:41.000 --> 00:13:48.000
These days, maybe you have 10 CPUs, and maybe you can use them all at the same time.
00:13:48.000 --> 00:13:57.000
Sometimes, the thing that you're constrained on is more about time than it is about processing power.
00:13:57.000 --> 00:14:06.000
Uh, for instance, there are things across a boundary that just take time to get.
00:14:06.000 --> 00:14:14.000
Uh, one of those things is, um… IO, data. If you have stuff on a disk.
00:14:14.000 --> 00:14:22.000
I don't care how fast your disk is. I mean, I guess I technically do, but nobody's made a disk fast enough to make this not be a concern.
00:14:22.000 --> 00:14:34.000
A disk is so far away and so slow compared to even the first-level cache in your computer that you are waiting for disk stuff to happen.
00:14:34.000 --> 00:14:47.000
So… Here's two different ways to solve the waiting problem on a computer. One way is you add more processing power.
00:14:47.000 --> 00:14:49.000
You really do run.
00:14:49.000 --> 00:14:57.000
Multiple hunks of code at the exact same time, but you do it on multiple processing units.
00:14:57.000 --> 00:15:07.000
Um, maybe those processing units are cores. Maybe those processing units are, um, I don't know if you remember, um.
00:15:07.000 --> 00:15:16.000
Uh, what did they call that special thing where, uh, you only had one core, but it had two processing threads?
00:15:14.000 --> 00:15:20.000
Oh, hyperthreading? That still exists. Oh yeah.
00:15:17.000 --> 00:15:28.000
hyperthread, or something, yeah. Uh, maybe you're running it on a GPU, maybe you're running it in a cluster. Uh, it's Kubernetes, or it's a Bay… who knows?
00:15:28.000 --> 00:15:41.000
That's one answer. Multiple pieces of code at the exact same time. That comes with problems of its own and I'm going to get to those. Or maybe your problem is that you're spending a lot of time waiting.
00:15:41.000 --> 00:15:56.000
So instead of being about running multiple pieces of code at the exact same time, your problem is about scheduling. Let me pick the right piece of code to run right at this minute.
00:15:56.000 --> 00:15:59.000
We have two different words.
00:15:59.000 --> 00:16:03.000
One of those words is concurrency.
00:16:03.000 --> 00:16:06.000
And the other word is parallelism.
00:16:06.000 --> 00:16:16.000
These are two different approaches for solving those problems. Concurrency is when you're going to use scheduling to solve it.
00:16:16.000 --> 00:16:21.000
You pick which piece of code is going to execute right now.
00:16:21.000 --> 00:16:30.000
Threads are a way of doing concurrency. They are an implementation for a concurrent solution.
00:16:30.000 --> 00:16:37.000
Multiple CPUs are an implementation for parallelism.
00:16:37.000 --> 00:16:49.000
Each of those two approaches comes with problems. There are words in the languages that you use that imply you are using one or the other.
00:16:49.000 --> 00:16:54.000
Some languages don't tell you what answer.
00:16:54.000 --> 00:17:09.000
that their language feature is going to leverage. And maybe they don't even choose what answer, um, that feature is going to leverage. Maybe it depends on how many CPU cores you have.
00:17:09.000 --> 00:17:15.000
Whether it's going to be threads, or cores, or whole separate processing units.
00:17:14.000 --> 00:17:15.000
But.
00:17:15.000 --> 00:17:17.000
But…
00:17:17.000 --> 00:17:18.000
Yes?
00:17:17.000 --> 00:17:23.000
So, what I'm hearing is, if I have a program that spawns threads.
00:17:24.000 --> 00:17:27.000
It's not going to take advantage of multiple cores.
00:17:27.000 --> 00:17:34.000
It depends on the language and the interface that it gives you, what the underlying implementation is.
00:17:34.000 --> 00:17:48.000
The typical answer is that a thread is lightweight and it runs in the CPU you are in. It is a scheduling problem. It is concurrency.
00:17:48.000 --> 00:17:54.000
But really, really good languages that can use both.
00:17:54.000 --> 00:18:05.000
have the option of saying, here's a thread I can run on a different processing unit. For instance, um, Go has Go routines.
00:18:05.000 --> 00:18:06.000
Okay.
00:18:05.000 --> 00:18:12.000
If necessary, a go routine could be a thread on your local CPU, on the CPU you're using.
00:18:12.000 --> 00:18:25.000
or Go can make a Go routine run on a separate CPU. Why is this an option? Because the communication between a Go routine and your main.
00:18:25.000 --> 00:18:39.000
hunk of executing code is a channel, not shared memory. And that means that it can send the data to the right place. If it's on the same CPU, maybe that's a real short trip.
00:18:39.000 --> 00:18:56.000
If it's on a different CPU, maybe it's a longer trip, but it doesn't have to worry about contention, which is one of the problems of both of these mechanisms when two different hunks of code want to access the same piece of memory.
00:18:56.000 --> 00:19:01.000
and coherency and access become a problem.
00:19:00.000 --> 00:19:11.000
Okay. In my mind, multiple threads always meant, uh, if there's more than one CPU, it'll just take advantage of it, but I guess I'm learning something today.
00:19:11.000 --> 00:19:16.000
Um, uh, which is great. Some days, some days I learn things.
00:19:11.000 --> 00:19:14.000
Well, here's an example.
00:19:15.000 --> 00:19:18.000
Um, JavaScript.
00:19:16.000 --> 00:19:17.000
Umm.
00:19:17.000 --> 00:19:18.000
Yeah.
00:19:18.000 --> 00:19:19.000
Yeah.
00:19:18.000 --> 00:19:24.000
JavaScript has async and await, as do several other languages.
00:19:21.000 --> 00:19:23.000
Sure.
00:19:24.000 --> 00:19:34.000
But unlike some other languages, JavaScript is explicitly designed to be one foreground thread.
00:19:34.000 --> 00:19:35.000
Right.
00:19:34.000 --> 00:19:46.000
That, like, that's the whole idea. When you say async and await in JavaScript, you are not getting multiple CPUs. That's not happening. What you are getting is…
00:19:43.000 --> 00:19:47.000
No, no, I don't think of that as threads at all.
00:19:47.000 --> 00:20:05.000
Uh, and yet, some people would describe it that way. But that's how it works. And there are other languages with async and await that actually do use threads. Um, and there's… a thing to know is there's multiple layers of threads. Like, threads are a thing.
00:19:50.000 --> 00:19:51.000
Okay.
00:19:55.000 --> 00:19:56.000
Sure.
00:20:05.000 --> 00:20:20.000
Uh, there's threads at the level of… let's talk about Unix and Linux for a second. Uh, we already know that they are divided into levels. There's the system level, and then above that, there's the user level.
00:20:20.000 --> 00:20:25.000
System-level threads are heavier than user-level threads.
00:20:25.000 --> 00:20:37.000
System-level scheduling is more resource-intensive, um, and, uh, more strongly opinionated than user-level scheduling.
00:20:37.000 --> 00:20:49.000
Um, so that it turns out there are system-level threads, and there are user-level threads, and your language might be designed to take advantage of one of those, but not both.
00:20:49.000 --> 00:20:50.000
Umm.
00:20:51.000 --> 00:21:06.000
Also interesting when you're talking about threads and async and await is this notion of There's a word people use. I'm not sure I agree with this particular terminology, but the idea of.
00:21:06.000 --> 00:21:09.000
Colored threads.
00:21:09.000 --> 00:21:10.000
Hmm.
00:21:09.000 --> 00:21:25.000
I don't know… you're mostly… you do a lot of stuff in Perl, so in Perl, you wouldn't have seen this, but in JavaScript, you would have. And there are other languages where you would see it, and certainly with respect to other language features. For instance.
00:21:25.000 --> 00:21:34.000
in C++, and I think in Java. I don't do a lot of Java. Um… There is the idea of const.
00:21:34.000 --> 00:21:39.000
When you mark a specific parameter const.
00:21:39.000 --> 00:21:46.000
or you declare a certain member function to be const.
00:21:46.000 --> 00:22:01.000
you can't call it with non-const data in certain contexts, in the way that you do it. When you declare a parameter to be const in C++, you can pass in anything, and the promise is.
00:22:01.000 --> 00:22:14.000
that it won't alter that underlying parameter. But if you declare the member function to be const, you are saying this cannot be called except on a const object.
00:22:14.000 --> 00:22:22.000
If you say this can only be called on a const object, then suddenly const starts to propagate.
00:22:23.000 --> 00:22:31.000
It can't just be that this is const. The thing that calls this has to be const. And that has to be const.
00:22:31.000 --> 00:22:42.000
And that's why they call it colored. The color spreads from the thing that you want. This happens in a thing people used to use in Microsoft.
00:22:42.000 --> 00:22:51.000
stuff called COM, the, um, Common Object Model, uh, other things. But the problem is that, uh.
00:22:46.000 --> 00:22:47.000
Umm.
00:22:47.000 --> 00:22:48.000
Umm.
00:22:51.000 --> 00:23:03.000
some people call it infection. One function infects the caller, and that infects the caller. Well, this happens with async and wait. Um, in a language like Python.
00:23:02.000 --> 00:23:04.000
Right.
00:23:03.000 --> 00:23:16.000
If you have a synchronous, a normal function, you can't just await some other function, because it needs you to be asynchronous to do that.
00:23:16.000 --> 00:23:30.000
So you need to have an asynchronous event loop to get that asynchronous function to execute. You need to be able to await it, and you can only do that from an asynchronous function.
00:23:30.000 --> 00:23:49.000
But in Go, for instance, um… It's, it's just a go routine. Any function that you want, you can call in this magic way by instead of saying, you know, the function is named foo. Instead of saying foo, open paren, some parameters, close paren, you say.
00:23:50.000 --> 00:23:51.000
Go.
00:23:51.000 --> 00:24:01.000
And it does its thing. No color, no infection, no spread. And now it communicates with a channel.
00:24:01.000 --> 00:24:17.000
That's pretty, pretty cool. That, like, you say that it's cool, it's this thing, you say that a lot. I don't say it very often. And I'm not a Go programmer. I can write stuff in Go if I have to, but it's not the first tool I reach for.
00:24:18.000 --> 00:24:26.000
But this is a reason to reach for Go. This is a really good thing that Go does.
00:24:26.000 --> 00:24:27.000
Um, Rust.
00:24:28.000 --> 00:24:33.000
Rust doesn't actually provide the mechanism.
00:24:33.000 --> 00:24:39.000
to do the threading, but it does provide the API.
00:24:39.000 --> 00:24:47.000
So, it has a way to communicate between, uh, things that is not unlike Go's channels.
00:24:47.000 --> 00:25:08.000
But you supply the underlying library that decides, is this a thread? Is it a… process? Is it a… what… what are you going to get? Um, which I think puts it even one step, uh, beyond Go. I think that's a fascinating answer. So, um…
00:25:06.000 --> 00:25:14.000
So, so with, with Rust, is it, is it moved more into like the standard library than in the language itself?
00:25:14.000 --> 00:25:19.000
The threading and that kind of thing?
00:25:19.000 --> 00:25:28.000
Um… You know, the main… so, I'm not a Rust expert, although I want to be.
00:25:28.000 --> 00:25:47.000
Uh, my understanding is that one of the most important Rust libraries in this, uh, neighborhood of solutions is called Tokyo, and it is about threading, and it is not in the standard library.
00:25:46.000 --> 00:25:50.000
Okay, but it's… it's a library.
00:25:47.000 --> 00:26:02.000
But it is so commonly used that it might it might as well be in the standard library Tokyo is as important to rust as a thing that used that we used to use all the time in Python called twisted.
00:26:03.000 --> 00:26:19.000
Used to be twisted is less important now, but Python and everything that came after learned a lot from twisted and I think there's a lot of code in rust that is greatly informed.
00:26:11.000 --> 00:26:12.000
Umm.
00:26:19.000 --> 00:26:21.000
Bye, Tokyo.
00:26:21.000 --> 00:26:22.000
Um, so.
00:26:23.000 --> 00:26:29.000
Two important concepts. Concurrency and parallelism.
00:26:29.000 --> 00:26:36.000
Concurrency is when you are waiting a lot, and the answer is scheduling.
00:26:36.000 --> 00:26:46.000
Parallelism is when you have more problem than you have computer, and the answer is more processing.
00:26:46.000 --> 00:26:51.000
And now let me give you one other really interesting word.
00:26:51.000 --> 00:26:57.000
Before, when I talked about optimization and measurement, and by the way.
00:26:58.000 --> 00:27:02.000
Measurement is a huge part of this world.
00:27:02.000 --> 00:27:12.000
how you are going to solve problems, uh, using these tools, you need to measure what is the constrained resource? What do I get when I do such and such?
00:27:12.000 --> 00:27:14.000
Well.
00:27:14.000 --> 00:27:25.000
Before, when talking about speeding up pieces of code, I talked about the idea that if you are trying to optimize some piece of your code.
00:27:25.000 --> 00:27:33.000
that takes 20% total of the execution time to solve some particular problem.
00:27:33.000 --> 00:27:39.000
then even if you make that part of your processing go away ENTIRELY.
00:27:39.000 --> 00:27:44.000
The most you can get out of that is a 20% speedup.
00:27:46.000 --> 00:27:56.000
It turns out that there is a similar effect in, uh, this whole idea of concurrency and parallelism, and that is.
00:27:56.000 --> 00:28:11.000
if the fraction of your program that can be parallelized… let's go the opposite direction for a second. Let's say that there is 20% of your program that can't be parallelized.
00:28:11.000 --> 00:28:16.000
then the best you can do is an 80% speedup.
00:28:16.000 --> 00:28:18.000
It has a name.
00:28:18.000 --> 00:28:33.000
And it turns out it's the same thing. It's the same name in concurrency and parallelism that it is in straight optimization. And that is Amdahl's law. And yes, it's that Amdahl.
00:28:33.000 --> 00:28:38.000
The one, uh, who is the computer expert and, uh…
00:28:36.000 --> 00:28:39.000
Yeah, Jean Amdahl.
00:28:39.000 --> 00:28:55.000
Yeah. Amdahl's law says that, you know, once you measure and you see what part can be sped up, that's the limit right there. Whether it's parallelism or concurrency or optimizing something away entirely.
00:28:39.000 --> 00:28:41.000
Right? Yeah.
00:28:55.000 --> 00:29:08.000
measure, you'll know what the limit is, and that limit, uh, is the expression of Amdahl's Law for your particular problem. It's not a law in the same way that, um.
00:29:08.000 --> 00:29:14.000
Uh, what's the one about, uh, number of transistors in a…
00:29:13.000 --> 00:29:16.000
Oh, Moore's Law.
00:29:14.000 --> 00:29:30.000
In a… Moore's Law. It's not, like, something independent of the problem in the way Moore's Law is. Um, it is super dependent on your problem and on your specific solution to the problem.
00:29:17.000 --> 00:29:18.000
Yah.
00:29:30.000 --> 00:29:42.000
Um, but it is its own thing. So… Now you know I do kind of want to talk about a couple of interesting.
00:29:42.000 --> 00:29:55.000
problems around the answers, um, and levels at which the answers occur. Like, there's parallelism that is beyond your ability to detect.
00:29:55.000 --> 00:29:59.000
Uh, for instance, inside a CPU.
00:29:59.000 --> 00:30:07.000
I can name at least two things that are, um… Uh, important one is the pipeline.
00:30:06.000 --> 00:30:08.000
Umm.
00:30:07.000 --> 00:30:24.000
Uh, the pipeline inside a CPU means that multiple instructions are happening at the same time, at various stages of their own execution. They don't exactly happen entirely at the same instant, but they are overlapping in a way.
00:30:24.000 --> 00:30:47.000
that makes you be executing more code than you would think you could. And there's branch prediction. Branch prediction isn't exactly getting two pieces of code to execute at the same time, unless it is, um… The kind of branch prediction where it does a bit of one side, but mostly has chosen the other.
00:30:47.000 --> 00:30:48.000
Umm.
00:30:48.000 --> 00:30:59.000
these are both a level of parallelism that you can't see, uh, unless you have special tools and you're working with a special version of the CPU.
00:30:59.000 --> 00:31:15.000
Um… you have a much better look inside concurrency, because concurrency is a scheduling problem, so you can actually look BEFORE it actually starts executing code. That's useful.
00:31:15.000 --> 00:31:18.000
So that's one thing.
00:31:18.000 --> 00:31:21.000
And a second thing is.
00:31:21.000 --> 00:31:29.000
Once your problem is going to be solved by some, uh, form of parallelism.
00:31:29.000 --> 00:31:44.000
Now you have another problem, and that problem is… communication. It's getting data from where you are to the place where you're going to solve a whole different part of it.
00:31:35.000 --> 00:31:37.000
Right.
00:31:44.000 --> 00:31:51.000
Um, if it's on a different CPU, in a different machine, across a cluster.
00:31:51.000 --> 00:32:06.000
Well, now you've got to send data across, you know, what TCP or who knows what, the fabric. If you're two machines in the same data center, you're not necessarily connected by TCP. You might be connected by an interconnect fabric or something.
00:32:05.000 --> 00:32:09.000
Yeah, was it a mirror net or there's there's various.
00:32:06.000 --> 00:32:24.000
which… There's tons. So there's that. Or maybe you're on a GPU. So now what matters is getting the data across the bus, which could be a PC.
00:32:09.000 --> 00:32:10.000
ways.
00:32:24.000 --> 00:32:27.000
IE bus or a P what whatever it is.
00:32:26.000 --> 00:32:28.000
PC yeah yeah.
00:32:28.000 --> 00:32:35.000
it's not nearly fast enough. None of it is fast enough. The great thing about parallelism is you've got.
00:32:35.000 --> 00:32:46.000
X times as much processing power, and the horrible thing about it is that you have to send the data there. Um, a nice compromise is on, uh, Apple Silicon.
00:32:46.000 --> 00:32:58.000
it uses shared memory, so there's not a communication problem, but on the other hand, where a NVIDIA GPU has.
00:32:58.000 --> 00:33:05.000
X thousands of cores, um, and they're simple and can do simple math.
00:33:05.000 --> 00:33:15.000
Where by simple math, I do mean that it can do reasonable amounts of linear algebra, the kind of stuff you'd want to do if it was video.
00:33:15.000 --> 00:33:21.000
But on a piece of Apple Silicon, maybe you get 40 cores.
00:33:21.000 --> 00:33:24.000
40 is not 10,000.
00:33:24.000 --> 00:33:46.000
They are significantly more interesting cores, individually. Um… Of course, you only get one kind on Apple Silicon at this moment, whereas on an NVIDIA card, there's three different levels of GPU cores, all the way from the smallest, fastest leaf cores.
00:33:46.000 --> 00:33:57.000
cores, up to the highest level ones where you only have 10 or 100, I don't even know how many of them, but that make big decisions and part out the work and whatnot.
00:33:57.000 --> 00:34:13.000
But zero communication overhead on the Apple Silicon and gigantic communication overhead on the NVIDIA system or whatever particular thing that it is because there's AMD and there's NVIDIA and there's this Apple Silicon.
00:34:13.000 --> 00:34:26.000
NVIDIA uses CUDA, and AMD uses ROCm, and Apple uses Metal, and all of them can be used to solve non-graphics problems.
00:34:26.000 --> 00:34:52.000
But once again, you need to look at your problem. You need to identify what is the band… what is the scarcity? What is the cost? What is the thing that you need to, uh… optimize for. Um… And then you need to pick the appropriate kind of parallelism and hardware on which you would like to solve it.
00:34:52.000 --> 00:35:13.000
IF you are allowed to choose. Just like when we talked about languages, we talked about the choices into which you could be forced. Well… if you're doing a video problem, and you want to run on Windows machines, because it's a game, and it's a AAA game, and you want people to buy it.
00:35:13.000 --> 00:35:24.000
Well, you already know what the answer is. The answer is, you're gonna solve it on an NVIDIA card, and your problem is communication, and you're gonna do video work, and that's all there is to it.
00:35:24.000 --> 00:35:38.000
If your problem is you want to search a strand of genetic material, and it is a million bases long, and you want to find something that's 60 bases long in that.
00:35:38.000 --> 00:35:52.000
Now you have a different problem, and you have different things to optimize for. So… Almost never do you have the right amount of hardware to solve the problem that you want.
00:35:53.000 --> 00:36:03.000
So you need to understand your hardware, and you need to understand your problem, and you need to understand if there's other ways to solve your problem.
00:36:03.000 --> 00:36:12.000
I wanted to mention, um… some of the interesting ways that we have done this in the past. Threads were easy.
00:36:12.000 --> 00:36:22.000
But, um… In the past, on Linux and Unix systems in particular, we have used fork.
00:36:22.000 --> 00:36:28.000
Fork is awesome, in some ways, and kinda terrible in others.
00:36:28.000 --> 00:37:01.000
At the beginning, Fork used to do a thing where it would copy the world. It would copy your entire process, including all file descriptors and any resources that you might use, and um… Then the only change would be that you are the child or you are the parent and you could do some particular thing. And sometimes what you really wanted was you didn't even actually want that copy. What you wanted was you wanted to be something else entirely.
00:37:01.000 --> 00:37:14.000
So, you did a fork with that giant copy, and then an exact, and suddenly you were something else! making that whole copy kind of a waste. Um, but that's a thing, that's what we used to do.
00:37:15.000 --> 00:37:18.000
Since that time, we've made fork.
00:37:18.000 --> 00:37:28.000
in a lot of cases, significantly faster, because we use a copy-on-write version of fork. So fork.
00:37:28.000 --> 00:37:37.000
acts exactly like it does a complete copy of your entire process, but it doesn't. It just starts running.
00:37:37.000 --> 00:37:41.000
And it doesn't actually copy anything.
00:37:41.000 --> 00:37:56.000
until you write. And since most of the thing that you just copied is executable code, and code is in volatile segments, they can't be modified.
00:37:56.000 --> 00:38:03.000
Therefore, your code doesn't get copied, but your data, which includes the file descriptors, does.
00:38:03.000 --> 00:38:04.000
Bats.
00:38:04.000 --> 00:38:18.000
the part that might get modified if, uh, you're gonna do something, and then that's the part that eventually will get copied. Everything is faster. Unless you're Python. If you're Python, and you do reference counting.
00:38:18.000 --> 00:38:19.000
Ugh.
00:38:19.000 --> 00:38:32.000
then instantly just looking at a variable changes its reference count, and then back again, so copy and write almost never helps you. And also, Python had this horrible thing called the GIL.
00:38:32.000 --> 00:38:47.000
the, uh, global interpreter lock. The global interpreter lock is a way of solving this problem of multiple stuff happening at the same time, um, by stopping two different pieces of code.
00:38:47.000 --> 00:38:53.000
from trying to have a coherence problem with the same piece of data.
00:38:53.000 --> 00:38:59.000
Now, in more recent versions of Python, I think starting at 3.14.
00:38:59.000 --> 00:39:17.000
You can run without the GIL, and I think in 3.15, running without the GIL will become the default. For some stuff, for single-threaded stuff, in 3.14, it wasn't necessarily faster. In fact, I might go so far as to say it was often slower.
00:39:17.000 --> 00:39:26.000
Uh, but if you actually were multi-threaded, it was… it was a win. But soon, it will be the case that, uh, life is just better with that.
00:39:26.000 --> 00:39:27.000
Umm.
00:39:27.000 --> 00:39:28.000
So.
00:39:29.000 --> 00:39:41.000
How do you know what you want? Uh, you have to decide. Are you IO-bound? If you're waiting for the disk, if you are waiting for the network, if you are waiting, you know, if you're waiting.
00:39:41.000 --> 00:39:46.000
Waiting is the key. Then threads.
00:39:46.000 --> 00:40:01.000
Concurrency, probably your answer. If you're CPU-bound, if you are, um… trying to calculate, uh, hashes or encrypt, um.
00:40:01.000 --> 00:40:07.000
I mean, in a way, you're data-bound for those, but… but if you're CPU-bound, if it's pure cal.
00:40:07.000 --> 00:40:10.000
Probably parallelism is the answer.
00:40:10.000 --> 00:40:17.000
There is a third choice, it doesn't… crop up a lot, but it's if you're memory-bound.
00:40:17.000 --> 00:40:25.000
If you're memory-bound, it's a lot like Jim's location problem with his base station.
00:40:25.000 --> 00:40:37.000
The answer to a problem of being memory-bound isn't about picking threads or picking multiple CPUs. It is about.
00:40:37.000 --> 00:40:44.000
probably layout of your data structures in memory. Can you get that right?
00:40:44.000 --> 00:41:10.000
Um… And the real hard part is when you want to share data. Is it something that you need to send between two things using communication? Is there coherency that you have to manage? Is there locking? Um… Different languages use different things. Um, there is one model that I didn't talk about, and that's the.
00:41:10.000 --> 00:41:16.000
actor model. Most of the languages that we use.
00:41:16.000 --> 00:41:33.000
don't leverage the actor model. This is kind of an Erlang thing. But it turns out Swift does use actors. An actor is when it's the object itself that owns the data and nobody else owns it.
00:41:33.000 --> 00:41:38.000
So you can only get to it by executing code in the object?
00:41:39.000 --> 00:41:49.000
That kind of solves the access problem, as long as you don't provide a bunch of escape hatches.
00:41:49.000 --> 00:42:02.000
Um… Anyway… I think we can make today be kind of a short episode if I give you the takeaways from this. Because really.
00:42:02.000 --> 00:42:08.000
It's just those three main things, I think. Is it a scheduling problem?
00:42:08.000 --> 00:42:11.000
Or is it a work problem, a CPU problem?
00:42:10.000 --> 00:42:11.000
Sure.
00:42:11.000 --> 00:42:16.000
Um, so the takeaways are, um.
00:42:16.000 --> 00:42:20.000
That either you have more computer than you have code.
00:42:20.000 --> 00:42:33.000
A bunch of cores going to waste. And if that's the case, the answer's concurrency. Or… You have more work to do than you have computer to do it.
00:42:33.000 --> 00:42:40.000
In which case, the answer is parallelism. Get more CPUs. Get more cores. Get more machines.
00:42:40.000 --> 00:42:42.000
Get more.
00:42:42.000 --> 00:42:51.000
You need to diag… just like every single episode we have, I'm going to harp on it, you need to diagnose.
00:42:51.000 --> 00:42:58.000
before you write the code. Start measuring in advance. Figure out where is the problem.
00:42:58.000 --> 00:43:07.000
And the actual enemy isn't how much work you have to do. It's not what you're waiting for. It's sharing state.
00:43:07.000 --> 00:43:14.000
It's communication and coherency. That is your enemy, no matter which mechanism you end up choosing.
00:43:14.000 --> 00:43:23.000
That's the thing you will fight. That is the thing you will get wrong the first three times. That is the thing the languages that help you with.
00:43:23.000 --> 00:43:28.000
That's the place they're going to help you the most.
00:43:28.000 --> 00:43:40.000
Go helps you because it uses channels. Rust helps you because it enforces strict ownership. Only one owner gets a mutable reference to something.
00:43:40.000 --> 00:43:45.000
Or, everybody has a read-only reference.
00:43:45.000 --> 00:43:55.000
Amdahl's Law is the thing that essentially, um, I guess the word here is humbles you. Um, you're gonna know in advance.
00:43:56.000 --> 00:43:59.000
You can't do better than this.
00:43:59.000 --> 00:44:05.000
If you are 5'2", it is unlikely you are going to be dunking.
00:44:05.000 --> 00:44:11.000
If only 20% of your program is parallelizable.
00:44:11.000 --> 00:44:12.000
Ben?
00:44:12.000 --> 00:44:22.000
is the fastest you will get to go. Is that 80% at normal speed, and then some speed up on that 20%.
00:44:22.000 --> 00:44:24.000
Python?
00:44:24.000 --> 00:44:31.000
Used to have lots of limits, and is having some of them… removed.
00:44:31.000 --> 00:44:40.000
Um… And… You should know when not to parallelize. If you're already fast enough.
00:44:40.000 --> 00:44:52.000
Maybe it's okay you have too much computer. Maybe. Or whatever. But it's key to know. We use this magic word. We talk about Heisenbugs.
00:44:52.000 --> 00:44:55.000
Bugs that only show up.
00:44:55.000 --> 00:45:02.000
Um, when they are least wanted, and are incredibly hard to reproduce under a debugger.
00:45:02.000 --> 00:45:12.000
Well, let me tell you where they live. They live in parallelized code. So, it could well be the case that.
00:45:12.000 --> 00:45:18.000
You have a problem where parallelization is an option?
00:45:18.000 --> 00:45:21.000
But it's not a guaranteed win?
00:45:22.000 --> 00:45:30.000
If the win isn't guaranteed, go spend your money on something where the win is guaranteed.
00:45:30.000 --> 00:45:34.000
That's what I think the takeaways are.
00:45:34.000 --> 00:45:36.000
What do you think, Jim?
00:45:34.000 --> 00:45:51.000
So I've, well, I've got an example of parallel parallelization that makes sense. I just read this earlier today and I don't remember it was on Hacker News or someplace talking about how Pixar.
00:45:51.000 --> 00:46:01.000
uh, renders their movies. Um, and, you know, when, when you're doing an animated, uh, uh, movie, um.
00:46:01.000 --> 00:46:08.000
the, the animations have gotten so good. If you, if you look back at, um, the one I can think of is, um.
00:46:08.000 --> 00:46:24.000
Finding Nemo, where they made the water look like water, and now, like, grass looks like grass. And I think even back when they did Hair Looks Like Hair, back when they did the movie Titanic.
00:46:17.000 --> 00:46:20.000
And hair, hair looks like hair.
00:46:24.000 --> 00:46:39.000
Um, rendering that water, uh, there was some, some new technology had to be created to do that. Anyway, um, the thing I read today said, uh, that a single frame of, of.
00:46:39.000 --> 00:46:43.000
those movies takes 24 hours to render.
00:46:44.000 --> 00:46:58.000
That's just one frame, and what are movies? 30 frames per second? I think they said the average movie is about 130,000 frames. Now, if you had 130,000 frames.
00:46:58.000 --> 00:47:01.000
And it took 24 hours per frame.
00:47:01.000 --> 00:47:14.000
um, there's not enough time for, for any of us to, to, you know, we would never be able to watch the movies because they would take that long to create. So they went to massive parallelization.
00:47:14.000 --> 00:47:29.000
Pixar developed this whole thing using… creating render farms, where they are, indeed, processing multiple frames, like, thousands or tens of thousands of frames simultaneously. So at the end of 24 hours.
00:47:29.000 --> 00:47:42.000
they could have tens of thousands of frames done, instead of just one. So that's a great place, a great example of running things in parallel, where your problem is bigger than your compute.
00:47:42.000 --> 00:47:57.000
Yeah, and, uh, like, some things to point out about that situation are the data that needs to be shared, um, first of all, is actually independent per frame, because, uh.
00:47:57.000 --> 00:48:11.000
The big data is all the pixels at the end. It's the output, not the input. The input is 3D models, and the 3D models are actually different per frame, because they've moved, things have moved.
00:48:11.000 --> 00:48:26.000
So it's not like there's a coherency problem. That's a big, big deal. Second, I wanted to say, remember when I can think of at least two movies back in the beginning time of computer animation and CGI.
00:48:26.000 --> 00:48:38.000
Tron was an early one, and so was The Last Starfighter. I thought those were both interesting. And I used to work at Trolltech, a company… it's now known as The Cute Company.
00:48:27.000 --> 00:48:28.000
Yep.
00:48:38.000 --> 00:48:53.000
And, uh, we had, you know, Qt is a gigantic C++ framework, especially back in the day. It was way bigger than the computers, but you needed the libraries to be pre-built.
00:48:53.000 --> 00:49:11.000
to do your thing, but we were constantly… I wasn't, I was the… I was the evangelist, so I don't want to confuse anybody that I was actually writing code there that was for part of the library. I was writing code, but it was, like, demo code and… And whatnot. Um, uh, anyway.
00:49:11.000 --> 00:49:13.000
They had a system.
00:49:13.000 --> 00:49:22.000
Which other people started to emulate, where, when you wanted to build a specific, um, version of the code.
00:49:22.000 --> 00:49:29.000
You would start… And every machine in the office.
00:49:29.000 --> 00:49:31.000
would help.
00:49:31.000 --> 00:49:48.000
you'd compile two files, that machine would compile five files, the other machine would compile three, and they were tied to the specific commit. It wasn't Git back then, I remember, I think maybe we were using Subversion, I can't remember.
00:49:48.000 --> 00:49:58.000
But it was tied to the specific commit, so later, when people wanted those object files, they could get the right one, and they'd already been compiled.
00:49:58.000 --> 00:50:07.000
That's a form of parallelism. And there, they get the data from the centralized repo.
00:50:07.000 --> 00:50:16.000
And there's no duplication, because you're compiling one file, and the other guy is compiling a different file.
00:50:16.000 --> 00:50:23.000
There's lots of problems where the answer is parallelism, or concurrency, or…
00:50:18.000 --> 00:50:20.000
Sure.
00:50:23.000 --> 00:50:25.000
Or who knows what.
00:50:23.000 --> 00:50:40.000
Sure, sure. Uh, make minus J. I do that all the time when I build Postgres or whatever. And… there's no communication between them, because they're all compiling different files. Every thread, uh… or every process is compiling a different file, so they don't need to talk to each other.
00:50:40.000 --> 00:50:54.000
But, uh, Make is a great tool for that. Make-j16 will, uh, spawn 16 threads to build your program much faster, assuming you have the CPU cores to do that.
00:50:54.000 --> 00:51:12.000
Right. You definitely want cores for that, as opposed to threads, because that's a problem that's CPU-bound, um, not IO-bound. It's not like you're waiting for that file. And if you had threads, and it was one CPU switching between compiling the different files.
00:51:03.000 --> 00:51:04.000
Yeah.
00:51:12.000 --> 00:51:21.000
it wouldn't be faster. In fact, it would be very slightly slower. So, minus J, really good when you have multiple cores.
00:51:21.000 --> 00:51:50.000
Sure. And, uh, you mentioned, very briefly, JavaScript. I've been doing a lot of programming in JavaScript lately. And, you know, JavaScript is single-threaded, and it's all built around an event loop. So when you… I'm doing it in Node, so I'm making database calls, and… and other things. And when I, when I go to submit a query, or, or, you know, execute a query, um, I, I use async and await, and, and that, uh, not always, but, um.
00:51:50.000 --> 00:51:58.000
When I feed a query to Postgres, my program is free to do other things.
00:51:58.000 --> 00:52:08.000
Right? It's, it's, uh, that, that, that, uh, database query's running in the background. Uh, it's… JavaScript itself is single-threaded.
00:52:08.000 --> 00:52:11.000
It's running in your background.
00:52:08.000 --> 00:52:14.000
But I can… What do you mean?
00:52:14.000 --> 00:52:29.000
I mean, the query has gone off, and now it is the primary importance to Postgres. It's running in Postgres' foreground. You're just waiting for an answer to come back. And the fact that you're waiting.
00:52:17.000 --> 00:52:19.000
Yes.
00:52:24.000 --> 00:52:26.000
Right, right.
00:52:26.000 --> 00:52:44.000
Right, and when I get an answer… Yeah. And if I don't do the async await, it'll run a callback for me. It'll basically run… my program will process the result of that query when it.
00:52:31.000 --> 00:52:32.000
Yeah.
00:52:44.000 --> 00:52:57.000
finishes doing whatever it's doing. Like, my program can be sitting there doing lots of things, and when the query finishes, my program will get that and use that. And I've recently.
00:52:55.000 --> 00:53:07.000
Which you can do either with threads, or with promises and futures. Um, both of those give you the same power, which is, you're waiting.
00:53:01.000 --> 00:53:03.000
Yeah.
00:53:07.000 --> 00:53:13.000
And why wait? Like, while you're waiting, let's go do something useful and interesting.
00:53:11.000 --> 00:53:26.000
Right, right. But something I'm using now, well, an issue I ran into, I can only queue up, I think it's four, I can only submit up to four queries to Postgres from that single thread.
00:53:26.000 --> 00:53:33.000
And all the other queries that I might queue up will just sit there in a queue, they won't actually run. But 4 of them will run.
00:53:33.000 --> 00:53:40.000
uh, the way, the way Postgres works with, with Node.js. So four of those queries will just be off running on their own.
00:53:40.000 --> 00:53:57.000
I'm working on a problem where I have a lot more queries to run, so I've resorted to worker threads, which, uh, exist in Node, and they also exist in browsers. Uh, you can, you can run, uh, it's called something else in browsers. It's, uh… Uh, I forget, worker something or other.
00:53:57.000 --> 00:54:00.000
There is a thing called Web Workers.
00:53:57.000 --> 00:53:59.000
Umm.
00:53:59.000 --> 00:54:11.000
Web workers. It's a web worker in the browser, it's a worker thread in Node.js, and I can spawn as many of those as I need. And, you know.
00:54:12.000 --> 00:54:27.000
This way, I could submit lots of queries to the database, and of course, I need a powerful enough database server to handle that. But it's… so far, it's working really, really well. So you can do multi-threaded stuff in JavaScript in Node.
00:54:27.000 --> 00:54:30.000
And I'm kind of having fun with that right now.
00:54:30.000 --> 00:54:34.000
But anyway, um… Boy, you covered a lot of stuff.
00:54:34.000 --> 00:54:37.000
Some real interesting stuff.
00:54:37.000 --> 00:54:56.000
I'm anxious to get this episode out there for people to listen to and offer some feedback. So again, feedback. Send your feedback to us, please. We love it. Feedback at runtimearguments.fm. You can go to our website at runtimearguments.fm.
00:54:56.000 --> 00:55:01.000
Uh, I forget, do we need the WWW on it? Or we don't?
00:54:59.000 --> 00:55:03.000
You absolutely do not want the WWW.
00:55:02.000 --> 00:55:07.000
We don't want the www.runtimearguerstaffer, that's not what you want.
00:55:04.000 --> 00:55:07.000
And you don't want the ass either.
00:55:07.000 --> 00:55:20.000
It's not HTTPS, because of where the website is hosted. But we will have show notes on the end of this episode, so you can check out all the links there, and a transcript, and um…
00:55:15.000 --> 00:55:17.000
And a transcript.
00:55:20.000 --> 00:55:23.000
Wow, it's been a great episode, Wolf.
00:55:23.000 --> 00:55:31.000
I had fun with this one. Thank you and thanks to everybody for listening. And Wolf, you got anything else to say or are we done?
00:55:31.000 --> 00:55:41.000
Uh, I think… we're almost done. I think we should put a link to the YouTube video about DapMux in our show notes.
00:55:40.000 --> 00:55:42.000
Yes, we'll do that.
00:55:41.000 --> 00:55:48.000
And it's possible to make a link that starts right at the beginning of the DATMUX part.
00:55:48.000 --> 00:55:51.000
Which is about 24 minutes in.
00:55:51.000 --> 00:55:53.000
You measured.
00:55:53.000 --> 00:55:54.000
Yeah.
00:55:53.000 --> 00:55:59.000
Well, it says, you know, it's right in the corner. I didn't have to, you know, go through it or anything.
00:55:55.000 --> 00:55:57.000
Right.
00:55:57.000 --> 00:55:58.000
Right.
00:55:58.000 --> 00:56:04.000
Right! Uh, I'll have to see how we can do that, but yeah, we'll do that.
00:56:04.000 --> 00:56:09.000
We'll do that. Uh, so thank you everybody. And until next time, uh, goodbye.
00:56:09.000 --> 00:56:11.000
Bye bye
00:00:00.000 --> 00:00:11.000
Alright, welcome to another episode of Runtime Arguments, the podcast where we talk about all kinds of cool, techy… Programming… things.
00:00:11.000 --> 00:00:13.000
Well, we, we.
00:00:13.000 --> 00:00:19.000
I'm Jim McQuillan, and uh, my partner here is Wolf. Say hello, Wolf.
00:00:19.000 --> 00:00:20.000
Hey everybody.
00:00:21.000 --> 00:00:37.000
So, uh, yeah, this is our, uh, episode 30, our 31st episode. And today, uh, we're going to talk about what to do when you have more problem than you have computer.
00:00:37.000 --> 00:00:52.000
We're gonna get into lots of cool things, uh, multi-processing, threading, uh, I don't know what all. Wolf's got all kinds of, all kinds of tricks up his sleeve, and we're gonna learn some of those today, so I'm kinda looking forward to that. So, how was your week?
00:00:52.000 --> 00:00:53.000
Wolf?
00:00:53.000 --> 00:01:08.000
Oh my god, so annoying! But, you know, in a nice way. I live in a rental and have for a very long time. And I have the greatest landlord of all time. He is a really terrific guy.
00:01:08.000 --> 00:01:18.000
And, uh, they went out of their way to replace the flooring over the major part of the house, and that meant.
00:01:18.000 --> 00:01:33.000
furniture going into a pod in the front yard and the family sort of splitting up into various locations. So my wife and me in a hotel with one of the three dogs.
00:01:33.000 --> 00:01:43.000
Uh, one of the boys living with grandma, with two of the dogs, and one of the boys here in the house, not to leave his room. So…
00:01:44.000 --> 00:01:46.000
He stuck it out, huh?
00:01:44.000 --> 00:01:54.000
They said… he did, he did. So they, they said this would be 2 days, and I said, okay, it'll probably be 5, I'll reserve a hotel room for 4.
00:01:55.000 --> 00:02:01.000
And we came back today and they were still doing a little bit of work. So I think I was right.
00:01:59.000 --> 00:02:04.000
And today was the fourth day? Is that right?
00:02:04.000 --> 00:02:13.000
It was like I spent Sunday night and this is Thursday as we speak, so Wednesday night was our last night there.
00:02:08.000 --> 00:02:10.000
Yah.
00:02:10.000 --> 00:02:11.000
Okay.
00:02:13.000 --> 00:02:16.000
So it's, it's, it's good to be back home, isn't.
00:02:16.000 --> 00:02:27.000
It is, and it looks good, it's gorgeous. I mean, of course, Jennifer is not as happy as me, because she sees things and notices things.
00:02:27.000 --> 00:02:31.000
So she can't be as easily satisfied as I am.
00:02:28.000 --> 00:02:37.000
And… And us… us guys, we just… I don't know what it is, we just don't pay attention, do we?
00:02:37.000 --> 00:02:40.000
We just, we walk in, we're happy.
00:02:37.000 --> 00:02:43.000
You know, in this way, ignorance, ignorance kind of is bliss.
00:02:43.000 --> 00:02:44.000
Yeah.
00:02:44.000 --> 00:02:46.000
Just be happy, right?
00:02:45.000 --> 00:02:47.000
How about you? How was your week?
00:02:46.000 --> 00:02:59.000
Uh, busy this week. I just got back from vacation. I spent, uh, 9 days, uh, gone. Uh, my buddy Ron and I, we drove up to see Scotty in Winnipeg.
00:03:00.000 --> 00:03:13.000
Which, uh, you know, we live in the Detroit area, so that's, uh, round trip, that's 2,500 miles in the car, and, uh, you know what? I enjoy every minute of it.
00:03:13.000 --> 00:03:20.000
I, I, I love the, I love the adventure. I love the drive. And we, we, uh, we had a good time. So, but I'm back.
00:03:18.000 --> 00:03:33.000
You're lucky it's summertime, because in the winter in Canada, you can often get pulled over by a snowplow. I don't know if you've noticed the color of the lights on the various vehicles, but they're yellow on…
00:03:27.000 --> 00:03:31.000
Or run over. Yeah.
00:03:31.000 --> 00:03:34.000
Yeah, you get run over.
00:03:33.000 --> 00:03:39.000
police vehicles and they're red and blue or something like that on snow plows.
00:03:37.000 --> 00:03:39.000
Yeah, I don't know.
00:03:39.000 --> 00:03:50.000
I don't know. But it was, it was a good time, but of course, uh, being gone for 9 days, it put me behind, uh, in my normal work, so this week was catch-up time.
00:03:50.000 --> 00:03:58.000
And, uh, so that's what I did. Right up until, like, 2 minutes before we turned on the microphones, I was busy working.
00:03:58.000 --> 00:04:04.000
Uh, but we're here now and we're gonna talk about some cool stuff. Uh, let's see, do we have feedback?
00:04:04.000 --> 00:04:22.000
Um, last episode, we talked about Wi-Fi, and uh, you know… uh, uh, we've got lots of friends on the podcast, and one of them is, is my friend Marlon, our friend, uh, he works at Meta, he's a brilliant, uh, engineer there, and, um.
00:04:22.000 --> 00:04:28.000
Uh, he listens and, uh, you know, we talked about, uh, different, uh, uh.
00:04:28.000 --> 00:04:44.000
levels of Wi-Fi, uh, different, uh, you know, 802.11 something. And I mentioned that the whole thing that drove me to that conversation was the fact that in my house, I, the signal doesn't reach everywhere I would like it to reach.
00:04:44.000 --> 00:04:58.000
And we talked about mesh networks, and various ways I can fix this, and one of the things I was thinking about doing is just getting a better access point that maybe does Wi-Fi 7, which is maybe a stronger signal.
00:04:58.000 --> 00:05:13.000
Marlin says, just move my access point. Move it to a more central location in the house and I'll be fine, because placement is probably the most important thing when it comes to Wi-Fi signal throughout the building.
00:05:13.000 --> 00:05:16.000
And, um, uh, yeah.
00:05:13.000 --> 00:05:21.000
Can I actually use a phrase? Is it location, location, location?
00:05:17.000 --> 00:05:19.000
Yes.
00:05:20.000 --> 00:05:38.000
That's exactly what it is, like it always is, right? They answer to everything. And the location of my access point is not a good location. Unfortunately, the way my house is laid out, moving it, drilling through walls, and drilling through floors.
00:05:38.000 --> 00:05:53.000
Um, it presents a challenge, so I'm gonna… I don't know, maybe over the next month or so, I'm gonna try to figure this thing out. I suspect, uh, I… I can get, uh, much better coverage without spending any money, and that… that… that appeals to me.
00:05:53.000 --> 00:06:12.000
Uh, but thanks, Marlon, for that advice. Uh, it's… it's kind of obvious, um… you know, I, I, the solutions I look for are usually on Amazon, uh, not on, uh, just common sense. But, uh, so yeah, that was, uh, that's the feedback we got from the last episode.
00:06:12.000 --> 00:06:22.000
Um, and of course, we love feedback, we encourage it, send us feedback, uh, you can send it to, uh, feedback at runtimearguments.fm.
00:06:22.000 --> 00:06:39.000
Uh, of course, we'll, uh, we'll repeat that again at the end, and we have show notes, and we have a website, we'll… we have all that contact information there, so you can, um… Tell us what you'd like to hear. Tell us what we said wrong. Sometimes tell us what we said right.
00:06:39.000 --> 00:06:50.000
uh, hopefully today, um, we've got some ideas that will, uh, cause people to tell us what they think. So, hopefully. Anyway.
00:06:49.000 --> 00:06:51.000
You, you know.
00:06:51.000 --> 00:06:55.000
Over the course of the last week, something really good did happen.
00:06:56.000 --> 00:06:58.000
Uh, what's that?
00:06:56.000 --> 00:07:01.000
And that is, we took my little open source idea.
00:07:01.000 --> 00:07:03.000
And.
00:07:02.000 --> 00:07:04.000
Right.
00:07:03.000 --> 00:07:13.000
for a DAP multiplexer, and, um, and we did a big demo of it, um, that is now recorded and available on YouTube.
00:07:14.000 --> 00:07:41.000
Yeah, you gotta go to mug.org, or go to the… mug.org is where you'll see the announcement. If you go to YouTube and just search for mug.org, you'll see Wolf's, um, talk on… uh, uh, DAP mux, which was really, really cool for, um, controlling a debugger and seeing variables and… all the kinds of things you want to do when you're debugging. Uh, so check that out.
00:07:41.000 --> 00:07:52.000
Um, you know, when we do these episodes, we, uh, we put together a little… sometimes it's an outline, sometimes it's, uh, it's never a script.
00:07:52.000 --> 00:07:59.000
But, uh, we sort of jot down the ideas of what we're going to talk about. So we did that with this episode.
00:07:59.000 --> 00:08:11.000
create a great outline, discussing all the things we're gonna discuss. And, uh, a couple of days ago, Wolf and I had a conversation about it, and it was kind of funny, because we… we got into a heated, uh, discussion.
00:08:11.000 --> 00:08:17.000
about one of the things. We'll get more into it later. But, you know.
00:08:17.000 --> 00:08:26.000
I had a realization, and this is true about a lot of things between Wolf and myself, we come at things from a whole different direction.
00:08:26.000 --> 00:08:27.000
Umm.
00:08:27.000 --> 00:08:43.000
Wolf, Wolf is a genius software developer and he thinks, I don't know what he thinks, like I haven't figured it out, but it's what's going on in his brain is different from what's going on in mine.
00:08:43.000 --> 00:08:54.000
I'll tell you what it was. We're going to cover it more later, but it's the idea of multiprocessing and clustering.
00:08:54.000 --> 00:09:09.000
And in Wolf's mind, they're exactly the same thing. And, and I understand from his point of view, yeah, they are, the more I think about it. But from my point of view, I think in terms of what's it going to take to implement this thing at the hardware level.
00:09:09.000 --> 00:09:18.000
Because I'm kind of a full solution provider, and I have to think about plugging things together. And that's where I think.
00:09:14.000 --> 00:09:15.000
Plus.
00:09:16.000 --> 00:09:26.000
And you sign the… you sign the checks. When… when you need 5 machines to do it, you have to sign a check that says 5 machines!
00:09:20.000 --> 00:09:21.000
Yes.
00:09:22.000 --> 00:09:24.000
You're right.
00:09:25.000 --> 00:09:36.000
Right. Right, right. So I'm thinking about it from a whole different standpoint, and to me, uh, multiprocessing and clustering are entirely different.
00:09:36.000 --> 00:09:48.000
And to Wolf, they're the same thing. It's just how we look at things. Um, and when I realized that, it's like, oh, that's what was going on the other day when we had that conversation.
00:09:48.000 --> 00:10:02.000
And really, like I said, that explains lots of our conversations. We just come at it from different points of view, and maybe that's what makes this podcast interesting, assuming it is. Um, but anyway…
00:10:00.000 --> 00:10:04.000
I mean, that's half the title.
00:10:03.000 --> 00:10:07.000
Yeah, right. Yeah.
00:10:07.000 --> 00:10:24.000
Yeah, I guess it's the purpose of the podcast, right? We have different ways of looking at things and sometimes it goes smoothly and sometimes not so smoothly, but this is going to be a great episode. I think I'm ready to move on. Are you?
00:10:24.000 --> 00:10:27.000
Uh, I think I'm ready.
00:10:25.000 --> 00:10:27.000
You ready to go?
00:10:27.000 --> 00:10:30.000
All right, let's get into it. Wolf, go ahead.
00:10:30.000 --> 00:10:50.000
Um, so… There's lots of ways to talk about this topic. We could be super… pedantic, and… I don't think that's valuable. What I think is valuable is getting to the essence of the problem.
00:10:50.000 --> 00:10:55.000
Um, and the problem is when, uh.
00:10:55.000 --> 00:11:00.000
You're solving some… Workload, some problem.
00:11:01.000 --> 00:11:09.000
Usually, you're at one of the extremes. Usually, you've got more computer than you have code.
00:11:09.000 --> 00:11:19.000
Um, for instance, uh, my Mac has… I don't know. What? How many cores does my Mac have? Does it, does it say up here if I pull it down?
00:11:17.000 --> 00:11:21.000
Oh, 20, I, I, a bunch.
00:11:21.000 --> 00:11:23.000
Yeah, a bunch of cores.
00:11:21.000 --> 00:11:24.000
I can't keep track of them anymore.
00:11:23.000 --> 00:11:44.000
like, maybe 18 or something, and 40 GPU cores, something like that, and uh… a regular, like, an NVIDIA GPU card has, you know, in the tens of thousands of cores, or something. They're all simple, but… but it's got so many. So… Almost all the time.
00:11:44.000 --> 00:12:04.000
You have one core doing your job. And… some huge number of cores not doing your job. If you're lucky, they're doing some other job that is also important to you, but a lot of times, all that hardware is just a waste.
00:12:04.000 --> 00:12:16.000
Or, you're in the opposite situation. You have a problem that is so big that you didn't bring enough computer to solve it.
00:12:17.000 --> 00:12:22.000
And it just seems like you're so rarely in the middle. So the question is.
00:12:22.000 --> 00:12:28.000
How do you balance this? And so I want to talk about.
00:12:28.000 --> 00:12:41.000
Essentially, this whole episode is… Um, when getting… Two hunks of code, both of them aimed at the same goal.
00:12:41.000 --> 00:12:49.000
to run at the same time. At least… From your point of view, it looks like they're running at the same time.
00:12:49.000 --> 00:12:54.000
Now, there's constraints. There's gonna be some reason.
00:12:54.000 --> 00:13:06.000
that they can or can't actually run at the same time. There's something that you don't have enough of. That's the whole reason why this is a problem that needs addressing.
00:13:06.000 --> 00:13:13.000
Maybe you don't have enough, um… Uh, CPU.
00:13:13.000 --> 00:13:27.000
your problem is that, you know, you… remember the old days, when we just had one CPU in our computer? And if you wanted to solve a problem that.
00:13:23.000 --> 00:13:24.000
Oh yeah.
00:13:27.000 --> 00:13:40.000
you know, would easily benefit from 10 CPUs or something. Instead of running it on 10 CPUs, what you did is you ran it, but it took 10 times as long, or worse.
00:13:40.000 --> 00:13:41.000
Umm.
00:13:41.000 --> 00:13:48.000
These days, maybe you have 10 CPUs, and maybe you can use them all at the same time.
00:13:48.000 --> 00:13:57.000
Sometimes, the thing that you're constrained on is more about time than it is about processing power.
00:13:57.000 --> 00:14:06.000
Uh, for instance, there are things across a boundary that just take time to get.
00:14:06.000 --> 00:14:14.000
Uh, one of those things is, um… IO, data. If you have stuff on a disk.
00:14:14.000 --> 00:14:22.000
I don't care how fast your disk is. I mean, I guess I technically do, but nobody's made a disk fast enough to make this not be a concern.
00:14:22.000 --> 00:14:34.000
A disk is so far away and so slow compared to even the first-level cache in your computer that you are waiting for disk stuff to happen.
00:14:34.000 --> 00:14:47.000
So… Here's two different ways to solve the waiting problem on a computer. One way is you add more processing power.
00:14:47.000 --> 00:14:49.000
You really do run.
00:14:49.000 --> 00:14:57.000
Multiple hunks of code at the exact same time, but you do it on multiple processing units.
00:14:57.000 --> 00:15:07.000
Um, maybe those processing units are cores. Maybe those processing units are, um, I don't know if you remember, um.
00:15:07.000 --> 00:15:16.000
Uh, what did they call that special thing where, uh, you only had one core, but it had two processing threads?
00:15:14.000 --> 00:15:20.000
Oh, hyperthreading? That still exists. Oh yeah.
00:15:17.000 --> 00:15:28.000
hyperthread, or something, yeah. Uh, maybe you're running it on a GPU, maybe you're running it in a cluster. Uh, it's Kubernetes, or it's a Bay… who knows?
00:15:28.000 --> 00:15:41.000
That's one answer. Multiple pieces of code at the exact same time. That comes with problems of its own and I'm going to get to those. Or maybe your problem is that you're spending a lot of time waiting.
00:15:41.000 --> 00:15:56.000
So instead of being about running multiple pieces of code at the exact same time, your problem is about scheduling. Let me pick the right piece of code to run right at this minute.
00:15:56.000 --> 00:15:59.000
We have two different words.
00:15:59.000 --> 00:16:03.000
One of those words is concurrency.
00:16:03.000 --> 00:16:06.000
And the other word is parallelism.
00:16:06.000 --> 00:16:16.000
These are two different approaches for solving those problems. Concurrency is when you're going to use scheduling to solve it.
00:16:16.000 --> 00:16:21.000
You pick which piece of code is going to execute right now.
00:16:21.000 --> 00:16:30.000
Threads are a way of doing concurrency. They are an implementation for a concurrent solution.
00:16:30.000 --> 00:16:37.000
Multiple CPUs are an implementation for parallelism.
00:16:37.000 --> 00:16:49.000
Each of those two approaches comes with problems. There are words in the languages that you use that imply you are using one or the other.
00:16:49.000 --> 00:16:54.000
Some languages don't tell you what answer.
00:16:54.000 --> 00:17:09.000
that their language feature is going to leverage. And maybe they don't even choose what answer, um, that feature is going to leverage. Maybe it depends on how many CPU cores you have.
00:17:09.000 --> 00:17:15.000
Whether it's going to be threads, or cores, or whole separate processing units.
00:17:14.000 --> 00:17:15.000
But.
00:17:15.000 --> 00:17:17.000
But…
00:17:17.000 --> 00:17:18.000
Yes?
00:17:17.000 --> 00:17:23.000
So, what I'm hearing is, if I have a program that spawns threads.
00:17:24.000 --> 00:17:27.000
It's not going to take advantage of multiple cores.
00:17:27.000 --> 00:17:34.000
It depends on the language and the interface that it gives you, what the underlying implementation is.
00:17:34.000 --> 00:17:48.000
The typical answer is that a thread is lightweight and it runs in the CPU you are in. It is a scheduling problem. It is concurrency.
00:17:48.000 --> 00:17:54.000
But really, really good languages that can use both.
00:17:54.000 --> 00:18:05.000
have the option of saying, here's a thread I can run on a different processing unit. For instance, um, Go has Go routines.
00:18:05.000 --> 00:18:06.000
Okay.
00:18:05.000 --> 00:18:12.000
If necessary, a go routine could be a thread on your local CPU, on the CPU you're using.
00:18:12.000 --> 00:18:25.000
or Go can make a Go routine run on a separate CPU. Why is this an option? Because the communication between a Go routine and your main.
00:18:25.000 --> 00:18:39.000
hunk of executing code is a channel, not shared memory. And that means that it can send the data to the right place. If it's on the same CPU, maybe that's a real short trip.
00:18:39.000 --> 00:18:56.000
If it's on a different CPU, maybe it's a longer trip, but it doesn't have to worry about contention, which is one of the problems of both of these mechanisms when two different hunks of code want to access the same piece of memory.
00:18:56.000 --> 00:19:01.000
and coherency and access become a problem.
00:19:00.000 --> 00:19:11.000
Okay. In my mind, multiple threads always meant, uh, if there's more than one CPU, it'll just take advantage of it, but I guess I'm learning something today.
00:19:11.000 --> 00:19:16.000
Um, uh, which is great. Some days, some days I learn things.
00:19:11.000 --> 00:19:14.000
Well, here's an example.
00:19:15.000 --> 00:19:18.000
Um, JavaScript.
00:19:16.000 --> 00:19:17.000
Umm.
00:19:17.000 --> 00:19:18.000
Yeah.
00:19:18.000 --> 00:19:19.000
Yeah.
00:19:18.000 --> 00:19:24.000
JavaScript has async and await, as do several other languages.
00:19:21.000 --> 00:19:23.000
Sure.
00:19:24.000 --> 00:19:34.000
But unlike some other languages, JavaScript is explicitly designed to be one foreground thread.
00:19:34.000 --> 00:19:35.000
Right.
00:19:34.000 --> 00:19:46.000
That, like, that's the whole idea. When you say async and await in JavaScript, you are not getting multiple CPUs. That's not happening. What you are getting is…
00:19:43.000 --> 00:19:47.000
No, no, I don't think of that as threads at all.
00:19:47.000 --> 00:20:05.000
Uh, and yet, some people would describe it that way. But that's how it works. And there are other languages with async and await that actually do use threads. Um, and there's… a thing to know is there's multiple layers of threads. Like, threads are a thing.
00:19:50.000 --> 00:19:51.000
Okay.
00:19:55.000 --> 00:19:56.000
Sure.
00:20:05.000 --> 00:20:20.000
Uh, there's threads at the level of… let's talk about Unix and Linux for a second. Uh, we already know that they are divided into levels. There's the system level, and then above that, there's the user level.
00:20:20.000 --> 00:20:25.000
System-level threads are heavier than user-level threads.
00:20:25.000 --> 00:20:37.000
System-level scheduling is more resource-intensive, um, and, uh, more strongly opinionated than user-level scheduling.
00:20:37.000 --> 00:20:49.000
Um, so that it turns out there are system-level threads, and there are user-level threads, and your language might be designed to take advantage of one of those, but not both.
00:20:49.000 --> 00:20:50.000
Umm.
00:20:51.000 --> 00:21:06.000
Also interesting when you're talking about threads and async and await is this notion of There's a word people use. I'm not sure I agree with this particular terminology, but the idea of.
00:21:06.000 --> 00:21:09.000
Colored threads.
00:21:09.000 --> 00:21:10.000
Hmm.
00:21:09.000 --> 00:21:25.000
I don't know… you're mostly… you do a lot of stuff in Perl, so in Perl, you wouldn't have seen this, but in JavaScript, you would have. And there are other languages where you would see it, and certainly with respect to other language features. For instance.
00:21:25.000 --> 00:21:34.000
in C++, and I think in Java. I don't do a lot of Java. Um… There is the idea of const.
00:21:34.000 --> 00:21:39.000
When you mark a specific parameter const.
00:21:39.000 --> 00:21:46.000
or you declare a certain member function to be const.
00:21:46.000 --> 00:22:01.000
you can't call it with non-const data in certain contexts, in the way that you do it. When you declare a parameter to be const in C++, you can pass in anything, and the promise is.
00:22:01.000 --> 00:22:14.000
that it won't alter that underlying parameter. But if you declare the member function to be const, you are saying this cannot be called except on a const object.
00:22:14.000 --> 00:22:22.000
If you say this can only be called on a const object, then suddenly const starts to propagate.
00:22:23.000 --> 00:22:31.000
It can't just be that this is const. The thing that calls this has to be const. And that has to be const.
00:22:31.000 --> 00:22:42.000
And that's why they call it colored. The color spreads from the thing that you want. This happens in a thing people used to use in Microsoft.
00:22:42.000 --> 00:22:51.000
stuff called COM, the, um, Common Object Model, uh, other things. But the problem is that, uh.
00:22:46.000 --> 00:22:47.000
Umm.
00:22:47.000 --> 00:22:48.000
Umm.
00:22:51.000 --> 00:23:03.000
some people call it infection. One function infects the caller, and that infects the caller. Well, this happens with async and wait. Um, in a language like Python.
00:23:02.000 --> 00:23:04.000
Right.
00:23:03.000 --> 00:23:16.000
If you have a synchronous, a normal function, you can't just await some other function, because it needs you to be asynchronous to do that.
00:23:16.000 --> 00:23:30.000
So you need to have an asynchronous event loop to get that asynchronous function to execute. You need to be able to await it, and you can only do that from an asynchronous function.
00:23:30.000 --> 00:23:49.000
But in Go, for instance, um… It's, it's just a go routine. Any function that you want, you can call in this magic way by instead of saying, you know, the function is named foo. Instead of saying foo, open paren, some parameters, close paren, you say.
00:23:50.000 --> 00:23:51.000
Go.
00:23:51.000 --> 00:24:01.000
And it does its thing. No color, no infection, no spread. And now it communicates with a channel.
00:24:01.000 --> 00:24:17.000
That's pretty, pretty cool. That, like, you say that it's cool, it's this thing, you say that a lot. I don't say it very often. And I'm not a Go programmer. I can write stuff in Go if I have to, but it's not the first tool I reach for.
00:24:18.000 --> 00:24:26.000
But this is a reason to reach for Go. This is a really good thing that Go does.
00:24:26.000 --> 00:24:27.000
Um, Rust.
00:24:28.000 --> 00:24:33.000
Rust doesn't actually provide the mechanism.
00:24:33.000 --> 00:24:39.000
to do the threading, but it does provide the API.
00:24:39.000 --> 00:24:47.000
So, it has a way to communicate between, uh, things that is not unlike Go's channels.
00:24:47.000 --> 00:25:08.000
But you supply the underlying library that decides, is this a thread? Is it a… process? Is it a… what… what are you going to get? Um, which I think puts it even one step, uh, beyond Go. I think that's a fascinating answer. So, um…
00:25:06.000 --> 00:25:14.000
So, so with, with Rust, is it, is it moved more into like the standard library than in the language itself?
00:25:14.000 --> 00:25:19.000
The threading and that kind of thing?
00:25:19.000 --> 00:25:28.000
Um… You know, the main… so, I'm not a Rust expert, although I want to be.
00:25:28.000 --> 00:25:47.000
Uh, my understanding is that one of the most important Rust libraries in this, uh, neighborhood of solutions is called Tokyo, and it is about threading, and it is not in the standard library.
00:25:46.000 --> 00:25:50.000
Okay, but it's… it's a library.
00:25:47.000 --> 00:26:02.000
But it is so commonly used that it might it might as well be in the standard library Tokyo is as important to rust as a thing that used that we used to use all the time in Python called twisted.
00:26:03.000 --> 00:26:19.000
Used to be twisted is less important now, but Python and everything that came after learned a lot from twisted and I think there's a lot of code in rust that is greatly informed.
00:26:11.000 --> 00:26:12.000
Umm.
00:26:19.000 --> 00:26:21.000
Bye, Tokyo.
00:26:21.000 --> 00:26:22.000
Um, so.
00:26:23.000 --> 00:26:29.000
Two important concepts. Concurrency and parallelism.
00:26:29.000 --> 00:26:36.000
Concurrency is when you are waiting a lot, and the answer is scheduling.
00:26:36.000 --> 00:26:46.000
Parallelism is when you have more problem than you have computer, and the answer is more processing.
00:26:46.000 --> 00:26:51.000
And now let me give you one other really interesting word.
00:26:51.000 --> 00:26:57.000
Before, when I talked about optimization and measurement, and by the way.
00:26:58.000 --> 00:27:02.000
Measurement is a huge part of this world.
00:27:02.000 --> 00:27:12.000
how you are going to solve problems, uh, using these tools, you need to measure what is the constrained resource? What do I get when I do such and such?
00:27:12.000 --> 00:27:14.000
Well.
00:27:14.000 --> 00:27:25.000
Before, when talking about speeding up pieces of code, I talked about the idea that if you are trying to optimize some piece of your code.
00:27:25.000 --> 00:27:33.000
that takes 20% total of the execution time to solve some particular problem.
00:27:33.000 --> 00:27:39.000
then even if you make that part of your processing go away ENTIRELY.
00:27:39.000 --> 00:27:44.000
The most you can get out of that is a 20% speedup.
00:27:46.000 --> 00:27:56.000
It turns out that there is a similar effect in, uh, this whole idea of concurrency and parallelism, and that is.
00:27:56.000 --> 00:28:11.000
if the fraction of your program that can be parallelized… let's go the opposite direction for a second. Let's say that there is 20% of your program that can't be parallelized.
00:28:11.000 --> 00:28:16.000
then the best you can do is an 80% speedup.
00:28:16.000 --> 00:28:18.000
It has a name.
00:28:18.000 --> 00:28:33.000
And it turns out it's the same thing. It's the same name in concurrency and parallelism that it is in straight optimization. And that is Amdahl's law. And yes, it's that Amdahl.
00:28:33.000 --> 00:28:38.000
The one, uh, who is the computer expert and, uh…
00:28:36.000 --> 00:28:39.000
Yeah, Jean Amdahl.
00:28:39.000 --> 00:28:55.000
Yeah. Amdahl's law says that, you know, once you measure and you see what part can be sped up, that's the limit right there. Whether it's parallelism or concurrency or optimizing something away entirely.
00:28:39.000 --> 00:28:41.000
Right? Yeah.
00:28:55.000 --> 00:29:08.000
measure, you'll know what the limit is, and that limit, uh, is the expression of Amdahl's Law for your particular problem. It's not a law in the same way that, um.
00:29:08.000 --> 00:29:14.000
Uh, what's the one about, uh, number of transistors in a…
00:29:13.000 --> 00:29:16.000
Oh, Moore's Law.
00:29:14.000 --> 00:29:30.000
In a… Moore's Law. It's not, like, something independent of the problem in the way Moore's Law is. Um, it is super dependent on your problem and on your specific solution to the problem.
00:29:17.000 --> 00:29:18.000
Yah.
00:29:30.000 --> 00:29:42.000
Um, but it is its own thing. So… Now you know I do kind of want to talk about a couple of interesting.
00:29:42.000 --> 00:29:55.000
problems around the answers, um, and levels at which the answers occur. Like, there's parallelism that is beyond your ability to detect.
00:29:55.000 --> 00:29:59.000
Uh, for instance, inside a CPU.
00:29:59.000 --> 00:30:07.000
I can name at least two things that are, um… Uh, important one is the pipeline.
00:30:06.000 --> 00:30:08.000
Umm.
00:30:07.000 --> 00:30:24.000
Uh, the pipeline inside a CPU means that multiple instructions are happening at the same time, at various stages of their own execution. They don't exactly happen entirely at the same instant, but they are overlapping in a way.
00:30:24.000 --> 00:30:47.000
that makes you be executing more code than you would think you could. And there's branch prediction. Branch prediction isn't exactly getting two pieces of code to execute at the same time, unless it is, um… The kind of branch prediction where it does a bit of one side, but mostly has chosen the other.
00:30:47.000 --> 00:30:48.000
Umm.
00:30:48.000 --> 00:30:59.000
these are both a level of parallelism that you can't see, uh, unless you have special tools and you're working with a special version of the CPU.
00:30:59.000 --> 00:31:15.000
Um… you have a much better look inside concurrency, because concurrency is a scheduling problem, so you can actually look BEFORE it actually starts executing code. That's useful.
00:31:15.000 --> 00:31:18.000
So that's one thing.
00:31:18.000 --> 00:31:21.000
And a second thing is.
00:31:21.000 --> 00:31:29.000
Once your problem is going to be solved by some, uh, form of parallelism.
00:31:29.000 --> 00:31:44.000
Now you have another problem, and that problem is… communication. It's getting data from where you are to the place where you're going to solve a whole different part of it.
00:31:35.000 --> 00:31:37.000
Right.
00:31:44.000 --> 00:31:51.000
Um, if it's on a different CPU, in a different machine, across a cluster.
00:31:51.000 --> 00:32:06.000
Well, now you've got to send data across, you know, what TCP or who knows what, the fabric. If you're two machines in the same data center, you're not necessarily connected by TCP. You might be connected by an interconnect fabric or something.
00:32:05.000 --> 00:32:09.000
Yeah, was it a mirror net or there's there's various.
00:32:06.000 --> 00:32:24.000
which… There's tons. So there's that. Or maybe you're on a GPU. So now what matters is getting the data across the bus, which could be a PC.
00:32:09.000 --> 00:32:10.000
ways.
00:32:24.000 --> 00:32:27.000
IE bus or a P what whatever it is.
00:32:26.000 --> 00:32:28.000
PC yeah yeah.
00:32:28.000 --> 00:32:35.000
it's not nearly fast enough. None of it is fast enough. The great thing about parallelism is you've got.
00:32:35.000 --> 00:32:46.000
X times as much processing power, and the horrible thing about it is that you have to send the data there. Um, a nice compromise is on, uh, Apple Silicon.
00:32:46.000 --> 00:32:58.000
it uses shared memory, so there's not a communication problem, but on the other hand, where a NVIDIA GPU has.
00:32:58.000 --> 00:33:05.000
X thousands of cores, um, and they're simple and can do simple math.
00:33:05.000 --> 00:33:15.000
Where by simple math, I do mean that it can do reasonable amounts of linear algebra, the kind of stuff you'd want to do if it was video.
00:33:15.000 --> 00:33:21.000
But on a piece of Apple Silicon, maybe you get 40 cores.
00:33:21.000 --> 00:33:24.000
40 is not 10,000.
00:33:24.000 --> 00:33:46.000
They are significantly more interesting cores, individually. Um… Of course, you only get one kind on Apple Silicon at this moment, whereas on an NVIDIA card, there's three different levels of GPU cores, all the way from the smallest, fastest leaf cores.
00:33:46.000 --> 00:33:57.000
cores, up to the highest level ones where you only have 10 or 100, I don't even know how many of them, but that make big decisions and part out the work and whatnot.
00:33:57.000 --> 00:34:13.000
But zero communication overhead on the Apple Silicon and gigantic communication overhead on the NVIDIA system or whatever particular thing that it is because there's AMD and there's NVIDIA and there's this Apple Silicon.
00:34:13.000 --> 00:34:26.000
NVIDIA uses CUDA, and AMD uses ROCm, and Apple uses Metal, and all of them can be used to solve non-graphics problems.
00:34:26.000 --> 00:34:52.000
But once again, you need to look at your problem. You need to identify what is the band… what is the scarcity? What is the cost? What is the thing that you need to, uh… optimize for. Um… And then you need to pick the appropriate kind of parallelism and hardware on which you would like to solve it.
00:34:52.000 --> 00:35:13.000
IF you are allowed to choose. Just like when we talked about languages, we talked about the choices into which you could be forced. Well… if you're doing a video problem, and you want to run on Windows machines, because it's a game, and it's a AAA game, and you want people to buy it.
00:35:13.000 --> 00:35:24.000
Well, you already know what the answer is. The answer is, you're gonna solve it on an NVIDIA card, and your problem is communication, and you're gonna do video work, and that's all there is to it.
00:35:24.000 --> 00:35:38.000
If your problem is you want to search a strand of genetic material, and it is a million bases long, and you want to find something that's 60 bases long in that.
00:35:38.000 --> 00:35:52.000
Now you have a different problem, and you have different things to optimize for. So… Almost never do you have the right amount of hardware to solve the problem that you want.
00:35:53.000 --> 00:36:03.000
So you need to understand your hardware, and you need to understand your problem, and you need to understand if there's other ways to solve your problem.
00:36:03.000 --> 00:36:12.000
I wanted to mention, um… some of the interesting ways that we have done this in the past. Threads were easy.
00:36:12.000 --> 00:36:22.000
But, um… In the past, on Linux and Unix systems in particular, we have used fork.
00:36:22.000 --> 00:36:28.000
Fork is awesome, in some ways, and kinda terrible in others.
00:36:28.000 --> 00:37:01.000
At the beginning, Fork used to do a thing where it would copy the world. It would copy your entire process, including all file descriptors and any resources that you might use, and um… Then the only change would be that you are the child or you are the parent and you could do some particular thing. And sometimes what you really wanted was you didn't even actually want that copy. What you wanted was you wanted to be something else entirely.
00:37:01.000 --> 00:37:14.000
So, you did a fork with that giant copy, and then an exact, and suddenly you were something else! making that whole copy kind of a waste. Um, but that's a thing, that's what we used to do.
00:37:15.000 --> 00:37:18.000
Since that time, we've made fork.
00:37:18.000 --> 00:37:28.000
in a lot of cases, significantly faster, because we use a copy-on-write version of fork. So fork.
00:37:28.000 --> 00:37:37.000
acts exactly like it does a complete copy of your entire process, but it doesn't. It just starts running.
00:37:37.000 --> 00:37:41.000
And it doesn't actually copy anything.
00:37:41.000 --> 00:37:56.000
until you write. And since most of the thing that you just copied is executable code, and code is in volatile segments, they can't be modified.
00:37:56.000 --> 00:38:03.000
Therefore, your code doesn't get copied, but your data, which includes the file descriptors, does.
00:38:03.000 --> 00:38:04.000
Bats.
00:38:04.000 --> 00:38:18.000
the part that might get modified if, uh, you're gonna do something, and then that's the part that eventually will get copied. Everything is faster. Unless you're Python. If you're Python, and you do reference counting.
00:38:18.000 --> 00:38:19.000
Ugh.
00:38:19.000 --> 00:38:32.000
then instantly just looking at a variable changes its reference count, and then back again, so copy and write almost never helps you. And also, Python had this horrible thing called the GIL.
00:38:32.000 --> 00:38:47.000
the, uh, global interpreter lock. The global interpreter lock is a way of solving this problem of multiple stuff happening at the same time, um, by stopping two different pieces of code.
00:38:47.000 --> 00:38:53.000
from trying to have a coherence problem with the same piece of data.
00:38:53.000 --> 00:38:59.000
Now, in more recent versions of Python, I think starting at 3.14.
00:38:59.000 --> 00:39:17.000
You can run without the GIL, and I think in 3.15, running without the GIL will become the default. For some stuff, for single-threaded stuff, in 3.14, it wasn't necessarily faster. In fact, I might go so far as to say it was often slower.
00:39:17.000 --> 00:39:26.000
Uh, but if you actually were multi-threaded, it was… it was a win. But soon, it will be the case that, uh, life is just better with that.
00:39:26.000 --> 00:39:27.000
Umm.
00:39:27.000 --> 00:39:28.000
So.
00:39:29.000 --> 00:39:41.000
How do you know what you want? Uh, you have to decide. Are you IO-bound? If you're waiting for the disk, if you are waiting for the network, if you are waiting, you know, if you're waiting.
00:39:41.000 --> 00:39:46.000
Waiting is the key. Then threads.
00:39:46.000 --> 00:40:01.000
Concurrency, probably your answer. If you're CPU-bound, if you are, um… trying to calculate, uh, hashes or encrypt, um.
00:40:01.000 --> 00:40:07.000
I mean, in a way, you're data-bound for those, but… but if you're CPU-bound, if it's pure cal.
00:40:07.000 --> 00:40:10.000
Probably parallelism is the answer.
00:40:10.000 --> 00:40:17.000
There is a third choice, it doesn't… crop up a lot, but it's if you're memory-bound.
00:40:17.000 --> 00:40:25.000
If you're memory-bound, it's a lot like Jim's location problem with his base station.
00:40:25.000 --> 00:40:37.000
The answer to a problem of being memory-bound isn't about picking threads or picking multiple CPUs. It is about.
00:40:37.000 --> 00:40:44.000
probably layout of your data structures in memory. Can you get that right?
00:40:44.000 --> 00:41:10.000
Um… And the real hard part is when you want to share data. Is it something that you need to send between two things using communication? Is there coherency that you have to manage? Is there locking? Um… Different languages use different things. Um, there is one model that I didn't talk about, and that's the.
00:41:10.000 --> 00:41:16.000
actor model. Most of the languages that we use.
00:41:16.000 --> 00:41:33.000
don't leverage the actor model. This is kind of an Erlang thing. But it turns out Swift does use actors. An actor is when it's the object itself that owns the data and nobody else owns it.
00:41:33.000 --> 00:41:38.000
So you can only get to it by executing code in the object?
00:41:39.000 --> 00:41:49.000
That kind of solves the access problem, as long as you don't provide a bunch of escape hatches.
00:41:49.000 --> 00:42:02.000
Um… Anyway… I think we can make today be kind of a short episode if I give you the takeaways from this. Because really.
00:42:02.000 --> 00:42:08.000
It's just those three main things, I think. Is it a scheduling problem?
00:42:08.000 --> 00:42:11.000
Or is it a work problem, a CPU problem?
00:42:10.000 --> 00:42:11.000
Sure.
00:42:11.000 --> 00:42:16.000
Um, so the takeaways are, um.
00:42:16.000 --> 00:42:20.000
That either you have more computer than you have code.
00:42:20.000 --> 00:42:33.000
A bunch of cores going to waste. And if that's the case, the answer's concurrency. Or… You have more work to do than you have computer to do it.
00:42:33.000 --> 00:42:40.000
In which case, the answer is parallelism. Get more CPUs. Get more cores. Get more machines.
00:42:40.000 --> 00:42:42.000
Get more.
00:42:42.000 --> 00:42:51.000
You need to diag… just like every single episode we have, I'm going to harp on it, you need to diagnose.
00:42:51.000 --> 00:42:58.000
before you write the code. Start measuring in advance. Figure out where is the problem.
00:42:58.000 --> 00:43:07.000
And the actual enemy isn't how much work you have to do. It's not what you're waiting for. It's sharing state.
00:43:07.000 --> 00:43:14.000
It's communication and coherency. That is your enemy, no matter which mechanism you end up choosing.
00:43:14.000 --> 00:43:23.000
That's the thing you will fight. That is the thing you will get wrong the first three times. That is the thing the languages that help you with.
00:43:23.000 --> 00:43:28.000
That's the place they're going to help you the most.
00:43:28.000 --> 00:43:40.000
Go helps you because it uses channels. Rust helps you because it enforces strict ownership. Only one owner gets a mutable reference to something.
00:43:40.000 --> 00:43:45.000
Or, everybody has a read-only reference.
00:43:45.000 --> 00:43:55.000
Amdahl's Law is the thing that essentially, um, I guess the word here is humbles you. Um, you're gonna know in advance.
00:43:56.000 --> 00:43:59.000
You can't do better than this.
00:43:59.000 --> 00:44:05.000
If you are 5'2", it is unlikely you are going to be dunking.
00:44:05.000 --> 00:44:11.000
If only 20% of your program is parallelizable.
00:44:11.000 --> 00:44:12.000
Ben?
00:44:12.000 --> 00:44:22.000
is the fastest you will get to go. Is that 80% at normal speed, and then some speed up on that 20%.
00:44:22.000 --> 00:44:24.000
Python?
00:44:24.000 --> 00:44:31.000
Used to have lots of limits, and is having some of them… removed.
00:44:31.000 --> 00:44:40.000
Um… And… You should know when not to parallelize. If you're already fast enough.
00:44:40.000 --> 00:44:52.000
Maybe it's okay you have too much computer. Maybe. Or whatever. But it's key to know. We use this magic word. We talk about Heisenbugs.
00:44:52.000 --> 00:44:55.000
Bugs that only show up.
00:44:55.000 --> 00:45:02.000
Um, when they are least wanted, and are incredibly hard to reproduce under a debugger.
00:45:02.000 --> 00:45:12.000
Well, let me tell you where they live. They live in parallelized code. So, it could well be the case that.
00:45:12.000 --> 00:45:18.000
You have a problem where parallelization is an option?
00:45:18.000 --> 00:45:21.000
But it's not a guaranteed win?
00:45:22.000 --> 00:45:30.000
If the win isn't guaranteed, go spend your money on something where the win is guaranteed.
00:45:30.000 --> 00:45:34.000
That's what I think the takeaways are.
00:45:34.000 --> 00:45:36.000
What do you think, Jim?
00:45:34.000 --> 00:45:51.000
So I've, well, I've got an example of parallel parallelization that makes sense. I just read this earlier today and I don't remember it was on Hacker News or someplace talking about how Pixar.
00:45:51.000 --> 00:46:01.000
uh, renders their movies. Um, and, you know, when, when you're doing an animated, uh, uh, movie, um.
00:46:01.000 --> 00:46:08.000
the, the animations have gotten so good. If you, if you look back at, um, the one I can think of is, um.
00:46:08.000 --> 00:46:24.000
Finding Nemo, where they made the water look like water, and now, like, grass looks like grass. And I think even back when they did Hair Looks Like Hair, back when they did the movie Titanic.
00:46:17.000 --> 00:46:20.000
And hair, hair looks like hair.
00:46:24.000 --> 00:46:39.000
Um, rendering that water, uh, there was some, some new technology had to be created to do that. Anyway, um, the thing I read today said, uh, that a single frame of, of.
00:46:39.000 --> 00:46:43.000
those movies takes 24 hours to render.
00:46:44.000 --> 00:46:58.000
That's just one frame, and what are movies? 30 frames per second? I think they said the average movie is about 130,000 frames. Now, if you had 130,000 frames.
00:46:58.000 --> 00:47:01.000
And it took 24 hours per frame.
00:47:01.000 --> 00:47:14.000
um, there's not enough time for, for any of us to, to, you know, we would never be able to watch the movies because they would take that long to create. So they went to massive parallelization.
00:47:14.000 --> 00:47:29.000
Pixar developed this whole thing using… creating render farms, where they are, indeed, processing multiple frames, like, thousands or tens of thousands of frames simultaneously. So at the end of 24 hours.
00:47:29.000 --> 00:47:42.000
they could have tens of thousands of frames done, instead of just one. So that's a great place, a great example of running things in parallel, where your problem is bigger than your compute.
00:47:42.000 --> 00:47:57.000
Yeah, and, uh, like, some things to point out about that situation are the data that needs to be shared, um, first of all, is actually independent per frame, because, uh.
00:47:57.000 --> 00:48:11.000
The big data is all the pixels at the end. It's the output, not the input. The input is 3D models, and the 3D models are actually different per frame, because they've moved, things have moved.
00:48:11.000 --> 00:48:26.000
So it's not like there's a coherency problem. That's a big, big deal. Second, I wanted to say, remember when I can think of at least two movies back in the beginning time of computer animation and CGI.
00:48:26.000 --> 00:48:38.000
Tron was an early one, and so was The Last Starfighter. I thought those were both interesting. And I used to work at Trolltech, a company… it's now known as The Cute Company.
00:48:27.000 --> 00:48:28.000
Yep.
00:48:38.000 --> 00:48:53.000
And, uh, we had, you know, Qt is a gigantic C++ framework, especially back in the day. It was way bigger than the computers, but you needed the libraries to be pre-built.
00:48:53.000 --> 00:49:11.000
to do your thing, but we were constantly… I wasn't, I was the… I was the evangelist, so I don't want to confuse anybody that I was actually writing code there that was for part of the library. I was writing code, but it was, like, demo code and… And whatnot. Um, uh, anyway.
00:49:11.000 --> 00:49:13.000
They had a system.
00:49:13.000 --> 00:49:22.000
Which other people started to emulate, where, when you wanted to build a specific, um, version of the code.
00:49:22.000 --> 00:49:29.000
You would start… And every machine in the office.
00:49:29.000 --> 00:49:31.000
would help.
00:49:31.000 --> 00:49:48.000
you'd compile two files, that machine would compile five files, the other machine would compile three, and they were tied to the specific commit. It wasn't Git back then, I remember, I think maybe we were using Subversion, I can't remember.
00:49:48.000 --> 00:49:58.000
But it was tied to the specific commit, so later, when people wanted those object files, they could get the right one, and they'd already been compiled.
00:49:58.000 --> 00:50:07.000
That's a form of parallelism. And there, they get the data from the centralized repo.
00:50:07.000 --> 00:50:16.000
And there's no duplication, because you're compiling one file, and the other guy is compiling a different file.
00:50:16.000 --> 00:50:23.000
There's lots of problems where the answer is parallelism, or concurrency, or…
00:50:18.000 --> 00:50:20.000
Sure.
00:50:23.000 --> 00:50:25.000
Or who knows what.
00:50:23.000 --> 00:50:40.000
Sure, sure. Uh, make minus J. I do that all the time when I build Postgres or whatever. And… there's no communication between them, because they're all compiling different files. Every thread, uh… or every process is compiling a different file, so they don't need to talk to each other.
00:50:40.000 --> 00:50:54.000
But, uh, Make is a great tool for that. Make-j16 will, uh, spawn 16 threads to build your program much faster, assuming you have the CPU cores to do that.
00:50:54.000 --> 00:51:12.000
Right. You definitely want cores for that, as opposed to threads, because that's a problem that's CPU-bound, um, not IO-bound. It's not like you're waiting for that file. And if you had threads, and it was one CPU switching between compiling the different files.
00:51:03.000 --> 00:51:04.000
Yeah.
00:51:12.000 --> 00:51:21.000
it wouldn't be faster. In fact, it would be very slightly slower. So, minus J, really good when you have multiple cores.
00:51:21.000 --> 00:51:50.000
Sure. And, uh, you mentioned, very briefly, JavaScript. I've been doing a lot of programming in JavaScript lately. And, you know, JavaScript is single-threaded, and it's all built around an event loop. So when you… I'm doing it in Node, so I'm making database calls, and… and other things. And when I, when I go to submit a query, or, or, you know, execute a query, um, I, I use async and await, and, and that, uh, not always, but, um.
00:51:50.000 --> 00:51:58.000
When I feed a query to Postgres, my program is free to do other things.
00:51:58.000 --> 00:52:08.000
Right? It's, it's, uh, that, that, that, uh, database query's running in the background. Uh, it's… JavaScript itself is single-threaded.
00:52:08.000 --> 00:52:11.000
It's running in your background.
00:52:08.000 --> 00:52:14.000
But I can… What do you mean?
00:52:14.000 --> 00:52:29.000
I mean, the query has gone off, and now it is the primary importance to Postgres. It's running in Postgres' foreground. You're just waiting for an answer to come back. And the fact that you're waiting.
00:52:17.000 --> 00:52:19.000
Yes.
00:52:24.000 --> 00:52:26.000
Right, right.
00:52:26.000 --> 00:52:44.000
Right, and when I get an answer… Yeah. And if I don't do the async await, it'll run a callback for me. It'll basically run… my program will process the result of that query when it.
00:52:31.000 --> 00:52:32.000
Yeah.
00:52:44.000 --> 00:52:57.000
finishes doing whatever it's doing. Like, my program can be sitting there doing lots of things, and when the query finishes, my program will get that and use that. And I've recently.
00:52:55.000 --> 00:53:07.000
Which you can do either with threads, or with promises and futures. Um, both of those give you the same power, which is, you're waiting.
00:53:01.000 --> 00:53:03.000
Yeah.
00:53:07.000 --> 00:53:13.000
And why wait? Like, while you're waiting, let's go do something useful and interesting.
00:53:11.000 --> 00:53:26.000
Right, right. But something I'm using now, well, an issue I ran into, I can only queue up, I think it's four, I can only submit up to four queries to Postgres from that single thread.
00:53:26.000 --> 00:53:33.000
And all the other queries that I might queue up will just sit there in a queue, they won't actually run. But 4 of them will run.
00:53:33.000 --> 00:53:40.000
uh, the way, the way Postgres works with, with Node.js. So four of those queries will just be off running on their own.
00:53:40.000 --> 00:53:57.000
I'm working on a problem where I have a lot more queries to run, so I've resorted to worker threads, which, uh, exist in Node, and they also exist in browsers. Uh, you can, you can run, uh, it's called something else in browsers. It's, uh… Uh, I forget, worker something or other.
00:53:57.000 --> 00:54:00.000
There is a thing called Web Workers.
00:53:57.000 --> 00:53:59.000
Umm.
00:53:59.000 --> 00:54:11.000
Web workers. It's a web worker in the browser, it's a worker thread in Node.js, and I can spawn as many of those as I need. And, you know.
00:54:12.000 --> 00:54:27.000
This way, I could submit lots of queries to the database, and of course, I need a powerful enough database server to handle that. But it's… so far, it's working really, really well. So you can do multi-threaded stuff in JavaScript in Node.
00:54:27.000 --> 00:54:30.000
And I'm kind of having fun with that right now.
00:54:30.000 --> 00:54:34.000
But anyway, um… Boy, you covered a lot of stuff.
00:54:34.000 --> 00:54:37.000
Some real interesting stuff.
00:54:37.000 --> 00:54:56.000
I'm anxious to get this episode out there for people to listen to and offer some feedback. So again, feedback. Send your feedback to us, please. We love it. Feedback at runtimearguments.fm. You can go to our website at runtimearguments.fm.
00:54:56.000 --> 00:55:01.000
Uh, I forget, do we need the WWW on it? Or we don't?
00:54:59.000 --> 00:55:03.000
You absolutely do not want the WWW.
00:55:02.000 --> 00:55:07.000
We don't want the www.runtimearguerstaffer, that's not what you want.
00:55:04.000 --> 00:55:07.000
And you don't want the ass either.
00:55:07.000 --> 00:55:20.000
It's not HTTPS, because of where the website is hosted. But we will have show notes on the end of this episode, so you can check out all the links there, and a transcript, and um…
00:55:15.000 --> 00:55:17.000
And a transcript.
00:55:20.000 --> 00:55:23.000
Wow, it's been a great episode, Wolf.
00:55:23.000 --> 00:55:31.000
I had fun with this one. Thank you and thanks to everybody for listening. And Wolf, you got anything else to say or are we done?
00:55:31.000 --> 00:55:41.000
Uh, I think… we're almost done. I think we should put a link to the YouTube video about DapMux in our show notes.
00:55:40.000 --> 00:55:42.000
Yes, we'll do that.
00:55:41.000 --> 00:55:48.000
And it's possible to make a link that starts right at the beginning of the DATMUX part.
00:55:48.000 --> 00:55:51.000
Which is about 24 minutes in.
00:55:51.000 --> 00:55:53.000
You measured.
00:55:53.000 --> 00:55:54.000
Yeah.
00:55:53.000 --> 00:55:59.000
Well, it says, you know, it's right in the corner. I didn't have to, you know, go through it or anything.
00:55:55.000 --> 00:55:57.000
Right.
00:55:57.000 --> 00:55:58.000
Right.
00:55:58.000 --> 00:56:04.000
Right! Uh, I'll have to see how we can do that, but yeah, we'll do that.
00:56:04.000 --> 00:56:09.000
We'll do that. Uh, so thank you everybody. And until next time, uh, goodbye.
00:56:09.000 --> 00:56:11.000
Bye bye
