Episode 311: Career Changes, GPUs & More with Zach Laine ၏ စာသားမှတ်တမ်းအပြည့်အစုံ

assembled the entire program instruction by instruction. 00:02:28.719 --> 00:02:29.759 Right. 00:02:30.319 --> 00:02:32.319 He's like, oh, definitely a zero goes here. 00:02:32.319 --> 00:02:33.759 Uh now this one's a one. 00:02:34.080 --> 00:02:34.400 Yeah. 00:02:34.400 --> 00:02:40.479 I guess there we joke, but there was a period of time when people did do that. 00:02:40.879 --> 00:02:42.000 Oh, yeah, yeah, absolutely. 00:02:42.000 --> 00:02:54.080 And in fact, like when I was maybe like high school age, my mom was dating a guy that was a little bit older than her, and um he had he literally been a rocket scientist at some point. 00:02:54.080 --> 00:02:59.599 He worked in NASA at the Apollo era in Houston, and you know, he still lived in Texas at the time and stuff. 00:02:59.599 --> 00:03:07.680 So I remember, you know, I was just getting out of high school and about to start college, and he was asking me what I was gonna major in, and I said, Computer science is like, what's the point? 00:03:07.680 --> 00:03:12.800 I mean, you just like there's a certain number of opcodes, and you got the you got the registers, you just look it up. 00:03:12.800 --> 00:03:15.439 Like, what what is the is it four years you're gonna study this? 00:03:15.439 --> 00:03:17.919 Like, uh, it's very different now, dude. 00:03:17.919 --> 00:03:32.560 That's uh you know, he was coming from the from the physics side, like so if to him it was just like this toolkit, and so he had very much uh a workmanlike uh approach to what programming is and could be and stuff, and I thought that was pretty funny. 00:03:32.879 --> 00:03:34.080 Yeah, it's it's simple, kid. 00:03:34.080 --> 00:03:38.319 Go go go re real go learn something useful like astrophysics. 00:03:38.639 --> 00:03:39.039 Right. 00:03:39.039 --> 00:03:43.039 Unuseful like what happens when two uh the galaxies collide with each other. 00:03:43.039 --> 00:03:44.240 Yeah, yes, yeah. 00:03:44.240 --> 00:03:46.240 You know, something practical. 00:03:49.199 --> 00:03:50.319 Alright, well I can't figure it out. 00:03:50.319 --> 00:03:58.000 We're on Chrome now, so hopefully uh if if something crashes for you, Bryce, it'll retain whatever amount of the Audacity file that you have. 00:03:58.000 --> 00:04:02.319 And yeah, so we it everyone's recording locally though, correct? 00:04:02.319 --> 00:04:04.560 Bryce? 00:04:04.879 --> 00:04:05.759 Yeah, I am, yeah. 00:04:06.080 --> 00:04:06.639 Alright. 00:04:06.639 --> 00:04:08.560 So I just looked it up, Zach. 00:04:08.560 --> 00:04:12.240 Did you know the last time you were on ADSP? 00:04:12.879 --> 00:04:13.280 Uh no. 00:04:13.280 --> 00:04:13.919 No, I don't. 00:04:13.919 --> 00:04:16.480 I'm gonna guess 2024. 00:04:16.800 --> 00:04:19.040 It was very close to 2024. 00:04:19.040 --> 00:04:23.199 It was the holiday special in December of 2023. 00:04:23.199 --> 00:04:35.199 So we are a couple months away from it being three years since we've had our one, two, three, four, five,

sixth most popular, or is it popular? 00:04:35.199 --> 00:04:37.439 Sixth most frequent guest, I guess. 00:04:37.439 --> 00:04:43.040 Uh the frequency does not necessarily 100% correlate with popular. 00:04:43.040 --> 00:04:45.120 I guess something's buzzing. 00:04:45.120 --> 00:04:46.879 What the hell is that? 00:04:46.879 --> 00:04:47.839 Is something buzzing? 00:04:49.199 --> 00:04:50.319 Oh sorry. 00:04:50.319 --> 00:04:51.360 What an idiot. 00:04:51.360 --> 00:04:54.639 I put my phone right behind the computer, and so it sounded like it's coming from the computer. 00:04:54.639 --> 00:04:55.600 Like, what is that sound? 00:04:55.600 --> 00:04:56.879 Why is my computer buzzing? 00:04:56.879 --> 00:04:58.639 It's my phone ringing on buzz. 00:04:58.639 --> 00:04:59.199 Okay, never mind. 00:04:59.759 --> 00:05:00.879 It's all in your head, Zach. 00:05:00.879 --> 00:05:02.959 You're starting to you're starting to start. 00:05:03.839 --> 00:05:05.759 Does anyone else smell burnt toast? 00:05:07.279 --> 00:05:08.879 Anyway, so it's been three years. 00:05:08.879 --> 00:05:10.319 We have a lot to catch up on. 00:05:10.319 --> 00:05:20.240 I don't think actually much has changed in the technology industry, so we'll probably just have to focus on uh but uh how are the kids? 00:05:20.319 --> 00:05:21.120 What else is it? 00:05:23.759 --> 00:05:24.639 How have you been? 00:05:24.639 --> 00:05:27.199 How have things uh how have things been? 00:05:27.199 --> 00:05:28.160 Good. 00:05:28.160 --> 00:05:39.439 I changed jobs since the last time I talked to you guys, so now I work with uh a bunch of people from the committee, and Robert Leahy is the one that was leading the team when I joined it. 00:05:39.439 --> 00:05:43.199 He's one of the committee members that you know I've gotten to know over the last few years. 00:05:43.199 --> 00:05:47.759 And then he bounced almost immediately after hiring me. 00:05:47.759 --> 00:05:54.560 But uh but it's a good team in the in the sense that um I mean particularly in the sense that like there's a lot of committee membership. 00:05:54.560 --> 00:05:57.920 So like we've got five programmers and three of us are on the committee. 00:05:57.920 --> 00:06:00.000 Oh no, six and and four of us are on the committee. 00:06:00.000 --> 00:06:04.560 Um and when Robert was on the team, it was more like five and six, which is pretty amazing. 00:06:04.560 --> 00:06:07.120 Like I've never worked with a team that had that much committee participation. 00:06:07.439 --> 00:06:09.199 And and what what is the company? 00:06:09.519 --> 00:06:23.519 So it's you know, used to be the May Street division, so a lot of those guys, like you might have heard of them working at May Street, but that's now like that name is pretty much gone from use now, but we've we're we got acquired before I even joined by uh London Stock Exchange. 00:06:23.519 --> 00:06:26.959 So we like provide data, but we don't do trading. 00:06:26.959 --> 00:06:30.560 So we're adjacent to trading kind of firm, so we don't do that stuff. 00:06:30.879 --> 00:06:50.399 So when I first met Zach, like the first three times I saw Zach were at C now's or maybe one other conference, and every t like it was a period in his life when he had changed he was been at like one job for a while, then he changed, and he was at another job for a short period of time, and then he changed to another job. 00:06:50.399 --> 00:06:54.800 But then like all I saw was just every time I see this guy, he's at a different job. 00:06:56.079 --> 00:06:56.800 Yep, yeah. 00:06:56.800 --> 00:07:10.319 And that's the thing is that like so you you first met me when I worked at my very first job, and then the fourth job I had was at NVIDIA, and that time period you're talking about, I think was like something like five or six conferences in a row. 00:07:10.319 --> 00:07:17.920 Like I either had just changed or I was just about to change uh jobs, and that was just because like yeah, there was a bunch of wacky stuff that happened. 00:07:17.920 --> 00:07:29.519 Like, so the first job I went to, like, I went there, and then out of the blue, one of the guys that I used to work with the previous job recruited me to work on this project that was super interesting and made me jump ship just because it was so interesting to work on, right? 00:07:29.519 --> 00:07:53.040 And then when I went there, almost immediately they got acquired by this big defense contractor, and everything changed, and the place was awful to work at, so I left there, and then I went to work at NVIDIA, and there my the manager that hired me also bounced and uh and then left me with this manager that was the only manager of my entire career I've been unable to get along with, and uh so I left the company because of him, and then I worked at Cadence for a very long time though. 00:07:53.040 --> 00:07:55.040 Once I landed there, yeah, yeah, you were you're a kid. 00:07:55.279 --> 00:07:56.720 I was I was so confused. 00:07:56.720 --> 00:08:00.560 I was young in my career, so I didn't know any better. 00:08:00.560 --> 00:08:04.800 Like I I just assumed that like Zach just had a new job every every time. 00:08:05.759 --> 00:08:08.160 And but you know, there are some people that do that though. 00:08:08.160 --> 00:08:15.120 I it's really odd to me that some people are like, you know, about every 18 to 24 months, they just like, yeah, I'm I'm done with this one, I'm going to the next one. 00:08:15.120 --> 00:08:19.279 And if you're good enough, you can kind of pull that off, right? 00:08:19.279 --> 00:08:28.480 Like some people will you will find someone to hire you in, you know, maybe not in the current market in the last uh year or so, but you know, for for a long time that was true. 00:08:28.959 --> 00:08:44.159 I mean, I think in for some people they like that because they get to go like I if you stay at a company, at least me, like I've been at NVIDIA for nine years, but I have not worked on the

same thing for nine years, right? 00:08:44.159 --> 00:08:53.279 It's been about that length of time, like 18 to 24 months, like between like has been like the average duration of like a phase of my career at NVIDIA. 00:08:53.279 --> 00:08:59.279 Like, you know, I worked on these sets of projects and the I started working on these sets of things, and I started working on these sets of things. 00:08:59.279 --> 00:09:08.320 There are some people who just work on one thing like their whole career, but then there's also people who will be at one company but will work on a bunch of different things. 00:09:08.320 --> 00:09:16.960 Um but uh oftentimes some of the best people, some of the best engineers, some of some of the best technical leaders, you're not gonna stay on one project the whole time. 00:09:16.960 --> 00:09:21.120 You stay in the project until the project doesn't need you anymore, then you go work on the next thing. 00:09:21.120 --> 00:09:28.080 It's just some people choose to move between companies, and it it can be really good for like your career growth. 00:09:28.080 --> 00:09:34.080 It can be easier to get level like up leveled and get your get a higher salary going somewhere else. 00:09:34.080 --> 00:09:35.360 So Yeah, yeah. 00:09:35.519 --> 00:09:42.080 And and that's the thing is I think a I think there's a certain number of people that do it just for pay bumps that you get, but uh I've I've never really done that. 00:09:42.080 --> 00:09:50.480 I've taken pay cuts twice moving jobs and didn't care about it as long as I mean usually I'm leaving because there's something that's making me unhappy, it's not about the pay usually, so Yeah. 00:09:50.879 --> 00:09:51.360 Yeah. 00:09:51.360 --> 00:09:53.360 How long were you at Cadence? 00:09:54.080 --> 00:09:54.559 Seven years. 00:09:54.799 --> 00:09:55.360 Yeah, that's right. 00:09:55.360 --> 00:09:55.840 Yeah, yeah, yeah. 00:09:56.000 --> 00:09:57.039 Yeah, that was that was another one. 00:09:57.039 --> 00:10:03.200 This case the manager was removed from the team because of, you know, just political whatever. 00:10:03.200 --> 00:10:10.080 And then the guy that took over the team had no idea what to do with it, didn't understand how we did things, didn't understand what we did really on the team. 00:10:10.080 --> 00:10:12.480 Like he was using the tool the team made, but he didn't really get it. 00:10:12.480 --> 00:10:14.159 So he kept telling us, like, why can't you do this thing? 00:10:14.159 --> 00:10:14.639 It's much simpler. 00:10:14.639 --> 00:10:17.360 We're like, well, let's explain the complexities that make that impossible. 00:10:17.360 --> 00:10:19.519 And he was like, No, no, just make it simpler. 00:10:19.519 --> 00:10:35.519 And the funny thing is, like, when they removed our manager from the team, I remember his manager talking to me, and everybody else on the team, he had these one-on-ones to announce this was happening, and and um he said, like, you know, you have anything you want to say or any concerns or anything? 00:10:35.519 --> 00:10:41.200 I said, Well, look, I mean, you know, because these this is the thing, is in the EDA industry, so everyone's worked at the same company their whole career. 00:10:41.200 --> 00:10:42.159 That's the way EDA works. 00:10:42.159 --> 00:10:45.759 It's a very small industry, like you either work at Cadence or Synopsis, and you kind of stick to one. 00:10:45.759 --> 00:10:48.879 Sometimes you jump ship, but usually you stick to the one. 00:10:48.879 --> 00:10:56.559 And um and there's also a lot of people that are in H1B visas, it just makes it harder to move, so they they tend to stick to the same place just because of, you know, logistics. 00:10:56.559 --> 00:11:06.080 And so I was like, look, I know you've only worked at the same company the whole time, but I've seen this happen at multiple places I've worked, and I'm telling you what's gonna happen is in 18 months, like the name of the team was the Atlas Demus. 00:11:06.080 --> 00:11:08.240 In 18 months, you're not gonna have an Atlas team anymore. 00:11:08.240 --> 00:11:11.840 Like you're getting rid of the guy that recruited every single person on the Atlas team to come work here. 00:11:11.840 --> 00:11:13.679 We all work for this team because we like him. 00:11:13.679 --> 00:11:15.039 We've all worked with him previously. 00:11:15.039 --> 00:11:16.879 This is the guy, he's the anchor of the team. 00:11:16.879 --> 00:11:19.279 And you get rid of him, the team's gonna dissipate. 00:11:19.279 --> 00:11:31.039 And then almost 18 months to the day I left the team and I was the second to last one, then the next guy left two weeks later, and then they didn't have an Atlas team, and now that manager and the manager they put in charge of us are no longer managers that got demoted. 00:11:31.039 --> 00:11:33.200 Like because they they imploded this team. 00:11:33.840 --> 00:11:36.320 Why did they uh why did they get rid of the manager? 00:11:36.320 --> 00:11:37.279 Like you made. 00:11:37.759 --> 00:11:48.399 So he was working for, you know, his skip used to be his manager, and when that manager got promoted, the guy that got promoted in place, the guy that I was having that conversation with, he couldn't get along with him. 00:11:48.399 --> 00:12:00.320 And it went on for about like 18 months or a year or something like this, a pretty long time where they were kept trying to and he was like, Look, you know, I know what the team is doing, I know what the team needs, I'm I'm you know, pretty self-sufficient. 00:12:00.320 --> 00:12:03.679 Can I just report back to the guy I was reporting to before? 00:12:03.679 --> 00:12:06.159 And they said, I'll do you one better, you're off the team. 00:12:06.159 --> 00:12:13.360 So they they they took him off the team, but they didn't fire him, so they put him on special projects kind of stuff, the stuff he was interested in working on, so that wasn't so bad. 00:12:13.360 --> 00:12:27.039

But then you know he ended up not staying for very very long, and then they laid off one other guy from the team, they moved another guy from the team, and we fired h finally hired someone to fill in a gap, they immediately like loaned him out to another team that he stayed on the entire time. 00:12:27.039 --> 00:12:30.720 He never once worked on the thing that we hired him to work on in our team, the Atlas Project. 00:12:30.720 --> 00:12:38.080 And so, you know, Atlas was this thing that was like kind of middleware that you know it's all like distributed workflows in in EDA, right? 00:12:38.080 --> 00:12:44.960 You have these really huge solutions you're trying to generate, they take days to run sometimes, and so everything's distributed. 00:12:44.960 --> 00:12:58.480 And so this was middleware that made the distribution of work really efficient, and there was one product that literally like not just like the part that they sped up, but the entire runtime of the product decreased by 50x. 00:12:58.480 --> 00:12:59.519 Okay? 00:12:59.519 --> 00:13:04.080 I think it was actually technically 45, but 50, you know, it's a round number about 50x, right? 00:13:04.080 --> 00:13:14.559 And so like if you tell a customer, like, well, through mismanagement, we just let all the guys go that maintain this thing, and so that's why recently the the product got twice as slow. 00:13:14.559 --> 00:13:19.120 We gave you this 50x speed up, but then we lowered it by 2x again because we don't know what we're doing. 00:13:19.120 --> 00:13:20.080 That's pretty hard. 00:13:20.080 --> 00:13:24.480 That's why those managers had like this come up and it's so rare that that happens. 00:13:24.480 --> 00:13:29.519 It's so satisfying to hear that those guys like actually had to pay a price for just destroying that team. 00:13:29.519 --> 00:13:30.399 It was it was really great. 00:13:30.399 --> 00:13:31.679 I just was great people to work with. 00:13:31.679 --> 00:13:33.120 I really enjoyed it too. 00:13:35.840 --> 00:13:37.120 Oh man, man. 00:13:37.120 --> 00:13:39.840 So wait, wait, when did when did you start at Cadence? 00:13:39.840 --> 00:13:41.600 Were you at NVIDIA before Cadence? 00:13:41.840 --> 00:13:44.559 Yeah, yeah, I left NVIDIA NVIDIA and went straight to Cadence. 00:13:44.960 --> 00:13:46.159 And what year was that? 00:13:46.480 --> 00:13:48.000 And oh, what year was that? 00:13:48.000 --> 00:13:54.000 So I was in NVIDIA for like just barely under three years, and I think I started there Well check your LinkedIn. 00:13:54.000 --> 00:13:55.200 Well it's easier to work backwards. 00:13:55.200 --> 00:14:02.480 So it was nine years ago that I started at NVIDIA because I was I mean at Cadence rather, I was I was I was there for seven years and I left two years ago. 00:14:02.799 --> 00:14:09.840 So that must be you left you left NVIDIA in May of 2017, says you LinkedIn. 00:14:09.840 --> 00:14:12.960 And I joined in September of 2017. 00:14:12.960 --> 00:14:14.879 So we

just barely did an overlap. 00:14:15.360 --> 00:14:19.840 Yeah, and it's it's not it's not a coincidence, it was in May, and that's when the CNO conference is. 00:14:19.840 --> 00:14:31.600 Because the the the final straw that maybe leave NVIDIA was the the manager, you know, I asked for time off to go to CNOW, which I've been to every single one of them, and he said no, and I was like, we'll see. 00:14:31.600 --> 00:14:38.399 I quit like the week before the conference happened, and I started it at Cadence the week after the conference is over. 00:14:38.799 --> 00:14:43.759 One of my great regrets is that Zach left NVIDIA before I was able to join NVIDIA. 00:14:43.919 --> 00:14:45.279 We could have had so much fun together. 00:14:45.279 --> 00:14:46.720 Yeah, it would have been great times. 00:14:46.720 --> 00:14:49.200 Yeah, and and and in fact, like I was really surprised. 00:14:49.200 --> 00:14:57.440 Like when I was leaving, I went around and like kind of said goodbye to people on the floor that you know I worked with and like people I knew from lunch and stuff I didn't work with necessarily super directly. 00:14:57.440 --> 00:14:58.720 But they were like, oh, why didn't you tell me? 00:14:58.720 --> 00:15:02.960 I could most of them on that in that area were like managers, and they were like, Oh, what didn't you tell me? 00:15:02.960 --> 00:15:13.200 You could have moved to you know, one of the teams that I managed, and I was like, I mean, they told me I couldn't make a move because of like the I got this bad performance review from that from that manager that made it impossible for me to move. 00:15:13.200 --> 00:15:14.799 And they said, Oh no, you can't move. 00:15:14.799 --> 00:15:19.120 And so I thought I couldn't, but um, yeah, I really liked working at at NVIDIA so much. 00:15:19.120 --> 00:15:21.039 Like the people there were super sharp. 00:15:21.039 --> 00:15:25.279 There was always someone that was like an expert in something I didn't know anything about, I could learn from. 00:15:25.279 --> 00:15:26.000 It was great. 00:15:26.000 --> 00:15:27.679 It was a really great place to work. 00:15:28.080 --> 00:15:29.200 Yeah, yeah, it was. 00:15:29.200 --> 00:15:32.320 It was yeah, I had a I had a great time there, yeah. 00:15:32.320 --> 00:15:35.120 It's uh a lot of people are kind of surprised that I was leaving. 00:15:35.120 --> 00:15:36.240 And I I was very happy. 00:15:36.240 --> 00:15:38.000 I just kind of had a unique opportunity come up. 00:15:38.000 --> 00:15:43.919 So I'm going to a I'm going to a startup called Modal, which is an AI infrastructure company. 00:15:43.919 --> 00:15:50.960 So they provide like a serverless platform for all of the like building blocks you need for AI applications. 00:15:50.960 --> 00:15:54.720 Like in particular, they're like most core primitive is being able to spin up a sandbox. 00:15:54.720 --> 00:16:02.559 So either a CPU sandbox or a GPU sandbox, and they can spin them up in like very, very fast start times. 00:16:02.559 --> 00:16:10.000 Like if you want to like run one function, one function on a GPU like for like 47 seconds, like they can do that. 00:16:10.000 --> 00:16:21.200 And then they can pause and snapshot the entire state of your whole sandbox, including like the CUDA driver, and like go and run somebody else's function on that GPU. 00:16:21.200 --> 00:16:31.440 And then you can come back like an hour later, and they can resume the entire state, like as if as if you'd had access to that GPU for that whole hour, and you can just run another function. 00:16:31.440 --> 00:16:35.120 And it takes like microseconds to do the resumes, maybe milliseconds, I don't know. 00:16:35.440 --> 00:16:35.840 That's wild. 00:16:36.159 --> 00:17:04.400 But uh yeah, and so they have a lot of cool, a lot of cool tech there, and then um they're now getting into the inference game, and they've got all these layers of infrastructure on top of it for like storage for these sandboxes, and all of these like building blocks that you need to build AI applications or build uh like things that are like systems that are a mix of like agentic and traditional software, and also to like train models to deploy inference, etc. 00:17:04.400 --> 00:17:06.799 And uh yeah, and it's just a cool group of people. 00:17:06.799 --> 00:17:23.119 It's part of the goal that you can like like uh basically timeshare hardware more efficiently without no nobody nobody has access to, you know, no nobody can get access to GPUs these days, so you may as well make sure that you have really efficient, really high utilization. 00:17:23.119 --> 00:17:27.440 Yeah, like I think across their fleet they get very, very high utilization. 00:17:27.599 --> 00:17:29.359 Um, that's really interesting. 00:17:29.599 --> 00:17:29.839 Yeah. 00:17:30.079 --> 00:17:32.079 Yeah, because that that's one of the kind of famous problems. 00:17:32.079 --> 00:17:44.000 In fact, when I worked on that that Atlas team, one of the things we had people asking us for was, you know, if we have part of a solution that runs on a GPU, can we share GPUs the way we share cores on a machine? 00:17:44.000 --> 00:17:48.880 And you know, usually there's some kind of layer like um oh I forget what those layers are called now, unfortunately. 00:17:48.880 --> 00:17:52.799 But but like there's there's this middleware that that people do where you have like a subscription. 00:17:52.799 --> 00:17:56.720 Uh maybe a hypervisor is in this category, but I'm I'm not sure. 00:17:56.720 --> 00:17:59.279 And another things we use were called a hypervisor, so I'm not sure. 00:17:59.279 --> 00:18:10.079 But the idea was you have some kind of like you know, user mode program that runs on the machine, and you you you you ask it as a service, like, hey, can you give me this much memory and this many cores? 00:18:10.079 --> 00:18:16.640 And if there's not other processes that are using the same subscription model that that are occupying

that space, then yes, you can run there. 00:18:16.640 --> 00:18:20.559 And we would sort of shop around for the machine that had the best fit kind of thing. 00:18:20.559 --> 00:18:22.480 And they were like, oh, can we just do that with GPUs? 00:18:22.480 --> 00:18:23.599 And we're like, ooh, not really. 00:18:26.000 --> 00:18:27.680 There's a couple reasons why it's harder. 00:18:27.680 --> 00:18:35.519 Partially Olivier said this to me early on in my career at NVIDIA, and it really stuck with me. 00:18:35.519 --> 00:18:43.440 And that what he said is our product, Nvidia's GPUs, they only run user mode software. 00:18:43.440 --> 00:18:46.400 They don't have to run system level software. 00:18:46.400 --> 00:18:53.200 And what he meant by that is you don't have like a CPU, a CPU has to be able to like boot the Linux kernel. 00:18:53.200 --> 00:19:01.599 And as a CPU, you have to expose controls for all the stuff that you need to run an operating system, to run a BIOS, to do all this low-level stuff. 00:19:01.599 --> 00:19:06.880 On the GPU, you're only ever running like user mode, you know, compute. 00:19:06.880 --> 00:19:12.720 You're not running any of the control, all the control code that you're running is only NVIDIA's control code. 00:19:12.720 --> 00:19:15.759 You're not running, you're not making it programmable to somebody else. 00:19:15.759 --> 00:19:24.559 And so that makes it a lot easier, but it makes it a lot easier for NVIDIA because it means you're just exposing less of an API service. 00:19:24.559 --> 00:19:36.880 However, it also means that GPUs were not really designed to be used for something like a timesharing system where you want to have some privileged user that's like distributing stuff. 00:19:36.880 --> 00:20:02.640 And and uh there's other aspects of GPU computing where the idea of like time sharing, the idea of like having to think about like a multi-tenant system and all the security implications is not something that we really had to think about prior to the start of the AI boom in the HPC era of you know NVIDIA, which was like maybe up until like 20 2017, 2018, 2019. 00:20:02.640 --> 00:20:07.279 In that era, it was not a big thing, you know, but people mostly got dedicated systems. 00:20:07.279 --> 00:20:12.400 You know, you you if you were using an HPC system, it's a shared system, but you'd allocate an entire node. 00:20:12.400 --> 00:20:19.279 And for the most part, the GPU was not directly exposed to untrusted input. 00:20:19.279 --> 00:20:23.359 You know, it wasn't like you were running a web server on the GPU. 00:20:23.359 --> 00:20:28.400 And just because of that, it was not a part of the the design of the stack from the start. 00:20:28.400 --> 00:20:43.599 Nowadays, there's a bunch of different there's sort of three different core technologies that you can use for this, where there's a there's a notion of like a VGPU, there's multiple different mechanisms in the CUDA stack for work sharing on GPUs. 00:20:43.599 --> 00:20:51.680 And like one is just doing it at a level of like, okay, you have a system with eight GPUs, we're gonna expose you know each one of those eight in a separate VM. 00:20:51.680 --> 00:21:09.920 There's another layer of like, okay, you want to share share this one GPU across multiple different you know users, and there's one form of that where it's like within a process, multiple submitters to the GPU, and there's another version where it's multiple processes submitting to the same GPU. 00:21:09.920 --> 00:21:25.599 But getting the security and the isolation right on all that is challenging, especially when it's a chip that's like primarily designed for performance and where people like multi-tenancy is not in a part of its DNA since background since day one, like it was for the CPU. 00:21:25.599 --> 00:21:27.119 Yeah. 00:21:27.119 --> 00:21:27.839 Yeah. 00:21:27.839 --> 00:21:54.160 So it's like, oh actually, one thing that that'll resonate with you, having been somebody who worked on the performance counters at NVIDIA on modal, since you're on these multi-tenant systems, for a long time the performance counters, the NVIDIA performance counters have not been enabled because do you know if I if I enable the performance counters for you, can you use the performance counters to find out something about something that's running on somebody else's GPU? 00:21:54.240 --> 00:21:54.480 Right. 00:21:54.640 --> 00:21:54.880 Right? 00:21:54.880 --> 00:21:59.839 And it's like, oh, it was like probably nobody thought about that when we were designing all the performance counters. 00:22:01.519 --> 00:22:01.920 Yeah. 00:22:02.559 --> 00:22:03.440 Yeah, for sure. 00:22:03.440 --> 00:22:13.680 Yeah, it's it's also interesting how like I feel like there's a bunch of people, at least in the area that of NVIDIA that I worked, like that left in the last two or three years. 00:22:13.680 --> 00:22:17.440 Almost no one that I worked with is still there. 00:22:17.440 --> 00:22:22.480 Um because they're all like multimillionaires because the NVIDIA stock price went so insane. 00:22:22.480 --> 00:22:29.279 And they, you know, they they they first started working there when the stock price was like in the single digits, and so they were like, okay, I think maybe I'll just retire. 00:22:29.279 --> 00:22:30.000 This is a lot. 00:22:30.000 --> 00:22:32.720 Because you know, it's a it's a it's a lot of hours in NVIDIA typically. 00:22:33.039 --> 00:22:37.200 A lot of them were in Texas, and a couple NVIDIA shares goes a long way. 00:22:37.200 --> 00:22:40.319 You can get a lot of land in Texas for a few NVIDIA shares. 00:22:40.559 --> 00:22:41.440 Yeah, it's definitely true. 00:22:41.440 --> 00:22:45.279 Even I mean, Austin is like the most expensive urban area in Texas. 00:22:45.279 --> 00:22:48.480 Even here, it's like nothing compared to like the coastal cities. 00:22:48.480 --> 00:22:49.359 Like it's very cheap. 00:22:49.759 --> 00:22:59.359 And

some of the the really some of the best teams at NVIDIA, like like the the um the performance, you know, uh counters team, uh I don't know what what was that team called? 00:22:59.359 --> 00:23:00.400 I don't know, I don't know what that was. 00:23:00.559 --> 00:23:01.119 Uh the DevTools. 00:23:01.119 --> 00:23:03.039 DevTools organization was the broader thing. 00:23:03.039 --> 00:23:04.319 I we're the DevTools team. 00:23:04.799 --> 00:23:13.839 So like that particular part of the DevTools team, like you look at that team and the average tenure of employees there as of like a couple years ago was like 20 years or something, right? 00:23:13.839 --> 00:23:16.079 Like these are people who just worked there for a long time. 00:23:16.079 --> 00:23:19.839 So it's not that people were like, oh, like we're unhappy, the company's changed. 00:23:19.839 --> 00:23:27.119 It's just like we've worked here for a long time, we're a little bit older, we've made a lot of money, you know, maybe we'll go retire now. 00:23:27.440 --> 00:23:27.839 Yeah. 00:23:27.839 --> 00:23:31.920 So one of the guys I worked with, guy named Jason, he's the guy that recruited me to go work there. 00:23:31.920 --> 00:23:38.400 I know you know Jason, but yeah, I remember he um he got married, uh I think it was two years ago, three years ago, something like that. 00:23:38.400 --> 00:23:43.599 And and I said, So, you know, what what are you guys doing for the the honeymoon? 00:23:43.599 --> 00:23:44.240 Are you going anywhere? 00:23:44.240 --> 00:23:46.559 And he was like, I mean, it's really hard to get time off of work. 00:23:46.559 --> 00:23:50.640 So uh in three months I'm just gonna quit my job and we're gonna go to Europe for a year. 00:23:50.640 --> 00:23:59.519 And I mean that that seems to be the kind of the the calculus for a lot of those folks is like they're like, okay, well it's really hard to get time off. 00:23:59.519 --> 00:24:05.200 And so even though there's like the all-you-can-eat buffet of of time off, like you just get your manager's approval, like no one takes it. 00:24:05.200 --> 00:24:14.160 And so a lot of those people are like, I've been working and working and working, and I I want a break, and why should wait, why should I take a break when I could just leave, right? 00:24:14.160 --> 00:24:14.960 And just retire. 00:24:14.960 --> 00:24:16.720 And so I think a lot of them are choosing that. 00:24:17.039 --> 00:24:17.519 Yeah. 00:24:17.519 --> 00:24:19.519 Yeah, well, and it's it was funny. 00:24:19.519 --> 00:24:27.599 I we did um so my last two weeks at NVIDIA, I went to visit Connor in Toronto and meet with a bunch of students. 00:24:27.599 --> 00:24:39.200 Then I went to Taiwan to give a keynote at PyCon Taipei, and then I went to NVIDIA HQ for this big face-to-face summit we were doing with some of the teams working on various Python DSLs. 00:24:39.200 --> 00:24:48.720 And then like we did a farewell reception, and uh the farewell reception, they give you like a bar passcode for like people in your group to order at the bar. 00:24:48.720 --> 00:24:53.440 And my bar passcode, I didn't pick it, but they were like, Bryce retirement is your bar passcode. 00:24:53.440 --> 00:25:01.839 And I'm like, these people, these people are all gonna think that I've got like tens of millions of dollars because they're gonna think this is my retirement party. 00:25:01.839 --> 00:25:05.279 It's like I haven't I've been there nine years, folks, not 20 years. 00:25:05.279 --> 00:25:06.880 I'm uh and I live in Manhattan. 00:25:06.880 --> 00:25:09.279 I'm not I'm not retiring anytime soon. 00:25:09.519 --> 00:25:11.200 Yeah, it doesn't go quite as far either. 00:25:11.200 --> 00:25:18.720 Yeah, the the folks that I worked there with, almost everyone on the team that I was there with, they had moved from a place here in Austin, what's it called? 00:25:18.720 --> 00:25:21.279 Um oh, I'm gonna forget the name of it. 00:25:21.279 --> 00:25:22.720 But anyway, they all worked the same place. 00:25:22.720 --> 00:25:25.359 It was like a place that did like kind of lower level stuff. 00:25:25.359 --> 00:25:29.279 It was like mostly circuit design kind of stuff, if I remember correctly. 00:25:29.279 --> 00:25:36.559 Anyway, so they all left there like as a big block because it was some kind of big shakeup and there was the you know, people getting laid off, there's all kinds of bad stuff going on there. 00:25:36.559 --> 00:25:42.160 So they all moved over, one guy moved over to NVIDIA and was like, this is great, you gotta all come over, and all of them followed him over. 00:25:42.160 --> 00:25:43.519 So almost everyone is there with. 00:25:43.519 --> 00:25:52.000 When I started there in like 2014, I think it was, they were all like had been working there for seven years already. 00:25:52.000 --> 00:25:56.720 So yeah, I think when you say 20 years, I think it's that that team in particular, I think it's almost exactly 20 years now. 00:25:57.039 --> 00:25:57.200 Yeah. 00:25:57.200 --> 00:25:57.599 Yeah. 00:25:57.599 --> 00:26:04.079 And I mean, and honestly, like, you know, like people think like, oh, it's like everybody at NVIDIA like made a ton of money and got super. 00:26:04.079 --> 00:26:09.759 A lot of those folks were, you know, 50s or so, you know. 00:26:09.759 --> 00:26:14.319 It wasn't like it was an unreasonable, unreasonably young retirements, right? 00:26:14.319 --> 00:26:18.640 You know, these are people who they really put in their time and and put in a lot of work. 00:26:19.039 --> 00:26:24.640 I mean, I think Jason is like early to mid-40s, so that's pretty young to retire. 00:26:25.440 --> 00:26:27.759 In the back of my head, I've been trying to name this episode. 00:26:27.759 --> 00:26:31.519 At one point I was thinking, ah, we can call it like career changes with Zach Lane. 00:26:31.519 --> 00:26:38.960 And then like you mentioned you left NVIDIA, and I was like, well, if I wanted to be clickbaity, I could just say leaving NVIDIA with Zach Lane, even though you left years ago. 00:26:38.960 --> 00:26:43.680 But now we've been talking about it for so long, like it's I gotta choose something clickbaity at this point. 00:26:43.839 --> 00:26:48.559 Uh call it Jason selling your RSUs at the wrong time, because I definitely did. 00:26:49.440 --> 00:26:53.200 I definitely, I definitely did a bit of selling my RSUs at the wrong time too. 00:26:53.200 --> 00:26:55.359 Do you still see have you seen Jason recently? 00:26:55.440 --> 00:27:08.160 I know you still uh I I usually like um a couple of times a couple three times a year, me and him and another guy that's actually a Python programmer that all three of us used to be regulars at the C meetup before it dissolved during COVID. 00:27:08.160 --> 00:27:11.359 We'll we'll get together sometimes and have dinner and just catch up and stuff. 00:27:11.359 --> 00:27:13.759 But I haven't seen him in maybe like six to nine months. 00:27:18.319 --> 00:27:19.119 Next time you see him. 00:27:19.119 --> 00:27:23.839 Speaking of C, so uh Oh yeah, this is a great transition. 00:27:24.079 --> 00:27:25.279 This is a great transition. 00:27:25.279 --> 00:27:26.240 The drama. 00:27:26.240 --> 00:27:27.519 The drama. 00:27:27.839 --> 00:27:37.599 Be sure to check these show notes either in your podcast app or at adspthepodcast.com for links to anything we mentioned in today's episode, as well as a link to a GitHub discussion where you can leave thoughts, comments, and questions. 00:27:37.599 --> 00:27:38.400 Thanks for listening. 00:27:38.400 --> 00:27:40.319 We hope you enjoyed and have a great day. 00:27:40.960 --> 00:27:43.680 Low quality, high quality. 00:27:43.680 --> 00:27:46.160 That is the tagline of our podcast. 00:27:46.480 --> 00:27:47.359 That's not the tagline. 00:27:47.359 --> 00:27:50.400 Our tagline is chaos with sprinkles of information.