TRANSKRIPTIO 🔗
WEBVTT
00:00:00.320 --> 00:00:07.679
And this is like a whole new field of agentic engineering, and and you know, people have been doing programming languages for a long time.
00:00:07.759 --> 00:00:20.960
It's a lot more matured, and I feel like there's a lot more to learn and discover here in the world of agentic engineering, and like how agentic engineering impacts our programming languages, our compilers, our tooling.
00:00:21.120 --> 00:00:23.440
Like that, that I'm very interested in.
00:00:23.600 --> 00:00:30.320
You know, I don't have answers yet for how agentic engineering impacts programming languages.
00:00:30.480 --> 00:00:37.520
I think I'm starting to see how it how it will impact, you know, tools, compilers, etc.
00:00:37.759 --> 00:00:41.039
But I don't yet know how it impacts programming languages.
00:00:57.119 --> 00:01:06.000
My name is Connor, and today with my co-host Bryce, we chat about GPU mode, are programming languages still important, post-AGI, wingspan, lawn care, and more.
00:01:13.439 --> 00:01:25.200
Well, so so so I think the the the biggest uh a colleague said to me the other day regarding out our research that like 90% of your effort should go into preventing cheating and reward hacking.
00:01:25.439 --> 00:01:27.519
And I I think that's more or less right.
00:01:27.760 --> 00:01:32.159
You know, uh back in the day there was this tool called C-Reduce before AI.
00:01:32.239 --> 00:01:40.560
And C-Reduce is this thing where you'd give it like C source and then you'd give it a test, and you'd use it for like reducing a compiler error.
00:01:40.719 --> 00:01:46.719
Like let's say that you had like an internal compiler error that you hit with some you know, code base with GCC.
00:01:47.040 --> 00:01:54.959
And so you would you would like create a pre-processed header of that like or you'd create a pre-processed source file of from the code base.
00:01:55.040 --> 00:02:16.080
It'd be like 500,000 lines of code, and you'd you'd give it to CReduce, and then what CReduce would do is it would go in and it would apply a bunch of these code transformations, like you know, like remove all the comments and like you know, try try removing this function and then this function and try renaming variables, etc., trying to minimize the reproducer of your error message.
00:02:16.560 --> 00:02:22.319
And the biggest challenge with writing good C reducers is the test script.
00:02:22.560 --> 00:02:36.080
So if you gave it a crappy test script, like you know, you give it a test script, you're like, okay, just like you know, the way that you test whether it still reproduces is just run GCC on this file and then like grep for the error message.
00:02:36.639 --> 00:02:41.919
Okay, well if you do that, then uh it's gonna give you back something that'll reproduce your error message.
00:02:42.159 --> 00:02:45.360
But the but there will be a dozen other error messages too.
00:02:45.599 --> 00:02:48.479
And like the rest of the code will be like ill-formed.
00:02:48.719 --> 00:03:04.560
And so it's like, okay, so what we really want is that the this is a valid reduction, a valid minimization if the code continues to produce the error message with GCC, but it also compiles fine with Clang.
00:03:04.879 --> 00:03:14.000
And then like, okay, then now it will only give you a minimizer where the code is valid C, but it happens to break with GCC.
00:03:14.240 --> 00:03:22.159
And then there were like other things where like if you were, if you did something subtly the wrong way and like a reducer, like it would just reduce your your source file down to like being like an empty file.
00:03:22.240 --> 00:03:24.639
And it's like, hey, this this passes your test.
00:03:24.879 --> 00:03:26.159
It passes your test.
00:03:26.400 --> 00:03:29.120
So, you know, this is the valid a valid reduction.
00:03:29.439 --> 00:03:39.520
And so like whenever I would run into a compiler error, I would go and like cr carefully craft this test script to test whether the bug still reproduced.
00:03:39.680 --> 00:03:42.479
And then I would just like put it into C reduce and wait a few hours.
00:03:42.719 --> 00:03:44.319
And it's the same thing with AI.
00:03:44.479 --> 00:03:53.280
It's the same exact process with AI that the uh in particular with these auto-research agentic, you know, uh loop sort of workloads.
00:03:53.520 --> 00:04:13.680
Ours is more what you and I are doing is truly more of what we I would call auto-research, which is something where it is iteratively following some some form of a scientific process where it's trying something, then testing whether that's whether it works, then evaluating a metric, and then deciding whether to keep it or or revert it.
00:04:13.919 --> 00:04:21.600
But anyways, the the the biggest and most important thing is the quality of your tests and your validation.
00:04:21.759 --> 00:04:26.800
Because if there are exploitable holes, it will it will go after those.
00:04:26.959 --> 00:04:45.439
And in particular, if you're doing something like you know, like like optimization or you know, even if it's not optimization, even if it's not performance, if it's just like correctness, like if you're testing for like three cases and there's like some other cases that your tests don't cover, like yeah, it's gonna write code for those three cases.
00:04:45.680 --> 00:04:50.079
It's not gonna care about the other cases, it's gonna break stuff for the other cases.
00:04:50.480 --> 00:04:55.360
And yeah, it it it's it's a challenging art and science.
00:04:55.519 --> 00:05:01.040
I I wish I could say more, but uh I don't think I can talk too much more about the techniques.
00:05:01.920 --> 00:05:02.959
Only matter of time.
00:05:03.120 --> 00:05:04.240
Only matter of time.
00:05:04.560 --> 00:05:08.720
Should we try and talk about some non-AI stuff for 30 minutes?
00:05:08.879 --> 00:05:09.360
What do you think?
00:05:09.439 --> 00:05:14.639
What is there stuff that we uh how's
Load full transcript
how's the how's the the kid?
00:05:14.800 --> 00:05:16.480
How's the kid coming along?
00:05:16.800 --> 00:05:17.680
The kid's good.
00:05:17.839 --> 00:05:19.680
Do we know of do we know a boy or girl?
00:05:19.920 --> 00:05:20.800
Yeah, yeah, it's a boy.
00:05:21.519 --> 00:05:22.639
Do we have names yet?
00:05:22.879 --> 00:05:26.160
Uh we have a couple contenders.
00:05:26.319 --> 00:05:27.759
You know, I have a top name.
00:05:28.000 --> 00:05:29.360
Shima has a top name.
00:05:29.600 --> 00:05:36.000
I think both are I think you should convince Shima to just let the the internet uh decide from the set of top names.
00:05:39.279 --> 00:05:42.399
And then yeah, baby baby's good.
00:05:42.560 --> 00:05:48.480
I'm in the middle of reading, I'm on my third parenting book, and see how many I get through in the next two and a half months.
00:05:48.639 --> 00:05:52.560
I I'd say probably I'll stop at like somewhere between 10 and 15.
00:05:52.800 --> 00:05:58.000
Not that that's necessary, but you know, it doesn't take me too long to to crush audiobooks.
00:05:58.240 --> 00:05:59.680
And I'm learning a lot, folks.
00:05:59.759 --> 00:06:06.160
There's a lot of a lot of academic literature out there about how to raise a baby.
00:06:06.560 --> 00:06:21.600
And um Olivier, uh, who we both know and who was at NVIDIA for a long time, when our when our other buddy JF was about to have his first kid, this was I was at the C committee meeting in Ulu, which was 2016.
00:06:21.759 --> 00:06:25.439
That was my first international trip, my second committee meeting.
00:06:25.600 --> 00:06:30.160
So I'd met Olivier and JF at the previous meeting, and we had quickly become quickest thieves.
00:06:30.319 --> 00:06:35.680
And so we're at we're in Ulu in the middle of uh, you know, the sort of middle part of Finland.
00:06:35.839 --> 00:06:51.120
And um JF's talking about, you know, like becoming a father soon, and Olivier tells him, like, listen, we can talk about parenting philosophy now, but as soon as you have a kid, it will be become impossible for us to talk about it.
00:06:51.279 --> 00:06:58.720
Because once you have a kid, you you develop your own, like, very deeply held beliefs about what's the correct way to parent.
00:06:58.959 --> 00:07:06.240
And there's like no point in us talking about it after that, because you'll have your worldview, I'll have my worldview, and they may not align.
00:07:06.639 --> 00:07:11.519
Alright, well, stay tuned to whether or not that uh that turns out to be true for me.
00:07:11.759 --> 00:07:15.040
I mean, I don't think that's really gonna be the case.
00:07:15.199 --> 00:07:24.639
I I always think that like if you're the kind of person that's worried, if you're well, it probably doesn't apply all the time, but if you're worried, like, oh am I being a good parent, blah blah blah.
00:07:24.800 --> 00:07:29.759
That art hey, we just took what it just uh it just won sort.
00:07:32.000 --> 00:07:34.480
Wow, that's actually it
won sort by a good bit.
00:07:34.639 --> 00:07:35.040
Nice.
00:07:35.199 --> 00:07:35.519
Okay.
00:07:35.759 --> 00:07:51.040
Alright, so so out of GPU mod leaderboard, I've now I'm now the the number one for sort, vector sum, prefix sum, histogram, grayscale, number two for convolution, and I haven't tried mapmaller vector ad yet.
00:07:51.680 --> 00:07:53.360
I'm sorry you were saying things.
00:07:53.519 --> 00:07:53.839
Yeah, yeah.
00:07:54.079 --> 00:07:57.439
On the B200, there's multiple GPUs, so no, no, no.
00:07:57.519 --> 00:07:59.920
I've uh look at the look at the H100.
00:08:00.000 --> 00:08:03.839
I think I've already taken more slots on H100 and on L4.
00:08:04.160 --> 00:08:06.079
And interestingly, sort you had don't have it.
00:08:06.319 --> 00:08:07.120
On H100.
00:08:07.360 --> 00:08:09.120
I don't think I've done sort on H100 yet.
00:08:09.279 --> 00:08:13.839
Like histogram, I'm pretty sure I now have all of them on Histogram, uh, which is the one I started with.
00:08:14.000 --> 00:08:15.519
Yeah, I have all of them on Histogram.
00:08:15.839 --> 00:08:23.040
It's a shame that you didn't choose a shorter username like B Lelback, because on mobile it cuts off the gold emoji.
00:08:23.360 --> 00:08:24.560
Uh, yeah.
00:08:24.879 --> 00:08:31.439
So so the interesting thing uh with GPU mode, there's different you know, leaderboards for each GPU type.
00:08:31.680 --> 00:08:39.039
And initially I did not instruct the model to only submit to the GPU type that it had locally.
00:08:39.120 --> 00:08:45.039
So all of my agents run on GPU nodes so that they can do you know local testing and evaluation.
00:08:45.279 --> 00:08:52.000
And what happened is you know, the models love to quit, you know, when things when the going gets tough, they love to give up.
00:08:52.080 --> 00:08:54.399
But they've got this directive to never quit.
00:08:54.559 --> 00:09:08.159
And so like once it feels like, well, I can't really make any more progress on an L4 GPU, it would start taking its its uh kernel for L4 and just submitting it to the uh to the other GPU types and uh would like get wins there.
00:09:08.320 --> 00:09:09.600
So I thought that was pretty funny.
00:09:09.759 --> 00:09:11.360
I have not run on A100 yet.
00:09:11.440 --> 00:09:23.440
I'm saving the A100 slots for a particular setup that uh will use Nemotron, and I want to be able to evaluate how well it does before I have all of the uh all of the slots.
00:09:23.759 --> 00:09:35.120
I I have the only reason I haven't gone after Mapmull and uh VectorAd is I don't think that they're particularly interesting problems because like the speed of light answer is gonna be pretty clear for both of them.
00:09:35.360 --> 00:09:37.360
But I don't know, maybe I'll be wrong on that.
00:09:37.519 --> 00:09:40.159
Oh man, I'm very happy with that sort result.
00:09:40.480 --> 00:09:49.279
Because that one, that sort one, was running with Deep Seek V4, and so that's the first time that I've gotten really good results out of Deep Seek V4 Pro.
00:09:49.360 --> 00:09:50.559
I'm very happy with that.
00:09:50.639 --> 00:09:56.399
That's really uh very encouraging for uh my future inference bills being substantially lower.
00:09:56.720 --> 00:09:57.600
Six out of eight.
00:09:57.679 --> 00:09:58.799
You only got two to go.
00:09:59.039 --> 00:09:59.440
Yeah.
00:09:59.679 --> 00:10:00.799
So we couldn't do it, folks.
00:10:00.879 --> 00:10:01.519
We couldn't do it.
00:10:01.759 --> 00:10:05.919
We tried to talk about not AI, but uh we ended up talking about it.
00:10:06.240 --> 00:10:06.639
Yeah.
00:10:06.960 --> 00:10:08.559
I don't what was I even saying?
00:10:10.159 --> 00:10:10.480
I don't know.
00:10:10.639 --> 00:10:11.519
Something about parenting.
00:10:11.759 --> 00:10:12.639
I don't know what to do.
00:10:12.879 --> 00:10:19.440
When I'm editing this before, I'll realize that I was mid-thought, and unfortunately the listener will never get to know the end of it.
00:10:19.679 --> 00:10:20.720
I mean, what else has happened?
00:10:20.799 --> 00:10:24.799
That uh you said you were at some conference, uh the Singularity Conference.
00:10:25.200 --> 00:10:26.960
Yeah, I was at that, I was at GoSim.
00:10:27.120 --> 00:10:28.799
Um in Paris, right?
00:10:29.039 --> 00:10:30.080
Yeah, yeah.
00:10:30.320 --> 00:10:32.159
Do you have other conferences coming up?
00:10:32.399 --> 00:10:35.120
Do we have programming language stuff to talk about?
00:10:36.320 --> 00:10:38.080
Programming languages.
00:10:38.799 --> 00:10:41.759
I don't know that I care about programming languages anymore, Connor.
00:10:41.919 --> 00:10:42.559
Honestly.
00:10:42.720 --> 00:10:44.960
I I don't know that I care about programming languages.
00:10:45.279 --> 00:10:49.200
Alright, maybe that's maybe that's what the title of this episode will be.
00:10:49.679 --> 00:10:51.840
Programming languages, do they matter?
00:10:52.159 --> 00:10:54.320
Or like, do programming languages matter anymore?
00:10:54.480 --> 00:10:55.200
Question mark?
00:10:55.360 --> 00:10:56.080
Go, Bryce.
00:10:56.320 --> 00:10:59.200
Talk for 20 minutes and uh round this uh episode out.
00:10:59.919 --> 00:11:05.679
I mean, uh yeah, I'm I'm sure they're still mat I I do think they still matter and are still important.
00:11:06.320 --> 00:11:12.320
But like agentic engineering just feels like a higher priority thing for me right now.
00:11:12.559 --> 00:11:21.120
Because like again, my my my mandate and mission is how do people program parallel platforms, specifically NVIDIA's parallel platform.
00:11:21.200 --> 00:11:22.960
But sort of my whole career has been around that.
00:11:23.120 --> 00:11:27.840
And like in the that that comes down to like how do people get performance out of a platform?
00:11:28.159 --> 00:11:35.519
And this is like a whole new field of agentic engineering, and and you know, people have been doing programming languages for a long time.
00:11:35.600 --> 00:11:48.720
It's a
lot more matured, and I feel like there's a lot more to learn and discover here in the world of agentic engineering, and like how agentic engineering impacts our programming languages, our compilers, our tooling.
00:11:48.960 --> 00:11:51.200
Like that, that I'm very interested in.
00:11:51.360 --> 00:11:58.080
You know, I don't have answers yet for how agentic engineering impacts programming languages.
00:11:58.320 --> 00:12:05.279
I think I'm starting to to see how it how it will impact, you know, tools, compilers, etc.
00:12:05.600 --> 00:12:08.559
But I don't yet know how it impacts programming languages.
00:12:08.879 --> 00:12:09.519
Yeah.
00:12:09.919 --> 00:12:32.159
I mean, I think I the the one the one thing I will pr I will posit here, i and we you and I talked about this in the past, is I I think there is this question of whether higher level abstractions are more token to f token efficient to reach good performance than lower level abstractions.
00:12:32.399 --> 00:12:46.720
If you're using something like Parrot or or Kutile or Triton, can you write good performant code at a significantly lower token cost?
00:12:47.200 --> 00:12:52.559
And then can you port that code to a new architecture at a significantly lower token cost?
00:12:52.799 --> 00:12:57.279
And I think I think there's a a lot of people believe that to be the case.
00:12:57.519 --> 00:13:03.200
I have not seen a comprehensive study published evaluating this.
00:13:03.360 --> 00:13:05.600
And I think it'd be great work for someone to do.
00:13:05.840 --> 00:13:14.799
And you could look at it, you could look at things like all these DSLs that popped up, but you could even just evaluate it solely on programming languages.
00:13:15.039 --> 00:13:27.600
Like I I think it might be sufficient to look at like, you know, trying to write like a some some code with like C ⁇ , C, Python, and Fortran.
00:13:28.480 --> 00:13:49.039
And uh, you know, my my guess is that like a language like Fortran, which has uh slightly like higher level abstractions, and you don't have to for suffer stuff like numerical code, you don't have to do as much low-level tweaking, but it still gets very good performance, like it might show very favorably there.
00:13:49.200 --> 00:13:54.799
So it'd be nice if somebody had like a uh an e-le suite or a benchmark to do these sorts of evaluations.
00:13:54.879 --> 00:13:57.360
There probably is something out there like this, I just don't know about it.
00:13:57.519 --> 00:14:16.159
But those are the sorts of questions that I think are are interesting is like how how token efficient is our existing languages, because I don't think we have enough information yet to know how we should evolve programming languages to be more token efficient and agent-friendly.
00:14:16.399 --> 00:14:18.080
And I do think the two are linked together.
00:14:18.240 --> 00:14:27.360
I think that like if you if you optimize your thing for token efficiency, you're implicitly making it more agent-friendly.
00:14:27.600 --> 00:14:30.080
Because what's the most token inefficient thing?
00:14:30.320 --> 00:14:43.200
Well, if the if the agent uses your thing and gets a compilation error or gets a tool failure because it it wasn't it was unclear how to use your thing or there were ambiguities or your thing was hard to use, well, that's gonna waste a bunch of tokens.
00:14:44.159 --> 00:14:49.840
So if you just minimize token usage, if that's just your objective function, then you'll make your thing more agent-friendly.
00:14:50.399 --> 00:14:52.639
Well, we should just put a loop around that.
00:14:53.200 --> 00:14:54.720
Put a loop around that?
00:14:55.039 --> 00:14:55.360
Yeah.
00:14:55.600 --> 00:15:01.600
Just like, okay, that that's that's gonna be my new answer for anybody who tells me how do I make my thing more agent-friendly.
00:15:01.759 --> 00:15:15.360
I'm just gonna say make some eval that uses your thing, and then just put an agentic loop around it that's minimizing that that that subjective function is minimize the number of you know tokens to uh to achieve this task.
00:15:18.000 --> 00:15:19.679
Yeah, I guess.
00:15:19.919 --> 00:15:34.960
You gotta isolate though to make sure that like each time you're launching your like prompts, like you're gonna run into like caching and you know context bleeding that could like affect things, right?
00:15:35.120 --> 00:15:38.320
Uh I don't think across session you have to worry about caching.
00:15:38.639 --> 00:15:39.279
Yeah, okay.
00:15:39.440 --> 00:15:40.879
Maybe if you don't have to worry about it.
00:15:40.960 --> 00:15:42.799
I'm pretty sure sessions will be isolated, yeah.
00:15:42.960 --> 00:15:48.240
I'm pretty sure I'm pretty sure uh I mean I could be wrong about that, but I'm pretty sure the sessions are isolated.
00:15:48.320 --> 00:15:50.320
If not, I'm sure there's a way that you can isolate.
00:15:50.480 --> 00:15:51.600
We didn't even talk about Rust.
00:15:51.679 --> 00:16:03.440
So so Rust would be the other language to to throw in the gauntlet there because Rust has a lot of guardrails on it, so it catches a lot of like like errors uh you know at compile time because it has this current record by construction model.
00:16:03.600 --> 00:16:19.840
So like the interesting eval would be like, okay, if you give Rust if you give an agent the same task and ask it to do it in Rust versus C, does it complete it in fewer tokens because in in Rust it's just harder to write bad code?
00:16:20.080 --> 00:16:21.039
The answer is probably.
00:16:21.200 --> 00:16:27.039
Again, I just haven't like I haven't seen the the the study in front of me proving that.
00:16:27.360 --> 00:16:28.000
Yeah.
00:16:28.320 --> 00:16:29.840
You work in research, you should go do that.
00:16:30.080 --> 00:16:32.639
There's already enough people working on Rust, and to be honest.
00:16:33.279 --> 00:16:34.799
No, no, no, but but it's not about Rust.
00:16:34.879 --> 00:16:41.679
It's it's
what I'm saying is somebody should evaluate the agent readiness of programming languages.
00:16:42.159 --> 00:16:46.000
Uh I mean that's adjacent to kind of some of the stuff that I'm working on.
00:16:46.159 --> 00:16:52.720
I'm working on more of like the abstraction layer, like level that you're targeting.
00:16:52.799 --> 00:16:58.240
Like not dissimilar to when you said Fortran versus C versus Python.
00:16:58.399 --> 00:17:02.240
But my more thing is like you could even do the same thing within a single language, right?
00:17:02.399 --> 00:17:06.640
Like, you know, some high-level thing like Parrot, then Thrust, then Cub, then just Raw to C.
00:17:07.119 --> 00:17:07.440
Yeah, yeah.
00:17:07.680 --> 00:17:19.279
Or just in general, like yeah, like if if something And I I think that's a that's a really interesting point that maybe I'm conflating abstraction level with programming language here.
00:17:19.599 --> 00:17:24.799
I do think that the abstraction level part is the more important question than the specifics of a programming language.
00:17:24.960 --> 00:17:28.880
I mean maybe they are to some degree orthogonal, but what I was really asking about was abstraction level.
00:17:29.200 --> 00:17:42.079
And the thing is, is in any language, like I'm I'm not too familiar with Fortran, but like even Python, you can like, you know, open up the hood of the car and and start doing some crazy stuff with basically like C APIs within Python.
00:17:42.319 --> 00:17:53.200
And my main thing is like which one of these layers, if you're targeting, is gonna result in like the best outcomes.
00:17:53.519 --> 00:17:58.640
And I have my own personal theories, but that's yeah, that's the direction.
00:17:58.799 --> 00:18:02.319
It just what you were saying though makes me just think, yeah, like, I don't know.
00:18:02.559 --> 00:18:07.440
Where where do you where do you think, like five years, ten years from now, where does this all end?
00:18:07.759 --> 00:18:16.559
Like I I listened to an interview with Boris Cherney, and it was he got asked, you know, what happens when we're post-AGE?
00:18:17.039 --> 00:18:22.640
And I won't I won't say his answer, but there's always there's there's always gonna be AI doctors.
00:18:22.880 --> 00:18:24.160
We're always gonna need AI doctors.
00:18:24.400 --> 00:18:25.440
That's that's what you think it is?
00:18:25.599 --> 00:18:26.799
Is Yeah.
00:18:27.279 --> 00:18:38.000
Yeah, no, I mean I think if we think about the state of models today, there's a couple different paradigm shifts that I can imagine happening.
00:18:38.319 --> 00:18:44.799
One is moving more of thinking and reasoning to happen in the latent space within the model itself.
00:18:45.039 --> 00:18:56.400
My understanding is that today most of the reasoning works by model generates an output and then takes that back in as an input instead of this all happening within the latent space.
00:18:56.559 --> 00:19:03.839
So you hear you hear a lot of people about like uh the idea of looped transformers and people claiming that maybe mythos is that.
00:19:04.000 --> 00:19:11.039
But like I think I think that is a thing that is you know likely to be a big paradigm shift that'll have big impacts.
00:19:11.200 --> 00:19:18.240
It it's not as not something that I would expect to impact the agentic engineering part too much.
00:19:18.400 --> 00:19:24.720
Although if the reasoning happens within the latent space, then it's going to be less visible to us than it is today.
00:19:24.880 --> 00:19:26.000
I mean, it's already probable.
00:19:26.319 --> 00:19:43.119
The thing that you see that's that are called like thinking traces or reasoning traces are already like kind of a lie because it's like a it's not really the reasoning trace, or it's not really like the internal process, it's just sort of like the model like generating for you its idea of what the internal process would be.
00:19:43.279 --> 00:19:45.519
Anyways, the other thing is around like memory.
00:19:45.680 --> 00:19:53.279
So there's all these various systems for memory today, a lot of which kind of boil down to like we have some series of markdown files somewhere.
00:19:53.519 --> 00:20:04.240
And I think that there's like a lot of room for more first-class memory systems for memory, longer-term memory of being more of a first-class part of the LLM.
00:20:04.480 --> 00:20:18.640
But there's a there's a there's an engineering problem here too, which is w how do you like manage and build reliable systems that have this like self-evolving memory?
00:20:18.799 --> 00:20:26.400
So almost all of the agenic workflows that I'm doing are running on ephemeral VMs.
00:20:26.640 --> 00:20:34.000
So the agent might run for two or three or four days or even a week, but then like I'm gonna shut down that VM.
00:20:34.160 --> 00:20:43.680
So any like information or learning or anything that I take out of that, I'm gonna put into skills, I'm gonna, I'm gonna put into my harness, my tools, etc.
00:20:44.000 --> 00:20:47.920
I'm not running like, you know, I come from the HPC world.
00:20:48.079 --> 00:20:49.519
I never run stuff on my laptop.
00:20:49.680 --> 00:20:52.640
I have like a 10-year-old laptop because my laptop's just a thin client.
00:20:52.720 --> 00:20:56.480
Because start of my career, I never ran anything on my laptop, anyways.
00:20:56.720 --> 00:20:59.599
My laptop is just an SSH terminal to some node somewhere.
00:20:59.759 --> 00:21:03.519
And so I've always sort of had a very different mindset than other developers.
00:21:03.599 --> 00:21:06.240
I've never been somebody who's built stuff locally.
00:21:06.319 --> 00:21:08.079
I'm always doing it somewhere in the cloud.
00:21:08.240 --> 00:21:12.559
And so I'm I'm never running an agent locally on my laptop.
00:21:12.799 --> 00:21:21.599
And and that means that I've always had to think about, you know, what data needs to be persisted.
00:21:21.759 --> 00:21:36.319
I think a lot of people
end up, you know, sort of building like an agent that has a personality over time, that has memories, longer-term memories, etc., that has, you know, all of these things that sort of allow them to customize it.
00:21:36.480 --> 00:21:42.559
For me, I have a lot less of that because anything I do like that, I need to like embed into my skills or my config.
00:21:42.799 --> 00:21:51.440
But like in the future, obviously, you know, agents that know that know and learn about you over time are going to be the most useful thing.
00:21:51.519 --> 00:21:52.400
Like I recognize that.
00:21:52.480 --> 00:21:58.720
I'm not you I'm not necessarily using that today, but like things that that have their own like memory system, those are gonna be the most useful things.
00:21:58.960 --> 00:22:12.240
But how How are we gonna like ensure that an agent that's been like running for six months or a year and has this like a complex memory system that it's built up, like how are we gonna ensure that that's like reliable and reproducible?
00:22:12.400 --> 00:22:14.640
And like how are you gonna debug that when something goes wrong?
00:22:15.039 --> 00:22:17.759
That seems like that seems like a real challenge in the future.
00:22:18.079 --> 00:22:22.960
I think we're moving more and more towards a world of like these like always-on, always running agents.
00:22:23.200 --> 00:22:29.039
Maybe in the future we'll even have agents that are basically like essentially always like reasoning or or inferring.
00:22:29.359 --> 00:22:31.759
You know, I mean, I don't know, maybe we're there already today.
00:22:31.920 --> 00:22:40.640
A lot of these loops tend to tend to be moving from like, you know, inference to sort of like waiting to like doing tool calls.
00:22:40.880 --> 00:22:45.440
But but maybe like AGI means like a system where it's basically like always thinking.
00:22:45.759 --> 00:22:50.240
And uh I just I think it's gonna be very interesting to make those things coherent.
00:22:50.319 --> 00:23:00.720
And I think I think going from tasks that run for days to weeks to tasks or agents that like run for like months to years is gonna be a very hard problem.
00:23:00.880 --> 00:23:07.200
In part because like how do we simulate and test an agent that runs for like months to years?
00:23:07.279 --> 00:23:07.759
I don't know.
00:23:08.160 --> 00:23:09.359
That seems challenging.
00:23:10.079 --> 00:23:13.359
That's maybe the end of like reliability in the software space.
00:23:13.759 --> 00:23:41.440
Yeah, I uh I always think about you know the fact that if you think about 30 years ago, you know, the internet is just like beginning, and now we're we all can like carry supercomputers and blah blah blah, and there's social networks and just like in 30 years the technology landscape has changed so much that it's like impossible to be able to predict like 30 years from now.
00:23:41.680 --> 00:23:57.279
But I I do I do think about like the Iron Man scene in the first Iron Man that came out where he's like building his suit with like a holographic projection where he's like exploding out all the pieces and then grabbing a piece and twisting it and blah blah blah.
00:23:57.440 --> 00:24:30.160
And I think about that applied to like mechanistic and interpretability, where it's gonna be like visual, like like a visual version of like mechanistic interp, where you get to go into some like you know, 3D holographic, you're gonna put on some you know futuristic version of the Apple Vision Pro, and then you're gonna be able to go in and look at some version of a three-dimensional knowledge graph that some AI system has built, and like there will be a way to interpret like its knowledge store or knowledge garden or whatever you want to call it.
00:24:30.480 --> 00:24:32.799
And that is gonna be so freaking cool.
00:24:32.880 --> 00:24:36.960
And like that, I think that like that could be a form of programming in the future, right?
00:24:37.039 --> 00:24:50.640
Like if you if you've got some system that's hill climbing on a bunch of whether it's CUDA kernels or whatever your thing is, and then you can go in and come up with a visual like representation of the different improvements and ways that it found its way.
00:24:50.799 --> 00:24:58.559
Like like have have you seen the the Carpathies thing about like knowledge bases and building knowledge bases from a couple months ago?
00:24:58.799 --> 00:24:58.880
No.
00:24:59.039 --> 00:25:01.359
Is it a talk or I don't know, it may have been a tweet.
00:25:01.440 --> 00:25:04.799
I get all I get all my information from tweets on what's going on in AI.
00:25:04.960 --> 00:25:06.160
But I think it was very similar.
00:25:06.240 --> 00:25:13.440
It's this idea of like, you know, maybe the next step is like how do we how do we have agents like compile and orchestrate like knowledge?
00:25:13.599 --> 00:25:29.839
Like a lot of like creation of like today, a lot of like creation of skills is still largely like driven by like humans and like the creation and organization of skills, and we create and organize skills like of a scale that like humans can go and like look and review through.
00:25:30.079 --> 00:25:45.440
And uh maybe in the future we have these like knowledge bases that are like curated and prepared by models, maybe that are viewable for humans, but also that are you know these bases of knowledge that the models can use for their future work.
00:25:45.759 --> 00:25:46.160
Yeah.
00:25:46.400 --> 00:25:48.319
Yeah, it's just I don't know.
00:25:48.400 --> 00:25:52.400
I don't know how I understand there's concerns, etc., but I just I'm so excited.
00:25:52.480 --> 00:26:02.880
Like, I mean, there's another there's another thing that just popped into my head is I at one point when we were talking to Sean Parent about her name was I think Jessie.
00:26:03.039 --> 00:26:04.400
Let me look it up right now.
00:26:04.640 --> 00:26:05.920
Or t or is it Terry?
00:26:06.000 --> 00:26:08.400
Actually, I have this as a potential future guest.
00:26:08.480 --> 00:26:14.319
If I just go to ADSP and I scroll down, it will say Jerry.
00:26:14.480 --> 00:26:15.200
So what did I say?
00:26:15.279 --> 00:26:16.880
I said Jesse and then Terry.
00:26:16.960 --> 00:26:25.519
It is Jerry Ellsworth, and she is the individual behind a holographic-esque kind of board game tool.
00:26:25.839 --> 00:26:34.240
And I I know even as far back on some CPP cast episode, they were talking about like Dungeons and Dragons and like augmented reality version of that.
00:26:34.319 --> 00:26:39.200
But like uh a literally like a real-world example today is Shima and I.
00:26:39.279 --> 00:26:40.640
We love this game Wingspan.
00:26:40.799 --> 00:26:42.480
We're both birders, we like board games.
00:26:42.640 --> 00:26:44.319
When Wingspan is a bird.
00:26:44.640 --> 00:26:45.759
I too have played this game.
00:26:45.920 --> 00:26:47.359
Yeah, it's a bird-flavored game.
00:26:47.680 --> 00:26:49.119
With Olivier and his kids, actually.
00:26:49.519 --> 00:26:49.759
Interesting.
00:26:49.839 --> 00:26:50.559
Yeah, it's a great game.
00:26:50.640 --> 00:27:17.599
I have a bunch of the expansions, but sometimes it's it's a very uh point salady, requires like five minutes of prep to set up, and because there's so many components, cards, eggs, etc., food, it's a little bit irritating to like keep track of everything, and like guaranteed one out of every two games, like we one of us doesn't have like the number of turncubes like in the right order, and we have to figure out who who didn't put a turncube in the right place.
00:27:17.839 --> 00:27:23.359
We also on boardgamarena.com play there sometimes, but it's without the expansions.
00:27:23.519 --> 00:27:33.200
And sometimes if we just we don't have the energy to set up the the physical board game, even though we prefer doing that, we'll just go and we'll play, she'll come to my office, she'll play on her laptop, I'll play on my computer.
00:27:33.519 --> 00:27:49.839
And I would love a basically like combination of the two where you have the expansions, but then also like have the augmented reality version, or even better, like it's it's like the Star Wars, you know, monster fighting game where they always play at the table and they've got the holographic monsters.
00:27:50.000 --> 00:28:02.160
I would love to not even be able to have to put on augmented reality glasses and just have some like little you know discs that you put at the corner of your dining table or wherever you're playing, and poof, you've got the you've got the holographic version.
00:28:02.319 --> 00:28:11.039
So you're still playing in person, the best way to do board games, and but like you don't need to do all of the point tracking and the point totaling and the moving of eggs.
00:28:11.119 --> 00:28:19.039
And oh, did you actually pay uh the two eggs that you were required to play in order to play, you know, the Kingrail as your fourth bird in
your wetland?
00:28:19.200 --> 00:28:24.720
And like how that wouldn't, and I you know, some people probably are being like, oh, you're ruining, like, it's nice to have a tech.
00:28:25.039 --> 00:28:29.279
Yeah, you're you're there there's so many, there's so many people who are gonna be so upset at the words you say.
00:28:29.680 --> 00:28:31.759
I'm just like saying that that would be amazing.
00:28:31.920 --> 00:28:41.200
And and also, too, like on the digital version on board game arena, you can play with a couple different settings because there's a few birds that are known as the OP birds.
00:28:41.359 --> 00:28:44.559
OP, for those that don't know the lingo, is uh overpowered.
00:28:44.720 --> 00:28:52.640
And the two ravens, the kill deer, Franklin's Gull, and then the Wood Duck are considered the Wood Duck's kind of like a tier two OP bird.
00:28:52.799 --> 00:29:01.440
But you can, when you're playing competitive, which I never do, but you can enter tournaments and stuff, they remove those birds because they're considered such an unfair advantage when you get them.
00:29:01.680 --> 00:29:05.680
And like you can imagine, like, oh, like let's play like a different version.
00:29:05.759 --> 00:29:12.079
You can, like, you'll be able to talk to some model and say, we're actually, can we play a variant of this game?
00:29:12.160 --> 00:29:15.599
And you just explain the rule change, and then it'll incorporate it.
00:29:15.680 --> 00:29:26.559
And you can like, you're gonna come up with a different version of Scrabble and a different version of chess or checkers, or like, you know, maybe you're playing with your kid and you want to have like a you know a handicap.
00:29:26.720 --> 00:29:28.160
Was that the right am I allowed to say that?
00:29:28.319 --> 00:29:29.039
Alright, whatever.
00:29:29.119 --> 00:29:32.079
Well, I apologize if that's the wrong term to say in 2026.
00:29:32.559 --> 00:29:35.359
I don't think that that's the right term.
00:29:35.519 --> 00:29:50.799
But whatever the right term is, I don't mean to be offensive, but like if you want to make it so that it's more competitive when you're playing with uh your kid in the future, and so you put some, you know, head start, you give your kid a head start, or you you double the points, you know, that your your kid is getting.
00:29:50.960 --> 00:29:59.920
And and then, you know, you can even you could even I was just thinking, well, isn't it valuable for your kid to be adding up the scores and doing the math him or herself?
00:30:00.079 --> 00:30:05.440
And it's like, well, you talk to the model and you say, you know what, we'd actually like to score ourselves.
00:30:05.519 --> 00:30:09.680
So give him a little scratch pad where he can draw with his finger and do the 10 plus 12.
00:30:09.839 --> 00:30:18.640
But if he ever makes a mistake or she makes a mistake, give a little like corrective thing and say, oh, you know, you made a classic mistake, you didn't carry the one, blah,
blah, blah.
00:30:18.799 --> 00:30:19.359
Amazing.
00:30:19.519 --> 00:30:20.480
Amazing, folks.
00:30:20.640 --> 00:30:21.119
Amazing.
00:30:21.279 --> 00:30:23.359
Look at the I just like I'm spitting off ideas here.
00:30:23.440 --> 00:30:25.279
This is this is this is gold here, you know.
00:30:25.359 --> 00:30:28.480
I I should start an augmented reality board game.
00:30:28.799 --> 00:30:40.079
Well, you've you've you've just broadcast the idea to all your listeners, so I mean, but if you want to invest in Connor's idea, no, I the thing is, I'm sure there's like 14 different companies, including Jerry, I know, was working on stuff like this.
00:30:40.160 --> 00:30:53.039
But like, I don't really have the energy or the time to do any of this stuff because I'm happy at NVIDIA doing the work that I'm doing, but I just I can't wait until this stuff comes out because it's guaranteed it's gonna exist in the next 30 years or something like this.
00:30:53.680 --> 00:31:00.000
It is like the last six months have been probably my best, some of my best times at NVIDIA.
00:31:00.559 --> 00:31:03.839
And like, yeah, this stuff is just so exciting and fun.
00:31:04.079 --> 00:31:04.480
Yeah.
00:31:04.720 --> 00:31:12.240
I don't think I've been quite so excited about the day-to-day engineering tasks since the start of my career.
00:31:12.799 --> 00:31:12.960
Yeah.
00:31:13.359 --> 00:31:18.480
And like, you know, I I'm sure there's some people that are thinking, blah, blah, blah, terrible for the world.
00:31:18.799 --> 00:31:20.160
It's not all terrible, folks.
00:31:20.319 --> 00:31:25.519
I have become a lawn what do you call a lawn enthusiast.
00:31:25.599 --> 00:31:26.079
You know?
00:31:26.400 --> 00:31:30.720
My Kentucky bluegrass, grass, front and backyard.
00:31:30.880 --> 00:31:31.680
How do I know that?
00:31:31.839 --> 00:31:32.640
Thanks to AI.
00:31:32.720 --> 00:31:36.079
You just send it photos and you say, listen, grade my lawn.
00:31:36.160 --> 00:31:37.119
I got a D plus.
00:31:37.440 --> 00:31:44.319
We got lots of creeping Charlie, we got lots of clover, and my wife and I are environmentally conscious, so we're not using any pesticides.
00:31:44.400 --> 00:31:45.440
It's gonna be a lot more work.
00:31:45.599 --> 00:31:49.920
We gotta use the This is this is the Connor equivalent of Bryce's couch too.
00:31:50.079 --> 00:31:50.720
Yeah, well, yeah.
00:31:50.880 --> 00:31:54.640
I mean, everyone, yeah, maybe you're gonna be the furniture guy and I'm gonna be the lawn care guy.
00:31:54.720 --> 00:31:56.400
Because honestly, I remember my dad.
00:31:56.640 --> 00:32:03.839
My dad built a table from some wood planks that lived on my grandfather and my grandmother's farm in Terrace, BC.
00:32:04.240 --> 00:32:05.519
Rest in peace to both of them.
00:32:05.759 --> 00:32:08.640
But they had a wonderful farm and they had these wood planks.
00:32:08.880 --> 00:32:10.000
My dad like took them.
00:32:10.160 --> 00:32:12.720
We drove them eight
hours to Prince George, where I grew up.
00:32:12.960 --> 00:32:22.960
He built this like beautiful table, and like he spent so many weeks, slash, months, and every single time he would sit down, he would like rub his hand on it to like look at like the quality.
00:32:23.119 --> 00:32:24.000
Was it smooth enough?
00:32:24.079 --> 00:32:27.839
Did we need to put like another layer of liqueur or whatever they call it on, a lacquer?
00:32:28.160 --> 00:32:33.920
And we would all all me and my three sisters would always make fun of him, like because he's always like we'd sit down to eat, and what is he doing?
00:32:34.079 --> 00:32:37.279
He's ah, you know, this table, uh, you know, I maybe got to do some stuff.
00:32:37.440 --> 00:32:48.880
I have now become my father because every day I basically walk out, especially if it's rained, and I am like assessing, you know, oh, did the iron kill enough of the uh the creeping Charlie or the clover or whatever?
00:32:49.039 --> 00:33:08.799
And it is like it brings me so much joy, and also we have a skunk, we named him Wilbur, and we had to have a discussion, Shima and I, about whether we were going to get rid of the grubs that are living beneath our grass because it is the lunch buffet and dinner buffet for Wilbur, the skunk, who comes and just digs up these little holes and feeds himself.
00:33:08.960 --> 00:33:14.960
And Shima said that uh she was very happy that our lawn was the uh skunk's lunch and dinner.
00:33:15.119 --> 00:33:17.920
So we're just gonna leave, let Wilbur tear up the lawn.
00:33:18.079 --> 00:33:23.519
But anyways, I have to go around and like fill in these little holes that Wilbur keeps on uh digging in our front and back lawn.
00:33:23.680 --> 00:33:26.000
And like all of this is thanks to AI.
00:33:26.160 --> 00:33:28.640
I didn't care before, and then I was like, you know what?
00:33:28.720 --> 00:33:33.039
I bet AI could tell me how to like you know improve the quality of my lawn.
00:33:33.279 --> 00:33:38.880
And uh sure enough, it's you get we're I'm gonna buy you a chair so you can go yell at the kids and say get off my lawn.
00:33:39.039 --> 00:33:39.680
No, no, no, no.
00:33:39.759 --> 00:33:42.079
If the kids want to play on my lawn, they can play on my lawn.
00:33:42.319 --> 00:33:45.599
Anyways, the point is it's not all bad, folks.
00:33:45.839 --> 00:33:50.319
If you, you know, give it a chance, it'll make you fall in love with your lawn as well.
00:33:50.559 --> 00:34:00.319
Be sure to check these show notes either in your podcast app or at adspthepodcast.com for links to anything we mentioned in today's episode, as well as a link to a GitHub discussion where you can leave thoughts, comments, and questions.
00:34:00.480 --> 00:34:01.119
Thanks for listening.
00:34:01.200 --> 00:34:02.880
We hope you enjoyed and have a great day.
00:34:03.680 --> 00:34:06.400
Low quality, high quality.
00:34:06.640 --> 00:34:08.880
That is the tagline of our podcast.
00:34:09.199 --> 00:34:10.079
That's not the tagline.
00:34:10.320 --> 00:34:13.039
Our tagline is chaos with sprinkles of information.