Gemini 3.5 Flash, Composer 2.5 is a Beast, Google IO, We Live in Exciting Times | Ep 16

WEBVTT
00:00:00.640 --> 00:00:12.800
Ladies and gentlemen, boys and girls, sit down because this is going to be a wild ride with your hosts Ray Fernando, Eric Provencea, and Adam Larson of the Rate Limited Podcast.
00:00:13.039 --> 00:00:21.199
Make sure you tune up the dial fully for this entire episode because we'll be covering many topics across what's happened this week with Google I.O.
00:00:21.600 --> 00:00:25.920
Composer 2.5, OpenAI, and a whole bunch more.
00:00:26.160 --> 00:00:29.280
And I can't believe all the stuff that's been happening this week, guys.
00:00:29.359 --> 00:00:30.239
I mean, what do you guys think?
00:00:30.399 --> 00:00:35.439
This has just been an incredible last couple of weeks that you know Carpathi is now at anthropic.
00:00:35.520 --> 00:00:36.560
You know, Google I.O.
00:00:36.719 --> 00:00:37.600
just cap came out.
00:00:37.759 --> 00:00:43.840
Uh, I'm using Composer 2.5 a lot, and we'll just be curious to hear what you guys are thinking and kind of what's going on.
00:00:44.000 --> 00:00:45.520
Eric, what's up?
00:00:45.920 --> 00:00:47.600
Well, the news never stops.
00:00:47.679 --> 00:00:48.479
It doesn't sleep.
00:00:48.560 --> 00:00:51.679
Uh, in the world of AI, every day is a new day.
00:00:51.840 --> 00:00:54.240
You gotta sit up and pay attention.
00:00:54.479 --> 00:01:02.320
Um, but you know, in this week with Google IO, there was a lot of news coming out, and um, a lot of it is probably like you know, vapor for now.
00:01:02.399 --> 00:01:09.840
Like, you know, we we saw this new like spark open claw competitor, but like it's only for testers, and then when it will be out, it's us only.
00:01:10.000 --> 00:01:11.680
So, I mean, we'll see where that ends up.
00:01:11.760 --> 00:01:16.159
But like the things you can use that did come out this with the this week with with the I.O.
00:01:16.319 --> 00:01:19.120
announcements are the new Gemini 3.5 flash.
00:01:19.359 --> 00:01:37.280
And man, like the the people are not pleased with it because it is expensive, and from what I've been seeing, uh running benchmarks with it, um, it's actually more expensive than GPT 5.5 in many cases, because it just burns tokens like nothing else.
00:01:37.359 --> 00:01:38.560
It's a token guzzler.
00:01:38.640 --> 00:01:47.680
So I think we're starting to see uh a change now with models where they have a sticker price, which is already 3x more than the old flash model, which is like a huge shame.
00:01:47.840 --> 00:01:51.200
Um, and then there's like, okay, well, you have a task you want to complete.
00:01:51.280 --> 00:01:53.680
How many tokens does it burn to get through that task?
00:01:53.840 --> 00:01:56.719
And 3.5 will burn tokens like nothing else.
00:01:56.879 --> 00:02:05.359
Um, so if you've seen if you've seen uh benchmarks on that, like just pay pay in mind, like maybe GPT-5.5 will get it done faster and cheaper, even though the price is higher.
00:02:05.439 --> 00:02:06.879
Uh, have any of you tried the model?
00:02:06.959 --> 00:02:08.639
Have you uh played with it at all, Adam?
00:02:08.800 --> 00:02:09.439
Oh, yeah.
00:02:09.759 --> 00:02:14.960
Actually, I'm a huge fan of the Gemini Flash series, like 2.0, 2.5.
00:02:15.439 --> 00:02:16.000
Those were great.
00:02:16.240 --> 00:02:18.719
Even three to a certain degree, their preview model.
00:02:18.879 --> 00:02:19.039
Yeah.
00:02:19.199 --> 00:02:33.360
I when I when I saw the 3.5 one, I was actually pretty excited about it because I these models are typically really good at agentic sub-agent workflows, things that you just need really fast turnaround time on, and you don't need a ton of reasoning.
00:02:33.680 --> 00:02:38.960
I feel like they have rebranded Flash to mean something different now.
00:02:39.039 --> 00:02:39.199
Yeah.
00:02:39.680 --> 00:02:41.840
Like they burned a lot more tokens.
00:02:41.919 --> 00:02:42.479
They are fast.
00:02:42.560 --> 00:02:43.280
I will give it that.
00:02:43.360 --> 00:02:48.960
Like, it does have fast CPS, but it's burning a lot more tokens, and it is substantially more expensive.
00:02:49.120 --> 00:02:53.199
Like almost to what uh like if you think about it, like what is it?
00:02:53.439 --> 00:02:55.759
66% of like a top-tier model.
00:02:55.840 --> 00:02:58.000
Like it is insane how costly they are.
00:02:58.240 --> 00:03:08.240
Even their caching price is actually pretty high compared to their the it's a dollar per million tokens per hour, so it's got an hour limit on it too, which is crazy.
00:03:08.479 --> 00:03:14.800
Um, I'm very disappointed, honestly, that they've really changed the meaning of flash.
00:03:14.960 --> 00:03:17.520
Now, the flashlight model, did you guys try that one?
00:03:17.680 --> 00:03:19.280
The Gemini 3.1 flashlight?
00:03:19.599 --> 00:03:21.919
3.1 flash I've tried, and I was pretty impressed with it.
00:03:22.000 --> 00:03:24.639
But it's just like not quite what I need.
00:03:24.719 --> 00:03:31.840
You know, it's just like just under the bar, and it's like the you need really niche workloads for it to be worth its time, you know.
00:03:31.919 --> 00:03:33.599
That's the thing with these tiny models.
00:03:33.919 --> 00:03:42.400
I'm hoping there is a Gemini 3.5 flash light model in the works that is substantially cheaper but better than 3.1 light.
00:03:42.560 --> 00:03:44.800
Uh I don't know, Ray, what are you thinking?
00:03:45.199 --> 00:03:51.599
Um I was just using inside of anti-gravity, and I don't know why, but it performs pretty well for me inside there.
00:03:51.919 --> 00:03:55.520
And I just but I had a really well spec'd out plan.
00:03:55.919 --> 00:04:03.919
And just breaking the plans down into much, much smaller pieces, even though it has a million token context window, I'm able to pull quite a bit out of it.
00:04:04.400 --> 00:04:09.919
It doesn't have that uh playful intelligence that 5.5 just has out of the box.
00:04:10.319 --> 00:04:15.759
And so I think that's kind of where 5.5, it's like I can transition without thinking too much.
00:04:15.840 --> 00:04:31.759
And I think whenever I have to offload thinking in terms of like work that needs to be done, you know, I I'm thinking about this more in this agent engineering phases of things, and I feel like I can see the the hints of it there, and it just needs it needs a smarter model to drive this this this model forward.
00:04:32.079 --> 00:04:36.879
Um because I feel like the more how do you say this?
00:04:37.040 --> 00:04:45.120
Okay, so if I have a very succinct plan that has a lot of details that are very well structured, I flash actually follows them extremely well.
00:04:45.279 --> 00:04:55.360
So one of my prompts is like 300 lines, but it's literally to build out like a a web page that has a home page, a blog page, a bunch of design notes that are in it.
00:04:55.920 --> 00:04:59.759
And Gemini even 3.1 Flash does extremely well with this.
00:05:00.079 --> 00:05:08.160
I found that Gemini Um Pro didn't really do Gemini 3 Pro didn't really do too well, or the pro versions of this stuff that didn't really do too well of it.
00:05:08.560 --> 00:05:11.920
And but the Flash versions always perform really well with instruction following.
00:05:12.480 --> 00:05:19.920
With like very special like, but I had to have Opus as a model kind of design this prompt um so they can work with a smaller model like this.
00:05:20.079 --> 00:05:22.160
Um and so those work really, really well.
00:05:22.399 --> 00:05:31.040
When I just vibe in terms of like speaking on the microphone and explaining things like of how I want them, that's kind of where the frustration really comes out.
00:05:31.199 --> 00:05:34.879
And so anti-gravity seems to I don't know if it aids in it.
00:05:34.959 --> 00:05:47.040
I don't know what they're doing behind the hood because I do select the model, I don't think it's changing the model underneath the hood, or maybe it is, um, but it seems to guide the the model a little bit more or like shepherds it across, like, oh, you should be doing this thing at this phase now.
00:05:47.120 --> 00:05:47.759
So I don't know.
00:05:48.079 --> 00:05:50.160
So here's here's a random question for both of you.
00:05:50.319 --> 00:05:53.040
Yeah, do you think they're making money on this model now?
00:05:53.360 --> 00:05:54.079
At the price they're gonna be able to do that.
00:05:54.240 --> 00:05:54.959
I think they're trying.
00:05:55.120 --> 00:06:02.720
I I think the thing with Google that I've understood is that they value margins really, really like intensely when they're charging money for something.
00:06:02.879 --> 00:06:06.879
And so if they're charging the price they're charging, it's because they want to have margins on it.
00:06:07.120 --> 00:06:13.279
Um, they're not like I I think just structurally, they're not in a place to try and like undercut people for AI.
00:06:13.360 --> 00:06:15.600
And I think they're very capacity constrained as well.
00:06:15.759 --> 00:06:19.199
And each of their products where they release them, they have very tiny allocations.
00:06:19.279 --> 00:06:23.600
Like, I think the anti-gravity side gets like a slice, and that's why they're trying to consolidate things.
00:06:23.680 --> 00:06:37.439
And part of this news as well is that they're killing off Gemini Cli, which had been around for a while, was open source, it supported ACP, which was meant you can use Gemini models and other IDEs and harnesses and stuff, and that was just fully removed.
00:06:37.680 --> 00:06:39.120
An anti-gravity 2.0 doesn't have it.
00:06:39.279 --> 00:06:48.399
TBD on whether they'll bring it back because they were actually one of the stewards of the of the of the uh pro of the the of the standard, but it seems like that's no longer the case.
00:06:48.639 --> 00:06:50.959
So that's uh that's unfortunate news for that.
00:06:51.199 --> 00:07:01.120
Um but yeah, I mean I I think they they are trying to to make a profit off of this, and I think that's probably why 3.5 is so expensive.
00:07:01.360 --> 00:07:02.560
Um so we'll see.
00:07:02.639 --> 00:07:04.319
But yeah, I'm a little disappointed with that.
00:07:04.720 --> 00:07:15.360
I think yeah, if I'm looking at their website, it they have it all organized, like project management style, like standard, badge, flex, priority, whether they train on the data or not.
00:07:15.600 --> 00:07:21.600
And you know, I think in terms of like profitability, they probably had profitability from down on up.
00:07:22.000 --> 00:07:25.920
And they just said, okay, if you're gonna give you this much more, you're just gonna pay that much more.
00:07:26.000 --> 00:07:26.639
Because check it out.
00:07:26.959 --> 00:07:36.240
If you get 3.5 flash on priority, the input price is$2.7 per million tokens, and then 16.20 for output for million tokens.
00:07:36.560 --> 00:07:40.160
Yo, that's that's the the price of Sonnet on honestly.
00:07:40.319 --> 00:07:41.519
Like, that's that's where you're getting at.
00:07:41.600 --> 00:07:49.439
Like, yeah, the input's a little cheaper, but you're you're paying, you're paying Sonnet prices for a model that is probably much smaller than Sonnet, to be honest.
00:07:49.680 --> 00:07:51.839
Um, I don't know, like it's weird.
00:07:51.920 --> 00:08:02.480
Like, I think Google's very efficient with how they make their models, but man, like I this is not this is not a a whopper model like you're gonna get with Opus or Fi5.
00:08:02.720 --> 00:08:07.279
Like, this is a small model that is competing at pricing that is not small prices.
00:08:07.439 --> 00:08:18.639
And you know, like one of the things that is a bit unfortunate with this is that like so reasoning models, there there's this, there's this thing where where they burn tokens to get intelligent answers, right?
00:08:18.879 --> 00:08:22.959
Um, and smaller models need to burn more tokens to get to the same intelligence.
00:08:23.120 --> 00:08:28.959
And the problem is like if you're burning more tokens, you're burning more of your context window on reasoning.
00:08:29.120 --> 00:08:33.039
And so then the deeper into the context window you get, the dumber the model.
00:08:33.360 --> 00:08:43.360
So if you're burning more tokens to get to that place, you're you're just degrading intelligence further and further as you get to announce it, and you can do less in a context window, which is a problem.
00:08:43.519 --> 00:08:49.759
And and Google tries to pad it with like the 1 million context window, but small models are worse at large context than bigger models.
00:08:49.840 --> 00:08:52.399
Uh, so you you you have this problem that that arises.
00:08:52.559 --> 00:08:58.240
That's why you're getting, you know, GPT-5.5, the breakthrough is that it's so efficient at reasoning.
00:08:58.399 --> 00:09:09.279
And so you you can get GPT-5.5 low, and it will reason for a fraction of the time with 3.5 flash and get to the same outcome, even with like max reasoning on 3.5 flash.
00:09:09.360 --> 00:09:14.159
So and it ends up being much cheaper as a result, faster and cheaper, even though you're getting lower tokens per second.
00:09:14.320 --> 00:09:24.159
Like these things are getting hard to judge and compare because yeah, like on the price on the sticker, like it looks like you're getting a better value, but you know, the amount of tokens you burn matters, and you're burning a lot with this.
00:09:24.240 --> 00:09:26.320
It's a it's a real token guzzler.
00:09:27.360 --> 00:09:39.360
Do you think it's a slingshot play in terms of where they need to just collect data and get people to use it from all these different use cases, and then eventually that'll get RL'd in, or you know, like I think they're trying.
00:09:39.440 --> 00:09:46.399
It seems like Google's thing is that they build like a behemoth model and then they distill it down, and they're getting really good at this distilling game.
00:09:46.480 --> 00:09:53.120
And I think Google's probably the best at distilling, but you know, I think there's diminishing returns on that, and at some point you need like I don't know.
00:09:53.200 --> 00:09:54.159
We'll we'll see where it goes.
00:09:54.320 --> 00:10:04.320
Like, I obviously they need usage, but also they're selling a lot of usage to anthropic with their with their big uh you know TPU play and and and trying to give give all that compute away.
00:10:04.399 --> 00:10:19.600
So I think internally they're underallocated on compute, and I think that's part of why things are so expensive, is because they have to like load balance demand uh with with availability, and I think charging higher is how you get to that equilibrium, and hopefully it makes sense for them.
00:10:19.759 --> 00:10:44.399
But like I, you know, I'm talking to people building on Flash, and they were building good businesses on Gemini 3 Flash, and it did the business case stops making sense, and the model performs worse in certain benchmarks for them in that case, and um, and and the thing that is really unfortunate too with with this move is that so 3.5 flash is uh a GA model, but the previous Gemini 3 Flash was a preview model, which means that it gets sunset way earlier.
00:10:44.559 --> 00:10:54.320
Um, so they're sunsetting all the cheaper ones, getting putting the new price up, and and it's just like I I just think people are gonna move away from Gemini models because it just doesn't make sense at this point.
00:10:54.559 --> 00:10:57.600
Um I mean, just to add on to that, there's there's a couple points here.
00:10:57.840 --> 00:11:00.000
You nailed it by saying what you did, Eric.
00:11:00.080 --> 00:11:15.519
There there are there are use cases that I built in my company that uh I know other companies are building on where they just need a very inexpensive workhorse of a model to do some lightweight like and there is kind of a gap in the market right now.
00:11:15.759 --> 00:11:23.919
Um I know the flashlight, but again, that performs in my experience worse than the flash models of prior generations.
00:11:24.080 --> 00:11:25.919
So it's uh there's a big gap.
00:11:26.000 --> 00:11:32.559
And the other thing is this model, the 3.5 flash version, seems to have a lot less information like in it.
00:11:32.639 --> 00:11:34.639
I'm not really sure how better to equate that.
00:11:35.120 --> 00:11:40.080
But some of the jobs that we typically would run require some knowledge outside of code.
00:11:40.159 --> 00:11:44.000
Like it's good at coding, but it's not a general purpose workhorse anymore.
00:11:44.080 --> 00:11:44.399
Yeah.
00:11:44.799 --> 00:11:50.320
And I actually was checking, and I think the it's uh knowledge base hasn't been updated.
00:11:50.399 --> 00:11:55.759
I still think it's dated to January 2025, which is kind of crazy, honestly.
00:11:56.000 --> 00:11:56.799
It's brutal.
00:11:57.200 --> 00:12:05.519
Yeah, honestly, the flash models always had less information in them than the pro ones, and I think that's what you're kind of getting with the pro model, the bigger model, bigger retention and world knowledge.
00:12:05.679 --> 00:12:13.120
Um, but you know, I think that's that was always the benefit of the Gemini models is that you get like Google's like world understanding in the model.
00:12:13.200 --> 00:12:17.279
And you know, at a flash model, you're not getting that as much and you rely on searching.
00:12:17.360 --> 00:12:25.679
And yeah, I see a variation in some of the evals that we run between like 2.5 flash and 3.5 flash.
00:12:25.840 --> 00:12:33.759
Where I think they're I think they've actually degraded the overall either they they more fine-tuned it for coding versus more like general purpose use cases.
00:12:34.080 --> 00:12:34.799
I think they have, yeah.
00:12:34.879 --> 00:12:36.159
I think I think that's definitely the case.
00:12:36.320 --> 00:12:38.240
They wanted to prioritize agentic use.
00:12:38.399 --> 00:12:50.080
And I think the thing, if you look at like what Google's announcing, is they were probably putting agents everywhere, and and the big thing with Spark is their like general purpose agent where you know it could you access your email, it can access all your Google services.
00:12:50.159 --> 00:12:57.120
And I think that's like a really promising thing, and I've been looking for that from Google, but they needed a model that is cost effective for them to run and serve in that kind of a harness.
00:12:57.360 --> 00:12:59.360
Um, and I think that's what they're trying to build there.
00:12:59.519 --> 00:13:03.519
Um, but it's just like and I think as a general agent, it's probably fine.
00:13:03.600 --> 00:13:11.919
I I just it's it's unfortunate, like it's just so expensive to use as a as a consumer, uh like in in enterprise paying API prices for for this.
00:13:12.000 --> 00:13:14.960
Like, I just it's just an unfortunate turn of events.
00:13:15.200 --> 00:13:21.600
Um, but yeah, as part of this, you know, Anti Gravity 2 is is is around, and uh that's definitely their new thing.
00:13:21.679 --> 00:13:28.000
And like we were mentioning, you know, compute allocation is a big thing for Google, and they're I think they're trying to consolidate where they allocate compute for people subscribing.
00:13:28.080 --> 00:13:29.519
Um, so TBD on that.
00:13:29.679 --> 00:13:36.879
Um, if you're not familiar, anti-gravity was the product that uh is the result of the Windsurf acquisition uh from last year.
00:13:37.039 --> 00:13:40.960
Uh so so they've actually gutted out all the IDE.
00:13:41.039 --> 00:13:46.720
And there is a crazy thing that happened this week with this, uh, which is I I was shocked, and a lot of I've seen people really upset about it.
00:13:46.799 --> 00:13:59.679
So people who are using Anti-Gravity 1, um one day this week they updated the app and it became anti-gravity two, and the IDE is gone, their settings are gone, uh those all broke.
00:13:59.759 --> 00:14:10.480
Uh and then all of a sudden they're using uh a kind of rip-off of the cursor, uh, sorry, of the codex app, uh, which now everyone else is ripping off, um, except just not as good and not as complete.
00:14:10.639 --> 00:14:20.000
And and like, sure, it works fine, I guess, but like, you know, if you were using it a certain way one day and then now you're using it this way, like all of it's gone.
00:14:20.080 --> 00:14:25.600
And and if you wanted to get that old experience back, you have to download a separate app and reset up all your settings again.
00:14:25.759 --> 00:14:27.519
Uh, because it's two separate settings files.
00:14:27.600 --> 00:14:35.519
So then you have all these like cluttered things, you have like annuity one and two like set config folders in your in your repos, and it's a complete mess.
00:14:35.840 --> 00:14:38.799
So I think they did not handle this well at all.
00:14:39.039 --> 00:14:41.440
Uh curious to see how that goes for them.
00:14:41.519 --> 00:14:44.960
But I, you know, if I was a user of any gravity, I would have turned from this.
00:14:45.039 --> 00:14:48.000
Like, I I just can't imagine continuing to trust them for this.
00:14:48.159 --> 00:14:51.279
And ah, anyway, start with the rant there.
00:14:51.440 --> 00:14:56.320
Yeah, real question on this how many people do you think actively use anti-gravity?
00:14:56.480 --> 00:14:58.320
Like, do you think it's actually I don't know.
00:14:58.399 --> 00:15:01.679
Um, probably less this week because of this, but I love it.
00:15:02.000 --> 00:15:11.519
I think in different countries because I know a lot of people, like in South America, that they're using that, you know, because mostly access to compute, because for them it's cost a lot of money compared to us.
00:15:11.759 --> 00:15:12.399
Yeah.
00:15:12.799 --> 00:15:25.360
I think one thing that is how they have going for them, and there's actually someone in my Discord who's a who's a user because of this, is that so they have the like Google One plan for the family with like the mega plan or whatever.
00:15:25.519 --> 00:15:29.120
And if you have that plan, then you have anti-gravity usage.
00:15:29.279 --> 00:15:32.960
So a lot of people are subscribing to Google Services uh anyway.
00:15:33.120 --> 00:15:37.679
Like you know, I subscribe to Google YouTube Premium and different things, and uh one Google Drive.
00:15:37.840 --> 00:15:45.039
So if you're in that ecosystem, oh well, you have this coding product available to you as well on top of it, and I think for that it could be good value.
00:15:45.279 --> 00:15:52.799
Um, but you know, if you're not in that ecosystem, there's not really an appeal to kind of go in just for the coding side, so TPD.
00:15:53.200 --> 00:15:56.879
Um yeah, I just I feel like it's such a weird decision.
00:15:57.120 --> 00:15:58.960
And and I know they they have all the data.
00:15:59.120 --> 00:16:02.000
So like I I'm all I'm on the outside, I can only see so much.
00:16:02.240 --> 00:16:04.559
I totally understand I could be wrong.
00:16:04.799 --> 00:16:10.720
But from my bubble of the world, which again is very limited, I don't really know anyone that actually uses anti-gravity.
00:16:10.879 --> 00:16:18.639
And and I feel like they're gonna have a hard time breaking into that market with the stranglehold that like VS Code has and Cursor has.
00:16:18.879 --> 00:16:21.840
Not saying it's not possible, but it is gonna be a lot of investment.
00:16:22.000 --> 00:16:33.360
And then to outright like stop development on Gemini CLI, which I think they actually had a better chance of making that a long-term play, yeah, just feels a very interesting decision making to me.
00:16:33.679 --> 00:16:44.480
Yeah, I'm yeah, the some the CLI thing is weird because on top of it, well, one thing that is interesting is that they're actually not sunsetting it for enterprise customers, but I think it's gonna effectively be on life support.
00:16:44.639 --> 00:16:49.120
Um, you know, but they just don't want to disrupt those customers, but then the consumers, those they don't care about.
00:16:49.200 --> 00:16:54.159
So you have 30 days to move away, or and then your your sub stops working with it, which is kind of weird.
00:16:54.399 --> 00:16:54.720
We'll see.
00:16:54.799 --> 00:17:00.720
I you know, there's a lot of people that built on it because of ACP, and then all those people are getting screwed over a little bit because of it.
00:17:00.799 --> 00:17:02.639
Um, so a lot of workflows are going away.
00:17:02.799 --> 00:17:06.079
They really want to funnel people to anti-gravity, and ah, we'll see how it goes.
00:17:06.319 --> 00:17:08.000
Anyway, I think that's enough about Google.
00:17:08.160 --> 00:17:10.160
Um, we'll we'll kind of move on from here.
00:17:10.240 --> 00:17:15.039
Um, another thing that came out this week was Composer 2.5, and Ray, you've been like a super huge fan of it.
00:17:15.119 --> 00:17:18.880
And before you dive into your thoughts, I just want to like kind of re-add this thing.
00:17:19.039 --> 00:17:32.720
So, Adam, you were saying, you know, like you were finding 3.5 flash is kind of losing world knowledge in favor of coding, and I think like Composer is in the same boat in terms of like the the way they train this model as a coding model, they don't care about the world knowledge as much.
00:17:32.880 --> 00:17:36.000
Um, when I was using two composer 2, I definitely felt that way.
00:17:36.160 --> 00:17:38.799
I'm curious, Ray, how are you feeling about Composer 2.5?
00:17:39.039 --> 00:17:43.680
Like, how's it affected your workflows and and are you using it as your daily driver?
00:17:44.000 --> 00:17:57.359
So before 2.5 came out, I was using Codex a lot, like as a primary driver for a lot of my coding and just you know, codex high, medium, low, and and even in fast mode because I like the speed.
00:17:58.079 --> 00:18:00.880
And I just started using composer.
00:18:01.200 --> 00:18:03.920
I liked composer 2, you know, for a certain task, especially.
00:18:04.000 --> 00:18:05.680
Like I was really impressed with the front end stuff.
00:18:06.319 --> 00:18:13.759
Uh and then 2.5 came out and it's literally taken over everything from planning to like I've unleashed a QA agent.
00:18:14.160 --> 00:18:17.200
Like to give it like skills, it builds the skill files.
00:18:17.279 --> 00:18:24.319
I did everything, and I just keep pushing this model and I'm just extremely impressed at uh its ability to get stuff done.
00:18:24.480 --> 00:18:29.440
It feels like Yeah, it feels like a cross between Opus and GPT 5.5.
00:18:30.400 --> 00:18:32.799
And uh it just it's a workhorse.
00:18:32.880 --> 00:18:48.799
It's it's a workhorse, and the the speed is probably the other factor that it's really influencing a lot of my uh uh bias here in some ways, because I didn't realize how much of a factor that would make in uh me wanting to continue to just get needed information.
00:18:49.440 --> 00:18:52.400
But also the when I generate plans, they're also very succinct.
00:18:52.880 --> 00:18:58.640
They're like not overly wordy like Opus, but they're not overly like GPT-ified, if you know what I mean.
00:18:58.720 --> 00:19:03.599
So GPT-ified likes to put a lot of headers, likes to like kind of also guesslight you back a little bit.
00:19:03.839 --> 00:19:06.000
This is just kind of like here's a layout of the land.
00:19:06.079 --> 00:19:07.359
I was like, okay, cool.
00:19:07.839 --> 00:19:09.599
Can you then take this and and run with it?
00:19:09.759 --> 00:19:10.960
Or can you do this and that?
00:19:11.279 --> 00:19:14.640
Um, you know, I was like, okay, how good is it at building a novel system?
00:19:14.799 --> 00:19:18.720
Like I I'm literally just building an entire Suite workflow inside of my app.
00:19:18.799 --> 00:19:30.960
I just built a new app from scratch, and then I said, okay, let's just pretend to be um let's just go through like a whole bug review process, and you're gonna be uh the person who's gonna write me bug reports, help me set that up.
00:19:31.279 --> 00:19:43.920
And so it's like it set up its own folders, it set up its own way of you know writing bug reports and templates and everything like that, and then it did a test pass and it launched multiple browsers and was clicking away in the browser super fast.
00:19:44.079 --> 00:19:50.319
I was like, Oh, and it wrote really good bugs, like it found bugs that I would normally find just testing by myself.
00:19:50.799 --> 00:19:54.240
And I was like, damn, and the speed was crazy.
00:19:54.400 --> 00:20:00.000
So it's like oh, I don't think we've even understood the impact of this yet.
00:20:00.240 --> 00:20:03.839
And not a lot of people have like this general SWE background.
00:20:03.920 --> 00:20:17.200
If you've worked at like really deep in QA engineering or worked with like project managers and all these different workflows, you can reproduce those workflows with this model, send it off, and do stuff in parallel.
00:20:17.680 --> 00:20:21.039
And like that's another thing that their harness has improved tremendously.
00:20:21.119 --> 00:20:30.079
That I know I feel like a cursor shill and stuff, but I've just been uh the team has been I've been like giving tons of feedback and they've been incorporating as fast as possible.
00:20:30.319 --> 00:20:37.759
Uh and so it sounds like they've rewrote their entire engine behind the scenes, uh, and everything is extremely performant and really, really fast.
00:20:37.920 --> 00:20:44.640
In fact, I don't think it's tied to the VS code stuff, and I think that's what's also driving their new CLI as well.
00:20:44.720 --> 00:20:50.000
So that that same underpinnings is driving the new UI interfaces, but also driving these the CLI.
00:20:50.079 --> 00:20:57.200
So you can do all the stuff in the CLI technically of what I'm doing with like multiple agents and parallelizing and uh testing and stuff.
00:20:57.519 --> 00:20:58.720
It's really interesting.
00:20:58.799 --> 00:21:07.440
I I hadn't realized that the browser use was like such a step up on Composer 2.5 because I think that's really like a bottleneck with big models, is like how slow they are at clicking around and doing things.
00:21:07.519 --> 00:21:13.759
And if you can have that like at really high TPS and the vision side of it is good enough to kind of drive that, like that's impressive.
00:21:13.839 --> 00:21:20.240
And uh I think I think that's definitely like if they can get that latency down for those kinds of workloads, it's a huge demand for that.
00:21:20.319 --> 00:21:23.599
Like that opens up new use cases for sure in in your loops.
00:21:23.759 --> 00:21:25.519
So curious to see how that evolves.
00:21:25.680 --> 00:21:27.200
Um, have you tried it at all, Adam?
00:21:27.279 --> 00:21:29.039
I know you're heavy uh cursor user.
00:21:29.440 --> 00:21:29.839
Oh, yeah.
00:21:30.079 --> 00:21:38.400
I feel like it somehow I've become a cursor shill as well, and I was very like negative on cursor early on, so I also feel kind of odd about that.
00:21:38.559 --> 00:21:41.839
But I will say um I agree with everything you said.
00:21:41.920 --> 00:21:46.160
Ray, I think cursor uh composer 2.5 is just phenomenal.
00:21:46.400 --> 00:21:59.759
I still wish I had an understanding of how they pull off the speed they do, because if I understand correctly, this is built off of what is it, Kimmy's K 2.5 model, which is notoriously hard to.
00:22:00.160 --> 00:22:13.039
Run like a it's like a trillion parameter, it's like a ridiculous model, and I typically get 15 TPS or 12 TPS, something ridiculous when I hit, yeah, and it's like this thing runs so fast, it's very smart.
00:22:13.599 --> 00:22:28.799
It um it isn't to the level of like GPT 5.5 for me, but when I when I'm on a call and we're brainstorming something, I literally can just real-time prototype something, and it doesn't even like while the conversation's happening, I'll be like, Oh, are you thinking like this?
00:22:28.960 --> 00:22:30.880
And and I can't do that with any other model.
00:22:30.960 --> 00:22:34.319
It's phenomenal it is honestly just a ton of fun.
00:22:34.400 --> 00:22:50.079
And I to your point earlier, Eric, I do see I do agree it has sacrificed world knowledge, but it's like maybe I'm unfairly rating Gemini based on this, because Gemini at Flash, I have typically used of like workhorse, like API-driven behind-the-scenes things.
00:22:50.319 --> 00:22:51.440
Not so much for coding.
00:22:51.599 --> 00:22:55.359
Composer 2.5, if I were to put it in that same position, I'd probably have the same complaint.
00:22:55.599 --> 00:23:00.400
So just to be totally fair, but for coding, I love this model, like it is incredible.
00:23:00.640 --> 00:23:02.000
Kudos to the team.
00:23:02.880 --> 00:23:04.160
Massive kudos to the team.
00:23:04.400 --> 00:23:10.240
Well, I'm excited to see what they do with Composer 3 and uh you know the Colossus partnership with with SpaceX.
00:23:10.480 --> 00:23:12.960
Like, could be a crazy turn of events for them.
00:23:13.200 --> 00:23:14.400
They might you know be a leader.
00:23:14.480 --> 00:23:15.119
We'll see how it goes.
00:23:15.359 --> 00:23:16.319
What do you guys think?
00:23:16.400 --> 00:23:27.119
Do you think for me I wrote this on X and I kind of believe it because I saw there was a bug report where the model is preferring to write motion instead of div or something in some type of the way that it was writing.
00:23:27.359 --> 00:23:28.640
This is Composer 2.5.
00:23:29.440 --> 00:23:37.119
So basically, within getting that report, the cursor team turned around a new checkpoint for the model and shipped it out within like a couple of hours.
00:23:37.519 --> 00:23:42.880
And so for me, if they're able to keep up at this speed, I feel like the speed is the moat.
00:23:43.200 --> 00:23:48.640
You know, if they have this type of pipeline to turn stuff around super fast, you know, I I don't know.
00:23:48.720 --> 00:24:02.640
Like I'm a little skeptical of of this like continuous improvement RL, because the thing is like if you're just like vibe vibe shipping checkpoints for for these models, like it's it's really hard to like prevent regressions.
00:24:02.720 --> 00:24:20.240
And like, yes, you have benchmarks that you check for, and you're if your benchmarks are like good, you can you can retain quality, but I just think in real world use, like the benchmarks don't tell the whole story, and when you test models, like I don't know, you want some consistency, especially if you're serving on the API.
00:24:20.319 --> 00:24:23.759
Like, I I guess like for cursor, like they're not doing that, so they don't care.
00:24:24.079 --> 00:24:32.960
But I'm I'm I'm just skeptical that like you you end up with like these spiral degradation loops of like just trying to overcompensate on one side versus another.
00:24:33.279 --> 00:24:34.400
But I don't know, we'll see.
00:24:34.480 --> 00:24:36.480
Like, I it seems to be working okay for them.
00:24:36.559 --> 00:24:50.400
Uh, but I and I I do think like at some point, probably some continuous iteration is is kind of gonna be the way that's standard, but I'm just not sure that we're quite there at the moment in terms of the reliability of it, so I guess we'll see.
00:24:50.720 --> 00:25:10.960
You know what's funny is I was working with uh a different team over this past week, and they've gone through and they're like, okay, we updated all of our prompts and our evals are now at a higher like a higher percentage, and you go to actually test it out and you're playing around with it, and you're like, Well, congratulations, you figured out that our evals suck because this thing is way worse than it was yesterday.
00:25:11.200 --> 00:25:14.160
Like that that's that's the reality of where we are right now.
00:25:14.319 --> 00:25:21.920
Like it is that it's not a useful it's useful to a point, but there's still that human subjective nature that you have to apply to it.
00:25:22.319 --> 00:25:31.680
But there's also like, you know, when you're um when you're working in uh as a user of these models, you have like all kinds of complexity in your in your setup.
00:25:31.759 --> 00:25:48.799
You've got skills that are installed that you want the model to auto-invoke, you've got like your your current prompts, your system prompts, you've got your agent files, agents MD, uh, you've got like just how you type it, your code base is a prompt too, like the code that the model runs into and engages with, like how is that laid out and set up.
00:25:48.960 --> 00:25:54.720
All of these things are kind of divorced from the evals that are kind of static and and and don't change.
00:25:54.799 --> 00:25:59.119
And so, yes, if you you know tune on that, maybe you get better scores.
00:25:59.279 --> 00:26:07.599
I think cursor to their credit, like they're trying to consider all of this in their setup, and they haven't shared too much about what their cursor bench does, but like I'm sure consider some of this.
00:26:07.920 --> 00:26:20.720
But you know, the real world is messy, and if you're continuously optimizing for a kind of score, you kind of lose out on some of these other things and and the messiness and you know the context clutter of of like how a model works day to day.
00:26:20.799 --> 00:26:24.640
And there's just a risk that like you know, you you're changing too much.
00:26:24.720 --> 00:26:33.680
And the thing is when your user gets used to a model, they try to like lay things out, change their prompt style, and kind of tune to themselves to the model if they're if they're a good power user.
00:26:33.839 --> 00:26:38.400
Um, but if the model's changing under the hood every day, like your prompt that worked yesterday might not work today.
00:26:38.559 --> 00:26:43.519
And uh I just think it's a risky thing to just constantly pull the model out from under people every day.
00:26:43.839 --> 00:26:45.200
Just like a lot to do.
00:26:45.440 --> 00:27:00.960
Um so so on that note, I wanted to dive a little bit more on this kind of idea that like a lot of people right now are are kind of really building up a lot of skills, you know, plugins and different tools connecting MCPs.
00:27:01.039 --> 00:27:09.680
And you know, Droid actually just released that tool to kind of uh clean up you know sessions so that they don't preload everything, they dynamically pull in tools as they come.
00:27:09.759 --> 00:27:16.480
You know, this is a notion called tool search that you know Codex is implementing some form of it, you have Claude implementing some form of it.
00:27:16.640 --> 00:27:32.000
Um my my personal take is that like tool search is a crutch for people loading in too much, um, because if the model has to find the tool and if it's gonna use that tool every time, then you're actually burning more tokens finding the tool than just preloading it.
00:27:32.079 --> 00:27:38.160
But you know, if you're not gonna use every tool, then you save context by you know pre by like deferring the loading.
00:27:38.240 --> 00:27:45.599
Um but if you're doing that, then you're losing efficiency, and then you know, we measure context usage, but we don't necessarily measure outcomes and quality.
00:27:45.680 --> 00:27:55.359
Um, so for me, I think we're in this awkward pay like phase where tools kind of suck, they're inefficient, people are loading too many of them, and we have to have these kinds of uh dynamic loading to make them work.
00:27:55.519 --> 00:28:02.160
Um But besides tools, there's skills, and skills kind of rely on certain tools, and people are loading in way too many skills.
00:28:02.240 --> 00:28:03.839
Like some people have like hundreds of skills.
00:28:03.920 --> 00:28:08.720
So just like uh you know, round the round table, like Adam, do you have a lot of skills set up in your setup?
00:28:08.799 --> 00:28:10.400
Like, do you use like dozens of skills?
00:28:10.480 --> 00:28:11.039
Are you very little?
00:28:11.119 --> 00:28:11.839
Like, what was your skill?
00:28:12.160 --> 00:28:16.079
I'm very I'm a very much minimal context type person.
00:28:16.240 --> 00:28:22.640
And I'm even this way when I'm building agents myself, I I actually like to lean um in that regard.
00:28:22.880 --> 00:28:38.319
But I will say it is a real problem because when you're trying to do when you're trying to build like this agentic workflow that has hundreds of tasks that it could possibly do, and you don't know what a user is going to type in or ask for, like how do you how do you actually build that?
00:28:38.400 --> 00:28:40.559
And so I've got a lot of ideas around that.
00:28:40.799 --> 00:28:50.559
The the one that comes to mind is I really like the way like the codec CLI um kind of approaches this, which is we we let the AI do what it's good at.
00:28:50.720 --> 00:28:55.599
It leans more into like uh terminal commands, it leans more into like writing code when it needs to.
00:28:55.839 --> 00:29:13.599
Actually, I really like that approach to things, and I think if we can start thinking about building our apps from that perspective, like what are the what are the end things that need to happen and how do we actually give the context so the tools know how to go get access to those things, yeah, versus like distinct tools for every particular action.
00:29:13.839 --> 00:29:20.799
You know, I've seen I've seen some um some agents where they'll have like 50 or 60 tools loaded in, and that is never gonna be good, man.
00:29:20.880 --> 00:29:21.920
That's just not gonna be good.
00:29:22.160 --> 00:29:26.799
So I'm I'm a vanilla setup type person for the most part, other than maybe a claw md file.
00:29:26.880 --> 00:29:28.720
That's about that's about my limit at this point.
00:29:28.960 --> 00:29:33.759
Yeah, well, if you have a Claude MD, the thing that's funny is that the model doesn't actually care about it that much.
00:29:34.079 --> 00:29:35.759
It will it will search it periodically.
00:29:35.920 --> 00:29:38.000
I'll be like, go check the Claude MD file.
00:29:38.640 --> 00:29:47.119
I do think actually this this has changed a little bit with 4.7, but I still find tool instruction following is just quite lacking on uh Claude versus GPT.
00:29:47.279 --> 00:29:48.240
What what about you, Ray?
00:29:48.319 --> 00:29:50.880
Are you are you like a skill maximalist at this point?
00:29:51.119 --> 00:29:52.960
Bro, I just raw dog skills all the time.
00:29:53.119 --> 00:29:53.440
All the time.
00:29:53.519 --> 00:29:55.920
Just skill, skills, no, I'm just kidding.
00:29:57.680 --> 00:30:03.359
No, it's like I kind of want to have like an opposing opinion here just for just for contrast sake, but like no.
00:30:03.759 --> 00:30:04.720
Uh I agree.
00:30:04.799 --> 00:30:17.839
I think my question to you guys is I think a lot of these, you know, harnesses, you know, like the Codex app, the web app on Claude, these different places do a lot of that work for us underneath the hood.
00:30:17.920 --> 00:30:19.519
And I guess that's my question to you guys.
00:30:19.839 --> 00:30:23.359
Because I notice when I do do conversations, it doesn't pull in the skills.
00:30:23.759 --> 00:30:29.039
Like and and cursor, they show now like what stuff you're actually pulling into the context window a lot better now.
00:30:29.279 --> 00:30:32.000
So you can tell like it's not even loading the skills and other things.
00:30:32.559 --> 00:30:38.720
Um and you know, they just have them there sitting, and then you can either call them directly or mention them by a keyword.
00:30:39.200 --> 00:30:45.359
And so I I don't um feel like I'm polluting things, but I only have like a handful of them to begin with.
00:30:45.440 --> 00:30:48.799
I I really hate uh having to think about what's in there.
00:30:48.880 --> 00:30:51.519
Um but I usually they're around a lot of business needs.
00:30:52.079 --> 00:30:55.759
Uh and I uh I'm spending less time thinking about the skills.
00:30:55.839 --> 00:31:04.880
I only turn stuff into skills that are repeatable workflows, and kind of the same with OpenClaw, like it will call it whenever I need something, but it's not necessarily always a thing I call.
00:31:05.599 --> 00:31:14.400
Um and so I I think uh I don't work with the API directly right now with like my writing my own agency carne, so that would probably be a pain in the butt.
00:31:14.480 --> 00:31:18.400
But um yeah, I'm curious to know, Eric, like how does that actually happen nowadays?
00:31:18.480 --> 00:31:23.759
Is what's like have they accounted for that issue um in these modern tool sets, or what is your experience?
00:31:24.000 --> 00:31:40.000
Well, I mean, skills the way they're done right now is that there's basically like um the descriptions of them are loaded to the model up front, and so all the description if you've if you have a lot of skills, then that description bloat can be real and you can actually like drain a lot on on your context usage.
00:31:40.160 --> 00:31:42.000
So too many skills does hurt.
00:31:42.160 --> 00:31:48.880
Um, but then there's also like the models have a tool in the harnesses that let them invoke these skills.
00:31:49.039 --> 00:31:54.559
Um, and so the model can choose to go and like since they know the description, they're like, oh, you know what?
00:31:54.640 --> 00:31:58.319
Like that sounds like a good fit for that skill based on the description, and they'll invoke it.
00:31:58.480 --> 00:32:03.440
The thing is, most models don't do this proactively, only codex seems to do it reliably.
00:32:03.599 --> 00:32:07.920
But then the thing is like, so so you have to be careful, like you know what model are you using?
00:32:08.079 --> 00:32:10.000
Are your skills kind of generic?
00:32:10.160 --> 00:32:16.559
And so if you're using the skills for Claude and you switch to Codex, like you have to be careful because the behavior around skills changes.
00:32:16.720 --> 00:32:27.200
Um, and then if you start having the model dynamically call skills, which like technically Claude is able to do, but just doesn't do it well, um, well, like you have to think about what's in those skills.
00:32:27.279 --> 00:32:29.680
Like, are there skills that conflict with each other?
00:32:29.839 --> 00:32:34.640
Are there skills that actually shouldn't be there at all and don't belong in this specific instance?
00:32:34.880 --> 00:32:43.680
Like, I think the way that you gotta have to think about this in the optimal way is that like you should kind of preload in a given chat which skills the model should be able to use.
00:32:43.759 --> 00:32:47.519
Um, but that's like a lot of overhead for the user, which is why it doesn't exist.
00:32:47.920 --> 00:32:56.160
So, you know, there's all this balance of like how much upfront work do you want the user to do versus what the model's gonna do, and you pay you pay costs for everything that you defer from the user.
00:32:56.240 --> 00:32:57.200
So that's kind of where I'm at.
00:32:57.440 --> 00:32:59.599
And that and one of the you touched on two things there.
00:32:59.759 --> 00:33:06.799
Um the first being when the model changes or the harness changes, there's very little portability between them.
00:33:06.960 --> 00:33:15.599
So you end up like in the situation where your skills kind of become fixed to a model, and for some that might not be you know the the biggest issue whatsoever.
00:33:15.759 --> 00:33:32.079
But the other thing that I've been thinking about is um there is a there is a latency issue here too, because what ends up happening is now you have to go through this whole reasoning step to determine what the intent is before you can actually begin actioning on the the job.
00:33:32.319 --> 00:33:37.200
And that may just be a necessary evil right now, but I feel like there's got to be a better way in the future.
00:33:37.440 --> 00:33:44.400
And then I've also played around with this idea of like bundles, so basically a skill being paired with like unique tools for that skill.
00:33:44.559 --> 00:33:48.960
Like, how can you actually kind of bundle these things together and then pull them all in?
00:33:49.039 --> 00:33:56.640
Um anyway, I think there's a lot of like, and I'm sure there's folks that are probably doing that already, but that's one of the things I've been uh thinking about experimenting with.
00:33:56.799 --> 00:34:09.920
Because I do think there is some value to having that intent to context building step, but I do worry that having that as part of the large model loop and trying to figure it out, it's just it is added overhead.
00:34:10.000 --> 00:34:13.280
And a lot of times you want the agent to be faster than you can do it.
00:34:13.440 --> 00:34:22.719
And when I start when I start seeing, hey, I want to do this particular thing, and it takes longer than if I would have just done it, and somebody's like genetic system, it starts to lose its value.
00:34:23.039 --> 00:34:25.440
What model are you using where you're feeling this the most?
00:34:25.679 --> 00:34:28.159
Uh I would say typically the anthropic models.
00:34:28.880 --> 00:34:29.280
Yeah.
00:34:29.440 --> 00:34:35.280
I I find honestly, I was using 4.7 the other day, and I was like, wow, this is like extremely slow.
00:34:35.440 --> 00:34:41.280
Like this model is just thinking and thinking and thinking, and it's it just overthinks every tool call.
00:34:42.159 --> 00:34:43.119
I just don't enjoy it.
00:34:43.199 --> 00:34:51.360
And and and that like the reason it's thinking so much because I said it's XI, but like that's what Anthropic says is what you should use like Opus 4.7 on.
00:34:51.440 --> 00:34:53.199
So like I'm using the recommendation.
00:34:53.679 --> 00:34:59.760
You know, I I'm using GPT-5.5 a lot, and sometimes I switch down to low and it just blazes through things.
00:34:59.840 --> 00:35:03.360
Like, I just don't find this issue at all with that model, and that's what I really like about it.
00:35:03.519 --> 00:35:08.880
Um, so even though it's a big model, like slow tokens, it's fast, which is quite nice.
00:35:09.119 --> 00:35:19.360
Um but yeah, I I think like just to circle back a little bit on your thing for the for the skills, because you were saying, like, oh, you know, maybe there's not like that much of an issue between models.
00:35:19.519 --> 00:35:30.800
The thing I noticed as well, like I had skills that worked well for GPT 5.4 initially, and then 4.7, Opus 4.7 came out, and it just screwed up the skill completely, didn't do it well at all.
00:35:30.960 --> 00:35:38.880
And I found like, okay, like certain words, the way that they were said, like the models hook hold on to them in different ways.
00:35:39.039 --> 00:35:48.880
And so I had a thing where okay, like 4.7, the well, the model had to go get some information and then prompt another model with some work to do.
00:35:49.119 --> 00:35:53.119
But because of the way it was worded, 5.4 was like, yeah, that's fine, it's just a quick check.
00:35:53.199 --> 00:35:54.719
I'm gonna do a couple things and go.
00:35:54.800 --> 00:36:02.400
But 4-7 was like, no, no, I need like a complete understanding, and so it would just go off for like minutes at a time before it did step two.
00:36:02.719 --> 00:36:05.280
Um, and it was just broken, the rogue flow is just broken.
00:36:05.360 --> 00:36:14.800
So I had to like change the prompt to like lighten the requirement for 4.7's like more anal retentiveness uh behavior, and then it was able to work.
00:36:14.880 --> 00:36:33.199
So so you you have like keywords that like and 4.7 loves this word as like load bearing, um, that they could be very load bearing for the model and and and like really change how the model behaves, and you have to really be careful because like some models will be way more sticky on certain words than others, and some models will just ignore all the words and then just not do any of it.
00:36:33.280 --> 00:36:36.480
So, yeah, I I think that's yo does like very tricky.
00:36:36.559 --> 00:36:42.320
Yeah, I heard you need a skill for your skills so that you can run your skill files with other skill files for your models.
00:36:42.719 --> 00:36:49.119
Yeah, I actually though, yeah, you need you need a skill that primes each model to how to use different skills.
00:36:49.280 --> 00:36:54.880
Uh I I guess we're getting too silly there, but like I I think that's why like it's it's tough to change models too much.
00:36:54.960 --> 00:37:05.519
And if you like listen to like you know, AMP's philosophy around like tuning the prompts, the tools for each model, and like take away that choice from the user, like I think it makes sense.
00:37:05.679 --> 00:37:20.400
Like a year ago, I would have said, well, the models are converging, you know, and so none of this matters, but over time, I think this part of that's true, but also part of it is really not, and and like the models really are diverging in how they listen to instructions and and behave.
00:37:20.480 --> 00:37:27.440
And I think there's a lot to like do, a lot of alpha and just really tuning your workflows to be as lean as possible and optimize for specific models.
00:37:27.679 --> 00:37:30.000
I think it's kind of the best way forward right now, anyway.
00:37:30.159 --> 00:37:33.280
Yeah, and I think there's two two two major schools of thought there.
00:37:33.440 --> 00:37:41.840
It's like, are we gonna let engineers have the ability to tune and manage things for themselves, or do we want the harness to do it?
00:37:42.000 --> 00:37:55.599
And like I'm much more of a tinkerer, so I like having a bit more flexibility myself, but I do I do I have come around a bit more on AMP strategy, even though they've they've come the other way and started adding additional models and things to their stuff.
00:37:55.760 --> 00:37:59.280
So but their additional models are like in different workflows, right?
00:37:59.360 --> 00:38:06.320
So they'll have like a rush mode, they have a deep mode, a smart mode, um, and each one is a specific model that it uses and is tuned for.
00:38:06.480 --> 00:38:13.440
So like depending on how interactive or in a hurry you are, you want to use certain ones, and if you're if you want to just let it go deep, you use the different one.
00:38:13.679 --> 00:38:24.400
Like, I mean, I I still prefer my my own setup, but like, you know, I I I think for someone who doesn't want to think about like the setup, I think it's a good way to go.
00:38:24.639 --> 00:38:30.239
You know, we're getting to a point where like, you know, if you really want to be on the bleeding edge, you have to tinker.
00:38:30.400 --> 00:38:32.559
Um, but then there's the diminishing returns.
00:38:32.639 --> 00:38:40.559
Like, if if your job is not to do this tinkering, then how much of your work is tinkering versus doing the work that you're supposed to do?
00:38:40.719 --> 00:38:49.039
And it's like a huge challenge, like because you have to stay up to date with the model changes, like when we're making a podcast that like every episode we have like new models coming out, which is crazy.
00:38:49.199 --> 00:38:59.119
And and it's just hard to stay up to date and and you have to tinker, but like how much tinkering is actually worth doing and you know, versus what what you can just grab off the shelf and you have to really ask yourself that question.
00:38:59.360 --> 00:39:02.000
Yeah, and at what point is it like, do I even care to try a new model?
00:39:02.079 --> 00:39:05.440
Because what I'm using right now is fine and uh and it's doing what I need it to do.
00:39:05.599 --> 00:39:07.440
Like it is it is tricky for sure.
00:39:07.760 --> 00:39:12.559
I think that's tough though because I mean what models do you guys gravitate to?
00:39:12.639 --> 00:39:17.199
Because I feel like we're trying a lot of them and we're definitely at the buffet table.
00:39:17.519 --> 00:39:24.480
I would just love to hear everyone's like, you know, I used X, but now I'm using Y, or what's your kind of like state of this week?
00:39:24.800 --> 00:39:36.880
I mean, I've I've been on 5.5, you know, since launch, and I've kind of tried I've been actually escalating my use of it more and more, and and using five five low in certain cases because it's just so fast and efficient.
00:39:37.119 --> 00:39:42.000
Um and and I've just been trusting it, it's reliable, it does what I need.
00:39:42.159 --> 00:39:48.960
Um, even you know, UX-wise, I found like it does a decent job in many cases, it lays out information really well.
00:39:49.039 --> 00:39:53.920
Like you might need a styling pass, but like the way it designs layouts is is really good.
00:39:54.239 --> 00:40:02.239
Honestly, like it's hard for me to use anything else at this point in just terms of intelligence, reliability, and and efficiency, like, yeah, that's where I'm at.
00:40:02.400 --> 00:40:03.199
What about you, Adam?
00:40:03.360 --> 00:40:04.239
Oh, you're on mute there.
00:40:04.480 --> 00:40:08.719
I would say that the majority of my usage has been on GPT-5.5 medium.
00:40:09.039 --> 00:40:13.760
My anthropic usage over the last two weeks, it's probably been near zero, just to be totally honest.
00:40:13.840 --> 00:40:15.840
Like, I haven't felt much of a need to go back to it.
00:40:16.000 --> 00:40:21.039
And then Composer 2.5, honestly, is just like my dream model for prototyping things out really quickly.
00:40:21.440 --> 00:40:23.280
Like, you can't get you can't get better than that.
00:40:23.519 --> 00:40:29.920
I will say I have been giving GPT-5.5 low uh a bit of a chance, and I've had good results with that.
00:40:30.079 --> 00:40:37.920
I do wonder if that could be a workhorse uh model at some point, like if that could take place, take the place of some of the flash model stuff that I've built in the past.
00:40:38.159 --> 00:40:38.639
Yeah.
00:40:38.880 --> 00:40:40.079
Honestly, it's really fast.
00:40:40.239 --> 00:40:41.519
Like it's it's crazy.
00:40:41.599 --> 00:40:46.480
Like, I've been using it as a as an explore agent, and it just kind of narrows in.
00:40:46.719 --> 00:40:52.400
You give it a short specific task, it it executes way faster than smaller models that you would think are faster.
00:40:52.480 --> 00:41:07.920
Like I've tried comparing it to like GPT Spark or GPT 5.4 low or sonnet or or opus as well, like using it in those cases, and all pretty much all those other models overthink the task and read way too much context and don't actually get to the solution fast.
00:41:08.000 --> 00:41:12.719
And like the fastest one is typically always 5.5 low, which is surprising, honestly.
00:41:12.880 --> 00:41:16.159
Um about you, Ray?
00:41:16.239 --> 00:41:18.719
You're you're you seem like very composer pilled at this point.
00:41:19.039 --> 00:41:27.679
Yeah, I'm composer pilled inside of cursor for coding, but that was also typically done with codecs um app as well.
00:41:27.840 --> 00:41:30.400
So like I'll do codex medium or low.
00:41:30.800 --> 00:41:41.280
And then if if if I needed like really like cross-check a problem with composer 2.5 or just say, hey, can you just take a look at this thing and give me a second opinion?
00:41:41.360 --> 00:41:46.719
I'll go like high or X high just for that one pass on GPT 5.5 is like my expert that I pull in.
00:41:47.119 --> 00:41:55.440
And then um I have other workflows, so for open claw, I have a GPT 5.5 with low or medium uh in fast mode.
00:41:55.840 --> 00:42:10.800
And I just talk to it in open claw using my subscription 200 bucks a month, and I don't even burn through half of it, and it's literally helping me with my health and fitness goals, and it doesn't feel like any different than what Opus was doing, and it's like actually very useful.
00:42:11.199 --> 00:42:24.880
And then I use Codex in the app to drive my computer and do like computer workflows, and that just is like usually medium or yeah, it's usually medium, and you know, that computer use control, all that stuff is just absolutely incredible.
00:42:24.960 --> 00:42:34.719
And I don't think that I think a lot of people are actually sleeping on that new workflow with codex, where I think codex may even be the wrong name for the app because it can do way more.
00:42:34.800 --> 00:42:47.519
I'm actually wanting to give it to like grandparents and parents and just say, talk to this little box here, and then this will get you all your information you need for um you know, like helping you navigate around the computer.
00:42:47.599 --> 00:42:51.519
You know, it's like it's it's that good at controlling the computer, which is kind of my workflow there.
00:42:52.000 --> 00:42:55.280
So, Ray, are you are you doing like a lot of web browsing or things like that with it?
00:42:55.440 --> 00:42:57.599
Are you doing like work on in the web with it?
00:42:58.000 --> 00:42:58.400
Everything.
00:42:58.480 --> 00:43:06.480
So, like literally going to a text message, it opens up and it's writing Apple script to scroll like the text messages and read context for business.
00:43:06.639 --> 00:43:18.079
It goes back into mail, like my actual mail app, because you can only connect one Gmail account in the plugin, but if you can tell it to launch the mail app, it starts reading your mail, and then you know I can uh the computer use is actually insane.
00:43:18.159 --> 00:43:20.159
The way they did it, like too good.
00:43:20.480 --> 00:43:31.440
And yeah, I can control I can do other stuff on my computer because it's it just launches the app in the background and it has a separate mouse that it's moving, and that's why you need to get a Mac, uh Adam, yeah, like ASAP.
00:43:32.559 --> 00:43:36.800
I actually don't think that stuff will work quite as reliably on that's insane.
00:43:36.960 --> 00:43:37.840
That's insane.
00:43:38.079 --> 00:43:44.400
Yeah, yeah, because uh the with with Claude's desk the computer use thing, it takes over the whole machine, you can't do anything else.
00:43:44.559 --> 00:43:45.840
Yeah, yeah, it's painful.
00:43:46.320 --> 00:43:47.280
Yeah, it's it's so painful.
00:43:47.360 --> 00:43:49.920
But here it's it's actually driving the app underneath the hood.
00:43:50.079 --> 00:43:50.800
In the background, yeah.
00:43:51.039 --> 00:43:55.039
In the background, and you see the little cursor moving down, like a separate cursor that's driving that.
00:43:55.519 --> 00:43:56.719
UX on it is insane.
00:43:56.880 --> 00:43:59.760
I think they really killed it on that, and now you can even use all of that.
00:44:00.159 --> 00:44:02.239
That from your phone, which is just insane.
00:44:02.480 --> 00:44:03.840
You can just ball it.
00:44:04.079 --> 00:44:07.039
Hey, check check my email app on my phone on my computer.
00:44:07.440 --> 00:44:09.280
I was like, can I I have this thought?
00:44:09.360 --> 00:44:12.000
Let me just punch it into my machine at home and just drive.
00:44:12.320 --> 00:44:15.360
You know, yeah, I want more Mac now.
00:44:16.079 --> 00:44:18.559
Yeah, if you haven't tried it on Mac, like you really gotta go back.
00:44:18.639 --> 00:44:21.360
The definitive experience on Codex app is is definitely on Mac.
00:44:21.599 --> 00:44:24.480
Like you can't use it's just not the same on Windows, unfortunately.
00:44:24.639 --> 00:44:25.280
But yeah.
00:44:25.920 --> 00:44:29.440
Have you guys been using goal mode at all in in Codex?
00:44:30.000 --> 00:44:31.199
How about you, Ray?
00:44:32.239 --> 00:44:33.760
Not as much as I would want to.
00:44:33.840 --> 00:44:34.400
I don't know why.
00:44:34.480 --> 00:44:36.719
I just feel like I want to drive more things right now.
00:44:36.880 --> 00:44:40.719
Um, you know, I I feel like Yeah.
00:44:40.960 --> 00:44:41.280
I don't know.
00:44:41.360 --> 00:44:46.559
I'll let you guys speak on more on your experience, but as of right now, for whatever reason, I want more hands-on control.
00:44:47.360 --> 00:44:48.400
Have you tried it, Adam?
00:44:48.719 --> 00:44:53.679
I've only ran a couple things through it, and I was like, um, I like where it's heading.
00:44:54.159 --> 00:44:59.280
And I I like the idea because I it's like you're going to bed, I'll kick off something, come back and see where it got to.
00:44:59.360 --> 00:45:00.559
So I I like that.
00:45:00.800 --> 00:45:04.320
And I do think there's something to it, but I haven't done anything meaningful with it yet.
00:45:04.559 --> 00:45:12.880
Like I've modified some stuff in my Shopify store, uh, which is really nice actually, because it would it it did a really good job at coming through.
00:45:12.960 --> 00:45:20.239
And of course, I still have to bring it back in and then iterate on it from that point, but it kept me from having to like baby step it across.
00:45:20.400 --> 00:45:26.880
Like it got a high-level goal, and I was able to go away for a few hours, come back, and it was further along than I would have been.
00:45:27.199 --> 00:45:31.360
And it also helped, you know how like when you get stuck in that dopamine rush of like, oh, this is so cool.
00:45:31.440 --> 00:45:32.559
I'm working, it's 2 a.m.
00:45:32.800 --> 00:45:33.679
It's like, okay, you know what?
00:45:33.760 --> 00:45:41.519
I'm gonna kick off a goal, and then I'm just gonna let it run for an hour or two, and I'll come back and now I can go sleep because I know something's working.
00:45:42.000 --> 00:45:56.480
So if you're not familiar, because we just jumped into it, but goal is a new feature in Codex app where basically the way that it works is there's like an outer loop where you have like a goal, and then when the model's done working, it kind of gets injected, like, hey, like did you complete this goal?
00:45:56.559 --> 00:46:02.559
And the model's like, uh yeah, I completed it and stops, or it's like, no, actually, there's more to do to hit that goal and it keeps going.
00:46:02.719 --> 00:46:04.639
Um, so it doesn't let it stop.
00:46:04.719 --> 00:46:07.440
It's like it's like a thing that injects another turn.
00:46:07.760 --> 00:46:18.800
If you use codecs in the past when they had like a queuing as the main default, if you were to type like 10 continues in in the in the queue, it like you just say continue, continue it every time it would hit the continue, it would get another continue.
00:46:18.960 --> 00:46:21.360
It's basically the same thing, but like a little more sophisticated.
00:46:21.519 --> 00:46:24.960
Um so it's got like a like a double-edged, right?
00:46:25.039 --> 00:46:32.159
Like on one end, you know, if you have a well-structured goal and you take time to really craft your goal and have good clear end conditions, you can get a lot done with it.
00:46:32.400 --> 00:46:35.280
Um, the thing is that it leans a lot on compaction, right?
00:46:35.360 --> 00:46:45.679
So if you have a lot of context and it's necessary and it's kind of hard to get the whole picture, um, you know, you you might struggle a little bit with the goal setup because the model has to come continuously compact and continue.
00:46:45.840 --> 00:46:59.760
If you have like a very clear, like iterate on this specific thing, it's really optimal for because then it's like there's not that much context to load in, there's clear conditions on what to do, it can try different approaches, it remembers what it tried through compaction, um, and then can keep iterating.
00:46:59.840 --> 00:47:01.360
I think that's like a kind of ideal way.
00:47:01.519 --> 00:47:06.960
If you have a big plan and you give a goal, it'll kind of keep plugging along until it completes it.
00:47:07.119 --> 00:47:09.039
Um, so I think that's like really cool.
00:47:09.199 --> 00:47:38.320
I do think though, like, you know, I've been you know hammering on orchestration as a concept for a while, and I think orchestration is still like a better approach than doing goal because you have like a higher level executor that kind of understands the problem, breaks down tasks, and issues it, and you know, the way that you handle um context efficiency with like a master context and subcontext is kind of a better setup, in my opinion, uh, than goal, which is just like one train that just keeps compacting and continuing.
00:47:38.480 --> 00:47:44.480
Um, you can combine compaction, sorry, um goal plus orchestration, but in many cases, that's like overkill.
00:47:44.559 --> 00:47:50.639
Like if you have like a crazy huge project um and you need it to run for like days at a time, then that's kind of a good way to go.
00:47:50.800 --> 00:48:00.239
But you know, I I think it's really hard to set up a task that that's like well-defined enough to kind of run that long in a good way that's productive, because oftentimes it'll degrade and kind of fall off.
00:48:00.400 --> 00:48:02.880
But um so I actually yeah, go ahead.
00:48:03.199 --> 00:48:08.159
It would be a fun test to do the same thing in an orchestration and a goal.
00:48:08.320 --> 00:48:08.480
Yeah.
00:48:08.639 --> 00:48:13.679
To just see the di the variance between like something big, like and then like run it, see what would happen.
00:48:13.840 --> 00:48:15.360
I I would love that experiment.
00:48:15.519 --> 00:48:16.559
Yeah, anyway, keep going.
00:48:16.639 --> 00:48:17.280
I was thinking about that.
00:48:17.440 --> 00:48:17.760
Yeah, yeah.
00:48:17.840 --> 00:48:21.199
No, I mean I was running some experiments with it like last week with it a bit.
00:48:21.280 --> 00:48:25.360
Um, and one of the things I actually did, so um, this is like a little nerdy side.
00:48:25.440 --> 00:48:35.119
Um, so I was uh I I I saw this new release came out, and it's the um if you've ever played uh Zelda Twilight Princess game came out, um, there was a decompilation effort.
00:48:35.199 --> 00:48:41.360
So someone spent five years kind of reverse engineering the code for this game, which is like a GameCube and Wii game.
00:48:41.519 --> 00:48:45.360
Um, and then they made it run in a way that like compiles everywhere.
00:48:45.440 --> 00:48:53.440
So you can run it on like uh an iPhone, you can run it on like uh any device really uh because it uses like web web GPU to render.
00:48:53.599 --> 00:48:55.039
Um it's quite cool.
00:48:55.199 --> 00:48:58.800
Uh so I was like, okay, well, you know, we got this nice DComp, it can run really well.
00:48:58.960 --> 00:49:01.119
It has like high frame rate support.
00:49:01.280 --> 00:49:02.960
What if I can get this to run in VR?
00:49:03.039 --> 00:49:06.320
Uh, because like there was like an old mod that came out a while back that where that was possible.
00:49:06.480 --> 00:49:10.719
So I like you know, use repo prompt, I set up a plan with it, I have this D plan.
00:49:11.119 --> 00:49:19.760
And then since it's like a VR is like kind of only for Windows, I like well hopped to my Windows computer, gave it the plan instead of goal mode, and then I just let it plug away.
00:49:19.840 --> 00:49:26.880
And it took a couple of iterations um where like I it executed, went back, did the plan again, and came back.
00:49:27.039 --> 00:49:30.400
Um, but with that clear plan, it I it got it done.
00:49:30.480 --> 00:49:34.320
Like we we we we shipped it, it worked in VR, and I was like, wow, like we we got it.
00:49:34.559 --> 00:49:39.280
Like I didn't need some steering a little bit, but like the goal got through like huge chunks of multi-hours of work.
00:49:39.440 --> 00:49:43.519
I think like around four hours was like kind of the peak of where I was like trusting it.
00:49:43.679 --> 00:49:55.760
One time my my goal was bad, and then the model got stuck in a loop and it just kept saying uh can't complete goal stuck, and then it would keep going and looping, and I think they're that's a known issue they're fixing.
00:49:56.000 --> 00:49:58.800
Um but yeah, like you can you can get a lot done.
00:49:58.880 --> 00:50:06.400
Uh, but if it fails, if your goal end case is bad, you will burn tokens for no reason on stupid shit where it just keeps iterating on nothing.
00:50:06.559 --> 00:50:08.079
Um, so you gotta watch out for that.
00:50:08.239 --> 00:50:13.119
Um, but yeah, I think these little primitives are coming together, and yeah, orchestration is the thing that will come too with it.
00:50:13.199 --> 00:50:17.679
And I think you probably want both, they're both kind of in individual pieces that make sense.
00:50:17.760 --> 00:50:22.960
It but I think orchestration is the kind of a more important primitive in my opinion, and uh we'll see where we go from there.
00:50:23.199 --> 00:50:24.079
That's so cool.
00:50:25.199 --> 00:50:28.079
Uh so I think uh, you know, we're getting uh to the end here.
00:50:28.159 --> 00:50:30.159
I just wanted to end on one quick note there.
00:50:30.239 --> 00:50:43.760
So it seems that uh Carparthy, if you're not familiar, one of the legendary um co-founders of OpenAI who went to Tesla, worked on self-driving there, uh, then went off to teaching for a while.
00:50:43.840 --> 00:50:46.800
Now he's at Anthropic, uh, which is crazy.
00:50:47.039 --> 00:50:49.440
Um so curious to see what he does there.
00:50:49.599 --> 00:50:52.559
Seems like it's gonna be uh some work on self-improvement.
00:50:52.719 --> 00:50:57.039
So seems Anthropic's really the hot lab everyone seems to want to join at this point.
00:50:57.119 --> 00:51:00.320
Um, so kind of interesting to see him just kind of hop over there.
00:51:00.480 --> 00:51:03.599
Um interesting thoughts on you know why making the jump now?
00:51:03.679 --> 00:51:06.239
And um, yeah, what what do you think, Ray?
00:51:06.320 --> 00:51:08.320
Why why do you think he's he's making the jump now?
00:51:08.559 --> 00:51:15.679
Uh I'll steal what was said earlier in the green room is that like he probably went to anthropic just so he can get access to mythos, right?
00:51:17.760 --> 00:51:18.159
Right on.
00:51:18.480 --> 00:51:25.039
I mean, you know, as bottles are getting closed off, like if you want to stay on the frontier, the only place to do it is at a lab, it seems like, right now.
00:51:25.119 --> 00:51:32.719
So I think I think there are a couple things that as an engineer, you don't as the systems keep evolving.
00:51:32.880 --> 00:51:38.559
Like, for me, I don't have access to Apple's internal source code, so I can't see what's going on there.
00:51:38.719 --> 00:51:43.840
You know, you just get the result and you use the thing, and you're literally a black box, and you kind of have to figure out what's going on.
00:51:44.079 --> 00:51:50.000
And I feel like in that podcast he did with Sarah Guo, uh, he had a great podcast, they had a great discussion.
00:51:50.079 --> 00:51:57.679
And in that podcast, he kind of sort of revealed that he misses out like being in the lab and talking to these people and figuring out these new techniques.
00:51:57.920 --> 00:52:02.960
And she kind of casually like laughed and said, Haha, uh, it looks like he's looking for a job, folks.
00:52:03.119 --> 00:52:09.280
If you guys are hiring, you know, let and now it's like maybe he was also already talking to Anthropic over a month ago plus about that.
00:52:09.440 --> 00:52:14.400
And I mean it takes time to do in these labs, sounds sounds like that's a plausible case, yeah.
00:52:14.719 --> 00:52:19.440
And so uh to me, it makes a lot of sense because he has all this knowledge.
00:52:19.760 --> 00:52:25.119
He drops like the wiki thing, he drops all this knowledge in terms of like, why don't we just try this?
00:52:25.199 --> 00:52:30.400
Just like as a kid who's playing with this stuff, and it's pretty obvious to him, but it's not obvious to the rest of the people.
00:52:30.719 --> 00:53:02.400
And I don't I think he just forgets how much experience he has because he just drops them like everyone should kind of know this stuff, and it's like and everyone just goes bonkers or like, oh my god, I built my auto research loop, I built my you know, wiki brain for my whatever thing, and it's like that's only scratching the surface when you have uh other researchers who are very deep in all these different topics sitting in a room with you, the iteration just to being co-located is actually uh underrated, and I think that's what a lot of people really forget slash miss out.
00:53:02.559 --> 00:53:26.320
Uh and it's sort of like the sparkle of what's been happening in the Bay Area in general is that people are uh are co-located and they're they're able to move a lot faster because it's just that at a coffee, you're thinking about it, you're on a run, you come back, and then you you you're sitting down with everyone late at night, and then just you know, something magical happens and uh or something gets revealed from a competitor and you start playing with it, and then you get inspired, and you're like, Oh, since that can happen, now this can happen.
00:53:26.480 --> 00:53:35.519
And so But you know, there there is a double-edged short to that though, which is it leads to a lot of groupthink and like everyone trying the same ideas and and just doing the same things.
00:53:35.679 --> 00:53:48.159
Um, so sometimes you know that that does really push and refines those ideas to like the the natural limit, but oftentimes, like if everyone's kind of convinced this is the right idea, they don't think about other ideas which might be better.
00:53:48.239 --> 00:53:49.920
Um, so it's a bit of a risk there.
00:53:50.000 --> 00:54:01.840
But I'm not saying that's all happening there, but you know, I've seen it, you know, being able to try my own ideas, working on coding harnesses that like people are just not trying certain things, and I'm like, it's works pretty well, and I'm curious why not.
00:54:01.920 --> 00:54:02.800
Um so anyway.
00:54:03.039 --> 00:54:04.559
Any uh thoughts to add there, Adam?
00:54:04.719 --> 00:54:07.760
Yeah, real quick, I just say like he strikes me as a technologist.
00:54:07.920 --> 00:54:15.119
Like it you there comes a point in your life where you're like, yeah, uh he's gonna get the upside of the IPO that's inevitable.
00:54:15.280 --> 00:54:22.639
Yeah, he's gonna get access to stuff, but really at the end of the day, like we all just want to work on things that matter and are gonna make an impact to human civilization.
00:54:23.119 --> 00:54:35.679
Like we you know, there's only so many hours that we have in our life to actually put on things, and do we really want to spend it uh in things that maybe we think we could be spending and having a higher impact somewhere?
00:54:35.760 --> 00:54:38.239
So honestly, I think that's probably where he's at in his life.
00:54:38.320 --> 00:54:43.039
He's like, because he he mentioned in that tweet that I saw where he was like, I'll get back to teaching.
00:54:43.199 --> 00:54:50.559
I think he's seeing this as like, okay, there's only a short window of time where this stuff is actually gonna be like a green space.
00:54:50.639 --> 00:54:55.920
Like uh you know, basically, like I have an opportunity to help shape it, to help drive it, to be a part of it.
00:54:56.079 --> 00:54:57.679
Like, why would I not do that now?
00:54:57.840 --> 00:54:59.760
Teaching will be there when when that's over.
00:54:59.920 --> 00:55:01.199
So that's where my head would be.
00:55:01.280 --> 00:55:19.440
And I honestly I'm in a very similar boat, uh not on his level, but I want to work on stuff that's impactful and meaningful, and you know, I and and so whatever company or thing that I join, it's going to be it's gonna be that's gonna be a thing that I care a lot about because I I wanna I wanna build things that people use.
00:55:19.679 --> 00:55:21.519
Yeah, yeah, I definitely feel the same way.
00:55:21.679 --> 00:55:22.880
Like it's an interesting time.
00:55:22.960 --> 00:55:29.679
There's not many moments in history where like change is at the kind of velocity that it is right now.
00:55:29.760 --> 00:55:36.960
And right if you're curious and you're working with AI, like there's just so much to learn, it's just never ending really the depth to which you can learn.
00:55:37.199 --> 00:55:43.440
We're kind of discovering whole new workflows and that that are like changing and being deprecated so quickly.
00:55:43.599 --> 00:55:56.960
Um, you know, you you like if you're okay with throwing out what you were doing yesterday, like it's the perfect time to be building, and I think it's hard to it's hard to find anything else as a technologist, kind of that can sp like it's the same the same kind of curiosity.
00:55:57.119 --> 00:55:57.360
All right.
00:55:57.679 --> 00:56:04.480
So, how would you guide people right now who are kind of entering this new era of like agentic engineering, quote unquote?
00:56:04.639 --> 00:56:13.679
And you guys have spent a lot of your recent careers, you know, kind of shifting in this direction of playing with agents, using them, actually trying to make products with them.
00:56:14.079 --> 00:56:17.119
Uh, where do you think people should kind of start uh thinking, right?
00:56:17.199 --> 00:56:19.280
Because I feel like it's a different way of thinking than before.
00:56:19.360 --> 00:56:28.079
Because before you'd like, go learn the code, go learn learn these systems, you know, build them yourself, uh, and then maybe you can scale your own system or something like that.
00:56:28.239 --> 00:56:30.719
Um I feel like these systems behave very differently now.
00:56:31.119 --> 00:56:35.280
So this is gonna sound very dumb, but I've been in I've been a technologist for a lot of years.
00:56:35.519 --> 00:56:49.119
There was a transition from synchronous to asynchronous, and you would not believe how hard it was to get engineers that weren't familiar with the asynchronous way of thinking to understand how asynchronous technology works.
00:56:49.280 --> 00:56:55.440
And I know this isn't it's a very loose like correlation or uh example here, but agentich is like that.
00:56:55.599 --> 00:56:57.679
Like it is so different in thinking.
00:56:57.760 --> 00:57:01.360
It is not you put in something and you get the exact response back.
00:57:01.519 --> 00:57:11.920
You have to design your systems around these uh probabilistic like behemoths of uh these large language models that may or may not do what you ask it to do.
00:57:12.079 --> 00:57:16.800
And it is actually like one of the most exciting times to actually be in technology.
00:57:16.960 --> 00:57:22.719
So if I if I was just coming into it today, I'd be spending all my time in understanding that.
00:57:22.880 --> 00:57:39.199
I'd be I'd be trying to understand what does it mean to actually work with probabilistic software and how can we get the outcomes that we want from it, and how can we actually start chaining these things together to start bringing systems that are these point solutions that are all over the place into end-to-end complete solutions.
00:57:39.360 --> 00:57:40.880
That's that's where I would spend my time.
00:57:41.039 --> 00:57:41.519
Yeah.
00:57:41.840 --> 00:57:52.880
I think for me, like uh one thing I'll say is like you know, you could be building crazy systems, but I think like just using them is already a whole thing, and you're trying to do your work with them.
00:57:53.039 --> 00:58:03.840
I think my advice to that is to try and just spend maybe an hour or two every week uh to just look back on how you've been working with with the tools this week and see like, okay, like well, what could be different next week?
00:58:04.000 --> 00:58:05.199
What can we automate this week?
00:58:05.360 --> 00:58:07.760
Like uh, you know, what what did I do several times?
00:58:07.840 --> 00:58:08.719
And there's tools.
00:58:08.800 --> 00:58:11.119
Um, one one of them in particular is called Cass.
00:58:11.360 --> 00:58:19.920
It's uh made by this guy who uh I think his GitHub, something something Dickleworth, uh Jeffrey, Jeffrey Manual on X, if you find out follow him.
00:58:20.079 --> 00:58:42.559
That guy has like he goes a little hard on the orchestration, and I don't fully agree with his his approaches, but he he makes some cool tools, and one of them is is CAS, where basically it allows you to kind of really search through all of your old threads really easily, allows your agents to kind of work through what did you do in codecs, what did you do in Claude, all these things, and helps you find like inefficiencies and things that you you could probably do better.
00:58:42.639 --> 00:59:00.480
And um, you don't have to use that tool, you know, just but just the idea of like looking back on your threads, looking back on how you worked, reading your threads and seeing like what could be better, what could be automated into a skill, uh, and trying to like just like reshape a little bit like the way you're prompting and doing things and trying to find small points of leverage.
00:59:00.559 --> 00:59:09.519
I think that's like you know, an important thing to do is you you just gotta constantly be reevaluating how you're working with these tools every week and um taking the time to do it is important.
00:59:09.760 --> 00:59:10.719
I love it.
00:59:11.440 --> 00:59:12.400
What about you, Ray?
00:59:12.480 --> 00:59:13.199
What's your advice there?
00:59:13.440 --> 00:59:14.000
Just prompt.
00:59:14.320 --> 00:59:18.639
No, I think they're um I've been talking to a lot of people and kind of getting them started.
00:59:18.800 --> 00:59:26.880
And to be honest, I think it's it doesn't really require code, and it's just requiring you to go in and think about a workflow.
00:59:27.119 --> 00:59:40.079
And so my challenge to a lot of people, even just getting started with AI, is like, you know, launch the Codex app actually and and go through the setup to control the computer and then just say, you know, I what is something that I'm doing?
00:59:40.159 --> 00:59:41.119
You know, check my mail.
00:59:41.440 --> 00:59:44.480
What is something I'm doing every morning that can save like five minutes of my time?
00:59:44.719 --> 00:59:46.320
And just start something basic like that.
00:59:46.400 --> 00:59:57.760
You'll be very surprised once you start thinking in this way and you watch it interact and do stuff, you'll then get the imagination, like, if the computer can then go do that, why don't I just delegate this to it?
00:59:57.920 --> 00:59:59.199
And why don't I delegate this?
00:59:59.519 --> 01:00:06.320
And it may start to get you kind of thinking like, why did I even sit down to do certain things all the time uh repetitively?
01:00:06.480 --> 01:00:11.840
And that's how you start to realize uh where the what's actually happening as a machine, you know, uh underneath it.
01:00:12.159 --> 01:00:20.159
So that actually reminds me, I was having a conversation with a friend of mine who's a uh he's a he's an animator for video games, um, and he works at a small studio.
01:00:20.400 --> 01:00:29.039
And you know, he I I I put him onto Codex and he started using it more, and I started describing like some of the automations you could do with like Blunder and other tools.
01:00:29.199 --> 01:00:34.239
Um and he was like, oh man, like he started playing with it and he's he felt the power.
01:00:34.559 --> 01:00:41.039
And you know, the more automations you could add, the more he was like he he compared it to mana if you've played like an RPG.
01:00:41.199 --> 01:00:52.159
Um, you know, if you get like or you watch like you know, Dnd, like you know, mana is this you know pool of energy of magic that you can you cast spells with, and like the more mana you accumulate, the more you can accomplish.
01:00:52.400 --> 01:01:02.480
Um, and like in some ways, like having codecs and these tools like do all these automations, it's like it's like an expression of mana where you have to you have to decide how you're gonna spend it.
01:01:02.639 --> 01:01:07.920
Um, but like it is power that you can assign to your life and do things for you.
01:01:08.000 --> 01:01:12.719
And if you think about it that way, like you try to increase your mana and try to increase how you spend your mana.
01:01:12.880 --> 01:01:14.320
Uh, like interesting thoughts there.
01:01:14.400 --> 01:01:16.639
So just just uh fun ways to compare it, yeah.
01:01:16.880 --> 01:01:17.599
I I love that.
01:01:17.760 --> 01:01:18.719
That's awesome.
01:01:18.960 --> 01:01:19.440
Yeah.
01:01:19.679 --> 01:01:20.000
All right.
01:01:20.079 --> 01:01:22.159
Well, I think uh it's a good place to stop for today.
01:01:22.320 --> 01:01:26.880
So um thanks to everyone for tuning in and uh I've been appreciating the feedback.
01:01:27.039 --> 01:01:28.719
And uh yeah, see you all next time.
01:01:28.960 --> 01:01:29.679
Take care, everybody.
01:01:29.840 --> 01:01:30.559
Cool, take it easy.