Unsupervised Learning x Latent Space Crossover Special

WEBVTT
1
00:00:00.060 --> 00:00:06.020
Today, my partner Jordan and I have a special episode of Unsupervised Learning, a crossover with one of our favorite AI podcasts, Latent Space.
2
00:00:06.420 --> 00:00:11.660
If you're not already a listener, Latent Space is a technical newsletter and podcast by and for AI engineers.
3
00:00:12.160 --> 00:00:19.020
It had over two million downloads in 2024, and it's become a go-to resource for anyone who wants to understand the cutting edge of AI infrastructure, tooling, and product.
4
00:00:19.560 --> 00:00:26.480
If you like this show, it's definitely worth checking out. Given we've all spent a lot of time talking to some of the sharpest minds in AI, we thought it'd be fun to interview each other.
5
00:00:26.900 --> 00:00:37.560
In this episode, we dig into the questions we're constantly thinking about: what surprised us most last year, what we're paying most attention to right now, how we think about defensibility at the app layer, and which public companies we're long or short on.
6
00:00:38.000 --> 00:00:46.400
It's a different kind of episode, and I think you'll really enjoy it. Now here's my conversation with Swyx and Alessio from Latent Space. [upbeat music] Well, thanks so much for doing this, guys.
7
00:00:46.440 --> 00:00:55.740
I feel like we've, uh, we've been excited to do a collab, uh, for a while. I love crossovers. Yeah. Yeah, this, this is great. Like- Yeah... the ultimate meta about just podcasters talking to other podcasters [laughs].
8
00:00:55.780 --> 00:01:03.380
Yeah, it's a lot. Podcast all the way up. I figured we'd have a pretty free-ranging conversation today, um, but brought a few conversation starters to, to, to kick us off.
9
00:01:03.740 --> 00:01:08.760
And so I figured one interesting place to start is, uh, you know, obviously it feels like this world is changing, like, every few months.
10
00:01:09.020 --> 00:01:18.000
Wondering as you guys reflect pa- on the past year, like, what surprised you the most? I think definitely reasoning models. We're kinda on the, on the right here. We're, like, causing- Oh, open source. Oh, that...
11
00:01:18.040 --> 00:01:23.980
Well, I, I, I think there's two. There's, like, the what surprised us in a good way and may- maybe in a, in a bad way, I would say.
12
00:01:24.020 --> 00:01:35.400
In a good way, reasoning models and, um, I think the release of them right after the NeurIPS scaling is dead talk by Ilya. I think there was maybe, like, a, a little it's so over and then we're so back- [laughs]...
13
00:01:35.440 --> 00:01:41.820
in, like, such a short, short period of time. It was really fortuitous timing that, like, right as pre-training died. I mean, obviously, I'm sure within the labs they knew pre-training was dying- Right...
14
00:01:41.920 --> 00:01:50.060
and had to find, uh, something, but it, you know, from the outside it was, uh, it, it felt like one right into the other. Yeah. Yeah, exactly. So that, that was a good surprise.
15
00:01:50.140 --> 00:01:58.240
I would say if you wanna make that comment about timing, I think it's suspiciously neat- [laughs]... that, like... 'Cause, uh, we know that Strawberry was being worked on for, like, two years-ish.
16
00:01:58.740 --> 00:02:03.660
Um, like, and we know exactly when Noam joined OpenAI, and that was obviously a big strategic bet by OpenAI.
17
00:02:04.240 --> 00:02:16.260
Um, so, like, to- for it to transition so, transition so nicely when, like, pre-training is kinda tapped out to into like, oh, now inference time is, is the new scaling law, is, like, conve- very convenient. I, I, I...
18
00:02:16.300 --> 00:02:25.460
Like, if there were an Illuminati, this would be what they planned [laughs]. [laughs] Or if we're living in a simulation or something. [laughs] Yeah. Then you said open source as well? Yeah.
19
00:02:25.500 --> 00:02:32.940
Well, no, I, I think, like, open source, w- yeah, we were discussing this on the negative. Uh, I would say- Mm... the relevance of open source, I was surprised- Specifically open models.
20
00:02:33.320 --> 00:02:42.040
Yeah, I was surprised by the lack- Like the Llamas of the world... um, by the lack of adoption and... Y- I mean, people use it obviously, but I would say nobody's really, like, a huge fanboy, you know?
21
00:02:42.120 --> 00:02:50.640
I think the local Llama community and some of the more obvious use cases really like it, but when we talk to, like, enterprise folks, it's like, "It's cool," you know?
22
00:02:50.680 --> 00:03:03.140
And I think people love to argue about licenses and all of that, but the reality is that it doesn't really change the adoption path of, of AI, so. Yeah, the specific stat that I got from Ankur from Braintrust- Mm-hmm...
23
00:03:03.280 --> 00:03:11.800
in one of the episodes that we did was I think he estimated that open source model usage in work, in enterprises is at, like, 5% and going down.
24
00:03:12.040 --> 00:03:18.280
And it feels like you're basically all these enterprises are in, like, use case discovery mode, where it's like, let's just take what we think is the most powerful model and- Mm...
25
00:03:18.300 --> 00:03:22.260
figure out if we can find anything that works and, you know, so much of, of, of it feels like discovery of that.
26
00:03:22.280 --> 00:03:26.480
And then right as you've discovered something, a new generation of models are out, and so you have to go do discovery with those.
27
00:03:26.520 --> 00:03:32.360
And, um, you know, I think obviously we're probably optimistic that the, uh, that the open source, uh, models increase in uptake.
28
00:03:32.680 --> 00:03:37.980
It's funny, I was gonna say my biggest surprise in the last year was open source related, but it was just how fast open source caught up on the reasoning models.
29
00:03:38.140 --> 00:03:44.140
It was kind of unclear to me, like, over time, whether there would be, you know, a compounding advantage for some of the closed source models- Mm...
30
00:03:44.160 --> 00:03:53.300
wherein, okay, in the early days of, of scaling, um, you know, there was a, a tight time loop, but over time, you know, would, uh, would the gap increase? And if anything, it feels like shrunk. Um- Mm...
31
00:03:53.340 --> 00:04:00.720
you know, and I think, uh, DeepSeek specifically was just really surprising in how, you know, in many ways if the value of these model companies is, like- Mm...
32
00:04:00.760 --> 00:04:08.600
you have a model for a period of time and you're the only one that can build products on top of that model while you have it, like, God, that time period is [chuckles] much shorter than, uh, than I thought it was gonna be a year ago.
33
00:04:08.920 --> 00:04:16.079
Yeah. Um, I mean, again, I, I, I don't like this label of how fast open source caught up because it's really how fast DeepSeek caught up. [laughs] Right?
34
00:04:16.420 --> 00:04:22.960
And now we have, like, I think some evidence that DeepSeek is basically gonna stop open sourcing models. Yeah. So, like, there, there's no team open source.
35
00:04:23.000 --> 00:04:33.600
There's just different companies, and they choose to open source or not. And, um, we got lucky with DeepSeek releasing something, and then everyone else is basically distilling from DeepSeek, and that those are...
36
00:04:33.840 --> 00:04:43.140
Distillations catching up is such an easier- Yeah... lower bar- Yeah... than, like, actually catching up, which is, like, you, you, like, from scratch you're training something that, like, is competitive.
37
00:04:43.340 --> 00:04:47.800
On that front, I don't know if that's happening. Like, basically the only player right now is we're waiting for Llama 4.
38
00:04:48.140 --> 00:04:57.340
I mean, it's always an order of magnitude cheaper to replicate what's already been done than to create something fundamentally new. And so that's why I think DeepSeek overall was overhyped, right?
39
00:04:57.440 --> 00:05:06.620
I mean, obviously it's a good open source new entrant, but at the same time, there's nothing new fundamentally there other than sort of doing it, executing what's already been done really well. Yeah. Right? So.
40
00:05:06.840 --> 00:05:15.640
Well, but I think the traces is, like, maybe the biggest thing. I think most previous open models is, like, the same model just a little worse and cheaper. Yeah.
41
00:05:15.680 --> 00:05:22.060
Like, R1 is, like, the first model that had the full traces, so I think that's, like, a net unique thing in open source. That's fair. But yeah.
42
00:05:22.120 --> 00:05:33.000
I, I think, like, we talked about DeepSeek in the, our end of year 2023 recap, and we're mostly focused on, um, cheaper inference. Like, we didn't really have DeepSeek, like- DeepSeek V3 was out then, and we were like...
43
00:05:33.060 --> 00:05:40.020
That was already, like, talking about fine-tuning, uh, mixture of experts and all that. Like, so very- That's a great receipt to have, to be like, yeah, end of year '23. We were there. Yeah. Let's go.
44
00:05:40.040 --> 00:05:47.280
That's a, that's, that's an impressive one. [laughs] You, you follow the right whale believers on Twitter. [laughs] It's, it's, like, pretty obvious. [laughs] I actually had, like...
45
00:05:47.360 --> 00:05:54.100
Uh, so, you know, I used to be in finance. I, and, and, uh, a lot, a lot of my hedge fund and PE friends called me up. They were like, "Why didn't you tip us off on DeepSeek?"
46
00:05:54.620 --> 00:06:03.140
And I'm like, "Well, I mean, w- it's been there." It's, it's actually, like, kinda surprising that, like, Nvidia- Mm... like, fell, like, what? 15% in one day. Yeah. Yeah. Uh, because of DeepSeek.
47
00:06:03.480 --> 00:06:07.768
And, uh, I, I think it's just, like, whatever the market, public market narrative...
48
00:06:07.808 --> 00:06:13.628
decides is a story becomes the story, but really, like the technical movements are usually one to two years in the making the, before that.
49
00:06:13.908 --> 00:06:20.848
Basically these people were telling on themselves that they didn't listen to your podcast. [laughs] They've been on the end of ear 2023. No, no, no. Like, uh, yeah, we weren't, we weren't like banging the drum.
50
00:06:21.048 --> 00:06:26.948
So like it's also on us to be like, "No, like this, this is an actual tipping point." And I think I, like as people who are...
51
00:06:26.968 --> 00:06:37.608
Like our function as podcasters and industry analysts is to raise the bar or focus attention on things that you think matter, and sometimes we're too passive about it, and I think I was too passive there. Mm-hmm.
52
00:06:37.628 --> 00:06:38.888
I'd, I'd be happy to own up on that.
53
00:06:38.948 --> 00:06:47.828
No, I feel like over time, uh, you guys have moved into this more general role of like taking stances of things that are or aren't important and, you know, uh, I feel like you've done that with MCP of late- Mm-hmm...
54
00:06:47.848 --> 00:06:54.608
and, and a bunch of things. Yeah. Uh, so like the, the general push is AI engineering, you know, like it's gotta- [laughs]... gotta rep the shirt. Um, and MCP is part of that.
55
00:06:54.648 --> 00:07:02.448
But like the, the general movement is what can engineers do, uh, above the model layer to, uh, augment model capabilities, and, uh, it turns out it's a lot.
56
00:07:02.768 --> 00:07:09.808
And turns out it w- we went from like making fun of GPT wrappers to now I think the overwhelming consensus GPT wrappers is the only thing that's interesting. Yeah.
57
00:07:09.828 --> 00:07:14.428
[laughs] I remember like Arvind from Perplexity came on our podcast, and he was like, "I'm proudly a wrapper." Yeah.
58
00:07:14.448 --> 00:07:22.768
Like, you know, it's like anyone that's like talking about like, you know, differentiation, like pre-product market fit is like a ridiculous thing to, to say. Like build something people want and then- Yeah...
59
00:07:22.848 --> 00:07:30.508
over time you can kind of worry about that. Yeah. I, I interviewed him in 2023, and I think he may have been the first person on our podcast to like proudly be a GPT wrapper. Yeah.
60
00:07:30.528 --> 00:07:37.588
And, um, yeah, and obviously he's built a huge business on that. [laughs] Well, now, now we, uh, now we all can't get enough of it. I have another one for- Yeah. On that...
61
00:07:37.608 --> 00:07:44.708
uh, so that was Alessio's one, and we, we prepped individual answers just to be interesting. Um- In the same Uber on the way up. Yeah. [laughs] You just like in the, in different- Oh, I was driving too.
62
00:07:44.768 --> 00:07:48.248
Oh, you were driving. So it was like actually- Oh, wow... I mean, it was a Tesla- Glad you made it here okay. That's why I mostly drove himself.
63
00:07:48.268 --> 00:07:55.048
[laughs] Uh, mine was actually it's interesting that low code builders did not capture the AI builder market, right?
64
00:07:55.168 --> 00:08:08.608
AI builders being Bolt and Lovable, low code builders being Zapier, Airtable, Retool, Notion, any of those like when you're not technical you can build software. Yeah. Uh, somehow not all of them missed it. Why? Yeah.
65
00:08:08.608 --> 00:08:17.267
[laughs] It's bizarre. Like they should have the DNA, I don't know. They should have... They already have the reach. They already have the, the distribution. Like why? I have, I have no idea.
66
00:08:17.488 --> 00:08:22.308
The ability to fast follow too. Like I'm surprised there's- Yeah. There's just nothing. Yeah. What do you make of that?
67
00:08:22.948 --> 00:08:28.768
Uh, I- it seems, and, you know, not to come back to the AI engineering PhD, like it takes a, a certain kind of,
68
00:08:29.688 --> 00:08:41.148
uh, founder mindset or AI engineer mindset to be like we will build this from whole cloth and not be tied to existing paradigms, I think. 'Cause I, like if I was, if I'm to...
69
00:08:41.288 --> 00:08:51.148
You know, you know Wade b- uh, or, um, who's, who's, who's the Zapier person that you know? Mike. Mike. Um, who has left Zapier. The, the Zapier person. Yeah. [laughs] Um, who's the... Yeah.
70
00:08:51.188 --> 00:08:57.368
Like, uh, you know, Zapier, when they decided to do Zapier AI, they were like, oh, you can use natural language to make Zapier actions, right?
71
00:08:57.748 --> 00:09:05.008
When Notion decided to do Notion AI, they were like, oh, you can like, you know, uh, write documents or, you know, fill in tables with N- with AI.
72
00:09:05.288 --> 00:09:12.208
Like they didn't do the, the op- the, the next step because they already had their base, and they were like, "Let's improve our baseline."
73
00:09:12.608 --> 00:09:23.908
And the other people who actually tried for, to, to create it from whole cloth were like, "We, we got no prior preconceptions," like, "Let's see what we can, what kind of software people can build with, like from scratch," basically.
74
00:09:24.448 --> 00:09:28.548
I don't know. That, that's my explanation. I don't know if you guys have any retros on the AI builders. Yeah.
75
00:09:28.588 --> 00:09:36.268
Or, or, or did they kind of get lucky getting, you know, uh, starting that product journey like right as the models were reaching the inflection point where they started to work? There's the timing issue.
76
00:09:36.588 --> 00:09:39.668
Yeah, yeah, yeah. Yeah. Um, yeah, I don't know. Like I, I...
77
00:09:39.688 --> 00:09:49.468
And to some extent I think the only reason you and I are talking about it is that they, both of them have reported like ridiculous numbers, uh, like 0 to 20 million in three months, basically both of them.
78
00:09:49.888 --> 00:09:57.268
Jordan, did you have a, a big surprise? Uh, yeah. I mean, some of what's already been discussed. I guess the only other thing would be, uh, on the Apple side in particular- Oof...
79
00:09:57.388 --> 00:10:05.408
I think, I think, uh, you know, for the last- Those text message summaries, like phew. [laughs] But they're funny. [laughs] They're funny and how bad- They're great... and how often. [laughs] They're viral.
80
00:10:05.408 --> 00:10:06.068
[laughs] Yeah.
81
00:10:06.128 --> 00:10:20.988
I mean, so like for the last couple of years we've seen so many companies that are trying to do personal assistants, like all these various consumer things, and one of the things we've always asked is, well, Apple is in prime position to do all this, and then with Apple Intelligence, they just totally messed up in so many different ways.
82
00:10:21.028 --> 00:10:33.688
And then the whole BBC thing, uh, saying that the guy shot himself when he didn't, and just like there's just so many things at this point that I would've thought that they would've ironed up their, their AI products better, but just didn't really catch on.
83
00:10:34.008 --> 00:10:41.868
You know, second on this list of, of generally overly broad opening questions would be anything that you guys think is kind of like over-hyped or under-hyped, uh, in the AI world right now?
84
00:10:41.968 --> 00:10:50.888
Um, over-hyped agents framework. [laughs] Sorry. Not naming any particular ones. I'm sorry. Not, not, not... Yeah, exactly. It's not... I, I would say they're just overall
85
00:10:51.868 --> 00:11:00.068
a chase to try and be the framework when the workloads are like in such flux- Yeah... that I just think it's like so hard to reconcile the two.
86
00:11:00.288 --> 00:11:04.788
I think what Harrison and LangChain has done so amazingly is like, uh, product velocity.
87
00:11:05.148 --> 00:11:14.828
Like, you know, uh, the initial abstractions were maybe not the ending abstraction, but like they were just releasing stuff every day trying to be on top of it. But I think now we're like past that.
88
00:11:15.288 --> 00:11:22.888
Like what people are looking for now is like something that they can actually build on- Mm-hmm... and stay on for the next couple of years.
89
00:11:22.948 --> 00:11:30.248
And we talked about this with Brett Taylor on our episode, and it feels like it's like the jQuery era- Yeah... of like agents and LLMs.
90
00:11:30.308 --> 00:11:38.628
It's like, it's kinda like, you know, single file, big frameworks, kinda like a lot of developers, but maybe we need React, and I think people are just trying to build still jQuery.
91
00:11:38.908 --> 00:11:49.868
Like I don't really see a lot of people doing React-like stuff. Yeah. Um, maybe the, the only modification I made about that is maybe it's too early even for frameworks at all. Yeah.
92
00:11:49.868 --> 00:11:56.748
And the thing that- Do you think there's enough stability, uh, in the underlying model layer and, and patterns- Right... to, to add this? The thing is the protocol and not the framework. Yeah.
93
00:11:56.868 --> 00:12:04.808
Um, because frameworks inherently embed protocols, but if you just focus on a protocol, maybe that works, and obviously MCP is the current leading- Mm-hmm... uh, area.
94
00:12:04.828 --> 00:12:11.268
And, you know, I think the comparison there would be instead of just jQuery, it is XML HTTP request, which is like the, the thing that enabled- Mm-hmm...
95
00:12:12.048 --> 00:12:19.752
AJAX, um, and that was the, the, the, the, the sort of inciting incident for JavaScript's, uh, being popular as a language. I would largely agree with that.
96
00:12:19.812 --> 00:12:25.592
I mean, I think on the, the React side of things, I think we're starting to see more frameworks sort of go after more of that.
97
00:12:25.972 --> 00:12:32.452
I guess like Mastro is sort of like on the TypeScript side and more of like a sort of- Mastro? Yeah. Yeah, yeah. The traction is really impressive there.
98
00:12:32.492 --> 00:12:40.072
And so I think, um, we're starting to see more surface there, but I think there's still a big opportunity. What do you have for, uh, for an over or underhyped?
99
00:12:40.292 --> 00:12:50.552
On the underhyped side, um, you know, actually, I, I know I mentioned Apple already, but I think the private cloud compute side with PCC, I actually think that could be really big. It's under the radar right now- Mm...
100
00:12:50.572 --> 00:13:01.912
but in terms of basically bringing the on-device sort of security to the cloud, um, they've done a lot of architecturally interesting things there. Who's they? Apple. Oh, okay. Um, on the PCC side.
101
00:13:02.072 --> 00:13:06.652
And so actually thinking of that. So negative on Apple Intelligence, but positive- But on the-... on Apple Cloud [laughs]...
102
00:13:06.832 --> 00:13:18.372
on the more of the local device, uh, sort of I think there will be a lot of workloads still on device, but when you need to speak to the cloud for larger LLMs, I think that Apple has done really interesting thing on the privacy side.
103
00:13:18.592 --> 00:13:23.231
Yeah. We did the seed of a company that does that, so- Yeah. Especially as things become more consumerized. Did you just set him up on purpose?
104
00:13:23.272 --> 00:13:31.672
That, that felt like a perfect- [laughs] Yeah, no, I was like, "Let's go, Jordan." [laughs] Like, come on. Did you have a meeting before this episode? [laughs] Tell me about that company after. [laughs] Yeah.
105
00:13:31.692 --> 00:13:42.332
We'll, we'll chat after. But, but yes, I, I think that's like the unique, um... the thing about LLM workloads is like you just cannot have everything be single tenant- Right... because you just cannot get enough GPUs.
106
00:13:42.572 --> 00:13:49.952
Like, even like large enterprises are used to having VPCs, and like everything runs privately, but now you just cannot get enough GPUs to run in a VPC.
107
00:13:50.132 --> 00:14:01.332
So I think you're gonna need to be in a multi-tenant architecture, and you need, like you said, like single tenant guarantees in multi-tenant environments. So, um, yeah, it's a interesting space. Yeah.
108
00:14:01.352 --> 00:14:15.592
What about you, Swyx? Underhyped, I want to say memory. Mm. Um, just like stateful AI. Um, uh, as part of my keynote on, on, uh... for... just like every, every conference I do, I do a keynote.
109
00:14:16.112 --> 00:14:23.372
And, uh, I try to do the task of like define an agent. [laughs] You know, always- Evergreen content- Evergreen content... for a, for a keynote.
110
00:14:23.652 --> 00:14:29.572
[laughs] Um, but I did it in a, in a way that it was like, um, I think like a, what a researcher would do.
111
00:14:29.611 --> 00:14:38.872
Like you, you survey what people say, and then you sort of categorize and, and go like, "Okay, this is the, the, uh... what everyone calls agents, and here are the groups of def- definitions. Pick and choose." Right?
112
00:14:38.972 --> 00:14:50.872
Um, and then it was very interesting that the week after that, OpenAI launched their agents SDK, um, and, uh, kind of formalized what they think agents are. Cloudflare also did the same with us. Um, and,
113
00:14:51.892 --> 00:14:59.432
uh, none of them had memory. Yeah. It's very strange. Mm. Um, the-- pretty much like the only big lab...
114
00:14:59.972 --> 00:15:09.012
uh, obviously there- there's conversation memory, but there's not memory, memory, like in like a, like a Let's Store- Yeah... a knowledge graph of facts about you and like, you know, exceed the, the context length.
115
00:15:09.632 --> 00:15:16.352
Um, and, uh, here's the o- the... if you, if you're lo- if you look closely enough, there's a really good implementation of memory inside of MCP.
116
00:15:16.552 --> 00:15:25.872
When they launched, uh, with the initial set of servers, they had a memory server in there, which I, I would recommend as like that's where you start with memory. But I think like, um, if there was,
117
00:15:26.872 --> 00:15:37.732
um, a better, uh, memory abstraction, then a lot of our agents would be smarter and could learn on, on the job, which is something that we all want. And for some reason, um, we've all just like
118
00:15:39.012 --> 00:15:45.912
ignored that because it's just convenient to. And- Do you feel like it's being ignored, or it's just a really hard problem and like lots of- [laughs]... and I feel like lots of people are working on it.
119
00:15:46.012 --> 00:15:53.692
Um, just feels like- Yeah... it's, it's proven more challenging, uh- Yeah. Yeah, yeah. So, so Harrison has LangMem, which, uh, I think now he's like, you know, relaunched again.
120
00:15:54.052 --> 00:16:01.272
And then, uh, we had Leta come speak at our- Mm-hmm... our, uh, conference. Um, I don't know Zep. I think there's a bunch of other memory guys.
121
00:16:01.392 --> 00:16:10.732
But like something like this I think should be normal in the stack, and basically, um, I think anything stateful should be interesting to VCs, uh, because it's databases and, you know, they...
122
00:16:10.832 --> 00:16:17.572
we know how those things make money. I think on the, uh, overhype side, the only thing I'd add is like I'm, I'm still surprised how many net new companies there are training models.
123
00:16:17.732 --> 00:16:25.772
I thought we were kind of like past that. Um- Hmm... and- I would say they died end of last year, now, now they've resurfaced. Yeah. I mean, that's one of the questions that you had down there of like- Yeah, sorry...
124
00:16:26.112 --> 00:16:30.632
uh, is there an opportunity for net new model players? Uh, I would've said no. Um, I don't know what you guys think.
125
00:16:31.112 --> 00:16:40.972
I, I don't have a reason to say no, but I also don't have a reason to say this is what is missing and you should have a new model company do it. But again, I'm an engineer. Like all these guys want to pursue AGI.
126
00:16:41.272 --> 00:16:47.172
You know, all-- they all want to be like, "Oh, we'll, we'll like hit, you know, sort out on all the benchmarks," and like they can't all do it. [laughs] Yeah.
127
00:16:47.692 --> 00:16:54.172
I mean, look, I don't know if Ilya has the secret, uh, secret approach up his sleeve of, uh, of something beyond, uh, test time compute. Mm-hmm. But it was funny. I...
128
00:16:54.192 --> 00:17:00.352
we had, um, Noam Shazeer on the podcast last week, and I was asking him like, "You know, is, is there like some sort of other algorithmic breakthrough? What do you make of Ilya?"
129
00:17:00.392 --> 00:17:06.772
And he's like, "Look," I think what he, uh, implicitly said was test time compute will get to the point where these models are doing AI engineering for us.
130
00:17:06.812 --> 00:17:12.932
And so, you know, at that point they'll figure out the next [chuckles] algorithm breakthrough- Yeah... uh, which I thought was, uh, was pretty interesting. I agree with you folks.
131
00:17:13.012 --> 00:17:20.472
I think that, um, we're most interested at least from our side in like, you know, foundation models for specific use cases and more specialized use cases. Mm.
132
00:17:20.532 --> 00:17:29.212
I guess the broader point is if there is something like that, that these companies can latch onto and being they're sort of known for being the best at, maybe there's a case for that.
133
00:17:29.552 --> 00:17:35.472
Largely though, I do agree with you that I don't think there should be at this point more model companies. That's just- I think it's like these unique data sets, right?
134
00:17:35.512 --> 00:17:37.792
Like I mean, obviously robotics has been an area we've been really interested in- Mm...
135
00:17:37.852 --> 00:17:46.432
which is an entirely different set of, uh, uh, data that's required, you know, on top of like a, a good VLM and then, you know, biology, material science- More the specific use cases. Yeah.
136
00:17:46.472 --> 00:17:55.032
But also specific like- Specific markets... a lot of these models are super generalizable but like, you know, uh, finding opportunities to, you know, where, you know, for a lot of these bio companies they have wet labs.
137
00:17:55.072 --> 00:17:58.812
Like they're literally running a ton of experiments or, um, you know, same on the material sciences side.
138
00:17:58.852 --> 00:18:07.172
And so I still feel like there's some, some opportunities there but, uh, the core kind of like LLM agent space is, uh, it's tough, tough to compete with the big ones. Yeah. Agree. Yeah.
139
00:18:07.212 --> 00:18:18.412
But they're moving more into product. Yeah. So I think that's the question is like if they could do better vertical models, why not do that instead of trying to do deep research and operator and these different things?
140
00:18:18.472 --> 00:18:26.352
Mm. I think that's what I'm- Mm... and in, in my mind it's like- And SuiteAgent's coming out too. Well, yeah, in my, in my mind it's like financial pressure, like they need to monetize in a much- Yeah...
141
00:18:26.392 --> 00:18:35.580
shorter timeframe because the costs are so high. Um- But maybe it's like it's not that easy to do. You think they would be-- that it'll be a better business model to like do it vertical?
142
00:18:35.580 --> 00:18:40.980
Well, it, it's more like why wouldn't they? You know? Like, you make less enemies if you're like a model builder, right? Yeah.
143
00:18:41.060 --> 00:18:54.060
Like, like now with deep research and like search, now Perplexity is like an enemy and like a, you know, Gemini deep research is like more of an em- enemy, um, versus if they were doing a finance model, you know- Mm-hmm...
144
00:18:54.120 --> 00:19:00.400
or whatever, like they would just enable so many more companies. And they always have... Like, they had Habia as one of the customer case studies for- Yeah...
145
00:19:00.440 --> 00:19:04.800
GPT search, but they're not building a finance-based model for them.
146
00:19:04.880 --> 00:19:12.900
So is it because it's super hard and somebody should do it or is it because the new models are gonna be so much better that like the vertical models are useless anyways? Yeah.
147
00:19:12.920 --> 00:19:20.560
Like the lesser- This is the lesser- Exactly. It still seems to be a somewhat outstanding question. I, I'd say like all the signs of the last few years seem to be like a general purpose model is like the way to go.
148
00:19:20.940 --> 00:19:28.940
Um, and you know, you know, tail- like training a hyper-specific model in a s- in, in a domain is like, you know, maybe it's cheaper and faster, but it's not gonna be like higher quality.
149
00:19:28.980 --> 00:19:35.100
But also like I think it's still an... I mean, we were talking to the- to, uh, Noam and Jack Ray from Google last week, and they were like, "Yeah, this is still an outstanding..."
150
00:19:35.140 --> 00:19:42.980
Like, we, we check this every time we have a new model, like whether there's, uh, you know, there-- that still seems to be holding. Um, I remember like a few years ago it felt like all the rage was like the...
151
00:19:43.180 --> 00:19:50.280
It was like the Bloomberg GPT model came out and everyone was like, "Oh, you gotta like, you know, take massive data and-" Yeah. I had the CPFI Bloomberg, uh, present on that. Yeah. Yeah.
152
00:19:50.360 --> 00:19:55.720
That must be a really interesting episode to go back on because I feel like- [laughs]... like very shortly thereafter, the next OpenAI model came out and just like- Yeah, yeah...
153
00:19:55.740 --> 00:20:04.240
beat it on all sorts of the- Uh, no, it, it was a talk. We haven't released it yet. But yeah, I mean, it's, uh, basically they concluded that the, the closed models were better, so they just- Okay, yeah... they stopped.
154
00:20:04.280 --> 00:20:08.560
So they stopped. Interesting. Exactly. Yeah. Sounds like that's been the, uh- Uh, but he's, uh, I, I would be...
155
00:20:09.080 --> 00:20:17.260
He's very insistent that the work that he did, the team he assembled, the data that he collected is actually useful for more than just the model. So like basically everything but the model survived.
156
00:20:17.620 --> 00:20:26.440
What are the other things? Uh, the data pipeline- Okay... the team that they, they, they assembled to, for like fine-tuning and im- implementing, uh, whatever models they, they ended up picking.
157
00:20:27.040 --> 00:20:36.600
Um, yeah, it seems like they are happy with that, and they're running with that. Uh, he runs like 12, 13 teams at Bloomberg just- Yeah... working GenAI across the company.
158
00:20:37.020 --> 00:20:48.160
I mean, I guess we've all, we've all kind of been alluding it, to it right now, but I guess because it's a natural transition, you know, the other broad opening I have is just what are we paying most attention to right now, and I think back on this like, you know, the model companies coming into the product area, I mean, I think that's gonna be like...
159
00:20:48.420 --> 00:20:56.520
I'm fascinated to see how that plays out over the next year and kind of these like front of me dynamics and- Yeah... it feels like it's gonna first boil up on like Cursor Anthropic and like the way- Mm-hmm...
160
00:20:56.560 --> 00:21:03.280
that plays out over the next- Mm-hmm... six months I think will be what we're- What, what is Cursor Anthropic? Uh, is- you mean Cursor versus Anthropic or?
161
00:21:03.340 --> 00:21:08.300
Yeah, I mean, I assume, you know, over time Anthropic wants to get more into the application side of coding. Uh-huh.
162
00:21:08.400 --> 00:21:18.560
Um, and you know, I assume over time Cursor will want to diversify off of, you know, just using the Anthropic model. It's interesting that now Cursor is now worth like 10 billion, nine, nine, 10 billion. Yeah.
163
00:21:18.680 --> 00:21:24.380
Um, and like they've made themselves hard to acquire. Like, I would have said like you should just- Yeah.
164
00:21:24.420 --> 00:21:32.200
You get yourself to five, six billion and join OpenAI, and like all the training data goes to OpenAI, and that's how they train their coding model. Uh, now it's not as complicated.
165
00:21:32.240 --> 00:21:33.660
Now they need to be an independent company.
166
00:21:34.080 --> 00:21:53.360
Uh, increasingly it seems the model companies wanna get into the product layer, and so seeing over the next six, 12 months, does having the best model, you know, uh, let you kind of start from a cold start on the product side and, and get something in market or are the, you know, companies with the best products, even if they eventually have to switch to a somewhat worse, tiny bit worse model, does it not, you know, where do the developers ultimately choose to go?
167
00:21:53.660 --> 00:22:03.120
I think that'll be super interesting. Yeah. But don't you think that Devin is more in trouble than Cursor? I, I feel like Anthropic, if anything, wants to move more towards-- I don't think they wanna build the IDE.
168
00:22:03.640 --> 00:22:11.060
Like, if I think about coding, it's like kinda like, you know, you look at it like a cube. It's like the IDE is like one way to get the code and then the agent is like the other side. Yeah.
169
00:22:11.120 --> 00:22:20.720
I feel like Anthropic wants more be on the agent side and then hand you off to Cursor when you wanna go in depth versus like trying to build the Claude IDE.
170
00:22:20.900 --> 00:22:29.900
I think that's not- I would say- I don't know how you think about it... the existence of Claude code doesn't show, doesn't support what you say. Like, maybe they would, but they haven't shown that yet.
171
00:22:29.980 --> 00:22:36.700
I assume all the... Like, I assume both just converge eventually where you wanna have- Yeah. Uh, where like you'll be able to do both. So, so in order to be, uh...
172
00:22:36.820 --> 00:22:42.560
So we're, we're talking about coding agents, whether it's sort of, um, [tsking] what is it? Inner loop versus outer loop, right? Yeah.
173
00:22:42.580 --> 00:22:49.340
Like inner loop is inside Cursor, inside your IDE between, uh, inside of a Git commit, and outer loop is between Git commits on, on the cloud.
174
00:22:49.820 --> 00:22:59.920
Um, and I think like to be an outer loop coding agent, you have to be more of a like we will integrate with your code base, we'll sign your whatever, you know, security thing that you need to sign- Yeah...
175
00:22:59.960 --> 00:23:07.540
that kind of schlep. Uh, I don't think the model ads wanna do that schlep. They just wanna provide models. [laughs] So that, that, that's, that would be my argument against- Yeah...
176
00:23:07.580 --> 00:23:19.580
like why cognition should still, uh, have, have, have some moat against Anthropic just simply because they would... Cognition would do the schlep and the biz dev and the infra that Anthropic doesn't really care about.
177
00:23:20.360 --> 00:23:27.580
I know the schlep is pretty sticky though once you do it. It's very sticky. [laughs] Yeah, yeah. Um, I mean, it's, it's, it's interesting like I, I think the natural winner of that should be Sourcegraph.
178
00:23:28.220 --> 00:23:34.900
Um, but like there's- Ah, another unprompted, uh, Metronome point portfolio. Unprompted. Nice. [laughs] Nice. We've been saying that, right? Yeah. I mean, they're, they're big supporters.
179
00:23:34.980 --> 00:23:44.180
Uh, like I, I'm very friendly with both Quinn and Viang, and, uh, they've, uh, they've done a lot of work with Cody, but like no, not much work on the outer loop stuff yet.
180
00:23:44.500 --> 00:23:52.380
But like any company where like they have already had like, "We've been around for 10 years. We, we like have all the enterprise contracts. They've-- You already trust us with your code base.
181
00:23:52.660 --> 00:23:59.880
Why would you go trust like Factory or Cognition as like, you know, two-year-old startups who had, like just came out of MIT?" Like, I don't know.
182
00:23:59.980 --> 00:24:09.280
[laughs] I guess switching gears to the, uh, to the application side, um, I'm curious, uh, for both of you, like how do you kind of characterize what has genuine product market fit in AI today?
183
00:24:09.620 --> 00:24:17.620
And, uh, I guess Alessio more and, and you're sort of on the investing side, like more interesting to invest in that category of the stuff that works today or, uh, kind of where the capabilities are going long term?
184
00:24:20.328 --> 00:24:27.988
That's hard. I'm just gonna do my job for you. Yeah, you're like, "Man, that's an easy [laughs] That's a layup." Tell us all your investment theses. Uh, yeah, yeah. Yeah, yeah. I, I, I would say we all...
185
00:24:28.028 --> 00:24:33.948
Well, we only really do mostly seed investing, so it's hard to invest in things that already work- Yeah, that's fair... because it means they're already late.
186
00:24:34.028 --> 00:24:42.448
[laughs] Uh, so we try to, but, but we try to be at the cusp of like, you know, uh, usually the investments we like to make, there's like really not that much market risk.
187
00:24:42.708 --> 00:24:52.928
It's like, if this works, obviously people are gonna use it, but like it's unclear whether or not it's gonna work, so that's kinda more what we skew towards. We try not to chase as many trends, and I don't know.
188
00:24:52.988 --> 00:24:59.748
I, you know, I was a founder myself, and sometimes I feel like it's easy to just jump in and do the thing that is hot.
189
00:25:00.168 --> 00:25:09.868
But like becoming a founder to do something that is like underappreciated or like doesn't yet work shows some level of like grit and self, like you, you actually really believe in the thing.
190
00:25:10.248 --> 00:25:14.048
So th- that alone for me is like, kinda makes me skew more towards that.
191
00:25:14.588 --> 00:25:23.668
And you do a lot of angel investing too, so I'm curious how- Um, yeah, but I, I don't regard, I don't have, I don't use, put, put that in my mental framework of things.
192
00:25:23.768 --> 00:25:31.928
Like, um, I come at this much more as a content creator or market analyst of like, yeah, it, it really does matter to me what has product market fit because people...
193
00:25:32.068 --> 00:25:35.628
I have to answer the question of what is working now- Yeah... uh, when, when people ask me.
194
00:25:35.908 --> 00:25:43.208
And- Do you feel like relative to the, the obviously the hype and discourse out there are like, you know, do you feel like there's a lot of things that have product market fit or like a few things?
195
00:25:43.288 --> 00:25:47.628
Like where- A few things. Yeah. Um, so- I was gonna say this... I have a list of like, uh, um,
196
00:25:48.868 --> 00:25:56.048
two years ago we, I wrote the Anatomy of Autonomy post where it was like the, the first like what's going on in agents and, and, um,
197
00:25:57.008 --> 00:26:04.428
and, and, and what is actually making money because I think there's a lot of gen AI skeptics out there that are like, "These, these things are toys. They're, they're unreliable.
198
00:26:04.488 --> 00:26:13.308
They're" and, you know, "Why, why, why are you dedicating your life to these things?" Um, and I think for me the product market fit bar at the time was $100 million, right?
199
00:26:13.348 --> 00:26:24.588
Like, what use cases can reasonably fit $100 million? And, uh, at the time it was like Copilot, um, it was Jasper, no longer. Mm-hmm. But, you know, in that category of like, uh, help you write- Yeah...
200
00:26:24.748 --> 00:26:32.388
which I think, I think was, was helpful. And then, um, uh, and Cursor I think was on there, uh, as, as, uh, as, as, as like a coding agent, uh, plus plus.
201
00:26:32.768 --> 00:26:40.148
Um, I think that list will just grow over time of like the form factors that we know to work, and then we can just adapt the form factors to a bunch of other things.
202
00:26:40.188 --> 00:26:45.348
So like the, the one that's the most recently added to this is deep research. Yeah. Right?
203
00:26:45.448 --> 00:26:55.768
Uh, where anything that looks like a deep research, uh, whether it's a Groq version, Gemini version, Perplexity version, whatever, um, he has an investment that, uh, that he likes called Brightwave that is basically deep research- Mm-hmm...
204
00:26:55.768 --> 00:26:56.668
for finance. Yeah.
205
00:26:56.748 --> 00:27:10.728
Um, and anything that where like, all right, it's like long-term agentic reporting, um, and it's, it's starting to take more and more of the job away from you and, and just give you a much more reasoned report, I think it's going to work, and, um, that has some PMF I think.
206
00:27:10.808 --> 00:27:21.788
Uh, obviously it has PMF. Like I, I would say, um, it's, uh, I, I went through this exercise of trying to handicap how much money OpenAI made from launching OpenAI deep research. I think it's billions.
207
00:27:22.748 --> 00:27:28.668
Um, like the, the, the amount, the, the sheer upgrade from like $20 to 200, um, it has to be billions in ARR.
208
00:27:28.728 --> 00:27:37.148
Maybe n- not all of them will stick around, but like that is some amount of PMF that is- Didn't they have to immediately drop it down to the $20 tier? They expanded access.
209
00:27:37.228 --> 00:27:39.668
I don't, I wouldn't say- Which I thought was really telling of the market, right?
210
00:27:39.708 --> 00:27:48.848
It's like where you have a, um, you know, I think it's gonna be so interesting to see what they're actually able to get in that 200 or $2,000 tier, which we all think- Yeah... is, is, you know, has a ton of potential.
211
00:27:48.908 --> 00:27:56.308
But it, I thought it was fascinating. I don't know whether it was just to get more people exposure to it or the fact that like- Yeah... Google had a similar product obviously, and, and other folks did too.
212
00:27:56.788 --> 00:27:58.688
But it's really interesting how quickly they dropped it down.
213
00:27:59.188 --> 00:28:08.508
I don't, I, I think that's just a more general policy of no matter what they have at the top tier, they always want to have smaller versions of that in the, in the lower tiers. Yeah, just get people exposure to it.
214
00:28:08.508 --> 00:28:15.988
Um, just, yeah, just get exposure. Uh, the brand of being first to market and, and like the default choice- Yeah... is paramount to OpenAI.
215
00:28:16.228 --> 00:28:22.548
Though I thought that whole thing was fascinating 'cause Google had the first product, right? Yeah. And no, like you, you know, I- We interviewed them. I, I- Yeah...
216
00:28:22.688 --> 00:28:29.328
I straight up to their faces, I was like, "OpenAI mocked you." Totally. And they were like, "Yeah." [laughs] Well, actually I'm curious, like what's it... This is totally off topic, but whatever.
217
00:28:29.528 --> 00:28:37.448
Like what is it going to take for Go- Google just released some great models like a, a few weeks ago. Um, like I feel like- It's happening... the stuff they're shipping is really cool. It's happening. Um- Yeah...
218
00:28:37.768 --> 00:28:47.828
but I, I also, I feel like at least in the, you know, broader discourse it's still like a drop in the bucket relative to- Yeah. Um, I mean, I, again, very front on this. I, I, but I, I think it's happening.
219
00:28:47.888 --> 00:29:00.588
I think it takes some time, but, uh, I am, m- like my Gemini usage is up. Like I, I use, I use it, uh, a lot more for, uh, anything from like summarizing YouTube videos to the native image generation- Yeah...
220
00:29:00.588 --> 00:29:03.728
that they just launched to like flash thinking. Um, so- The multi-modal stuff's great.
221
00:29:03.768 --> 00:29:13.448
Yeah, I run, uh, you know, and, and I run like a daily sort of news recap, uh, called AI News that is, uh, 99% generated by models, and I do a bake-off between all the frontier models every day.
222
00:29:14.088 --> 00:29:25.608
Um, and it's, it's, it's- Every day? Yes. Does it switch? I manual- Yes, it does switch. [laughs] Uh, and I man- I manually do it. Yeah. And, uh, Flash is, Flash wins most days. Um, so, so like I think it's happening.
223
00:29:25.688 --> 00:29:35.568
I think, uh, I was thinking, I was thinking about tracking myself, um, like number of opens of ChatGPT versus Gemini, and at some point it will cross. I think that, uh, Gemini will be my main.
224
00:29:36.068 --> 00:29:44.208
Um, a- and, uh, it, it, I, I, like it, that will slowly happen for a bunch of people and, and, and then that'll, that'll shift. I, I think that's, that's, uh, really interesting.
225
00:29:44.608 --> 00:29:55.208
For developers, this is a different question. Yeah. It's Google getting over itself of having Google Cloud versus Vertex versus AI Studio. They're all these like five different brands like slowly consolidating it.
226
00:29:55.728 --> 00:30:06.808
Um, it'll happen, just slowly I guess. Yeah. Yeah. I, I mean, another good example is like you cannot use, uh, the thinking models in Cursor, and I know Logan Kilpatrick said they're working on it.
227
00:30:06.848 --> 00:30:17.928
But, um, I, I think there's all these small things where like, uh, if I cannot easily use it, I'm really not gonna go out of my way to do it. But I do agree that when you do use them, their models are, are great.
228
00:30:17.988 --> 00:30:26.408
So, uh- Yeah... yeah, they just need better, better bridges. You had one of the questions in the prep, what public company are you long and short? And mine is Google- Yeah... versus, versus Apple.
229
00:30:26.708 --> 00:30:32.288
Like long Google, short Apple. That was also my- Yeah... my combo. I, I feel like, yeah, I mean, I- [laughs]... it does feel like Google's really cooking right now. Yeah.
230
00:30:32.892 --> 00:30:42.392
Um, yeah, so okay, coming back to what is product market fit, um- Yeah, now, now that we come back to my complete total sidetrack, um [laughing] Uh, there's also customer support. Uh, we were talking on the- Yeah...
231
00:30:42.412 --> 00:30:49.792
on the car about Decagon and Sierra, obviously. Brett, Brett Taylor is founder of Sierra. And, uh, yeah, it seems like there's just this...
232
00:30:49.832 --> 00:31:03.812
these layers of agents that'll, like, y- I think you just look at, like, the income statement or, like, the, the org chart of any large-scaled company, and you start picking them off one by one, uh, what, like, is interesting knowledge work, um, and they would just kinda eat things slowly from the outside in.
233
00:31:04.332 --> 00:31:12.612
Yeah. If that makes sense. I, I mean, the episode with the-- with Brett, he's so passionate about developer tools, and yet he did not do a developer tools company.
234
00:31:12.612 --> 00:31:21.252
[laughing] We spent, like, two hours talking about developer tools and, like, all, all of that stuff, and he's like, "I did a customer support company." [laughing] I'm like, man, that says something.
235
00:31:21.452 --> 00:31:27.452
You know what I mean? Yeah. It's like when you have somebody like him who can, like, raise any amount of money from anybody to do anything- Yeah...
236
00:31:27.512 --> 00:31:40.032
to pick customer support as the market to go after while also being the chairman of OpenAI, like, that shows you that, like, these things have moats and have long-standing... Like, they're gonna stick around, you know?
237
00:31:40.192 --> 00:31:51.512
Otherwise, he's smarter than that. So, um, yeah, that's a, that's a space where maybe initially, you know, I would've said I don't know if it's, like, the most exciting thing to, to jump into.
238
00:31:51.552 --> 00:32:02.052
But then if you really look at the shape of, like, how the workforces are structured and, like, how the cost centers of, like, the business really end up, especially for more consumer-facing businesses, like, a lot of it goes into customer support.
239
00:32:02.472 --> 00:32:08.532
All the AI story of the last two years has been cost-cutting. Yeah. I think now we're gonna switch more towards growth. Revenue. You know?
240
00:32:08.692 --> 00:32:19.232
Like you've seen Jensen, like last year at GTC was saying the more you buy, the more you save. This year he said, "The more you buy, the more you make." So, um- Oh... we're- Hot off the press. Yeah. We were there.
241
00:32:19.252 --> 00:32:19.672
We were there.
242
00:32:20.032 --> 00:32:31.972
I do think, I do think that's one of the most interesting things about the- this first wave of apps, where it's, like, almost the easiest thing that you could, um, you could get real traction with was stuff that, you know, for lack of a better way to frame it, like, stuff that people had already been comfortable outsourcing to BPOs or something- Yeah...
243
00:32:32.052 --> 00:32:37.892
and kind of implicitly said, like, "Hey, this is a call center. Like, we are willing to take some performance cut for cost in the past."
244
00:32:38.452 --> 00:32:52.312
You know, the, the irony of that, or what I'm really curious to see how it plays out is, you know, you, you could imagine that is the area where price competition is going to be most fierce because it's already stuff that, uh, you know, that people have said, "Hey, we don't need the, like, 100% best version of that."
245
00:32:52.612 --> 00:33:02.412
And, and I wonder, you know, this next wave of apps may prove actually even more defensible as you get these capabilities that actually are, you know, increased top line or whatnot, where you're like, you know, take AI go-to-market, for example.
246
00:33:02.472 --> 00:33:08.732
Like, you're w- You pay, like, twice as much for something that brought, like... 'Cause there's just a kind of very clean ROI story to it.
247
00:33:09.172 --> 00:33:16.452
Um, and so I wonder ultimately whether the, like, this next set of apps, um, actually ends up being more interesting than the, than the first wave. Yeah.
248
00:33:16.832 --> 00:33:23.692
I think a lot of the voice AI ones are interesting too, because you don't need 100% precision recall to actually, you know, have a great product.
249
00:33:23.732 --> 00:33:33.172
And so for example, we looked into a bunch of, um, you know, scheduling intake companies, for example, like home services, right, for electricians and stuff like that. Today, they miss 50% of their calls.
250
00:33:33.452 --> 00:33:38.392
So even if the AI- [laughing]... is only effective, say, 75% of the time- [laughing] Yeah, it's crazy, right?
251
00:33:38.412 --> 00:33:46.772
[laughing] So if it's effective 75% of the time, that's totally fine because that's still a ton of increased revenue for the customer, right? And so you don't need that 100% accuracy. Yeah.
252
00:33:46.792 --> 00:33:53.972
And so as the models and the reliability of these agents are getting better, it's totally fine because you're still getting a ton of value in the meantime. Yeah.
253
00:33:54.052 --> 00:33:59.712
Uh, what-- This is-- I don't know how related this is, but, uh, one of my favorite meetings at- Yeah. It is related.
254
00:33:59.712 --> 00:34:08.472
[laughing] [laughing] One, one of my favorite meetings at AI Engineering Summit, 'cause, like, like, I do these... This was our first one in New York, and I, I just, like, met the different crew than, than you meet here.
255
00:34:08.692 --> 00:34:14.012
Like, everyone here is, loves developer tools, loves infra. Over there, they're actually more interested in applications, so it's kinda cool.
256
00:34:14.312 --> 00:34:22.412
I met this, like, bootstrap team that, like, they're only doing appointment scheduling for vets. They, they [laughing] Yeah. And, like, they're like, "This is a, this is an anomaly.
257
00:34:22.452 --> 00:34:27.392
We don't usually come to engineering summits, 'cause we usually go to vet summits." [laughing] "And, like, talk to the..."
258
00:34:27.412 --> 00:34:33.392
They're, they're like, you know, they're, they're, they're literally the object- I'm sure it's a massive pain point. [laughing] They're willing to pay a lot of money. [laughing] Yeah.
259
00:34:33.432 --> 00:34:44.872
But, but, but this is, like, my point about saving versus making more. It's like if an electrician takes 2X more calls, do they have the bandwidth to actually do 2X more in-house? Then they get hired. Is it-- Well, it...
260
00:34:44.912 --> 00:34:53.392
Yeah, exactly. That's the thing is, like, I don't think today most businesses are, like, structured to just, like, [clicks tongue] overnight 2, 3X demand, you know? I think that's, like, a startup thing.
261
00:34:53.432 --> 00:35:01.672
Like, mo-most businesses can't do it, so- Then you make a electrician agent. Well, no, totally. [laughing] Well, how do you do, how do you do a recruiting agent for electrician?
262
00:35:01.672 --> 00:35:06.472
[laughing] Or, like, uh, electrician training? That's a good point. Yeah. How do you do Lambda school for electrician? I don't know.
263
00:35:06.532 --> 00:35:12.572
It's like there's some- It's gonna be a very human whack-a-mole for the bottlenecks in these businesses, like, as you- Yeah, exactly... like, "Oh, now we have a ton of demand. Like, cool, like, where do we go?" Yeah.
264
00:35:12.592 --> 00:35:22.752
Yeah. So, uh, just to f-round out the, the, this PMF thing, um, I think this is relevant in a c- in a sense of, like, it's pretty obvious that the killer agents are coding agents, support agents, deep research, right?
265
00:35:23.292 --> 00:35:29.912
Roughly, right? We've covered all those three already. Then, like, then, then you have to sort of be of g- turn to offense and go, like, "Okay, what's next?"
266
00:35:30.292 --> 00:35:35.732
And, like, what- Well, I mean, I mean, I also just like summarization of, of voice and conversation- Yep. Absolutely. We actually had that on there. Yeah. It...
267
00:35:35.892 --> 00:35:47.092
I just, I didn't put it as agent 'cause seems less agentic, you know? But yes. Still, still a good AI use case. Uh, the-- that one I, I've seen, uh, I would mention Granola, and what's the other one? Um- I don't know...
268
00:35:47.112 --> 00:35:54.132
Mo-Monterey. I think a bridge was the one you wanted to mention. I was gonna say Bridge. Yeah. Bridge? Bridge? Okay. [laughing] [laughing] So, uh, I would just... I would call out what I had on my slides- Yeah...
269
00:35:54.152 --> 00:36:04.172
for, for the agent engineering thing. Um, so it was screen sharing, which I think is actually, uh, kinda, kinda underrated. Like, people, like, an AI watching you as you do your work and just, like, offering assistance.
270
00:36:04.552 --> 00:36:12.492
Um, outbound sales, so instead of support just being more outbounds. Hiring- You, you say outbound sales has product market fit? No, it, it, it will... It's coming. Oh, on the com- yeah, yeah. It's coming.
271
00:36:12.512 --> 00:36:23.452
I totally agree with that. Yeah. Yeah. Um, hiring, um, like the recruiting side. Uh, education, like the, the sort of, like, personalized teaching, I think. Um- I'm kinda shocked we haven't seen more there. Mm. Yeah.
272
00:36:23.472 --> 00:36:31.412
It's- I don't know if that's, like- It's, like, Duolingo is the thing. Amigo. Yeah, I mean, Speak and some of these, like, you know- Speak... practice, yeah. Yeah. Yeah, interesting.
273
00:36:31.492 --> 00:36:38.512
Um, and then, uh, finance I-- there's a, there's a ton of finance cases that we can talk about that. Uh, and then personal AI, which, uh, we also had a little bit of that.
274
00:36:38.552 --> 00:36:47.132
But the, the-- I think personal AI is a l- a harder to monetize. But, like, I think those would be, like, what I would say is up and coming in terms of, like, that's what I'm currently focusing on.
275
00:36:47.652 --> 00:36:51.220
I feel like this question's been asked a few different ways, uh- But I'm, I'm curious what you guys think.
276
00:36:51.260 --> 00:37:01.040
It's like, is it, like if we just froze model capabilities today, like is there, you know, trillions of dollars of application value to be unlocked? Like, like AI education, like if we just stopped today- Yeah...
277
00:37:01.080 --> 00:37:05.800
all model development, like with this current generation of models, we could probably build some pretty amazing education apps.
278
00:37:06.200 --> 00:37:19.480
Um, or like how much of this, how much of, of all this is like contingent upon just like, okay, people have had two years with GPT-4 and like, you know, I don't know, six months with the reasoning models, like how much is contingent upon it just being more time with these things versus like the models actually have to get better?
279
00:37:20.380 --> 00:37:27.320
I don't know. It's a hard question, so I'm gonna just throw it to you. Yeah. [laughs] Well, I think the societal thing is maybe harder, especially in education. You know?
280
00:37:27.360 --> 00:37:34.880
Like can you basically like doge the education system? [laughs] Probably you should, but like can you?
281
00:37:34.960 --> 00:37:42.960
I, I think it's more of a human- But people pay for all sorts of like get ahead things outside of class and, you know, certainly in other countries there's a ton of consumer spend in education.
282
00:37:42.980 --> 00:37:49.280
It feels like the market opportunity is there. Yeah, and in private education I think. Um, yeah, public s- public is very different. Yeah.
283
00:37:49.320 --> 00:37:58.540
One, one of my i- most interesting quests from last year was kind of reforming Singapore's education system to be more sort of AI native. Just what you were doing on the side while you were- Yes.
284
00:37:58.660 --> 00:38:01.800
[laughs] While waiting for OpenAI. That's a, that's a great, that's a great side quest.
285
00:38:01.800 --> 00:38:08.280
[laughs] Uh, th- my stated goal is for Singapore to be the first country that has Python as a first language, as a, as a national language.
286
00:38:08.900 --> 00:38:19.200
Um, anyway, so but uh, the, the, the, the defense, the pushback I got from the Ministry of Education was that the teachers would be unprepared to do it. Hm. Uh, so it's like, it was like the def- the like the, uh...
287
00:38:19.300 --> 00:38:27.160
It was really interesting, like immediate pushback was the, the de facto teachers union being like resistant to change, and like, okay. Yeah. That, that's par for the course.
288
00:38:27.720 --> 00:38:32.320
Um, anyway, so not, not to, not to dwell too much on that, but like, uh, yeah, I mean like I, I think like
289
00:38:33.380 --> 00:38:38.540
education is one of those things that pe- everyone like has strong opinions on because we all have kids, all been through the education system.
290
00:38:39.060 --> 00:38:50.540
Um, but it like, I think it's gonna be like the, the domain specific like, like speak like such a amazing example of like top-down, like we will go through the idea maze and we'll go to Korea and teach them English.
291
00:38:50.720 --> 00:39:00.299
Like [laughs] it's like what the hell? Um, and uh, I would love to see more examples of that, like just- Yeah... like really focused, like no one try to solve everything, just, just do your thing really, really well.
292
00:39:00.660 --> 00:39:06.180
On this trend of, uh, of, of difficult questions that come up, I'm gonna just ask you the one that my partners like to ask me every single Monday, which is- [laughs]...
293
00:39:06.360 --> 00:39:14.680
uh, how do you think about defensibility at the, uh, at the app layer? Oh, yeah. That's great. Just give me an answer- Uh... I can copy/paste and just like, you know- Network effects... have auto, auto response.
294
00:39:14.760 --> 00:39:28.000
Uh, honestly, like network effects. I, I think people don't prioritize those enough because they're trying to make the single player experience good, but then, then they neglect the multiplayer experience.
295
00:39:28.060 --> 00:39:29.520
I think one of the, um...
296
00:39:29.620 --> 00:39:37.660
I always think about like load-bearing episodes, like, you know, as, as podcasts you do one a week and like, you know, some of those you don't really talk about ever again, and others you keep mentioning every single podcast.
297
00:39:38.080 --> 00:39:45.980
And one of the- This is obviously gonna be the latter. Yeah. [laughs] The, I think the recap episodes for us are pretty load-bearing. Like we, we refer to them every three months or so.
298
00:39:46.400 --> 00:39:56.040
Um, and like, uh, one of them I think for, for us is Chai, for me is Chai Research. Um, even though that wasn't like a super popular one among the broader community outside of Chai, the Chai community.
299
00:39:56.560 --> 00:40:06.240
Uh, for those who don't know, Chai Research is basically a character AI competitor. Yeah. Right? They were, uh, bootstrapped, they were founded at the same time, and they have outl- outlasted character of de facto. Yeah.
300
00:40:06.300 --> 00:40:12.720
Right? Uh, it's funny, like I, I would love to ask Milosh there a bit more about like the whole character thing, but- Good luck getting past the Google cops.
301
00:40:12.720 --> 00:40:19.540
[laughs] [laughs] Um, but like so he, like he, like he doesn't have his own models basically.
302
00:40:19.780 --> 00:40:30.160
He has his own network of people submitting models to be run, and I think like that is like short term going to be hurting him because he doesn't have proprietary IP.
303
00:40:30.440 --> 00:40:35.660
But long term he has the network, network effect to make him robust to any changes in the future.
304
00:40:35.880 --> 00:40:51.600
And I think like I want to see more of that, where like he's basically looking himself as kind of a marketplace and he's identified the choke point, which is build the app or the, the sort of protocol layer that, um, interfaces between the users and the model providers, and then make sure that the money kind of flows through.
305
00:40:52.100 --> 00:41:02.660
And that works. I, I wish that more AI builders or AI founders emphasize network effects, 'cause that, that's the only thing that you're gonna have at the end of the day. Yeah.
306
00:41:02.680 --> 00:41:07.740
And like brand leads into network effects as well. Yeah. I guess, uh, you know, harder, uh, in, in the enterprise context, right?
307
00:41:07.800 --> 00:41:17.180
But I mean, I feel it's funny, we do this exercise and I feel like we talk a lot about like, you know, obviously there's, you know, uh, kind of the velocity and the breadth you're able to kind of build, um, of product surface area.
308
00:41:17.360 --> 00:41:19.060
There's just like the ability to become a brand in the space.
309
00:41:19.100 --> 00:41:26.860
Like I'm shocked even in like six, nine months how an individual company can become synonymous with like an entire category, and like then they're in every room for customers- Yeah...
310
00:41:27.360 --> 00:41:31.380
and like some of the other startups are like clawing their way to try and get in like one, you know, 20th of those rooms.
311
00:41:31.620 --> 00:41:39.900
There's a bunch of categories where we talk about in IC and it's like, oh, pricing compression's gonna happen, not as defensible, and so ACVs are gonna go down over time.
312
00:41:40.260 --> 00:41:48.940
In actuality, some of these, the ACVs have doubled we've seen. And the reason for that is just, you know, people go to them and pay for that premium of being that brand. Yeah.
313
00:41:48.980 --> 00:41:54.840
I mean, what I'm struck by is there's been s- there was such a head fake in the early days of, of AI apps where people were like, "We want this amazing defensibility story."
314
00:41:54.860 --> 00:41:56.260
And then what's the easiest defensibility story?
315
00:41:56.280 --> 00:42:03.180
It's like, oh, like total unique data set or like train your own model or something, and I feel like that was just like a total head fake where I don't think that's actually useful at all. Mm-hmm.
316
00:42:03.200 --> 00:42:13.200
It's the much le- you sound much less articulate when you're like, "Well, the defensibility here is like the thousand small things that this company does to make like the user experience design everything just like delightful- Mm-hmm...
317
00:42:13.220 --> 00:42:22.800
and just like the speed at which they move to kind of both create a really broad product, but then also every three, six months when a new model comes out, it's kind of an existential event for like any company, 'cause if you're not the first to like- Yeah...
318
00:42:22.900 --> 00:42:25.940
figure out how to use it, someone else will. Yeah. And so velocity really matters there.
319
00:42:25.980 --> 00:42:31.860
And it's funny in, in, in kind of our internal discussions we've been like, man, that sounds pretty similar to like how we thought about like application SaaS companies.
320
00:42:32.260 --> 00:42:39.600
Um, that there isn't some like revolutionary reason. You don't sound like a genius when you're like, "Here's applications, like application SaaS company A is so much better than B."
321
00:42:39.960 --> 00:42:44.000
Um, but it's like a lot of little things that compound over time. What about the infrastructure space, guys?
322
00:42:44.040 --> 00:42:54.020
Like I'm curious, uh, you know, what, how do you guys think about where the interesting categories are here today and, you know, like where, where, where do you want to see more startups or, or where do you think there are too many?
323
00:42:54.320 --> 00:43:00.540
Yeah. Yeah, we call it kind of the LLMOS. Um, but I would say- Not we, I mean Andre, Andre calls it LLMOS.
324
00:43:00.540 --> 00:43:09.140
[laughs] Well, but yeah, we- [laughs] Well, we have on, uh, on our four worst- You, you and Andre, the three of you call it the LLMO. Well, we have this like four worst of AI, uh, framework- Yeah, yeah...
325
00:43:09.240 --> 00:43:17.680
that, that we use, and LLMOS is one of them. But, um, yeah, I mean, code execution is one. We've been banging the drum. Everybody now knows we're investors in E2B. Mm-hmm.
326
00:43:17.880 --> 00:43:26.720
Uh, memory, you know, is one that, uh, we kind of touched on before. Um, super interesting. Search, uh, we talked about. Um, I, I think those are more-
327
00:43:27.892 --> 00:43:38.172
Not traditional infra, not like the bare metal infra, it's more like the infra around the model- Tools for agents. Yeah... you know? Uh, which I think is where a lot of the value is gonna be, um, but- The app ones? Yeah.
328
00:43:38.212 --> 00:43:41.932
Yeah, yeah, in cyber security, I mean, um, there's so much, uh- Yeah...
329
00:43:41.972 --> 00:43:53.952
to be done there, and it's more like basically any area where AI is being used by the offense, AI needs to be applied on the defense side, like email security, you know, identity, like all these different things.
330
00:43:54.532 --> 00:44:05.212
Um, so we've been doing a lot there. A- as well as, you know, having to rethink things that used to be costly, like red teaming and maybe used to be a checkbox in the past, today they can be actually helpful- Yeah...
331
00:44:05.512 --> 00:44:14.012
uh, to make you secure your app. And there's this whole idea of like semantics, right? That now the models can be good at. Um, you know, in the past, everything is about syntax.
332
00:44:14.152 --> 00:44:16.692
It's kinda like very basic, you know, constraint rules.
333
00:44:17.152 --> 00:44:26.512
I think now you can start to infer semantics from things that are beyond just, like, simple recognition to, like, understanding why certain things are happening a certain way.
334
00:44:26.572 --> 00:44:31.972
So in the security space, we're seeing that with, um, binary inspection, for example.
335
00:44:32.011 --> 00:44:43.032
Like, there's kinda like the syntax, but then there are, like, uh, semantics of, like, understanding what is this code overall really trying to do, even though this individual syntax is, like, saying something specific.
336
00:44:43.432 --> 00:44:52.632
Not to get too technical, but, um, yeah, I, I think infra overall is, like, a, a super interesting place if you're making use of the model. If you're just...
337
00:44:52.952 --> 00:44:59.872
I, I'm less bullish, not, not that it's not a great business, but I think it's a very capital-intensive business, which is, like, serving the models. Mm-hmm. Yeah.
338
00:44:59.932 --> 00:45:09.332
I think that infra is, like, great, people will make money, but, um, yeah, I, I, I don't think there's as much of a, um, interest from, from us- Yes... at least.
339
00:45:09.492 --> 00:45:17.012
How, how do you guys think about what OpenAI and the big research labs will encompass as part of the developer and infra category? Yeah. That, that's why I, I would say,
340
00:45:18.292 --> 00:45:26.912
um, Search is the first example of one of the things we used to mention on w- you know, we had Axon on the podcast, and Perplexity obviously as a- Mm-hmm...
341
00:45:26.972 --> 00:45:36.272
as an API, um- The basic idea is, uh, if you go into, like, the ChatGPT custom GPT builder, like what are the checkboxes, each of them is a startup. Yeah. Yeah. Yeah. And, and now they're also APIs. Yes. Yeah.
342
00:45:36.292 --> 00:45:40.732
So now Search is also an API. Um, we'll see what the adoption is.
343
00:45:40.792 --> 00:45:55.732
There's the, you know, in traditional infra, it's like everybody wants to be multi-cloud, so maybe we'll see the same where, um, ChatGPT Search or OpenAI Search API is, like, great with the OpenAI models because you get it all bundled in, but their price is very high.
344
00:45:56.252 --> 00:46:08.592
If you compare it to, like, you know, Axon I think is, like, five times the, the price for the same amount of research, which makes sense if you have a big OpenAI contract, but maybe if you're just, like, pick, invest, and breed, you wanna compare different ones.
345
00:46:08.992 --> 00:46:18.892
Yeah. Um, yeah. They don't have a code execution one. I'm sure they'll release one soon. Um, so they wanna own that too, but, um, yeah. The, the same question we were talking about before, right?
346
00:46:19.032 --> 00:46:25.232
Do they wanna be an API company or a product company? Do you make more money building ChatGPT Search or selling Search API? Yeah.
347
00:46:25.272 --> 00:46:33.212
The, the broader lesson instead of, like, going, we did applications just now and then what do you think is interesting infrastructure, like, it's not 50/50. It's not, like, equal weighted. Yeah.
348
00:46:33.232 --> 00:46:38.932
Like, it, it's just very clearly the application layer has, like, been way more interesting. [laughs] Yeah.
349
00:46:38.972 --> 00:46:47.432
Like, yes, there, there's interesting in- uh, infrastructure plays, and I even wanna, like, push back on, like, the, the, the whole GPU serving thing because, like, Together AI is doing well, Fireworks is doing well.
350
00:46:47.472 --> 00:46:53.472
No, that's, that's- Oh, yeah, yeah, yeah... how it works. It's like data centers and inference providers for the- Yeah, yeah... you know. I think it's all, like, the capital. I see, I see. Yeah.
351
00:46:53.492 --> 00:46:59.492
For, for, for, uh, again- Capital efficiency, yeah... you guys have a much larger font, so you, I'm sure you have GPU pods, but- Yeah.
352
00:46:59.532 --> 00:47:21.252
Yeah, so, um, y- that's, that, that is one thing I have been learning in, in that, um, you know, I think I have s- historically had a dev tools and infra bias, and so has he, and we've had to learn that applications actually are very interesting and also maybe kind of the killer application of models in the sense that you can charge for utility and offer cost, right?
353
00:47:21.332 --> 00:47:30.692
Which where, like, most infrastructure reduces to cost plus. Yeah. Right? So, and, like, that's not where you wanna be for AI. [laughs] Um, so that's, that's interesting for, for me.
354
00:47:31.032 --> 00:47:41.572
Um, uh, I thought it'd be interesting for me to be the only non-VC in the room to be saying what is not investable 'cause, like, then I, then, you know, you can, uh... I, I won't be canceled for saying, like- [laughs]...
355
00:47:41.632 --> 00:47:46.392
your, your whole category is, like, not investable. This thing's not investable, and then, like, three months later we're desperately chasing people. [laughs] Exactly.
356
00:47:47.152 --> 00:47:53.552
So, so you don't wanna be on the record- This space changes so fast. Yeah, yeah. It's like every opinion you hold, you have to, like, hel- hold it quite loosely. I'm happy to be wrong in public, you know?
357
00:47:53.652 --> 00:48:02.252
I think that's how you learn the most, right? Yeah. Um, so, like, fine-tuning companies is something I struggled with, and still, like, I don't see how this becomes a, a, a big thing.
358
00:48:02.552 --> 00:48:16.212
Like, um, uh, you kind of have to wrap it up in a broader ser- broader, um, enterprise AI company, like services company, like a Writer AI, where, like, they will fine-tune as part of the overall offering, but, like, that's not where you spike.
359
00:48:16.872 --> 00:48:22.752
Um, yeah, it's kind of interesting. And then the... I'll, I'll just kind of group, uh, AI DevOps, and, like, there's a lot of AI SRE out there.
360
00:48:23.152 --> 00:48:31.232
Um, seems like there's a lot of data out there that, that should be able to be plugged into your code base or, or, or your app to sort of self-heal or whatever.
361
00:48:31.532 --> 00:48:38.512
It's just I don't know if that's, like, been a thing yet, and you guys can correct me if you're, if I'm wrong. And then the, the last thing I'll mention is voice real-time infra.
362
00:48:39.012 --> 00:48:48.912
Um, again, like, very interesting, very, very hot, but again, how big is it? Those are the, the main three that I'm thinking about for things I'm struggling with. Yeah. I guess, couple comments.
363
00:48:49.012 --> 00:48:51.532
On the AI SRE side, I actually disagree with that one. Yeah.
364
00:48:51.652 --> 00:49:04.552
Um, I think that the reason they haven't sort of taken off yet is because the tech is just not there quite yet, and so it goes back to the earlier question, do we think about investing towards where the companies will be when the models improve versus now?
365
00:49:04.952 --> 00:49:11.532
I think that's going to be, in short term we'll get there, but it's just not there just yet. But I think it's an interesting opportunity overall. Yeah.
366
00:49:12.052 --> 00:49:20.832
It, it, it's, um, m- my pushback to you is while it's monitoring a lot of logs, right? Yeah. And it's basically anomaly detection rather than, um...
367
00:49:21.512 --> 00:49:27.822
Like, there's, there's a whole bunch of, like, stuff that can happen after you detect the anomaly, but it's really just ano- anomaly detection, and we've always had that. You know?
368
00:49:28.242 --> 00:49:37.382
Like it's, this is like not a transformer's LLM use case, this is just regular anomaly detection. It's more in terms of, like, it's not going to be an autonomous SRE for a while. Yeah.
369
00:49:37.422 --> 00:49:45.362
And so the question is how, how much can the latest sort of AI advancements increase the efficacy of going, bringing your MTTR down? Yeah. And- I see...
370
00:49:45.422 --> 00:49:51.482
even if it's 10%, uh, improvement on beforehand, that's still potentially a lot of revenue. Okay. Right.
371
00:49:51.522 --> 00:50:02.102
That's the way at least I, I think I would think about it now, and then, you know, a few years from now if it's actually an autonomous SRE just replacing altogether, then that's a totally different thing. Hmm. Cool.
372
00:50:02.182 --> 00:50:11.622
I, I will look out for it. Yeah. Hmm. Yeah. Um, you know, I guess, uh, switching back to overly broad questions, like, what do you feel like is the biggest unanswered question, um, in AI today- Ah...
373
00:50:11.662 --> 00:50:22.342
you know, that has an, you know, large implications for the ecosystem? Um, yeah, I, I've been banging the drum on RL, and I think it's clear that you can do RL successfully on verifiable domains. Yeah.
374
00:50:22.442 --> 00:50:32.442
Um, I would say whether or not we can figure out how to do that in non-verifiable one, so law is a g- great example. Totally. Like, can you do RL on contracts and documents?
375
00:50:33.002 --> 00:50:44.342
Um, marketing, sales, going back to outbound sales, like can you do RL to like simulate what an outbound and, and kinda like the conversation leads to? Um, yeah, it's unclear.
376
00:50:44.502 --> 00:50:55.722
If not, then I think we'll be stuck with like you're gonna have agents in the more verifiable domains, and then you'll just kinda have co-pilots in the non-verifiable ones because you'll, you'll still need a person to be the tastemaker, so to speak.
377
00:50:55.762 --> 00:51:11.742
I had the exact same thing and I feel like it's like the que- I, I just, I'm trying to think of the implications where if it doesn't work, like the world could be weird where like you have like fully autonomous AI coders and like, you know, no one does any software or math or even like, you know, some areas of science, but then like to write the most basic sales email, it's still like, uh- Yeah...
378
00:51:12.082 --> 00:51:19.762
it, it like it just, it's always so hard to predict how the [laughs] world... Like that is such a weird, of all the sci-fi that was written, you know, 50 years ago, I don't think anybody foresaw that future.
379
00:51:19.802 --> 00:51:28.622
Like that is- It is... a really weird future. Yeah, yeah. Do either of you have a different one for that? Uh, well, uh, yeah. No, go for it. We'll, we'll go back and forth. [laughs] Um, biggest unanswered question.
380
00:51:28.682 --> 00:51:29.262
I guess, um,
381
00:51:31.102 --> 00:51:42.262
I don't know if this is a good answer, but you know, Bob, uh, McGrew we had on the podcast and he was talking about like the rule of nines they have at OpenAI where to go from 90% reliability to 99, it's an order of magnitude increase in compute.
382
00:51:42.302 --> 00:51:53.862
And then 99 to 99.9, order of magnitude increase, and that happens every two to three years. And so I think how are we going to scale sort of accordingly, um, this sort of next part?
383
00:51:54.002 --> 00:52:03.682
I think there's a lot of unanswered questions just like from a hardware perspective. And then I think as part of that, from the availability perspective, like is Nvidia just going to continue to be dominant?
384
00:52:03.722 --> 00:52:13.462
Like obviously AWS is going hard, um, into, uh, what's their Trainium chips? Mm-hmm. Uh, I'm blank on it. Thank you. Um, [laughs] and so I think like
385
00:52:14.402 --> 00:52:22.282
there's a big ecosystem around Cuda that's obviously allowed Nvidia to remain dominant, but just what's going to happen and is there anyone who's going...
386
00:52:22.582 --> 00:52:32.402
Is there anyone that's going to come sort of combat that to increase the availability of GPUs or are we just gonna be constrained going forward when we actually need way more compute going forward? Yeah.
387
00:52:33.242 --> 00:52:43.862
My quick thoughts, uh, I've, I've been, uh, I'm the only individual named as an investor in Madex. [laughs] Which is kind of like really funny 'cause everyone else was funds, and then it was just me.
388
00:52:43.862 --> 00:52:47.782
[laughs] Uh, a- and it's, it's, uh, it's a, there's a, there's an interest...
389
00:52:47.842 --> 00:52:54.182
There, like there are all these like Nvidia startups, like, uh, sorry, uh, dedicated Silicon startups that are coming up and, and, uh, trying to challenge that.
390
00:52:54.222 --> 00:53:02.502
And the simple answer is, um, like these GPUs are the most general things possible b- by design. That's why they do gaming and crypto- Mm-hmm... and AI.
391
00:53:02.882 --> 00:53:11.462
And, uh, I think as long as the architecture seems, seems stable, it seems like there's a case to be made for, uh, for that. Um, the only question is who will win that?
392
00:53:11.502 --> 00:53:20.322
And, uh, y- and obviously there's a whole bunch of competitors including I think AMD's trying to, you know- Yeah... to, to make a play for it. But so will AWS and so will, you know, every other...
393
00:53:20.342 --> 00:53:27.542
Like Microsoft has a chip, Facebook has a chip. Um, so who knows who will win that? Um, it, it's just, it's very interesting that like
394
00:53:28.482 --> 00:53:36.982
this seems to be such a valuable prize, like it's freaking Nvidia that you're competing with. Yeah. And no one has really made a real dent there yet.
395
00:53:37.102 --> 00:53:47.101
[laughs] Uh, but so I, I kind of, I kind of agree with you, but like I think that it basically is all about stability of workload, and as long as... It's a bet on like the death of transformers basically.
396
00:53:47.462 --> 00:53:55.262
Um, and if you're fine with that, like even, uh... And I think like the, even the state space model people would agree that like it, it wouldn't really change that much.
397
00:53:55.742 --> 00:54:06.102
Um, and probably, uh, I think the, the overall consensus is that you don't even use state space models in- individually. Like you would use them in a mixture with transformers anyway. Yeah.
398
00:54:06.202 --> 00:54:15.542
Um, so then like, yeah, just go bet on transformers, bake it into the chip and you will have much, way more, uh, you know, basically ASICs like for transformers, and that's fine.
399
00:54:15.862 --> 00:54:23.522
And, and, um, so like prima facie there should be a company that wins that. Yeah. I don't know who will [laughs] win that. Yeah. I wish we knew. [laughs] Yeah.
400
00:54:23.562 --> 00:54:35.782
I, I think that, I think anyone, you have to start basically after 2019 or 2020 because anyone started before that will still be too general. Mm-hmm. Yeah. Because you, transformers hadn't won yet, won at the time.
401
00:54:36.122 --> 00:54:42.922
Uh, I have one more. It, the, the, I think the, the most emergent one that came out of the New York conference that I did was a- agent authentication. Hmm.
402
00:54:43.142 --> 00:54:56.222
Um, I think literally the, like the information just published, uh, like this is something that they're worried about which is, um, when operator or whoever accesses your website on behalf of you, how does it indicate that it's not you but it's, it's an agent of you?
403
00:54:56.742 --> 00:55:07.802
And I think like, um, my general phil- philosophy on agent experience or any of the sort of like reinvention of every part of the stack for agents is they're all not, not necessary except for this agent auth thing.
404
00:55:08.042 --> 00:55:14.102
Like we, we really need, uh, to, to be able to like ne- new SSO effectively for agents. Is it gonna be crypto?
405
00:55:14.282 --> 00:55:23.922
Both crypto people are really amped about the, uh, uh- You know, like it really, it's really frustrating when Sam Altman is right, but like maybe you have to scan your eyeballs. [laughs] Like maybe you sh- just have to.
406
00:55:23.922 --> 00:55:34.122
[laughs] You just, you, like maybe he saw this like five years ago and he was like, "You gotta scan your eyeballs," and like the rest of us are just behind him as usual. [laughs] Oh, I love it.
407
00:55:34.162 --> 00:55:44.854
Well, okay, now I'll move to the quick fire round- Cool... um, where we'll, we'll go around the horn and get a, get quick takes on things. So the first is gonna be, uh, dream podcast guest. John Carmack. Yeah.
408
00:55:45.414 --> 00:55:55.894
Uh, he's six- John is, like, six steps away from solving AGI apparently, so we just ask him how long- [laughs] -before it launches. Exactly. Uh, for us, it's Andre. Uh, for me, it's Andre. Um, he had...
409
00:55:56.214 --> 00:56:04.214
He's a listener and supporter of the pod, and like, basically when I launched the, the whole AI engineer push that we, we, that we have, he was the basically the first one to legitimize it.
410
00:56:04.274 --> 00:56:09.594
He was like, "You know, there will be more AI engineers than ML engineers." And I think that made everyone else pay attention.
411
00:56:09.654 --> 00:56:18.994
So like, Latent Space only exists because, you know, he, he helps- he and other people helped to promote it. Yeah. I also had Andre, so [laughs] Yeah, me too. I guess we're thinking the same thing there.
412
00:56:19.434 --> 00:56:22.374
I basically- Mine's a little bit cheating, but I think at some point there will...
413
00:56:22.574 --> 00:56:27.854
So, like, they're writing a book about OpenAI now, and at some point, like, somebody probably acquired, we'll get to do the acquired OpenAI episode.
414
00:56:27.894 --> 00:56:38.154
But if unsupervised learning could, like- [laughs] There's probably just, like, so many amazing stories of, like, the last five, six years, um, and so- So do you know about Doomers? Uh, it's a play.
415
00:56:38.454 --> 00:56:48.034
I'm actually going to it this Saturday. [laughs] Uh, yeah, they, they, they, they... Someone made a play about the board drama of last year. Really? Of, like, yeah, two years ago. Wow. Yeah. Wow. [laughs] That's wild.
416
00:56:48.034 --> 00:56:51.234
Let us know how it is. Yeah, let us know. There, there will be- Maybe we should have- There'll be logs- -the director of that on the pod. Yeah.
417
00:56:51.354 --> 00:57:02.854
[laughs] [laughs] No, I, I think it's a lot of fan fiction basically, [laughs] but like, someone will write the accounts and, and it will be interesting and fascinating, and a lot of, a lot of it will be fake because it's a complex beast, right?
418
00:57:02.914 --> 00:57:06.334
You're just getting an oral history of what happened. Yeah. Yeah.
419
00:57:06.434 --> 00:57:15.774
Um, all right, for the next one I figured you could shout out either, like, a news source you use to stay up to date or a startup that's, that you're not invested in that you're excited about. Oh. Or you can do both.
420
00:57:15.874 --> 00:57:25.894
Um, my news source is Sh- Sean. [laughs] That's what I was gonna say. Yeah, no. I literally wrote Swyx's Twitter. [laughs] So in our, in our disc- We have a Latent Space- Oh, yeah... we have a Latent Space Discord.
421
00:57:26.914 --> 00:57:38.014
Any link that ever matters on the internet, Sean's gonna post it into Discord. [laughs] So all I do, I open Discord and, and- Wow... we have like, you know- Yeah, yeah... 40, 50 different channels by topic.
422
00:57:38.034 --> 00:57:48.434
Actually, that's, that's very true. That is very true. I open Discord and I'm like, okay, AI, and then I go developer tools, then I go creator economy, then I go stock and macro, then I go... And all there, so thank you.
423
00:57:48.654 --> 00:57:56.994
Yeah. We, we actually met because of the Discord. Yeah. Uh, it, it was like a COVID thing 'cause everyone's at home and just started a Discord and yeah, that was the origin of Latent Space, just- Mm...
424
00:57:57.034 --> 00:58:07.414
chatting on the Discord. It used to be called Dev/Invest. Yeah. So it was all about developer tools investing. Um- Yeah... and then we were at OpenAI in October of 2022. Uh, we were like, "Maybe we should do a podcast."
425
00:58:07.534 --> 00:58:18.574
[laughs] And then OpenAI was the first guest. The rest is history. Yeah, yeah. Um, yeah. Yeah. Um, I, [laughs] uh, I was not prepared about the, the news sources thing. Uh, I think maybe...
426
00:58:19.794 --> 00:58:24.194
It's hard, it's really shitty to say, but like, just in-person conversations. Yeah.
427
00:58:24.314 --> 00:58:35.494
Um, and, uh, I think the reason I have to be here in SF is because I make friends with people who know things and are smarter than me, and we go for chats, and they're nice enough to share some stuff.
428
00:58:36.014 --> 00:58:49.134
And so sometimes I wish-- I, I worry that I am being used in order to put things out there that are maybe not true, but you know. So I have to exercise my own judgment as to- Yeah... what that is.
429
00:58:49.134 --> 00:58:56.034
I think one of the cool things about the podcast in general is just, like, the opportunity to take these conversations that happen in, like, closed rooms and, and try and bring them on to, to the airwaves.
430
00:58:56.094 --> 00:59:07.674
I'm curious, like, how much of what you... How, how much do you feel like the private discourse is similar to the, to the public discourse? In, in many ways the, it is, uh, surprisingly similar. Yeah.
431
00:59:08.234 --> 00:59:14.734
Uh, as in people at OpenAI learn about things about OpenAI from us [laughs] which is interesting.
432
00:59:15.174 --> 00:59:23.914
Uh, and then there are some ways in which it is drastically not, uh, drastically dissimilar, and those are the things I just cannot repeat until it's public. This has been super fun. I hope it lived up to...
433
00:59:23.934 --> 00:59:30.394
We were looking forward to this for a while. Yeah, this was great. [laughs] We want to make sure, uh, everyone around the horn gets an opportunity to plug whatever they want to plug.
434
00:59:30.494 --> 00:59:37.574
Um, so we'll leave the last word to, uh, to all of us, I guess. Um, where can folks go to learn more about Latent Space and all the exciting things you do?
435
00:59:37.894 --> 00:59:48.674
Um, want to make sure our listeners have a good sense of everything. Yes. So we have a Substack. Latent.space is the website. And then please subscribe on YouTube. Uh, we're doing a lot of YouTube.
436
00:59:48.834 --> 00:59:57.314
We're trying to do better video and all that, so- He set our OKRs and, uh- Come-... it's, it's basically all YouTube. Come, come watch us on YouTube. [laughs] It's very important for me personally.
437
00:59:57.334 --> 01:00:05.854
[laughs] So even if you don't care, just open it. You gotta, you gotta hit those OKRs. Just view it. Well, we have to increase our production value. Look at this. I know. [laughs] I know. We only have three cameras.
438
01:00:05.854 --> 01:00:15.594
[laughs] Um, yeah, and then, um, Sean does a lot of the writing outside of the podcast- Yeah... on the newsletter. Yeah. So, um- Yeah, so it's, like, trying to be a
439
01:00:16.534 --> 01:00:24.354
newsletter and community and podcast and whatever else that we do. Um, yeah, so, uh, I, I guess our...
440
01:00:24.394 --> 01:00:30.534
for, for me, I guess there's Latent Space, but then there's also the other big piece which is the, the conference that I run. Yeah.
441
01:00:30.534 --> 01:00:38.894
Um, and the idea is that I think sometimes you just get the, the good stuff from people if you just put them in front of a lot of people. Uh, and that's really...
442
01:00:39.074 --> 01:00:43.334
Like, I'm mining people for content, and sometimes you put a mic in front of them and they yap for an hour.
443
01:00:43.794 --> 01:00:52.694
Uh, other, other times you have to put them in front of, like, a prestigious conference and then they drop some alpha. And so the next one for us is gonna be June. It's, uh, the AI Engineer World's Fair.
444
01:00:53.014 --> 01:01:04.354
Um, and it, it should be the largest technical conference, uh, for AI. And ours is simple. Just, uh, we, we, we're, we just run a humble podcast. So, uh, [laughs] No. Uh, subscribe to Unsupervised Learning on YouTube.
445
01:01:04.474 --> 01:01:12.054
Um, also Swyx, thanks so much. This was, uh, this was awesome. Thanks for having us. Yeah, it was fun. It was good to see you guys. Thanks for coming on. Hey guys, this is Jacob. Just one more thing before you take off.
446
01:01:12.494 --> 01:01:19.294
If you enjoyed that conversation, please consider leaving a five-star rating on the show. Doing so helps the podcast reach more listeners and helps us bring on the best guests.
447
01:01:19.934 --> 01:01:30.194
This has been an episode of Unsupervised Learning, an AI podcast by Redpoint Ventures where we probe the sharpest minds in AI about what's real today, what's going to be real in the future, and what it means for businesses in the world.
448
01:01:30.614 --> 01:01:39.774
With the fast-moving pace of AI, we aim to help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Thank you for listening, and see you next episode.
449
01:01:40.114 --> 01:01:52.494
[outro music]