WEBVTT
1
00:00:00.100 --> 00:00:15.060
[on-hold music] All right. We are here for a very special edition of Lane Space with my buddy, Jared Palmer, uh, SVP at GitHub and VP at Core AI at Microsoft. That's correct. Dual title. Yeah. Twice the fun.
2
00:00:15.080 --> 00:00:26.320
Is it weird to have two jobs? [laughs] No. I'm only on-- I'm only... The full disclaimer, I'm only on day thirteen, I think. Yeah. So early days, so, so far, so good. So far, so good.
3
00:00:26.400 --> 00:00:34.720
Uh, we've been trying to get you on the podcast for two years, I think. I think so, yeah. You- We've been dying for it. Yeah. You're a busy guy. We only ever do it in person, so... Yeah, we have to do it in person.
4
00:00:34.760 --> 00:00:45.760
Yeah, exactly. It's way better. Uh, I should also plug that you have-- if, if... Jared Palmer fans should dig into your previous podcasts with Ken Muller. [laughs] How about that? It's from, from, from the interim. Okay.
5
00:00:45.840 --> 00:00:56.920
Uh, so, um, but- Shout out to Ken. Shout out to Ken. Uh, before that, you were building, um, I, I guess, like, v0 and AI SDK, and you were just sort of VP of AI at- At Vercel, yeah. Vercel.
6
00:00:56.980 --> 00:01:06.120
Yes, all, uh, AI initiatives and vibes. Yeah. And I feel like basically you went from, like, you sort of building one coding agent to now being the home for all coding agents.
7
00:01:06.540 --> 00:01:16.360
Is that, is that, like, the, the general vibe of Agent HQ? I, I think that's right, yeah. So backing up, um, I spent the last sort of two years or so building v0 at Vercel and AI SDK.
8
00:01:17.000 --> 00:01:30.200
Uh, and then this summer took time off, and now joined GitHub, and today we launched Agent HQ, among other things here at Universe. And, uh, yeah, it's gonna be the home, we hope, of not only agents, but also developers.
9
00:01:30.440 --> 00:01:41.580
And it seems like, uh, the gravity well of this new collaboration space that we're trying to build. Yeah. What do you think, like, basically that GitHub can do that you couldn't do at v0?
10
00:01:42.140 --> 00:01:50.060
GitHub is an enormous platform, right? So these are- Hundred and eighty-three million. It's some amount of- Yeah, it's under a hundred million developers. Um, it's just the scale is immense, right?
11
00:01:50.160 --> 00:01:55.540
Um, and v0 was focused on not only one language, but one framework, right?
12
00:01:56.260 --> 00:02:11.200
And a specific problem space with a built-in renderer and, you know, um, you know, for those who are n-not aware of v0, it's, it's like a bolt or lovable, but it's built by Vercel, and, uh, it's focused on building Next.js apps, specifically Next.js apps.
13
00:02:11.880 --> 00:02:19.760
That constraint was rather liberating for the team at the time, and it lets us really, like, laser focus. It does. You rolled at it. I hope. Thank you. I hope so.
14
00:02:19.860 --> 00:02:29.320
Um, and obviously, at GitHub, you know, we're the home of all languages and, you know, and, and frameworks and, and developers. And so the scope is broadened.
15
00:02:29.860 --> 00:02:38.340
And yeah, it's just, it's just a different, a different part of the map, if you will, right? Yeah. How I've seen it-- So you, you've been basically covering the entire journey of coding agents- Yeah.
16
00:02:38.560 --> 00:02:46.860
-for, you know, from, from the start. Like, what do you think-- Uh, what's your jour-- personal journey through coding agents, right? Like- Sure. -is-- We started out with Copilot, obviously GitHub- Sure.
17
00:02:46.880 --> 00:02:57.960
-started the Copilot trend. Sure, sure. Uh, when-- Tell us about the origin story of v0- Yeah. -and then how that develops and, and maybe where, where-- like, what you wanna see next with- So- -history.
18
00:02:58.260 --> 00:03:05.560
It's, it's funny you ask that. As I've told this story multiple times, I feel like I've unlocked different parts of it in my brain- Yeah. -by going back.
19
00:03:05.600 --> 00:03:12.360
You know, like, so maybe we have to figure out how retrieval memory works. Which is, by the way, interesting how memory works for agents. Totally. That's why I brought it up. This is small.
20
00:03:12.380 --> 00:03:24.400
Like, kind of amazing as you- Yeah. Sometimes you discover new paths, right? Yeah. Anyway, um, the story goes like this. So when, um, ChatGPT first came out, obviously it was incredible, right?
21
00:03:24.540 --> 00:03:35.780
Like, world-changing immediately, fastest growing product ever. I look back at, like, the timeline and dates, and, um, we were very early, like in, when I was at Vercel, uh, jumping into AI stuff.
22
00:03:36.180 --> 00:03:45.900
But the journey kinda went like this. So at the time, actually, there was no AI division, there was no AI group. I was actually the director of engineering for all of Vercel Frameworks, and I was helping Next.js- Svelte.
23
00:03:46.300 --> 00:03:53.300
Uh, Svelte, right. Yeah, so Next.js, Svelte, uh, TurboRepo, TurboPack, Webpack, and all internal dev tools at Vercel.
24
00:03:53.840 --> 00:04:02.360
And I was helping the Next.js team dog food, um, and test the initial implementation of server actions. And instead of building a to-do app,
25
00:04:03.580 --> 00:04:13.020
Guillermo, the CEO of Vercel, was like, "Why don't you build, like, a playground?" And I was like, "Okay, cool." So that led to the AI playground, which is now just part of AI SDK. We'll get there in a second.
26
00:04:13.400 --> 00:04:22.040
Which, by the way, iconic for, like, side by side, but also the- Right, so nat.dev. So, so I'll, I'll- It's nat.dev. Nat.dev. So, so G, G told me the nat day of... I got a, I got a DM.
27
00:04:22.079 --> 00:04:29.200
It was like-- I remember 'cause I was at a bachelor party, and Guillermo, totally online, sends me a note like, "Nat.dev is launching on Monday. You have to ship."
28
00:04:29.700 --> 00:04:37.680
And I had been working on it previously, and so it was like- So it was the same idea. It wa-- Yeah. So he got wind of it, I guess. Uh- Right. So I, I definitely had to, like, jump into motion. And I didn't think we...
29
00:04:37.720 --> 00:04:44.140
We didn't even ship chat first. Guillermo sends me this, this, uh, DM over the weekend. I'm at a bachelor party, and he's like, "Nat.dev," this, the side by side.
30
00:04:44.180 --> 00:04:49.600
He sends me the link and I play with it, and I'm like, "Ciao." Okay, so I spring into gear, ship, um, the AI playground.
31
00:04:49.620 --> 00:05:01.400
What was cool about the AI playground was it, it, it forced me to go through every single model provider's API docs and figure out their s-- their quirks of their nuanced streaming, 'cause at the time it wasn't like everybody used OpenAI.
32
00:05:01.520 --> 00:05:09.400
Yeah. It was like all little quirks. Some of them kinda were compatible. So that was my first foray into it. And then launched AI Playground. That shot to the top of Hacker News. And I remember we didn't...
33
00:05:09.440 --> 00:05:19.040
I didn't even implement chat, 'cause that chat wasn't actually like a fa... like, wasn't as important. It was just like completion completions. So eventually we, we factored chat, and that, that project,
34
00:05:19.980 --> 00:05:29.760
uh, out of that came AI SDK, because I had already looked at all the model providers and all the, um, the combinations, and was like, "Okay, here's that chunk of streaming code you need."
35
00:05:30.240 --> 00:05:38.920
And then AI SDK found that sort of niche of, like, how do we focus on the part that we're gonna be good at, which is, like, that UI aspect of it, but then also knock it in your way.
36
00:05:39.500 --> 00:05:50.620
So that, we shipped AI Playground, then AI SDK launched, and then, um, you know, we're always about, um, demos and having great starter templates at Vercel, and I h- and, and I remember writing Guillermo.
37
00:05:50.620 --> 00:06:03.148
I was like: "You know what'd be cool? This guy Shad CN. Oh, man, he seems, like, amazing, and his UI library is doing great. Why don't we team up and ship a ChatGPT clone?" Open source. And we did.
38
00:06:03.528 --> 00:06:07.868
We shipped this, this, this awesome template, which is now called Chat SDK, but it's great.
39
00:06:08.448 --> 00:06:23.328
And, uh, what that did, though, at Vercel was it, um, set us up for, like, rapid experimentation because we had this really good, like, pretty full-featured ChatGPT ready to rock with all the latest features. Yeah.
40
00:06:23.428 --> 00:06:29.028
So when it came to, like, rapid prototyping that summer, now we're summer '23, it was so great. It was, it was, like, liberating.
41
00:06:29.128 --> 00:06:44.668
So I remember at that point, I had gained some momentum internally and pivoted almost entirely to AI. Um, and I had Shu Ding, uh, who you're friends with, um, and Max later, and, uh, Shad Cian now were cooking.
42
00:06:45.248 --> 00:06:53.528
And I think at that point, code execution had just come out. I think that's my timeline here. Yeah, they called Sandbox, uh, Interpreter. Interco Interpreter, that's what they called it at the time.
43
00:06:53.808 --> 00:07:03.588
And I had a very-- As soon as I saw this, I had a very ambitious idea and proposal to present to Guillermo, which was like, what if there was some... And mind you, tool calls don't exist.
44
00:07:03.608 --> 00:07:07.488
The context window is four thousand- Wow... tokens. Yeah. So, like, there's not much here.
45
00:07:08.108 --> 00:07:26.388
Um, what if we had this thing where, like, you could prompt, and sometimes it would do code interpretation, and then maybe we could sort of-- Sometimes it would do code interpretation, but then other times it would choose to render, like, UI, or then it would render sometimes, like, a document or- Right, inline in the chat.
46
00:07:26.448 --> 00:07:33.508
Yeah. And like, it would just have different sort of render modes. Like a generative UI type thing. Yeah, and maybe you could pipe them together, so, like, the output of one could pipe into...
47
00:07:33.568 --> 00:07:43.728
So if we did code interpretation, we coerce it to always emit, uh, like, tabular data. Yeah. Maybe we could pass that to another prompt that was like a UI, and just, like, some idea there. It's kind of crazy.
48
00:07:43.988 --> 00:07:51.588
But if it sounds like these are just tool calls, that's exa-that's exactly what these are. That's what it was, yeah. Um, so it became pretty obvious that, like, the...
49
00:07:51.968 --> 00:08:00.668
So I came to Guillermo, and we-- at the time, we just had a sec-secure-- we had a sort of security debate, like should we code interpretation with the ability to fetch data?
50
00:08:01.148 --> 00:08:09.288
Um, like giving it internet access was, like, kind of... Now they're like, "Fine, whatever, whatever, do whatever you want, wild, wild west." But at the time, it was, like, a little scary.
51
00:08:09.848 --> 00:08:23.968
So we kind of said, "Okay, no to the code interpretation, but this UI thing is pretty neat." And so that, this, like, prompt to UI, that was, like, the aha moment of v0, but the models were not very good, right?
52
00:08:24.028 --> 00:08:36.168
So-- Or relative to the way they are now. And it was, uh, for GPT-4 era? This is just into the GPT-4 era, and now we're probably at a, uh, sixteen thousand token context window, so you can't really do chat.
53
00:08:36.468 --> 00:08:49.488
So we had to kind of invent this kind of new paradigm of, like, fake it with completion, but that forced us to do sort of the click the... Well, we-- The, the initial v0, which launched I think in September '23, um,
54
00:08:50.908 --> 00:08:59.588
it would look more Midjourney. In fact, if you go back to the original suite, it was like Midjourney for React because it was all very visual, and you could click on different, uh, components and elements and re-prompt.
55
00:08:59.788 --> 00:09:07.408
But it was, again, we're, we're kind of hacking this because we didn't have chat, and we didn't have tool calls. Um, and then fast-forward again, um,
56
00:09:08.448 --> 00:09:18.288
you know, that launches, and then probably, like, nine months later, um... It took us, like, nine months to, uh, uh, get to, like, a million ARR, uh, this little team.
57
00:09:18.348 --> 00:09:30.028
But then the models progressed, and from, you know, GPT-4, GPT-4 32K, big, big, big boy. We never really got GPT-4 Turbo working. I don't know why. Just never happened.
58
00:09:30.448 --> 00:09:35.388
And then switched to oth-other frontier models and, and then started doing our own models and stuff like that.
59
00:09:35.888 --> 00:09:47.128
Uh, but fast-forward another ten months or nine months or so, and then we, we rebased towards chat, and now the models finally could do chat. Um, and the artifact pattern had evolved, so it was time to rewrite.
60
00:09:47.568 --> 00:09:56.868
When we launched v0, the chat version or the new v0, whatever you want to call it, uh, it, it's like fourteen days, another million MRR, fourteen days, another million MRR. Right. It was like a rocket ship after that.
61
00:09:56.928 --> 00:10:04.068
Yeah. And, um, that just proceeded, uh, and we just kept cooking. And so that's been the journey. Uh, we just kept perfecting.
62
00:10:04.088 --> 00:10:10.188
And, and what was really, um, liberating for us was actually the focus on just one stack or one framework.
63
00:10:10.208 --> 00:10:19.628
When everybody else was trying to do general purpose coding agent, we were like, "No, we're just gonna focus on Next.js front-end and ShadCN," and, and that was really-- That allowed the team to focus.
64
00:10:19.668 --> 00:10:27.588
So that's the, that's the story arc. I mean, to be fair, like, bolt, loadable, like, because Next.js is so dominant, basically everyone has to be good at Next.js. Right.
65
00:10:27.648 --> 00:10:33.248
But being focused on, like, even right down to the UI library and component stuff, like, that, that actually helps a lot.
66
00:10:33.428 --> 00:10:40.248
We also started working with all the frontier model labs to help because it was in our, Vercel's best interest to have them be great at Next.js. Yeah.
67
00:10:40.628 --> 00:10:45.908
And also because of the post-train models, and you can read about this on the Vercel blog, like, we can, uh, um...
68
00:10:47.168 --> 00:10:56.468
The post-training, um, like, harness that we created, we were started sharing and stuff with other model labs and stuff like that, and we had all our data in a very hygienic state- Yeah... to work with them, so.
69
00:10:56.808 --> 00:11:00.148
Did you ever debate internally, a-and because, uh, you know, I've--
70
00:11:01.128 --> 00:11:12.628
I-- My-- From my seat at Cognition, I can also see this, where you should pick the best qualities of every model and string them together in the v0, or you have the model selector, and you let customer's choose. Yeah.
71
00:11:12.648 --> 00:11:20.528
We went back and forth, and I think- Yeah... um, you know, we launched... We, we went back and forth on all this. I think at the end of the day, there's pros and cons. Yeah.
72
00:11:21.028 --> 00:11:30.368
One of the benefits of having, uh, your own branded model or synthetic or composite is that you can stitch these things together. Yeah. There's like- You can get higher levels of...
73
00:11:30.408 --> 00:11:37.768
And, and, and, and now it's a little different with, with the, say, agentic flow, but even look at what you g- uh, like, what you guys launched recently with- These are actually rep, right?
74
00:11:37.828 --> 00:11:50.608
So search is gonna be a different model than what generation, but you know, the genesis, but, like, 'cause it's, it's a shorter... But, like, search and genesis are two different mo- like, like, entire subsystems, right?
75
00:11:50.688 --> 00:12:00.548
So you can have search evals that are gonna be totally different. And so how do you... So where we ended up now, uh, or we, wh-where, where we've probably ended up now is, like, um-
76
00:12:02.348 --> 00:12:06.668
For a long time, we didn't have model selector, and then we had our own models, which were composites, which we talked about.
77
00:12:06.948 --> 00:12:19.108
And then like, um, and that would allow us to s- you know, mix, match, and I think that's probably if what-- It's also nice 'cause you get as a-- this is like the product hap- you get to brand it. Yes. Right?
78
00:12:19.668 --> 00:12:28.948
And you can decouple it from the launch of the Frontier Lab. Yes. How do you guys-- how does cognition even bill for it? Like- ACUs, uh, compute, compute hours. Right. Some synthetic unit, right? Yeah.
79
00:12:29.368 --> 00:12:38.468
So it gets a little wonky. Yeah, yeah. We can go on for pricing this, uh, this stuff. It's, it gets challenging. But the nice thing about having the, like, brand name model is that, like,
80
00:12:39.408 --> 00:12:50.248
you get to co-launch with the provider, and they'll hype you up. So, uh, but your billing needs to then is capped at whatever retail is, right? Or, or, or some- Right. Right, right, right.
81
00:12:50.428 --> 00:12:57.408
Or- You can't really charge a, uh, too much of a premium. You can, but- Right. But then people are like, "Why am I paying for my own API key?" Right? And it's like, well,
82
00:12:58.388 --> 00:13:00.968
then how do we charge you for suite rep or something, right? Going forward, so. Yeah, yeah, yeah.
83
00:13:00.968 --> 00:13:10.528
And, and so like I, I think it's some part of it is the cynical, like you wanna m- create a sustainable business and independence from the, the model labs, but the other part is genuinely you actually do get better performance- Right...
84
00:13:10.648 --> 00:13:18.288
if you like string together all these things. Yes. And so it's, it's tough. I think, uh, uh, well we've switching gears to GitHub, like we are all about model choice now- Right...
85
00:13:18.308 --> 00:13:31.888
and like making sure that, uh, and what's cool is that like it's, we also have Copilot, which is our harness and Copilot CLI, but we also have third party harnesses like Cloud Code and, and Codex and Cognition now like in a, uh, Agent HQ.
86
00:13:31.948 --> 00:13:43.308
So you kinda get the best of both worlds. Yeah. And like, I think that's gonna be awesome and ultimately what people want. Yeah. I think, uh, also the model layer is not the right abstraction to do the switcher anymore.
87
00:13:43.588 --> 00:13:51.488
It's-- Which is weird because that's where you started with the AI SDK. Yeah, yeah, yeah. Exactly. Um, but now it's like the model and the agent have to be strictly tied together, like very, very strongly balanced.
88
00:13:51.528 --> 00:14:00.888
You can't loosely bound it and just do a generic interface 'cause then you're just gonna have the lowest common denominator of all the models. If you're, if we're in agent world- Yeah, yeah...
89
00:14:00.928 --> 00:14:06.388
which may just be better than chat world, like in general, like- Yeah, yeah... better. A-agent world is, is a much better abstraction.
90
00:14:06.428 --> 00:14:14.808
I'm, I'm calling it agent world, but I mean by like a loop with maybe compute runtime- Yeah... and like files. I do- So that's, that's now your definition of agent? You're dropping your official definition here?
91
00:14:14.968 --> 00:14:23.468
No, don't put me on the spot. [laughs] But, um, maybe. Ma-- No. So my, my initial definition of agent for, for AI is, 'cause like I actually fought, I was dying on this hill, 'cause AI SDK,
92
00:14:24.388 --> 00:14:29.728
everyone else is an agent framework, and I think maybe they, they actually went to like this, I don't know what it says on the front page now, but like, who knows?
93
00:14:30.428 --> 00:14:41.008
But like- They say like six- An agent is, uh, you know, an agent is, uh, orchestrating, uh, ch-- you know, an API request with a queue and a for loop. Okay, but a coding agent now has meant so much more.
94
00:14:41.048 --> 00:14:51.928
There's like these, you know, coding, coding agent SDKs, and you've got sandboxing and file systems and tool calls and, and I do think that is a uniquely d-- I'll call that agent world and when charging coding agents here.
95
00:14:52.488 --> 00:15:05.588
And yeah, I think that that seems to be where things are going. And even I believe the, the Claude Excel agent is basically, I was talking, um, to, to Mike Reager backstage, like I think it's like related to Claude Code.
96
00:15:06.748 --> 00:15:12.288
Uh, it could be. I actually don't know how it's implemented under the hood. Yeah. It wouldn't surprise me if it was. Yeah, yeah, yeah.
97
00:15:12.308 --> 00:15:24.888
They seem very all, all, all in on skills, which is kind of like an interesting- What do you think of skills? Uh, it's kind of DXT, which is like the, the sort of bundled f- uh, version of MCPs. Okay. I've heard of that.
98
00:15:24.908 --> 00:15:33.068
Uh, it wasn't-- Yeah, it wasn't-- The, the reason you don't know about it is because it wasn't very popular. Okay. Uh, so skills is kinda like the, the, the second shot that is very LLM pills.
99
00:15:33.208 --> 00:15:43.688
Like it's like, just read my markdown and just read this, this directory of files and go nuts. Uh, as, as long as like it can understand that you have the capability to run code, to read files, you're good.
100
00:15:44.308 --> 00:15:50.868
Uh, and, and actually that, that is the universal interface, which is a file system. Right. Uh- Back to agent files. Yeah [chuckles], which is, which is kinda cool.
101
00:15:51.248 --> 00:16:02.608
Uh, yeah, so I mean, like I, I think like what you're hitting at is, uh, you know, th-this philosophy of like our understanding of what coding agents, the like the minimum bar is over the last two years. Yeah. Right?
102
00:16:02.648 --> 00:16:10.448
Like you've, you've lived this journey and like now you're basically kind of like the kingmaker or like the [laughs]- I don't know about that... you, you run- You have phrase. Yes.
103
00:16:10.488 --> 00:16:17.268
You run, you run Agent HQ and, and like, I mean, I, I, I imagine you have other projects too, but like Agent HQ is like the big one that we're talking about here.
104
00:16:17.768 --> 00:16:33.748
Um, like what are you seeing from like the different agents? Like what do you want to-- this to become? Such a good question. Um, I think that Agent HQ and GitHub itself need to like co-evolve and,
105
00:16:34.708 --> 00:16:42.928
you know, one of the things that Microsoft has done really well is by putting things that are, um, alike closer together. And so you think about the new core, core AI organization- Yeah...
106
00:16:42.928 --> 00:16:50.888
you've got Visual Studio- Okay... Visual Studio Code, GitHub, and parts of Azure all in one. Um,
107
00:16:51.808 --> 00:16:57.168
and obviously the GitHub team and the VS Code team have been working closely together for a long time, but now we're really close together.
108
00:16:57.628 --> 00:17:06.428
And I think for me, one of the cooler things that, um, Agent HQ can sort of offer is this seamlessness, this fluidity with your workflow, right?
109
00:17:06.488 --> 00:17:20.888
So if you saw in the demo today, we saw a, um, a demonstration of, uh, you use Agent HQ, you fire off a task, and it creates a PR, but you can also open that PR up in VS Code in one click. And that's, that's awesome.
110
00:17:20.908 --> 00:17:38.068
And I think my-- the, the vision for, for, for GitHub as it evolves is to look at those touch points where AI can be sprinkled in, you know, uh, salt-based style, um, into the native workflow, whether you're assigning an issue or maybe some new stuff that I think we should like focus on.
111
00:17:38.088 --> 00:17:49.848
Maybe it could be like, how do we resolve a merge conflict? Oh my God. Right? Like, how do we maybe pop open an action or like get in, right? And so it's like- I think solving merge conflicts is my definition of AGI.
112
00:17:50.248 --> 00:17:58.948
Sh- totally. But like you get that error on an action and you're like, we've all been in that, we've all been in that sort of flow where like actions kind of don't work lo-locally versus that tool act.
113
00:17:59.048 --> 00:18:02.548
If you ever try it and you're like, "Oh, I don't have that set up on my machine. I haven't done this in a while."
114
00:18:02.848 --> 00:18:12.304
So you're like pushing up and you're, you're this like, "Okay, what if we could just put like, oh, you know, comment or kick off a task to solve this for, for you or-" Sure things there.
115
00:18:12.384 --> 00:18:24.444
I think what I-- what I'm trying to describe is this like this workflow where it's just like seamless and fluid, and you can stay in a flow state across whether you're acro-- like, across all devices, mobile, web, on github.com, or in your local editor.
116
00:18:24.544 --> 00:18:33.464
And I think that's where, you know, my focus is gonna be in the next- Yeah... six, six months or so. Yeah, yeah. Um, just a side tangent on this. Uh, so
117
00:18:34.364 --> 00:18:44.544
one of the things that Microsoft also owns, I don't know if it's Microsoft or GitHub, is Dev Containers. And I think, like, a very important concept for sandboxing environments- Yeah...
118
00:18:44.564 --> 00:18:56.404
whatever you call it, uh, it is kind of a light version of what Docker containers are, kind of. Right. Uh, do you see that as a standard that we should invest in as like a, like a thing?
119
00:18:56.524 --> 00:19:05.784
Like, 'cause it's supported in VS Code. I don't think it's just that popular outside of VS Code. Yeah. Uh, it's used internally at GitHub too, for like d-development at GitHub. Oh, yeah, yeah. Which is cool.
120
00:19:06.164 --> 00:19:15.384
Um, yeah, I think they were so far ahead almost. Like it was... But now there's sand-- this, like, sandboxes, there's so many of these days, right? So I think Cloudflare just launched theirs. Yeah. Uh, there's Daytona.
121
00:19:15.544 --> 00:19:25.504
Vercel. They're here. Vercel. Um, Mo-Modal- Yeah... uh, which I think Lovable uses. Uh, I have no idea. I don't know. Yeah, I mean, you probably have your own. I don't know. What do you, what do, what do you guys use?
122
00:19:25.584 --> 00:19:29.104
Uh, just some Kubernetes pods. Okay. You guys are rolling it yourself. Very cool.
123
00:19:29.504 --> 00:19:38.464
I think that's maybe the runtime, but, but there's, there's work and discussion about what that runtime should be, even internally at Microsoft, and we've got a couple different competing things.
124
00:19:38.524 --> 00:19:45.844
So we'll figure it out in the next, you know, in the next cycle here. Right. But, uh, there's a great point. Like, there, there's a lot of cool stuff that is in a Dev Container.
125
00:19:45.864 --> 00:19:55.024
You've already got VS Code loaded, you've got a file system- Yeah... you've got a sandbox, you've got the security protocol. Yeah. It's also, like, wired into GitHub Enterprise- Yeah... and, like, ready to be packaged.
126
00:19:55.224 --> 00:20:04.664
So there's lots of goodness there. Yeah. I see, like, the, the number one pain point that Cognition has, but also Codex, also presumably the other guys, is repo setup. Yeah.
127
00:20:04.664 --> 00:20:15.324
Which is effectively what Dev Containers and a Docker file does for you, is like, run this thing, then that thing, set this up, do that thing. Uh, why is it so hard? Like, why haven't we solved it?
128
00:20:15.324 --> 00:20:23.704
[chuckles] I don't know. I think it's hard because you can't predict what's in the, what's in the repo, right? So it's like, and- Yeah... you don't know when they've bundled FFmpeg. You don't know. You just don't know.
129
00:20:23.784 --> 00:20:32.464
Right. It, it's nice when, like, if it's just Next.js, you just run PMPM install. [chuckles] Correct. Correct. You, you can, like, do special k- Like, there's obviously cons...
130
00:20:32.764 --> 00:20:42.124
through constraints, you can make optimizations, and I think the general purpose container is just, like, challenging. That being said, though, I think there's probably some work to do on,
131
00:20:43.164 --> 00:20:51.444
you know, auto-detection and preempting and, and stuff like it to be done there. But it's just a bigger, it's a broader problem space, right? Yeah. Uh, and, uh, yeah. So, so fun fact.
132
00:20:51.624 --> 00:21:01.604
When I was at Netlify, I actually wanted to reach out to Vercel to do, like, a standardized open source auto-detection thing of frameworks. Oh, yeah. And I, I...
133
00:21:01.804 --> 00:21:10.244
Like, we, we never, we never really, like, got internal momentum on that. It was an idea. I was like- Okay... shouldn't this be open source, you know? Yeah. Like, auto-detection is a, is a common utility- Yes...
134
00:21:10.384 --> 00:21:20.284
that everyone needs. Yes, yes. I remember that. I'm having a flashback. It's like [chuckles] half [stutters] Yeah. Everyone builds their, their- Yes. Yeah... probably we shouldn't all build it. Right. No, it's, um...
135
00:21:20.764 --> 00:21:30.404
And then also, like, what are your defaults? They're not exactly the same, which would be better to, like, just having even the same, like, uh, preference stack- Yeah... of defaults- Yeah... is the right- Yeah, yeah...
136
00:21:30.504 --> 00:21:37.324
um, would be great, 'cause then we could move the whole ecosystem together- Right... from, like, the PMPM run, right? [both laughing] Like, um, okay. So
137
00:21:38.544 --> 00:21:48.344
are there other, um, movements or protocols or standards that you're interested in? Like MCP was a big winner this year. Yeah. There's other, like, I don't know, AC...
138
00:21:48.504 --> 00:21:56.684
A2A, ACP, all this, all these, like- That'd been interesting. I'm not as familiar with, um- ACP, the payments one or the, is that- Right, right... Redone? Oh, no, there's that one. The, uh- That one. Okay.
139
00:21:57.244 --> 00:22:06.504
And then the, the payment that, was it Stripe or Coinbase? Stripe. Stripe? Yeah. Yeah, that's very cool. We've had them on the pod. Okay. Yeah, that's very cool. Um, but it'd be interesting to see if that takes off.
140
00:22:07.124 --> 00:22:17.164
Um- I mean, it's Stripe. [laughs] So... [laughs] Yeah, but it still needs to be adopted by, like, the clients, right? Yeah. And, and, and I think that's, that's fascinating. Uh, but MCP is huge.
141
00:22:17.204 --> 00:22:19.624
It seems like it is the way that a lot of the...
142
00:22:20.424 --> 00:22:26.464
Especially when it comes to, like, digital transformation or some of our enterprise customers, it's where they're gonna able to be a- or where they are able to add context.
143
00:22:26.844 --> 00:22:29.344
In addition to that, we also have custom agents that we announced today too.
144
00:22:29.384 --> 00:22:36.924
So, like, you can work with prompts and stuff within your, within your agent HQ and, and customize these agents for different tasks and, and those can have MCPs and such.
145
00:22:36.964 --> 00:22:44.904
And I think that's gonna be really powerful from like a, a platform perspective. Yeah. Gets me excited. That's what, like, I think is, like, shipping now in, in the next...
146
00:22:44.964 --> 00:22:55.024
But we're always on the lookout for, like, the next thing and, uh, I don't know. What, what's on your, what's, what's top of mind for you? Uh, for, like, standards or- Yeah, standard. Standards? Or should it be built?
147
00:22:55.064 --> 00:23:05.364
Uh, Dev Container is like- Dev Container. A container... like, is the... Look, I think Dev Container just has a PR problem. Right. It's a great idea. Right, right. Just no one makes it interesting. Yeah, yeah.
148
00:23:05.504 --> 00:23:08.884
I think you can do it, basically is what. [chuckles] Okay. Add it to my list.
149
00:23:09.464 --> 00:23:20.244
But before that, probably you have a bunch of other stuff that I, I wa- I do wanna get to, uh, but just staying on the AI stuff, like, uh, I, I think we're, we're, we're actively exploring computer use as, as a thing.
150
00:23:20.324 --> 00:23:30.184
Like, because it, it kinda got going a little bit. People were very excited, and then they found out it was slow and bad and inaccurate. It is computationally intensive- Intensive... from my understanding.
151
00:23:30.504 --> 00:23:41.924
It's getting better. Yeah. Uh, especially with open vision models like DeepSeek OCR and, and OMO OCR. Uh, it, it, it... Like, just give it a few more turns of the- Yeah, yeah... scaling. It seems very...
152
00:23:41.944 --> 00:23:48.164
And it seems like it's, like, you need that edge case, uh, and primary. It just seems like a modality worth, worth pursuing.
153
00:23:48.324 --> 00:24:01.564
Yeah, I think a lot of people are, uh, on the CodeGen side, the code agent side, a lot of people are trying to think about, all right, we had this evolution from copilot to, like, uh, you know, a more agentic, uh, sort of cl- like a cloud code situation is where I think that the status is.
154
00:24:02.044 --> 00:24:10.624
Like, what's next, right? What's, what's the, what's the, the, the, the obvious next step? Oh, you go. I mean, making them good. Making them good. Yeah. You don't like cloud codes? No, no, I, I don't.
155
00:24:10.804 --> 00:24:21.448
It's more just like, you know, the, the, the devil's in the details. Like- Yeah... going from ninety percent, going to, like- But hill climbing, it, it, it gets steeper- Yeah... in my opinion.
156
00:24:21.568 --> 00:24:29.688
It gets steeper, and so going from ninety percent success to ninety-five, to ninety-eight, to ninety-nine, to nines of success- Yeah... I mean, really hard.
157
00:24:29.748 --> 00:24:38.048
Paying Mercora a lot of money for, uh, expert programmers of open source maintainers to, like, you know- And then you realize along the way maybe the users aren't that good at it. Right. Yeah.
158
00:24:38.048 --> 00:24:41.548
[laughs] Like, um, no, but it-- I, I just think there's a lot of work to do to finish the swing.
159
00:24:42.268 --> 00:24:47.888
Um, and there's a big difference between ninety-eight percent and ninety-nine percent correct, uh, and that's, like, noticeable.
160
00:24:48.588 --> 00:24:53.928
Uh, and this used to hit, you know, if, if you're working on an AI product, you, you probably don't realize how...
161
00:24:54.448 --> 00:25:03.548
You've probably seen this, like most people are blind, like living in la-la land about how, um, poor quality, uh, their, their AI product likely is.
162
00:25:03.768 --> 00:25:09.728
Unless they're really measuring, like, the number of error-free sessions, like how many errors are coming from the in-infra, infra providers.
163
00:25:09.788 --> 00:25:14.568
Like, you know, how, how many requests are dropped, how fast, how fast these things are.
164
00:25:15.188 --> 00:25:23.608
Um, and so that's something that we cared about at Vercel quite a bit, and, like, we'll care- What other pro-- Do you have, like, a daily review of your dashboard or how, how- Yeah... I don't know. Daily?
165
00:25:23.688 --> 00:25:34.388
Daily would be slow. Yeah. Oh, okay. I thought you were gonna say daily's too much. [laughs] Daily's... No, and even thinking, it's like, uh, every three hours, k- uh, roll-up of, of key metrics and stats. Yeah. Um,
166
00:25:35.468 --> 00:25:42.808
and, like, one of them was, like, error-free sessions and other things like that, that was, like, really important because, you know, in the-- especially now with agents that are like multi-turn.
167
00:25:44.208 --> 00:25:53.868
Um, I have a tweet about this that was, like, in 2024, which is that, like, agents will really only work when we get to not only, like, the in- more intelligent models, but better reliability of the infrastructure providers, right?
168
00:25:53.968 --> 00:25:59.848
These aren't, these are not-- Inference is not like a database, like- Yeah... update uptime, right? So there's still differences between providers.
169
00:25:59.868 --> 00:26:05.728
There's still differences between performance and difference uptimes, and that's why you see things like OpenRouter being very successful- Mm...
170
00:26:05.808 --> 00:26:16.268
and, and different gateway products because, like, reliability, you need to switch. They go down all the time. Um, so long story short, yeah, we would, we would do, like, you know, uh...
171
00:26:16.328 --> 00:26:23.368
It was almost like video game style. Like, we'd have, like, all the data coming in all the time. Yeah. Uh, and that allowed-- We, uh, I used to joke it's my mood ring, like good day, bad day.
172
00:26:23.948 --> 00:26:27.288
Um, so it was very successful for us.
173
00:26:27.548 --> 00:26:48.248
Uh, I think other teams should adopt that, like, data-driven approach, um- I think one thing that's surprising is the lack, the relative lack I still see on data analyst agents, where you can sort of chat it, uh, like add a Slack bot for, uh, the, the precise analytics that you want to generate, um, because I think we're still in the BI era.
174
00:26:49.528 --> 00:26:57.988
Yeah. Isn't that weird? It-- Yeah, I totally agree. Um, it's interesting that that space hasn't been, like, captured as much. Yeah. Like, I guess maybe now.
175
00:26:58.388 --> 00:27:09.328
Actually, I'm interested in this, like, shift to away into, in, like, into knowledge work tasks with coding agents. Okay. I wonder- Using coding agents for non-coding tasks? Correct. Do you, do you do that personally?
176
00:27:09.428 --> 00:27:10.648
I do, yeah, yeah. Yeah. What, what do you do?
177
00:27:11.188 --> 00:27:17.908
Well, like, I was-- Like, this summer, I was doing, like, I was helping-- I was trying to automate some of my dad's workflows and stuff like that, and just, like, some of his...
178
00:27:17.928 --> 00:27:28.548
He's got some Excel spreadsheets and, uh, for, like, accounting, like, like, uh, financial accounting or managerial accounting, I guess. Um, and, uh, yeah, just like point Cloud code to that stuff and see what happens.
179
00:27:28.588 --> 00:27:41.148
And, like, I, it's, it ends up doing Python, um, and, and, and generating some scripts, and it kinda got off down other hairs, but it was like even he saw that it was better at it than the chat client than Excel.
180
00:27:41.288 --> 00:27:49.488
Super obvious. It, it- It became kinda obvious. Yeah. It, it felt better. I wonder if he can try Cloud for Excel and see. Yeah. Yeah. That's it. You got me, right. The, um...
181
00:27:50.148 --> 00:27:57.708
And then, of course, you got the browser, the... I don't even want-- Not browser. Browser-based agents, but not, not computer use, but browsers with agents. Agent browsers. Wait, wait, wait, wait.
182
00:27:57.768 --> 00:28:05.888
Oh, right, right, right. Agent browsers. LogMeIn, Perplexity. Yeah, everybody's coming with that. Yeah, yeah. It's like, is that the better... If that's true, then maybe the, the general purpose injection point is there.
183
00:28:06.128 --> 00:28:11.048
Yeah. Uh, what do you, or have-- Which-- Have you tried any of the agent browsers? All of them. All of-- Which one? What, what's your tip?
184
00:28:11.068 --> 00:28:21.608
I, I am very, uh, I'm currently maining Atlas, mostly because I just wanna give ChatGPT a fair go. Okay. Um, I-- But I'm very stuck to the Arc and, like, the vertical tabs. Oh, okay.
185
00:28:21.688 --> 00:28:29.868
I think, like, any pro user, like I have multiple businesses. How many tabs do you have? I'm context switching, right? I, I have hundreds of tabs open. I made a, I made an open source tool called Chrome Dump.
186
00:28:29.908 --> 00:28:39.128
You can find it on my GitHub, uh, where it literally dumps all the tabs open. It summarizes them, and then I can close it by deleting them on Markdown. That's pretty cool. And it syncs, it syncs to...
187
00:28:39.128 --> 00:28:46.248
[laughs] So you just go on, like, a- Oops... you just go on, like a, like a, like a, a bender. Yeah. And then you just dump it. Yeah. Nice.
188
00:28:46.268 --> 00:28:55.348
I mean, it should be as easy to close as Markdown, and Chrome isn't, isn't that good at the performance side of things yet. And you were working on, like, some browser comparisons or something before. I, I was.
189
00:28:55.368 --> 00:29:04.048
But, so I tried to build it in Tauri. Okay. How'd that go? And Tauri explicitly doesn't want you to build a browser, and I, I tried to fight it too much. I see. Yeah, I see. Very cool.
190
00:29:04.228 --> 00:29:11.168
So just to wrap things up, because I, I know we're, we're around about time. There are other side projects, uh, tasks, and things that you, you've, uh, announced here.
191
00:29:11.588 --> 00:29:19.308
Uh, first of all, redesigned GitHub homepage, which a lot of people don't even know GitHub has a homepage. [laughs] I, I leg-- I'm legitimately one of them.
192
00:29:19.328 --> 00:29:28.548
That tweet, which is, uh, re-- was this tweet, like, printed out. Like- What? You see the tweet. Like, there's a tweet, um, from Riz, and it's like- Love it... no one uses- No one, you know, GitHub homepage.
193
00:29:28.608 --> 00:29:32.628
None-- All of this stuff is totally useless. I'll, I'll pull it up. Like- Yeah, yeah, yeah.
194
00:29:32.808 --> 00:29:37.668
I, I have-- I pulled it up today, 'cause we were like, when we launch- [laughs] Let, let, let me get it right, 'cause I gotta do it right. Hold on. Okay.
195
00:29:37.788 --> 00:29:51.748
Uh, was, um, "Incredible how pretty much the entire GitHub homepage is useless." [laughs] And it's one point three million views and nineteen thousand likes. And this was, uh, May two '25. So heard,
196
00:29:52.668 --> 00:30:01.728
uh, the team, the team made improvements, and today they launched a new GitHub homepage- Yeah... which I'm very proud of. Um, and they should be really proud of. It's got tasks at the top. It's got recent PRs.
197
00:30:01.908 --> 00:30:09.308
Some stuff is still there, like your recent repositories. I still f-- I think there's more work to do, but it's, like, really overhauled, and they did an amazing job with it. Yeah. So they nailed it.
198
00:30:09.728 --> 00:30:21.068
Uh, but you know, more work to do. Never done and, like, hopefully we can keep iterating with, uh, the community and, and everyone and, uh, and keep going. The last thing I want to hit you on is Stack Dips. Oh, yeah.
199
00:30:21.148 --> 00:30:29.648
You asked everyone when you joined, "What should I work on?" or something. Yeah, what happened- I don't know if this is your job specifically. It wasn't. But- It wasn't... why do people want Stack Dips so much?
200
00:30:29.848 --> 00:30:39.664
Let's-- I, I think you have some history there. Yes. Anyone who's interacted with anyone at Facebook knows- Right... about Fabricator. They're religious about it, yeah. Uh, so can you explain why, why it's been so...
201
00:30:39.784 --> 00:30:55.644
Like, what it is, why is it so hard? Okay, so this, um, this concept of a pull request, which we're all familiar with, uh, you write some commits, you open a PR, and then you merge the PR, and you go about your day.
202
00:30:56.304 --> 00:31:03.464
So as you scale, like, larger organizations, um, and, like, you look at your history, and, and, and there are people who are, like, very
203
00:31:04.944 --> 00:31:11.544
r- I'll say, like, have, like, r- near religious beliefs about how to Git, how to do Git right. Rebase vers- Yeah... rebase versus merge.
204
00:31:11.624 --> 00:31:20.984
There's, there's a crowd that wants to fast-forward the repository so to preserve all the history, and then there's a crowd that wants to squash and, and, and merge into the main thing. I, I'm team squash. Okay.
205
00:31:21.164 --> 00:31:31.024
Anyway, Facebook, and I've, I've never worked Facebook, but, um, in my previous, uh, startup before Vercel, Turborepo, I did a lot of research on build systems.
206
00:31:31.604 --> 00:31:39.304
And at Facebook, they have, not only they have their custom build tools called Buck, they also have a custom file system, and they don't use Git.
207
00:31:39.704 --> 00:31:53.124
They use, uh, Mercurial, which, and then now it's sort of custom, and it's all wired together. Um, and at, at, at Facebook, they don't use pull requests. They have a different sort of philosophy. You can, um...
208
00:31:53.324 --> 00:32:05.264
It's sort of like the best way to ana- like, think about this is, like, imagine every PR just had one commit in it. Yeah. You could branch them, um, and res- and the critical thing is you can restack them.
209
00:32:06.384 --> 00:32:17.604
And then, um, if you restack or make a change later, like earlier in the stack than later in the stack, and these stacks are just diffs, right? The commits are just diffs, and it's the term stack diffs.
210
00:32:18.144 --> 00:32:22.564
You can then collapse them and merge the last one and merge them towards the end. It just gives you a little bit of a nicer workflow.
211
00:32:23.124 --> 00:32:34.524
And it's what people who, if you work on a monorepo or you work on a very, very large code base, it's a really, really nice way to work, especially if you've got a system that will re- that will d- automatically restack.
212
00:32:35.204 --> 00:32:48.544
Um, and then if you think even more deeply about it, like, and get really deeper into the weeds, uh, you can decide which, um, diffs in the stack CI should run against, if you get fan- if you get fancy. Okay.
213
00:32:48.764 --> 00:32:51.584
May not be- By, by like certain- Yeah... commit messages and, or- There's just always...
214
00:32:51.624 --> 00:32:58.444
Yeah, you could decide, like, maybe this one doesn't need it or skip that one or, or whatever, and you end up getting these, like, sort of these groups, these, these stacks.
215
00:32:58.904 --> 00:33:07.064
And it's really nice from a code review perspective because, um, when you go to update or you can update a different part of the stack, it just makes it a little bit more fluid, and so it's what people want.
216
00:33:07.404 --> 00:33:15.144
There are a couple tools out there in the market that do this kind of behavior. One's called Graphite. There are a couple others. Too many tools called Graphite. There is- There's another Graphite right here. Yeah, yeah.
217
00:33:15.164 --> 00:33:26.224
[laughs] Um, and just, it's got, it's a great workflow. And so it's been the top pull request, or sorry, the top, um- Feature request... feature request, thank you, uh, at GitHub for years.
218
00:33:26.264 --> 00:33:35.104
From, from the community's perspective. I don't know at GitHub, but... And then as s- as soon as I joined, the first, the first [laughs] thing I did was go look this up. [laughs] And, uh... Well, not the first thing.
219
00:33:35.184 --> 00:33:38.344
I asked how I should make it up better, and it was the top feature request, right?
220
00:33:38.904 --> 00:33:48.204
Uh, and then I went to go, like, okay, investigate, like any good product person would, and it, there's been multiple attempts at this in- internally, um, going back to, like, twenty twenty. Okay.
221
00:33:48.704 --> 00:34:00.484
And there was one very, very, very, um, polished attempt too in twenty twenty too, and it just... I, I don't have all the context, so, but it was, it was, there was a pretty good implementation.
222
00:34:00.564 --> 00:34:11.404
All of the work was done on the client, and it reintroduced this new concept called stacks outside the pull request into GitHub, and it was a little too risky. It was sort of deemed too risky, too big of a change.
223
00:34:11.944 --> 00:34:23.264
That's just what I was told. So anyway, we're, we, we had a couple meetings internally already, and, uh, we're trying to weave it into planning and the roadmap, and so hopefully we'll be able to share more updates soon.
224
00:34:23.304 --> 00:34:29.484
But, like, it's at top of the list known feature request. As, a- again, heard. Yeah. And so- We're at, and, like, we're, we're working on it.
225
00:34:29.864 --> 00:34:35.104
Uh, obviously, like, something the size of GitHub to move to, like, support this kind of new, this new feature is, like,
226
00:34:36.144 --> 00:34:44.144
not just, like, um, a walk in the park because of the size of GitHub's and, a- and GitHub's Git implementation, but, uh, it's something that we're actively exploring. Yeah.
227
00:34:44.484 --> 00:34:54.724
Well, I think, you know, just to wrap all that up, you know, I think it's really nice for someone who's so deeply e- en- engaged and, like, coming from, like, one of us literally- Yeah, yeah...
228
00:34:54.784 --> 00:34:59.364
that you now run things at, at GitHub, and we can just at you. You know what I mean? [laughs] You can do that, I mean.
229
00:34:59.384 --> 00:35:07.444
Yeah, I think, like, uh, Ajay Karpathy was, uh, the other day was saying, like, every company needs one of these where you can just like, "Hey, like, this, this really should exist at GitHub. We love GitHub.
230
00:35:07.464 --> 00:35:15.524
We use GitHub, but, like, come on," and... [laughs] Well, yeah. Feature requests welcome. Like, my DMs are always open. I'd... Like- Oh, careful. I don't know. [laughs] Hundred and eighty million developers.
231
00:35:15.824 --> 00:35:22.824
I, I like- [laughs] Hundred... Whatever. I, I am of the philosophy that, like, all feedback is a gift. Like, it's all a signal. Yeah.
232
00:35:22.924 --> 00:35:32.964
Um, and the more signal we can collect, the better decisions we can make, and the, truly build this really, really useful website, uh, and company, like, together, and that's gonna be the future.
233
00:35:32.984 --> 00:35:40.944
And if we focus just on that, we're gonna be okay. Yeah. We're gonna be okay. All right. Yeah. Well- Thanks so much, Eric. This was- Yeah. It's a real pleasure- Awesome... catching up. Yep, likewise. Congrats.
234
00:35:41.024 --> 00:35:51.084
[outro music]
1
00:00:00.100 --> 00:00:15.060
[on-hold music] All right. We are here for a very special edition of Lane Space with my buddy, Jared Palmer, uh, SVP at GitHub and VP at Core AI at Microsoft. That's correct. Dual title. Yeah. Twice the fun.
2
00:00:15.080 --> 00:00:26.320
Is it weird to have two jobs? [laughs] No. I'm only on-- I'm only... The full disclaimer, I'm only on day thirteen, I think. Yeah. So early days, so, so far, so good. So far, so good.
3
00:00:26.400 --> 00:00:34.720
Uh, we've been trying to get you on the podcast for two years, I think. I think so, yeah. You- We've been dying for it. Yeah. You're a busy guy. We only ever do it in person, so... Yeah, we have to do it in person.
4
00:00:34.760 --> 00:00:45.760
Yeah, exactly. It's way better. Uh, I should also plug that you have-- if, if... Jared Palmer fans should dig into your previous podcasts with Ken Muller. [laughs] How about that? It's from, from, from the interim. Okay.
5
00:00:45.840 --> 00:00:56.920
Uh, so, um, but- Shout out to Ken. Shout out to Ken. Uh, before that, you were building, um, I, I guess, like, v0 and AI SDK, and you were just sort of VP of AI at- At Vercel, yeah. Vercel.
6
00:00:56.980 --> 00:01:06.120
Yes, all, uh, AI initiatives and vibes. Yeah. And I feel like basically you went from, like, you sort of building one coding agent to now being the home for all coding agents.
7
00:01:06.540 --> 00:01:16.360
Is that, is that, like, the, the general vibe of Agent HQ? I, I think that's right, yeah. So backing up, um, I spent the last sort of two years or so building v0 at Vercel and AI SDK.
8
00:01:17.000 --> 00:01:30.200
Uh, and then this summer took time off, and now joined GitHub, and today we launched Agent HQ, among other things here at Universe. And, uh, yeah, it's gonna be the home, we hope, of not only agents, but also developers.
9
00:01:30.440 --> 00:01:41.580
And it seems like, uh, the gravity well of this new collaboration space that we're trying to build. Yeah. What do you think, like, basically that GitHub can do that you couldn't do at v0?
10
00:01:42.140 --> 00:01:50.060
GitHub is an enormous platform, right? So these are- Hundred and eighty-three million. It's some amount of- Yeah, it's under a hundred million developers. Um, it's just the scale is immense, right?
11
00:01:50.160 --> 00:01:55.540
Um, and v0 was focused on not only one language, but one framework, right?
12
00:01:56.260 --> 00:02:11.200
And a specific problem space with a built-in renderer and, you know, um, you know, for those who are n-not aware of v0, it's, it's like a bolt or lovable, but it's built by Vercel, and, uh, it's focused on building Next.js apps, specifically Next.js apps.
13
00:02:11.880 --> 00:02:19.760
That constraint was rather liberating for the team at the time, and it lets us really, like, laser focus. It does. You rolled at it. I hope. Thank you. I hope so.
14
00:02:19.860 --> 00:02:29.320
Um, and obviously, at GitHub, you know, we're the home of all languages and, you know, and, and frameworks and, and developers. And so the scope is broadened.
15
00:02:29.860 --> 00:02:38.340
And yeah, it's just, it's just a different, a different part of the map, if you will, right? Yeah. How I've seen it-- So you, you've been basically covering the entire journey of coding agents- Yeah.
16
00:02:38.560 --> 00:02:46.860
-for, you know, from, from the start. Like, what do you think-- Uh, what's your jour-- personal journey through coding agents, right? Like- Sure. -is-- We started out with Copilot, obviously GitHub- Sure.
17
00:02:46.880 --> 00:02:57.960
-started the Copilot trend. Sure, sure. Uh, when-- Tell us about the origin story of v0- Yeah. -and then how that develops and, and maybe where, where-- like, what you wanna see next with- So- -history.
18
00:02:58.260 --> 00:03:05.560
It's, it's funny you ask that. As I've told this story multiple times, I feel like I've unlocked different parts of it in my brain- Yeah. -by going back.
19
00:03:05.600 --> 00:03:12.360
You know, like, so maybe we have to figure out how retrieval memory works. Which is, by the way, interesting how memory works for agents. Totally. That's why I brought it up. This is small.
20
00:03:12.380 --> 00:03:24.400
Like, kind of amazing as you- Yeah. Sometimes you discover new paths, right? Yeah. Anyway, um, the story goes like this. So when, um, ChatGPT first came out, obviously it was incredible, right?
21
00:03:24.540 --> 00:03:35.780
Like, world-changing immediately, fastest growing product ever. I look back at, like, the timeline and dates, and, um, we were very early, like in, when I was at Vercel, uh, jumping into AI stuff.
22
00:03:36.180 --> 00:03:45.900
But the journey kinda went like this. So at the time, actually, there was no AI division, there was no AI group. I was actually the director of engineering for all of Vercel Frameworks, and I was helping Next.js- Svelte.
23
00:03:46.300 --> 00:03:53.300
Uh, Svelte, right. Yeah, so Next.js, Svelte, uh, TurboRepo, TurboPack, Webpack, and all internal dev tools at Vercel.
24
00:03:53.840 --> 00:04:02.360
And I was helping the Next.js team dog food, um, and test the initial implementation of server actions. And instead of building a to-do app,
25
00:04:03.580 --> 00:04:13.020
Guillermo, the CEO of Vercel, was like, "Why don't you build, like, a playground?" And I was like, "Okay, cool." So that led to the AI playground, which is now just part of AI SDK. We'll get there in a second.
26
00:04:13.400 --> 00:04:22.040
Which, by the way, iconic for, like, side by side, but also the- Right, so nat.dev. So, so I'll, I'll- It's nat.dev. Nat.dev. So, so G, G told me the nat day of... I got a, I got a DM.
27
00:04:22.079 --> 00:04:29.200
It was like-- I remember 'cause I was at a bachelor party, and Guillermo, totally online, sends me a note like, "Nat.dev is launching on Monday. You have to ship."
28
00:04:29.700 --> 00:04:37.680
And I had been working on it previously, and so it was like- So it was the same idea. It wa-- Yeah. So he got wind of it, I guess. Uh- Right. So I, I definitely had to, like, jump into motion. And I didn't think we...
29
00:04:37.720 --> 00:04:44.140
We didn't even ship chat first. Guillermo sends me this, this, uh, DM over the weekend. I'm at a bachelor party, and he's like, "Nat.dev," this, the side by side.
30
00:04:44.180 --> 00:04:49.600
He sends me the link and I play with it, and I'm like, "Ciao." Okay, so I spring into gear, ship, um, the AI playground.
31
00:04:49.620 --> 00:05:01.400
What was cool about the AI playground was it, it, it forced me to go through every single model provider's API docs and figure out their s-- their quirks of their nuanced streaming, 'cause at the time it wasn't like everybody used OpenAI.
32
00:05:01.520 --> 00:05:09.400
Yeah. It was like all little quirks. Some of them kinda were compatible. So that was my first foray into it. And then launched AI Playground. That shot to the top of Hacker News. And I remember we didn't...
33
00:05:09.440 --> 00:05:19.040
I didn't even implement chat, 'cause that chat wasn't actually like a fa... like, wasn't as important. It was just like completion completions. So eventually we, we factored chat, and that, that project,
34
00:05:19.980 --> 00:05:29.760
uh, out of that came AI SDK, because I had already looked at all the model providers and all the, um, the combinations, and was like, "Okay, here's that chunk of streaming code you need."
35
00:05:30.240 --> 00:05:38.920
And then AI SDK found that sort of niche of, like, how do we focus on the part that we're gonna be good at, which is, like, that UI aspect of it, but then also knock it in your way.
36
00:05:39.500 --> 00:05:50.620
So that, we shipped AI Playground, then AI SDK launched, and then, um, you know, we're always about, um, demos and having great starter templates at Vercel, and I h- and, and I remember writing Guillermo.
37
00:05:50.620 --> 00:06:03.148
I was like: "You know what'd be cool? This guy Shad CN. Oh, man, he seems, like, amazing, and his UI library is doing great. Why don't we team up and ship a ChatGPT clone?" Open source. And we did.
38
00:06:03.528 --> 00:06:07.868
We shipped this, this, this awesome template, which is now called Chat SDK, but it's great.
39
00:06:08.448 --> 00:06:23.328
And, uh, what that did, though, at Vercel was it, um, set us up for, like, rapid experimentation because we had this really good, like, pretty full-featured ChatGPT ready to rock with all the latest features. Yeah.
40
00:06:23.428 --> 00:06:29.028
So when it came to, like, rapid prototyping that summer, now we're summer '23, it was so great. It was, it was, like, liberating.
41
00:06:29.128 --> 00:06:44.668
So I remember at that point, I had gained some momentum internally and pivoted almost entirely to AI. Um, and I had Shu Ding, uh, who you're friends with, um, and Max later, and, uh, Shad Cian now were cooking.
42
00:06:45.248 --> 00:06:53.528
And I think at that point, code execution had just come out. I think that's my timeline here. Yeah, they called Sandbox, uh, Interpreter. Interco Interpreter, that's what they called it at the time.
43
00:06:53.808 --> 00:07:03.588
And I had a very-- As soon as I saw this, I had a very ambitious idea and proposal to present to Guillermo, which was like, what if there was some... And mind you, tool calls don't exist.
44
00:07:03.608 --> 00:07:07.488
The context window is four thousand- Wow... tokens. Yeah. So, like, there's not much here.
45
00:07:08.108 --> 00:07:26.388
Um, what if we had this thing where, like, you could prompt, and sometimes it would do code interpretation, and then maybe we could sort of-- Sometimes it would do code interpretation, but then other times it would choose to render, like, UI, or then it would render sometimes, like, a document or- Right, inline in the chat.
46
00:07:26.448 --> 00:07:33.508
Yeah. And like, it would just have different sort of render modes. Like a generative UI type thing. Yeah, and maybe you could pipe them together, so, like, the output of one could pipe into...
47
00:07:33.568 --> 00:07:43.728
So if we did code interpretation, we coerce it to always emit, uh, like, tabular data. Yeah. Maybe we could pass that to another prompt that was like a UI, and just, like, some idea there. It's kind of crazy.
48
00:07:43.988 --> 00:07:51.588
But if it sounds like these are just tool calls, that's exa-that's exactly what these are. That's what it was, yeah. Um, so it became pretty obvious that, like, the...
49
00:07:51.968 --> 00:08:00.668
So I came to Guillermo, and we-- at the time, we just had a sec-secure-- we had a sort of security debate, like should we code interpretation with the ability to fetch data?
50
00:08:01.148 --> 00:08:09.288
Um, like giving it internet access was, like, kind of... Now they're like, "Fine, whatever, whatever, do whatever you want, wild, wild west." But at the time, it was, like, a little scary.
51
00:08:09.848 --> 00:08:23.968
So we kind of said, "Okay, no to the code interpretation, but this UI thing is pretty neat." And so that, this, like, prompt to UI, that was, like, the aha moment of v0, but the models were not very good, right?
52
00:08:24.028 --> 00:08:36.168
So-- Or relative to the way they are now. And it was, uh, for GPT-4 era? This is just into the GPT-4 era, and now we're probably at a, uh, sixteen thousand token context window, so you can't really do chat.
53
00:08:36.468 --> 00:08:49.488
So we had to kind of invent this kind of new paradigm of, like, fake it with completion, but that forced us to do sort of the click the... Well, we-- The, the initial v0, which launched I think in September '23, um,
54
00:08:50.908 --> 00:08:59.588
it would look more Midjourney. In fact, if you go back to the original suite, it was like Midjourney for React because it was all very visual, and you could click on different, uh, components and elements and re-prompt.
55
00:08:59.788 --> 00:09:07.408
But it was, again, we're, we're kind of hacking this because we didn't have chat, and we didn't have tool calls. Um, and then fast-forward again, um,
56
00:09:08.448 --> 00:09:18.288
you know, that launches, and then probably, like, nine months later, um... It took us, like, nine months to, uh, uh, get to, like, a million ARR, uh, this little team.
57
00:09:18.348 --> 00:09:30.028
But then the models progressed, and from, you know, GPT-4, GPT-4 32K, big, big, big boy. We never really got GPT-4 Turbo working. I don't know why. Just never happened.
58
00:09:30.448 --> 00:09:35.388
And then switched to oth-other frontier models and, and then started doing our own models and stuff like that.
59
00:09:35.888 --> 00:09:47.128
Uh, but fast-forward another ten months or nine months or so, and then we, we rebased towards chat, and now the models finally could do chat. Um, and the artifact pattern had evolved, so it was time to rewrite.
60
00:09:47.568 --> 00:09:56.868
When we launched v0, the chat version or the new v0, whatever you want to call it, uh, it, it's like fourteen days, another million MRR, fourteen days, another million MRR. Right. It was like a rocket ship after that.
61
00:09:56.928 --> 00:10:04.068
Yeah. And, um, that just proceeded, uh, and we just kept cooking. And so that's been the journey. Uh, we just kept perfecting.
62
00:10:04.088 --> 00:10:10.188
And, and what was really, um, liberating for us was actually the focus on just one stack or one framework.
63
00:10:10.208 --> 00:10:19.628
When everybody else was trying to do general purpose coding agent, we were like, "No, we're just gonna focus on Next.js front-end and ShadCN," and, and that was really-- That allowed the team to focus.
64
00:10:19.668 --> 00:10:27.588
So that's the, that's the story arc. I mean, to be fair, like, bolt, loadable, like, because Next.js is so dominant, basically everyone has to be good at Next.js. Right.
65
00:10:27.648 --> 00:10:33.248
But being focused on, like, even right down to the UI library and component stuff, like, that, that actually helps a lot.
66
00:10:33.428 --> 00:10:40.248
We also started working with all the frontier model labs to help because it was in our, Vercel's best interest to have them be great at Next.js. Yeah.
67
00:10:40.628 --> 00:10:45.908
And also because of the post-train models, and you can read about this on the Vercel blog, like, we can, uh, um...
68
00:10:47.168 --> 00:10:56.468
The post-training, um, like, harness that we created, we were started sharing and stuff with other model labs and stuff like that, and we had all our data in a very hygienic state- Yeah... to work with them, so.
69
00:10:56.808 --> 00:11:00.148
Did you ever debate internally, a-and because, uh, you know, I've--
70
00:11:01.128 --> 00:11:12.628
I-- My-- From my seat at Cognition, I can also see this, where you should pick the best qualities of every model and string them together in the v0, or you have the model selector, and you let customer's choose. Yeah.
71
00:11:12.648 --> 00:11:20.528
We went back and forth, and I think- Yeah... um, you know, we launched... We, we went back and forth on all this. I think at the end of the day, there's pros and cons. Yeah.
72
00:11:21.028 --> 00:11:30.368
One of the benefits of having, uh, your own branded model or synthetic or composite is that you can stitch these things together. Yeah. There's like- You can get higher levels of...
73
00:11:30.408 --> 00:11:37.768
And, and, and, and now it's a little different with, with the, say, agentic flow, but even look at what you g- uh, like, what you guys launched recently with- These are actually rep, right?
74
00:11:37.828 --> 00:11:50.608
So search is gonna be a different model than what generation, but you know, the genesis, but, like, 'cause it's, it's a shorter... But, like, search and genesis are two different mo- like, like, entire subsystems, right?
75
00:11:50.688 --> 00:12:00.548
So you can have search evals that are gonna be totally different. And so how do you... So where we ended up now, uh, or we, wh-where, where we've probably ended up now is, like, um-
76
00:12:02.348 --> 00:12:06.668
For a long time, we didn't have model selector, and then we had our own models, which were composites, which we talked about.
77
00:12:06.948 --> 00:12:19.108
And then like, um, and that would allow us to s- you know, mix, match, and I think that's probably if what-- It's also nice 'cause you get as a-- this is like the product hap- you get to brand it. Yes. Right?
78
00:12:19.668 --> 00:12:28.948
And you can decouple it from the launch of the Frontier Lab. Yes. How do you guys-- how does cognition even bill for it? Like- ACUs, uh, compute, compute hours. Right. Some synthetic unit, right? Yeah.
79
00:12:29.368 --> 00:12:38.468
So it gets a little wonky. Yeah, yeah. We can go on for pricing this, uh, this stuff. It's, it gets challenging. But the nice thing about having the, like, brand name model is that, like,
80
00:12:39.408 --> 00:12:50.248
you get to co-launch with the provider, and they'll hype you up. So, uh, but your billing needs to then is capped at whatever retail is, right? Or, or, or some- Right. Right, right, right.
81
00:12:50.428 --> 00:12:57.408
Or- You can't really charge a, uh, too much of a premium. You can, but- Right. But then people are like, "Why am I paying for my own API key?" Right? And it's like, well,
82
00:12:58.388 --> 00:13:00.968
then how do we charge you for suite rep or something, right? Going forward, so. Yeah, yeah, yeah.
83
00:13:00.968 --> 00:13:10.528
And, and so like I, I think it's some part of it is the cynical, like you wanna m- create a sustainable business and independence from the, the model labs, but the other part is genuinely you actually do get better performance- Right...
84
00:13:10.648 --> 00:13:18.288
if you like string together all these things. Yes. And so it's, it's tough. I think, uh, uh, well we've switching gears to GitHub, like we are all about model choice now- Right...
85
00:13:18.308 --> 00:13:31.888
and like making sure that, uh, and what's cool is that like it's, we also have Copilot, which is our harness and Copilot CLI, but we also have third party harnesses like Cloud Code and, and Codex and Cognition now like in a, uh, Agent HQ.
86
00:13:31.948 --> 00:13:43.308
So you kinda get the best of both worlds. Yeah. And like, I think that's gonna be awesome and ultimately what people want. Yeah. I think, uh, also the model layer is not the right abstraction to do the switcher anymore.
87
00:13:43.588 --> 00:13:51.488
It's-- Which is weird because that's where you started with the AI SDK. Yeah, yeah, yeah. Exactly. Um, but now it's like the model and the agent have to be strictly tied together, like very, very strongly balanced.
88
00:13:51.528 --> 00:14:00.888
You can't loosely bound it and just do a generic interface 'cause then you're just gonna have the lowest common denominator of all the models. If you're, if we're in agent world- Yeah, yeah...
89
00:14:00.928 --> 00:14:06.388
which may just be better than chat world, like in general, like- Yeah, yeah... better. A-agent world is, is a much better abstraction.
90
00:14:06.428 --> 00:14:14.808
I'm, I'm calling it agent world, but I mean by like a loop with maybe compute runtime- Yeah... and like files. I do- So that's, that's now your definition of agent? You're dropping your official definition here?
91
00:14:14.968 --> 00:14:23.468
No, don't put me on the spot. [laughs] But, um, maybe. Ma-- No. So my, my initial definition of agent for, for AI is, 'cause like I actually fought, I was dying on this hill, 'cause AI SDK,
92
00:14:24.388 --> 00:14:29.728
everyone else is an agent framework, and I think maybe they, they actually went to like this, I don't know what it says on the front page now, but like, who knows?
93
00:14:30.428 --> 00:14:41.008
But like- They say like six- An agent is, uh, you know, an agent is, uh, orchestrating, uh, ch-- you know, an API request with a queue and a for loop. Okay, but a coding agent now has meant so much more.
94
00:14:41.048 --> 00:14:51.928
There's like these, you know, coding, coding agent SDKs, and you've got sandboxing and file systems and tool calls and, and I do think that is a uniquely d-- I'll call that agent world and when charging coding agents here.
95
00:14:52.488 --> 00:15:05.588
And yeah, I think that that seems to be where things are going. And even I believe the, the Claude Excel agent is basically, I was talking, um, to, to Mike Reager backstage, like I think it's like related to Claude Code.
96
00:15:06.748 --> 00:15:12.288
Uh, it could be. I actually don't know how it's implemented under the hood. Yeah. It wouldn't surprise me if it was. Yeah, yeah, yeah.
97
00:15:12.308 --> 00:15:24.888
They seem very all, all, all in on skills, which is kind of like an interesting- What do you think of skills? Uh, it's kind of DXT, which is like the, the sort of bundled f- uh, version of MCPs. Okay. I've heard of that.
98
00:15:24.908 --> 00:15:33.068
Uh, it wasn't-- Yeah, it wasn't-- The, the reason you don't know about it is because it wasn't very popular. Okay. Uh, so skills is kinda like the, the, the second shot that is very LLM pills.
99
00:15:33.208 --> 00:15:43.688
Like it's like, just read my markdown and just read this, this directory of files and go nuts. Uh, as, as long as like it can understand that you have the capability to run code, to read files, you're good.
100
00:15:44.308 --> 00:15:50.868
Uh, and, and actually that, that is the universal interface, which is a file system. Right. Uh- Back to agent files. Yeah [chuckles], which is, which is kinda cool.
101
00:15:51.248 --> 00:16:02.608
Uh, yeah, so I mean, like I, I think like what you're hitting at is, uh, you know, th-this philosophy of like our understanding of what coding agents, the like the minimum bar is over the last two years. Yeah. Right?
102
00:16:02.648 --> 00:16:10.448
Like you've, you've lived this journey and like now you're basically kind of like the kingmaker or like the [laughs]- I don't know about that... you, you run- You have phrase. Yes.
103
00:16:10.488 --> 00:16:17.268
You run, you run Agent HQ and, and like, I mean, I, I, I imagine you have other projects too, but like Agent HQ is like the big one that we're talking about here.
104
00:16:17.768 --> 00:16:33.748
Um, like what are you seeing from like the different agents? Like what do you want to-- this to become? Such a good question. Um, I think that Agent HQ and GitHub itself need to like co-evolve and,
105
00:16:34.708 --> 00:16:42.928
you know, one of the things that Microsoft has done really well is by putting things that are, um, alike closer together. And so you think about the new core, core AI organization- Yeah...
106
00:16:42.928 --> 00:16:50.888
you've got Visual Studio- Okay... Visual Studio Code, GitHub, and parts of Azure all in one. Um,
107
00:16:51.808 --> 00:16:57.168
and obviously the GitHub team and the VS Code team have been working closely together for a long time, but now we're really close together.
108
00:16:57.628 --> 00:17:06.428
And I think for me, one of the cooler things that, um, Agent HQ can sort of offer is this seamlessness, this fluidity with your workflow, right?
109
00:17:06.488 --> 00:17:20.888
So if you saw in the demo today, we saw a, um, a demonstration of, uh, you use Agent HQ, you fire off a task, and it creates a PR, but you can also open that PR up in VS Code in one click. And that's, that's awesome.
110
00:17:20.908 --> 00:17:38.068
And I think my-- the, the vision for, for, for GitHub as it evolves is to look at those touch points where AI can be sprinkled in, you know, uh, salt-based style, um, into the native workflow, whether you're assigning an issue or maybe some new stuff that I think we should like focus on.
111
00:17:38.088 --> 00:17:49.848
Maybe it could be like, how do we resolve a merge conflict? Oh my God. Right? Like, how do we maybe pop open an action or like get in, right? And so it's like- I think solving merge conflicts is my definition of AGI.
112
00:17:50.248 --> 00:17:58.948
Sh- totally. But like you get that error on an action and you're like, we've all been in that, we've all been in that sort of flow where like actions kind of don't work lo-locally versus that tool act.
113
00:17:59.048 --> 00:18:02.548
If you ever try it and you're like, "Oh, I don't have that set up on my machine. I haven't done this in a while."
114
00:18:02.848 --> 00:18:12.304
So you're like pushing up and you're, you're this like, "Okay, what if we could just put like, oh, you know, comment or kick off a task to solve this for, for you or-" Sure things there.
115
00:18:12.384 --> 00:18:24.444
I think what I-- what I'm trying to describe is this like this workflow where it's just like seamless and fluid, and you can stay in a flow state across whether you're acro-- like, across all devices, mobile, web, on github.com, or in your local editor.
116
00:18:24.544 --> 00:18:33.464
And I think that's where, you know, my focus is gonna be in the next- Yeah... six, six months or so. Yeah, yeah. Um, just a side tangent on this. Uh, so
117
00:18:34.364 --> 00:18:44.544
one of the things that Microsoft also owns, I don't know if it's Microsoft or GitHub, is Dev Containers. And I think, like, a very important concept for sandboxing environments- Yeah...
118
00:18:44.564 --> 00:18:56.404
whatever you call it, uh, it is kind of a light version of what Docker containers are, kind of. Right. Uh, do you see that as a standard that we should invest in as like a, like a thing?
119
00:18:56.524 --> 00:19:05.784
Like, 'cause it's supported in VS Code. I don't think it's just that popular outside of VS Code. Yeah. Uh, it's used internally at GitHub too, for like d-development at GitHub. Oh, yeah, yeah. Which is cool.
120
00:19:06.164 --> 00:19:15.384
Um, yeah, I think they were so far ahead almost. Like it was... But now there's sand-- this, like, sandboxes, there's so many of these days, right? So I think Cloudflare just launched theirs. Yeah. Uh, there's Daytona.
121
00:19:15.544 --> 00:19:25.504
Vercel. They're here. Vercel. Um, Mo-Modal- Yeah... uh, which I think Lovable uses. Uh, I have no idea. I don't know. Yeah, I mean, you probably have your own. I don't know. What do you, what do, what do you guys use?
122
00:19:25.584 --> 00:19:29.104
Uh, just some Kubernetes pods. Okay. You guys are rolling it yourself. Very cool.
123
00:19:29.504 --> 00:19:38.464
I think that's maybe the runtime, but, but there's, there's work and discussion about what that runtime should be, even internally at Microsoft, and we've got a couple different competing things.
124
00:19:38.524 --> 00:19:45.844
So we'll figure it out in the next, you know, in the next cycle here. Right. But, uh, there's a great point. Like, there, there's a lot of cool stuff that is in a Dev Container.
125
00:19:45.864 --> 00:19:55.024
You've already got VS Code loaded, you've got a file system- Yeah... you've got a sandbox, you've got the security protocol. Yeah. It's also, like, wired into GitHub Enterprise- Yeah... and, like, ready to be packaged.
126
00:19:55.224 --> 00:20:04.664
So there's lots of goodness there. Yeah. I see, like, the, the number one pain point that Cognition has, but also Codex, also presumably the other guys, is repo setup. Yeah.
127
00:20:04.664 --> 00:20:15.324
Which is effectively what Dev Containers and a Docker file does for you, is like, run this thing, then that thing, set this up, do that thing. Uh, why is it so hard? Like, why haven't we solved it?
128
00:20:15.324 --> 00:20:23.704
[chuckles] I don't know. I think it's hard because you can't predict what's in the, what's in the repo, right? So it's like, and- Yeah... you don't know when they've bundled FFmpeg. You don't know. You just don't know.
129
00:20:23.784 --> 00:20:32.464
Right. It, it's nice when, like, if it's just Next.js, you just run PMPM install. [chuckles] Correct. Correct. You, you can, like, do special k- Like, there's obviously cons...
130
00:20:32.764 --> 00:20:42.124
through constraints, you can make optimizations, and I think the general purpose container is just, like, challenging. That being said, though, I think there's probably some work to do on,
131
00:20:43.164 --> 00:20:51.444
you know, auto-detection and preempting and, and stuff like it to be done there. But it's just a bigger, it's a broader problem space, right? Yeah. Uh, and, uh, yeah. So, so fun fact.
132
00:20:51.624 --> 00:21:01.604
When I was at Netlify, I actually wanted to reach out to Vercel to do, like, a standardized open source auto-detection thing of frameworks. Oh, yeah. And I, I...
133
00:21:01.804 --> 00:21:10.244
Like, we, we never, we never really, like, got internal momentum on that. It was an idea. I was like- Okay... shouldn't this be open source, you know? Yeah. Like, auto-detection is a, is a common utility- Yes...
134
00:21:10.384 --> 00:21:20.284
that everyone needs. Yes, yes. I remember that. I'm having a flashback. It's like [chuckles] half [stutters] Yeah. Everyone builds their, their- Yes. Yeah... probably we shouldn't all build it. Right. No, it's, um...
135
00:21:20.764 --> 00:21:30.404
And then also, like, what are your defaults? They're not exactly the same, which would be better to, like, just having even the same, like, uh, preference stack- Yeah... of defaults- Yeah... is the right- Yeah, yeah...
136
00:21:30.504 --> 00:21:37.324
um, would be great, 'cause then we could move the whole ecosystem together- Right... from, like, the PMPM run, right? [both laughing] Like, um, okay. So
137
00:21:38.544 --> 00:21:48.344
are there other, um, movements or protocols or standards that you're interested in? Like MCP was a big winner this year. Yeah. There's other, like, I don't know, AC...
138
00:21:48.504 --> 00:21:56.684
A2A, ACP, all this, all these, like- That'd been interesting. I'm not as familiar with, um- ACP, the payments one or the, is that- Right, right... Redone? Oh, no, there's that one. The, uh- That one. Okay.
139
00:21:57.244 --> 00:22:06.504
And then the, the payment that, was it Stripe or Coinbase? Stripe. Stripe? Yeah. Yeah, that's very cool. We've had them on the pod. Okay. Yeah, that's very cool. Um, but it'd be interesting to see if that takes off.
140
00:22:07.124 --> 00:22:17.164
Um- I mean, it's Stripe. [laughs] So... [laughs] Yeah, but it still needs to be adopted by, like, the clients, right? Yeah. And, and, and I think that's, that's fascinating. Uh, but MCP is huge.
141
00:22:17.204 --> 00:22:19.624
It seems like it is the way that a lot of the...
142
00:22:20.424 --> 00:22:26.464
Especially when it comes to, like, digital transformation or some of our enterprise customers, it's where they're gonna able to be a- or where they are able to add context.
143
00:22:26.844 --> 00:22:29.344
In addition to that, we also have custom agents that we announced today too.
144
00:22:29.384 --> 00:22:36.924
So, like, you can work with prompts and stuff within your, within your agent HQ and, and customize these agents for different tasks and, and those can have MCPs and such.
145
00:22:36.964 --> 00:22:44.904
And I think that's gonna be really powerful from like a, a platform perspective. Yeah. Gets me excited. That's what, like, I think is, like, shipping now in, in the next...
146
00:22:44.964 --> 00:22:55.024
But we're always on the lookout for, like, the next thing and, uh, I don't know. What, what's on your, what's, what's top of mind for you? Uh, for, like, standards or- Yeah, standard. Standards? Or should it be built?
147
00:22:55.064 --> 00:23:05.364
Uh, Dev Container is like- Dev Container. A container... like, is the... Look, I think Dev Container just has a PR problem. Right. It's a great idea. Right, right. Just no one makes it interesting. Yeah, yeah.
148
00:23:05.504 --> 00:23:08.884
I think you can do it, basically is what. [chuckles] Okay. Add it to my list.
149
00:23:09.464 --> 00:23:20.244
But before that, probably you have a bunch of other stuff that I, I wa- I do wanna get to, uh, but just staying on the AI stuff, like, uh, I, I think we're, we're, we're actively exploring computer use as, as a thing.
150
00:23:20.324 --> 00:23:30.184
Like, because it, it kinda got going a little bit. People were very excited, and then they found out it was slow and bad and inaccurate. It is computationally intensive- Intensive... from my understanding.
151
00:23:30.504 --> 00:23:41.924
It's getting better. Yeah. Uh, especially with open vision models like DeepSeek OCR and, and OMO OCR. Uh, it, it, it... Like, just give it a few more turns of the- Yeah, yeah... scaling. It seems very...
152
00:23:41.944 --> 00:23:48.164
And it seems like it's, like, you need that edge case, uh, and primary. It just seems like a modality worth, worth pursuing.
153
00:23:48.324 --> 00:24:01.564
Yeah, I think a lot of people are, uh, on the CodeGen side, the code agent side, a lot of people are trying to think about, all right, we had this evolution from copilot to, like, uh, you know, a more agentic, uh, sort of cl- like a cloud code situation is where I think that the status is.
154
00:24:02.044 --> 00:24:10.624
Like, what's next, right? What's, what's the, what's the, the, the, the obvious next step? Oh, you go. I mean, making them good. Making them good. Yeah. You don't like cloud codes? No, no, I, I don't.
155
00:24:10.804 --> 00:24:21.448
It's more just like, you know, the, the, the devil's in the details. Like- Yeah... going from ninety percent, going to, like- But hill climbing, it, it, it gets steeper- Yeah... in my opinion.
156
00:24:21.568 --> 00:24:29.688
It gets steeper, and so going from ninety percent success to ninety-five, to ninety-eight, to ninety-nine, to nines of success- Yeah... I mean, really hard.
157
00:24:29.748 --> 00:24:38.048
Paying Mercora a lot of money for, uh, expert programmers of open source maintainers to, like, you know- And then you realize along the way maybe the users aren't that good at it. Right. Yeah.
158
00:24:38.048 --> 00:24:41.548
[laughs] Like, um, no, but it-- I, I just think there's a lot of work to do to finish the swing.
159
00:24:42.268 --> 00:24:47.888
Um, and there's a big difference between ninety-eight percent and ninety-nine percent correct, uh, and that's, like, noticeable.
160
00:24:48.588 --> 00:24:53.928
Uh, and this used to hit, you know, if, if you're working on an AI product, you, you probably don't realize how...
161
00:24:54.448 --> 00:25:03.548
You've probably seen this, like most people are blind, like living in la-la land about how, um, poor quality, uh, their, their AI product likely is.
162
00:25:03.768 --> 00:25:09.728
Unless they're really measuring, like, the number of error-free sessions, like how many errors are coming from the in-infra, infra providers.
163
00:25:09.788 --> 00:25:14.568
Like, you know, how, how many requests are dropped, how fast, how fast these things are.
164
00:25:15.188 --> 00:25:23.608
Um, and so that's something that we cared about at Vercel quite a bit, and, like, we'll care- What other pro-- Do you have, like, a daily review of your dashboard or how, how- Yeah... I don't know. Daily?
165
00:25:23.688 --> 00:25:34.388
Daily would be slow. Yeah. Oh, okay. I thought you were gonna say daily's too much. [laughs] Daily's... No, and even thinking, it's like, uh, every three hours, k- uh, roll-up of, of key metrics and stats. Yeah. Um,
166
00:25:35.468 --> 00:25:42.808
and, like, one of them was, like, error-free sessions and other things like that, that was, like, really important because, you know, in the-- especially now with agents that are like multi-turn.
167
00:25:44.208 --> 00:25:53.868
Um, I have a tweet about this that was, like, in 2024, which is that, like, agents will really only work when we get to not only, like, the in- more intelligent models, but better reliability of the infrastructure providers, right?
168
00:25:53.968 --> 00:25:59.848
These aren't, these are not-- Inference is not like a database, like- Yeah... update uptime, right? So there's still differences between providers.
169
00:25:59.868 --> 00:26:05.728
There's still differences between performance and difference uptimes, and that's why you see things like OpenRouter being very successful- Mm...
170
00:26:05.808 --> 00:26:16.268
and, and different gateway products because, like, reliability, you need to switch. They go down all the time. Um, so long story short, yeah, we would, we would do, like, you know, uh...
171
00:26:16.328 --> 00:26:23.368
It was almost like video game style. Like, we'd have, like, all the data coming in all the time. Yeah. Uh, and that allowed-- We, uh, I used to joke it's my mood ring, like good day, bad day.
172
00:26:23.948 --> 00:26:27.288
Um, so it was very successful for us.
173
00:26:27.548 --> 00:26:48.248
Uh, I think other teams should adopt that, like, data-driven approach, um- I think one thing that's surprising is the lack, the relative lack I still see on data analyst agents, where you can sort of chat it, uh, like add a Slack bot for, uh, the, the precise analytics that you want to generate, um, because I think we're still in the BI era.
174
00:26:49.528 --> 00:26:57.988
Yeah. Isn't that weird? It-- Yeah, I totally agree. Um, it's interesting that that space hasn't been, like, captured as much. Yeah. Like, I guess maybe now.
175
00:26:58.388 --> 00:27:09.328
Actually, I'm interested in this, like, shift to away into, in, like, into knowledge work tasks with coding agents. Okay. I wonder- Using coding agents for non-coding tasks? Correct. Do you, do you do that personally?
176
00:27:09.428 --> 00:27:10.648
I do, yeah, yeah. Yeah. What, what do you do?
177
00:27:11.188 --> 00:27:17.908
Well, like, I was-- Like, this summer, I was doing, like, I was helping-- I was trying to automate some of my dad's workflows and stuff like that, and just, like, some of his...
178
00:27:17.928 --> 00:27:28.548
He's got some Excel spreadsheets and, uh, for, like, accounting, like, like, uh, financial accounting or managerial accounting, I guess. Um, and, uh, yeah, just like point Cloud code to that stuff and see what happens.
179
00:27:28.588 --> 00:27:41.148
And, like, I, it's, it ends up doing Python, um, and, and, and generating some scripts, and it kinda got off down other hairs, but it was like even he saw that it was better at it than the chat client than Excel.
180
00:27:41.288 --> 00:27:49.488
Super obvious. It, it- It became kinda obvious. Yeah. It, it felt better. I wonder if he can try Cloud for Excel and see. Yeah. Yeah. That's it. You got me, right. The, um...
181
00:27:50.148 --> 00:27:57.708
And then, of course, you got the browser, the... I don't even want-- Not browser. Browser-based agents, but not, not computer use, but browsers with agents. Agent browsers. Wait, wait, wait, wait.
182
00:27:57.768 --> 00:28:05.888
Oh, right, right, right. Agent browsers. LogMeIn, Perplexity. Yeah, everybody's coming with that. Yeah, yeah. It's like, is that the better... If that's true, then maybe the, the general purpose injection point is there.
183
00:28:06.128 --> 00:28:11.048
Yeah. Uh, what do you, or have-- Which-- Have you tried any of the agent browsers? All of them. All of-- Which one? What, what's your tip?
184
00:28:11.068 --> 00:28:21.608
I, I am very, uh, I'm currently maining Atlas, mostly because I just wanna give ChatGPT a fair go. Okay. Um, I-- But I'm very stuck to the Arc and, like, the vertical tabs. Oh, okay.
185
00:28:21.688 --> 00:28:29.868
I think, like, any pro user, like I have multiple businesses. How many tabs do you have? I'm context switching, right? I, I have hundreds of tabs open. I made a, I made an open source tool called Chrome Dump.
186
00:28:29.908 --> 00:28:39.128
You can find it on my GitHub, uh, where it literally dumps all the tabs open. It summarizes them, and then I can close it by deleting them on Markdown. That's pretty cool. And it syncs, it syncs to...
187
00:28:39.128 --> 00:28:46.248
[laughs] So you just go on, like, a- Oops... you just go on, like a, like a, like a, a bender. Yeah. And then you just dump it. Yeah. Nice.
188
00:28:46.268 --> 00:28:55.348
I mean, it should be as easy to close as Markdown, and Chrome isn't, isn't that good at the performance side of things yet. And you were working on, like, some browser comparisons or something before. I, I was.
189
00:28:55.368 --> 00:29:04.048
But, so I tried to build it in Tauri. Okay. How'd that go? And Tauri explicitly doesn't want you to build a browser, and I, I tried to fight it too much. I see. Yeah, I see. Very cool.
190
00:29:04.228 --> 00:29:11.168
So just to wrap things up, because I, I know we're, we're around about time. There are other side projects, uh, tasks, and things that you, you've, uh, announced here.
191
00:29:11.588 --> 00:29:19.308
Uh, first of all, redesigned GitHub homepage, which a lot of people don't even know GitHub has a homepage. [laughs] I, I leg-- I'm legitimately one of them.
192
00:29:19.328 --> 00:29:28.548
That tweet, which is, uh, re-- was this tweet, like, printed out. Like- What? You see the tweet. Like, there's a tweet, um, from Riz, and it's like- Love it... no one uses- No one, you know, GitHub homepage.
193
00:29:28.608 --> 00:29:32.628
None-- All of this stuff is totally useless. I'll, I'll pull it up. Like- Yeah, yeah, yeah.
194
00:29:32.808 --> 00:29:37.668
I, I have-- I pulled it up today, 'cause we were like, when we launch- [laughs] Let, let, let me get it right, 'cause I gotta do it right. Hold on. Okay.
195
00:29:37.788 --> 00:29:51.748
Uh, was, um, "Incredible how pretty much the entire GitHub homepage is useless." [laughs] And it's one point three million views and nineteen thousand likes. And this was, uh, May two '25. So heard,
196
00:29:52.668 --> 00:30:01.728
uh, the team, the team made improvements, and today they launched a new GitHub homepage- Yeah... which I'm very proud of. Um, and they should be really proud of. It's got tasks at the top. It's got recent PRs.
197
00:30:01.908 --> 00:30:09.308
Some stuff is still there, like your recent repositories. I still f-- I think there's more work to do, but it's, like, really overhauled, and they did an amazing job with it. Yeah. So they nailed it.
198
00:30:09.728 --> 00:30:21.068
Uh, but you know, more work to do. Never done and, like, hopefully we can keep iterating with, uh, the community and, and everyone and, uh, and keep going. The last thing I want to hit you on is Stack Dips. Oh, yeah.
199
00:30:21.148 --> 00:30:29.648
You asked everyone when you joined, "What should I work on?" or something. Yeah, what happened- I don't know if this is your job specifically. It wasn't. But- It wasn't... why do people want Stack Dips so much?
200
00:30:29.848 --> 00:30:39.664
Let's-- I, I think you have some history there. Yes. Anyone who's interacted with anyone at Facebook knows- Right... about Fabricator. They're religious about it, yeah. Uh, so can you explain why, why it's been so...
201
00:30:39.784 --> 00:30:55.644
Like, what it is, why is it so hard? Okay, so this, um, this concept of a pull request, which we're all familiar with, uh, you write some commits, you open a PR, and then you merge the PR, and you go about your day.
202
00:30:56.304 --> 00:31:03.464
So as you scale, like, larger organizations, um, and, like, you look at your history, and, and, and there are people who are, like, very
203
00:31:04.944 --> 00:31:11.544
r- I'll say, like, have, like, r- near religious beliefs about how to Git, how to do Git right. Rebase vers- Yeah... rebase versus merge.
204
00:31:11.624 --> 00:31:20.984
There's, there's a crowd that wants to fast-forward the repository so to preserve all the history, and then there's a crowd that wants to squash and, and, and merge into the main thing. I, I'm team squash. Okay.
205
00:31:21.164 --> 00:31:31.024
Anyway, Facebook, and I've, I've never worked Facebook, but, um, in my previous, uh, startup before Vercel, Turborepo, I did a lot of research on build systems.
206
00:31:31.604 --> 00:31:39.304
And at Facebook, they have, not only they have their custom build tools called Buck, they also have a custom file system, and they don't use Git.
207
00:31:39.704 --> 00:31:53.124
They use, uh, Mercurial, which, and then now it's sort of custom, and it's all wired together. Um, and at, at, at Facebook, they don't use pull requests. They have a different sort of philosophy. You can, um...
208
00:31:53.324 --> 00:32:05.264
It's sort of like the best way to ana- like, think about this is, like, imagine every PR just had one commit in it. Yeah. You could branch them, um, and res- and the critical thing is you can restack them.
209
00:32:06.384 --> 00:32:17.604
And then, um, if you restack or make a change later, like earlier in the stack than later in the stack, and these stacks are just diffs, right? The commits are just diffs, and it's the term stack diffs.
210
00:32:18.144 --> 00:32:22.564
You can then collapse them and merge the last one and merge them towards the end. It just gives you a little bit of a nicer workflow.
211
00:32:23.124 --> 00:32:34.524
And it's what people who, if you work on a monorepo or you work on a very, very large code base, it's a really, really nice way to work, especially if you've got a system that will re- that will d- automatically restack.
212
00:32:35.204 --> 00:32:48.544
Um, and then if you think even more deeply about it, like, and get really deeper into the weeds, uh, you can decide which, um, diffs in the stack CI should run against, if you get fan- if you get fancy. Okay.
213
00:32:48.764 --> 00:32:51.584
May not be- By, by like certain- Yeah... commit messages and, or- There's just always...
214
00:32:51.624 --> 00:32:58.444
Yeah, you could decide, like, maybe this one doesn't need it or skip that one or, or whatever, and you end up getting these, like, sort of these groups, these, these stacks.
215
00:32:58.904 --> 00:33:07.064
And it's really nice from a code review perspective because, um, when you go to update or you can update a different part of the stack, it just makes it a little bit more fluid, and so it's what people want.
216
00:33:07.404 --> 00:33:15.144
There are a couple tools out there in the market that do this kind of behavior. One's called Graphite. There are a couple others. Too many tools called Graphite. There is- There's another Graphite right here. Yeah, yeah.
217
00:33:15.164 --> 00:33:26.224
[laughs] Um, and just, it's got, it's a great workflow. And so it's been the top pull request, or sorry, the top, um- Feature request... feature request, thank you, uh, at GitHub for years.
218
00:33:26.264 --> 00:33:35.104
From, from the community's perspective. I don't know at GitHub, but... And then as s- as soon as I joined, the first, the first [laughs] thing I did was go look this up. [laughs] And, uh... Well, not the first thing.
219
00:33:35.184 --> 00:33:38.344
I asked how I should make it up better, and it was the top feature request, right?
220
00:33:38.904 --> 00:33:48.204
Uh, and then I went to go, like, okay, investigate, like any good product person would, and it, there's been multiple attempts at this in- internally, um, going back to, like, twenty twenty. Okay.
221
00:33:48.704 --> 00:34:00.484
And there was one very, very, very, um, polished attempt too in twenty twenty too, and it just... I, I don't have all the context, so, but it was, it was, there was a pretty good implementation.
222
00:34:00.564 --> 00:34:11.404
All of the work was done on the client, and it reintroduced this new concept called stacks outside the pull request into GitHub, and it was a little too risky. It was sort of deemed too risky, too big of a change.
223
00:34:11.944 --> 00:34:23.264
That's just what I was told. So anyway, we're, we, we had a couple meetings internally already, and, uh, we're trying to weave it into planning and the roadmap, and so hopefully we'll be able to share more updates soon.
224
00:34:23.304 --> 00:34:29.484
But, like, it's at top of the list known feature request. As, a- again, heard. Yeah. And so- We're at, and, like, we're, we're working on it.
225
00:34:29.864 --> 00:34:35.104
Uh, obviously, like, something the size of GitHub to move to, like, support this kind of new, this new feature is, like,
226
00:34:36.144 --> 00:34:44.144
not just, like, um, a walk in the park because of the size of GitHub's and, a- and GitHub's Git implementation, but, uh, it's something that we're actively exploring. Yeah.
227
00:34:44.484 --> 00:34:54.724
Well, I think, you know, just to wrap all that up, you know, I think it's really nice for someone who's so deeply e- en- engaged and, like, coming from, like, one of us literally- Yeah, yeah...
228
00:34:54.784 --> 00:34:59.364
that you now run things at, at GitHub, and we can just at you. You know what I mean? [laughs] You can do that, I mean.
229
00:34:59.384 --> 00:35:07.444
Yeah, I think, like, uh, Ajay Karpathy was, uh, the other day was saying, like, every company needs one of these where you can just like, "Hey, like, this, this really should exist at GitHub. We love GitHub.
230
00:35:07.464 --> 00:35:15.524
We use GitHub, but, like, come on," and... [laughs] Well, yeah. Feature requests welcome. Like, my DMs are always open. I'd... Like- Oh, careful. I don't know. [laughs] Hundred and eighty million developers.
231
00:35:15.824 --> 00:35:22.824
I, I like- [laughs] Hundred... Whatever. I, I am of the philosophy that, like, all feedback is a gift. Like, it's all a signal. Yeah.
232
00:35:22.924 --> 00:35:32.964
Um, and the more signal we can collect, the better decisions we can make, and the, truly build this really, really useful website, uh, and company, like, together, and that's gonna be the future.
233
00:35:32.984 --> 00:35:40.944
And if we focus just on that, we're gonna be okay. Yeah. We're gonna be okay. All right. Yeah. Well- Thanks so much, Eric. This was- Yeah. It's a real pleasure- Awesome... catching up. Yep, likewise. Congrats.
234
00:35:41.024 --> 00:35:51.084
[outro music]
