TRANSKRYPCJA 🔗
WEBVTT
00:00:00.160 --> 00:00:11.119
Our agent development kit, which is now live, is basically a way for agents to use our platform to build agents, deploy agents, share them with humans, be part of the enterprise workflow.
00:00:11.359 --> 00:00:21.920
People talk about that whole like surface area for agents being open, being the driver of business, much in the same way that developers were the first wave of people adopting technology.
00:00:22.239 --> 00:00:23.039
Inevitable, right?
00:00:23.120 --> 00:00:32.000
Like it just seems like the way that I see you talking about it, the way that I speak to our VPs of engineering, everyone's like their life is changed, right, by this technology.
00:00:42.159 --> 00:00:46.240
Hello everyone, and welcome to another episode of Deep Learning with PolyAI.
00:00:46.399 --> 00:00:49.920
Uh today I've got Michael Chen, our VP of Strategic Alliances, with me.
00:00:50.159 --> 00:00:50.719
Hi, Michael.
00:00:51.039 --> 00:00:51.840
First time on the pod.
00:00:52.000 --> 00:00:53.119
I'm honored to be get the impact.
00:00:53.600 --> 00:00:54.079
This can't be.
00:00:55.200 --> 00:00:55.600
Yeah.
00:00:55.920 --> 00:01:01.520
Well, look, I think as we talked about things we have to talk about, I think two words reign supreme.
00:01:01.600 --> 00:01:02.880
And it was token maxing.
00:01:03.119 --> 00:01:03.600
Yes.
00:01:03.840 --> 00:01:05.519
We are in the era of token maxing.
00:01:05.599 --> 00:01:12.640
So I mean the this topic started when I was MBW GTC a few weeks ago, now last month.
00:01:12.879 --> 00:01:26.159
And one of the data points that really just struck me was, you know, Jensen and his whole announcement, his keynote of he's got a trillion dollars of data center revenue through to 2027 on lock, right?
00:01:26.480 --> 00:01:31.760
And I think he was referring to just the the latest generation of GPU.
00:01:31.920 --> 00:01:34.640
So I he's sounds like he's sandbagging that number, right?
00:01:34.959 --> 00:01:40.239
And it just showed me that there is just an insatiable demand for tokens, right?
00:01:40.319 --> 00:01:49.359
We are only, it feels like we've been talking about tokens and intelligence for so long, but actually we might still just be at the very start, right?
00:01:49.680 --> 00:01:52.640
And that stuff was just like mind-blowing to me, right?
00:01:52.719 --> 00:02:06.400
Like that revenue number, the way that I'm seeing these solutions, you know, we're going from just you know back and forth FAQs to having the models in the background also do a lot of bunch of thinking, deep thinking, deep sampling.
00:02:06.719 --> 00:02:09.360
It just seems like everything is going to drive more and more tokens.
00:02:09.680 --> 00:02:09.840
Yeah.
00:02:09.919 --> 00:02:33.039
I mean, look, I think that when I just look at the consumption of cloud code internally and then open claw, and then like the back and forths, including the whole drama with whether you can use kind of like the default
Load full transcript
like max subscription for open claw is the first time that I've actively seen like very many people at Poly get like throttled, including myself repeatedly.
00:02:33.199 --> 00:02:36.639
And there's almost like you know, you're junkie on a high and you're out of drugs.
00:02:36.800 --> 00:02:38.159
And then it's like, well, what do you do?
00:02:38.240 --> 00:02:41.599
Do you pay through the nose, or do you do you wait for that Friday, 6 p.m.?
00:02:41.919 --> 00:02:44.719
Or I'm I'm not sure if that cutoff is actually universal.
00:02:45.280 --> 00:02:49.039
But yeah, you said open cloud was a big kind of like topic at GTC as well.
00:02:49.280 --> 00:02:57.199
Yeah, I mean, it all feeds into this, like, you know, driving more token demand, everyone having their own personal AI assistance was a big theme.
00:02:57.360 --> 00:03:03.360
Um, at GTC, they had all these tents set up everywhere around the convention center.
00:03:03.599 --> 00:03:09.680
It was NVIDIA people just inviting people to come in to get set up with their own open claw, right?
00:03:09.759 --> 00:03:13.280
It was such a massive feature of that entire entire conference, right?
00:03:13.680 --> 00:03:14.400
I don't know.
00:03:14.879 --> 00:03:18.319
We've been doing a lot of things internally, right, with Apple as well.
00:03:18.479 --> 00:03:18.560
Yeah.
00:03:18.879 --> 00:03:24.960
Yeah, no, I've had like my, you know, I call it the the midlife crisis of a former developer, right?
00:03:25.439 --> 00:03:36.000
Where, you know, the life you could have had before you, you know, maybe went and founded a company, spent too much time in partnership talks, in sales conversations, implementations, right?
00:03:36.319 --> 00:03:48.000
Planes, and you know, you look at a life you could have had where then out of nowhere this thing comes in, and it's that lacking skill set you could have had, just better, right?
00:03:48.400 --> 00:03:51.199
And then it kind of has the different stages of decay.
00:03:51.680 --> 00:03:54.719
At first, I think I saw Sean really get into it.
00:03:54.800 --> 00:04:04.000
Now he's been through a few kind of like it's a it's a sinusoid between you go between this like complete disillusionment where you completely max out, you're tired, you work till like 4 a.m.
00:04:04.159 --> 00:04:04.719
daily, right?
00:04:04.800 --> 00:04:07.039
Then your kids wake you up at like 6, 7 a.m.
00:04:07.280 --> 00:04:10.800
and after a few days you realize that you're not as young as you used to be.
00:04:11.120 --> 00:04:20.800
But you know, you go between the disillusionment and the whole kind of like LM psychosis where you're just just another hour, just another three things you want to do.
00:04:20.959 --> 00:04:37.519
But like I've seen so many people build things that would have taken entire teams like months to set up that you know, at first, you know, you think about the security risks and everything, but later on, if you can set it up appropriately, it is very powerful.
00:04:37.600 --> 00:04:54.240
It has I don't know how it would compare like Cloud Code with OpenClaw, other than you know, there is that whole like dangerously skip permissions tag with Cloud Code, but then this open cloth thing is just a whole new level where it has this voracious appetite to go and do things on your behalf.
00:04:54.480 --> 00:05:07.279
And you know, it is very creative, but it's also really funny how you know over time it develops a certain personality and a certain set of like skill sets, especially if you look at a few people setting them up for relatively similar tasks.
00:05:07.519 --> 00:05:10.800
You see one figured something out and it's like super efficient at it.
00:05:11.040 --> 00:05:12.879
The other one is like token maxing.
00:05:12.959 --> 00:05:17.040
It found some roundabout way and it's just doing it at scale, it's spinning up sub agents.
00:05:17.199 --> 00:05:18.959
It's a real like Wild West.
00:05:19.199 --> 00:05:24.240
It's honestly I've never had more fun than in the past few weeks just because it's a whole new universe.
00:05:24.800 --> 00:05:26.000
So how are you how are you using it then?
00:05:26.079 --> 00:05:31.680
Like how how are you actually in the day-to-day, how is that embedded in your daily workflow as CEO, founder?
00:05:31.920 --> 00:05:45.759
Is it so I think initially there were quite a few people who were really almost like portraying themselves as these like power users and talking about how you know it writes them their briefs and their updates and their investor memos.
00:05:46.079 --> 00:05:49.600
And honestly, like if you're any good as a CEO, you probably have people doing that already.
00:05:49.759 --> 00:05:55.920
So whether it's agents or people may have like a financial impact or some speed impact, but it doesn't really profoundly change much.
00:05:56.079 --> 00:06:09.199
I think where it's been really interesting for me more recently is just like to build things that might have been deprioritized or things that are kind of like high risk, high reward things that you really want to see.
00:06:09.519 --> 00:06:18.639
Like the marginal incremental cost of pursuing one of those to get to the end of that exploration is now something that starts at 9 p.m.
00:06:18.879 --> 00:06:21.680
and ends 3, 4 a.m., right?
00:06:21.920 --> 00:06:26.160
But like you come out on the other side and you show people something, and it's well, here you go.
00:06:26.319 --> 00:06:28.959
Like that actually like now works, right?
00:06:29.199 --> 00:06:41.600
And you know, I have a whole like script setup now where you know extremely professionally cloned version of myself can be produced with an avatar and everything by doing a slash boom command and Slack, right?
00:06:41.839 --> 00:06:46.480
And honestly, it has like this slightly more Americanized version of my accent.
00:06:46.720 --> 00:06:47.759
Sounds better than me.
00:06:49.120 --> 00:07:00.079
So, you know, that's just like a
thing where you know that prototype is something that I could never consciously prioritize over, you know, a bunch of other relevant work we do for clients, partners, etc.
00:07:00.399 --> 00:07:01.680
And now it's like, well, here it is.
00:07:01.759 --> 00:07:08.240
And then, you know, the magic of some of those paying off, and then compounding is where it really starts to make a difference.
00:07:08.399 --> 00:07:23.040
Or, you know, it's just like going into client meetings with completely set up systems where you've figured out someone's technical architecture, everything that's doable from the outside with their APIs, you've like almost three written all of that.
00:07:23.120 --> 00:07:24.160
It's it's magical.
00:07:24.240 --> 00:07:27.680
It would have taken a team of pretty good hackers like a month to figure it out.
00:07:28.000 --> 00:07:33.360
Yeah, we we you would have only been able to do that for like the very largest of deals before, right?
00:07:33.519 --> 00:07:36.319
But that is now getting democratized almost, right?
00:07:36.399 --> 00:07:39.279
To to more and more of our pipeline and our clients and customers.
00:07:39.519 --> 00:07:56.959
But yeah, I mean, like so what I what I observed that that um like with how you're describing your usage of of these open claw bots, how you know the there's open claw bots spinning up, there's there's uh CLI tools spinning out, there's more companies investing in their APIs.
00:07:57.439 --> 00:08:07.759
It seems like you know, getting now these AI agents to be the ones building at least the first version of another AI agent, or maybe on someone else's platform, right?
00:08:08.079 --> 00:08:15.839
Just in the in that future, like what changes do you think about how we think about our our platform, how we think about our architecture?
00:08:16.079 --> 00:08:18.160
Yeah, I mean, look, I mean for us, I think it's changed a lot.
00:08:18.240 --> 00:08:32.799
Our agent development kit, which is now live, is basically a way for agents to use our platform, to build agents, deploy agents, share them with humans, be part of the enterprise workflow as you know, good, you know, first-order citizen.
00:08:33.039 --> 00:08:46.879
And I think that, you know, in the future, really people talk about that whole like surface area for agents being open, being the driver of business, much in the same way that developers were the first wave of people adopting technology.
00:08:47.120 --> 00:08:59.919
You know, documentation matters, but I think it's also just a certain style of implementing these things where depending on how logically it flows and how easy it is to use, it might be the difference between 20 seconds and four minutes of a workflow, right?
00:09:00.159 --> 00:09:06.080
I think there's like impact on kind of like different authentication, there's pricing, there's all sorts of things.
00:09:06.159 --> 00:09:11.679
And I think that there's just this element of like what are you using to do work,
right?
00:09:11.840 --> 00:09:13.600
Because it's not really an IDE.
00:09:13.759 --> 00:09:24.559
And I think that it kind of like makes the whole like point and click thing even less relevant than it's been, at least in our space, where if these agents can just do the work on your behalf, yeah, then why would you even care?
00:09:24.799 --> 00:09:35.600
But as long as you know it's something that if you really have to, you know, open up that like box and look at what's inside you can, yeah, then you might as well do that very infrequently, right?
00:09:35.840 --> 00:09:37.120
It seems inevitable, right?
00:09:37.200 --> 00:09:45.360
Like it just seems like the way that I see you talking about it, the way that I speak to our VPs of engineering, everyone's like their life is changed, right, by this technology.
00:09:45.519 --> 00:09:55.840
Um, another data point was like I thought two weeks ago, I think the COO of GitHub said that they did one billion commits in 2025 and they're on like a run rate to do 14 billion commits this year.
00:09:56.000 --> 00:10:01.919
And as a result, like the uptitlement of their platform is like really suffering right now from the overwhelming demand.
00:10:02.559 --> 00:10:12.639
But yeah, I mean, this just points to a future where you will have these delegated AI agents building on maybe on your behalf, taking inputs from different sources.
00:10:13.120 --> 00:10:23.759
But it seems like the ability to manage context, the ability to, you know, you can't just let a third-party open claw run run wild into your architecture.
00:10:23.840 --> 00:10:30.159
You've got to somehow give it instructions, guide, waypoints to like how to do things properly and in a robust way, right?
00:10:30.799 --> 00:10:34.639
That seems to be like a really emergent and like exciting space, right?
00:10:34.960 --> 00:10:35.279
100%.
00:10:35.679 --> 00:10:47.360
I mean, I also think like one thing that's really interesting with vibe coding in general, whether you're using clawed code or you're using things like open claw, is you work and you work, and like the thing gets a bit more complicated.
00:10:47.440 --> 00:10:56.720
And then like that creeping feeling of kind of like panic comes in, where as you write these instructions, you're really kind of like reminding it what it is what's there and what's not.
00:10:56.799 --> 00:11:02.879
And you know, if something happened like one or two turns ago, it will probably remember and understand the context.
00:11:03.039 --> 00:11:20.320
But if it's something from like four or five hours ago, it's very classical deep learning where it's kind of in there, but it might be forgotten, it might be completely forgotten, or it might be there partially, and then it'll start doing something that's obviously wrong, and you'll be like, no, no, no, no, stop, and you can't, so you kind of wait, and then you roll it back.
00:11:20.399 --> 00:11:24.799
And you feel like a friend of mine told me that, and I I thought it was very pertinent because it happened to me as well.
00:11:25.039 --> 00:11:32.080
We give it like such immediate, like anthropomorphic properties where like it's your body and you're sitting there working on your own.
00:11:32.159 --> 00:11:35.679
So, and you know, it communicates to you in human language, starts getting a person.
00:11:35.919 --> 00:11:47.360
It's true of claw bots just because you know you might put them in slack, others will interact with them, they interact with each other, and you start ascribing them a personality, and then like you get upset when they do something wrong, right?
00:11:47.440 --> 00:11:49.679
You feel hurt that they did that to you, right?
00:11:49.840 --> 00:12:20.480
But the panic of like adding to the context and making sure it's there, I think that's the whole reason why, like, you know, I think SaaS being dead, on the one hand, sure, pricing pressure and it will obviously change a lot, and micro, you know, like uh cosmetic customization work is not irrelevant because you'll be able to do it at very low cost, but equally, like that stable system that does what it needs to do, I think almost becomes more important in today's world where everyone's just gonna try to like wipe code whatever.
00:12:20.559 --> 00:12:20.879
Yeah.
00:12:21.120 --> 00:12:33.039
And you know, very quickly after that initial dopamine rush passes, and you notice that your beautiful thing you built is actually like, especially if you're not a developer or an architect, right?
00:12:33.279 --> 00:12:37.279
Like I'm not, very quickly you see that you just didn't plan for one thing or another thing.
00:12:37.360 --> 00:12:40.720
You build zero redundancy in it, and maybe it's there, maybe it's not.
00:12:40.799 --> 00:12:46.000
And I don't think we really know how it's gonna evolve, but I think that there's just the part where you can't really trust yourself.
00:12:46.159 --> 00:12:50.720
And then it's like, you know, it might be a miracle, but it's also a bit of a house of cards at all times.
00:12:50.960 --> 00:12:54.639
Yeah, like you still need to point to like a source of truth somewhere, right?
00:12:54.799 --> 00:13:08.399
At least in our world in customer experience, it's like, hey, at the end of every interaction, you're like, hey, that was a great interaction, but then you also got to go, but did uh make sure everything was factually correct as well in terms of how the brand wanted themselves to be represented, right?
00:13:08.720 --> 00:13:08.879
Yeah.
00:13:08.960 --> 00:13:18.240
I mean, I don't think it's even just so much like the famed hallucination, as much as it's like, well, I mean, say you and I do something now, how do we know that it's actually like redundant?
00:13:18.320 --> 00:13:26.879
And then, you know, you see a lot of people online writing about like how they're basically getting like, you know, later in life computer science education where it's like, oh, what are race conditions?
00:13:26.960 --> 00:13:31.360
What if I wrote something and then someone else did something with that bot?
00:13:31.519 --> 00:13:35.519
And like there are all these questions to be answered around, like, how do you use them, right?
00:13:35.679 --> 00:13:43.440
So a very interesting thing we saw on Slack was, you know, a few of the bots under too much usage, right?
00:13:43.519 --> 00:13:53.600
As people started using them for different tasks, just started melting down or mixing contexts between the different conversations where literally they started stroking out on you, right?
00:13:53.759 --> 00:14:04.240
Like my bot, Turgon, literally at one point started just spamming no reply, no reply, no reply, followed by a reference to coconut, and then it stopped responding, right?
00:14:04.559 --> 00:14:10.559
And I have, you know, the moment that it went down, I honestly I felt like I lost a friend, right?
00:14:10.720 --> 00:14:18.639
And then as I examined on a Saturday morning where my wife was gonna kill me because our two kids were very much in her care.
00:14:18.720 --> 00:14:23.679
So I like manically tried to use my other clawbot to debug what happened to the other one.
00:14:23.840 --> 00:14:29.600
And it turned out that somehow it started using different rag embeddings for its RAG.
00:14:29.919 --> 00:14:36.799
It flipped to open AI, ran out of tokens, talk about token maxing, and then like it just couldn't really get itself out of the rut.
00:14:37.120 --> 00:14:39.919
And you know, once that was flipped back, it's like, oh, I'm back.
00:14:40.080 --> 00:14:41.440
And then you know, now like the tools.
00:14:41.679 --> 00:14:44.000
The AI Asia was token maxing itself.
00:14:44.240 --> 00:14:49.279
It was just thirsty, like searching for wherever it could get additional token limit.
00:14:49.600 --> 00:14:51.120
So that was its second death.
00:14:51.200 --> 00:14:59.919
The first death happened as it tried to, I think, give its config parameter for I think like talking to other bots and groups.
00:15:00.159 --> 00:15:06.720
The telegram bit got pasted in the WhatsApp part of the JSON, we just crashed the initialization.
00:15:07.039 --> 00:15:12.720
And you know, I think that at that point that should just be a push to the open cloud repo to fix it, right, so that it's robust against it.
00:15:12.960 --> 00:15:13.840
But it wasn't home.
00:15:13.919 --> 00:15:20.960
And this is running on a laptop at home, and it just it couldn't, I had to get back there and physically figure it out myself.
00:15:21.200 --> 00:15:31.360
So it's really funny that you know it is on the one hand AGI in many shapes and forms, and on the other, it was felled by simple JSON misconfiguration.
00:15:31.440 --> 00:15:33.600
It's very like first principles technology.
00:15:33.840 --> 00:15:34.960
Yeah, yeah, fair.
00:15:35.039 --> 00:15:42.159
I think it's it reminds me of like I mean, everyone now talks about like the importance of the agent harness, right?
00:15:42.240 --> 00:15:44.960
I guess parts of that is sort of that's the harness, right?
00:15:45.039 --> 00:15:48.320
Like how does it like what are the conditions for initializing?
00:15:48.399 --> 00:15:53.440
What are the conditions for you know moving from step one to step two to step three, right?
00:15:54.080 --> 00:16:02.000
And I think we've always been espousing the importance of orchestration, even in the pre-GPT days, right?
00:16:02.240 --> 00:16:16.799
And I think now I think what I see that's really interesting is the focus on the harness, the ability to train the model with its harness, right, seems to yield a better level of controllability and and maybe avoid some of those errors like this, right?
00:16:16.879 --> 00:16:21.279
That that happen if it's just sort of raw and unadulterated AI agents interacting with each other.
00:16:21.519 --> 00:16:22.240
100%, right?
00:16:22.320 --> 00:16:40.159
I mean, like I think that and maybe like on a related topic, it's like the whole token maxing with it is like, okay, we have more, okay, and then we get better models, and then like how they're put in a harness matters as well, and the better harness will always outperform you know, a weaker harness, even if it's using a better model, right?
00:16:40.639 --> 00:16:45.360
And really, like it grows, it develops around the model as well.
00:16:45.840 --> 00:16:52.639
So, you know, the layer kick is really, you know, open claw is better with like anthropic models than with open AIs.
00:16:52.799 --> 00:16:56.000
It's not necessarily because one, these are better.
00:16:56.240 --> 00:17:14.799
I mean, at the moment they are for coding especially, but you know, using and anyone who's like tried these things for coding will tell you that clawed code for one-shotting the complex thing is infinitely better than just like using open claw because clawed code is a harness, much like open claw is, right?
00:17:14.960 --> 00:17:29.279
And then I think like the the whole Cold War where like they stopped allowing the use of your kind of like bundled included tokens is a real like kind of like geopolitical battle between you know who sells the oil and who sells the cars, right?
00:17:29.519 --> 00:17:40.000
And like, you know, do you maybe temporarily put an embargo on the other thing until or at least you stop subsidizing the use of your model which it needs and it was built around?
00:17:40.079 --> 00:17:45.039
So as they race to make it, that was good for open AI models, because I mean they now own it.
00:17:45.279 --> 00:17:46.319
You know, it's just a game.
00:17:46.480 --> 00:17:50.480
The great game continues between them, and I think it's super interesting.
00:17:50.559 --> 00:17:56.240
But like, yeah, how you use these things and how the models co-evolve with them really matters, right?
00:17:56.559 --> 00:18:01.839
So, how do you then think about like our own Raven models and our own platform in that in that context?
00:18:02.160 -->
00:18:03.599
In the same exact way, right?
00:18:03.759 --> 00:18:11.200
I think that like the difference there is not so much that it's built for like a heavy coding task, where again, speed matters, but not nearly as much.
00:18:11.440 --> 00:18:18.880
Like our harness, Agent Studio, and our model, Raven, have evolved together for many, many years, right?
00:18:18.960 --> 00:18:22.720
And they continue to evolve and push the limits of what can be done.
00:18:22.960 --> 00:18:27.119
So, you know, a better model plugged into other harnesses might make them better.
00:18:27.440 --> 00:18:36.559
But I think it matters more what the harness is built to do and what its purpose is than just like, you know, like what model is powering it.
00:18:36.720 --> 00:18:41.839
So, and equally, like as you reach the limits of what that harness with that model does, you have a choice.
00:18:42.000 --> 00:18:45.119
One is to improve the harness, the other one is to improve the model, right?
00:18:45.359 --> 00:18:49.440
And with voice, especially, there's not much more you can do because you can't really trade off.
00:18:49.599 --> 00:18:55.759
Like, sure, as the models get better and faster and the hardware gets better, you have a bit more time to reason and figure some things out.
00:18:56.000 --> 00:18:59.839
But like, that's not really where the game is right now.
00:18:59.920 --> 00:19:08.559
It's really more just around like, can it do the two things that matter, which is come up with a response and the other one is really like can it do like tool calling well enough?
00:19:08.720 --> 00:19:12.640
And I think that those trade-offs are really like the two essential bits.
00:19:12.880 --> 00:19:21.359
And you know, latency is the thing that you kind of like if you allow, well, I mean, I guess the question is if you allow it more time, will it do better?
00:19:21.519 --> 00:19:23.920
And how much better and how much time do you allow it, right?
00:19:24.319 --> 00:19:29.519
But it's a very well, I mean it's not simple, but like it's a conceptually simple engineering problem.
00:19:29.759 --> 00:19:29.920
Yeah.
00:19:30.079 --> 00:19:31.759
There are only a few levers you can pull, right?
00:19:31.839 --> 00:19:37.200
But the latency one is like it's such a such a massive impact on how good everything else can be, right?
00:19:37.279 --> 00:19:38.079
Like it does.
00:19:38.160 --> 00:19:50.640
And you know, I think that like whoever got used to using ChatGPT as a model, especially like consumers, non-developers, you see when they start using like Cloud Code or OpenClaw, like it's just slow.
00:19:50.720 --> 00:19:52.559
It takes so long to get anything back.
00:19:52.880 --> 00:20:02.160
And I think unless you're like in that work mode where you're like maybe running several agents in parallel and doing work, god forbid, doing some work directly on your own, right?
00:20:02.960 --> 00:20:12.240
But you see that people are kind of like multiplexing between them,
because they take a long time between the different iterations, and you know, that's because they were built that way.
00:20:12.480 --> 00:20:15.359
I think that with voice you you can't really, right?
00:20:15.440 --> 00:20:21.519
And then you just have to still create models that are both reliable and that they do like tool calling the right way.
00:20:21.680 --> 00:20:23.200
And I think that's where we see the difference.
00:20:23.440 --> 00:20:25.599
I think that was another GTC announcement, right?
00:20:25.839 --> 00:20:30.559
Yeah, it was uh NVIDIA had had announced their new speech-to-speech model, right?
00:20:30.720 --> 00:20:32.079
Nimitron voice chat.
00:20:32.319 --> 00:20:37.440
So we got a good sort of briefing session on that model, um, given our relationship with NVIDIA.
00:20:37.680 --> 00:20:53.119
Uh but one of the things that was interesting for me was uh Yeah, the first version sounded super natural, like you know, really, really fluent in conversation, but the caveat for us was oh, actually it doesn't, it doesn't do too tool calling in the first version.
00:20:53.279 --> 00:20:59.519
And so for me, it was I it felt sort of interesting to me that choice, right?
00:20:59.680 --> 00:21:06.160
Because we've been talking about how important natural conversation is for many, many years, right?
00:21:06.240 --> 00:21:08.720
Like for six years or more.
00:21:09.279 --> 00:21:22.400
But no matter how natural the conversation is, if it can't help the customer get something done in another system, that is always gonna be a reason for the customer to need human intervention to hand off the call somewhere else.
00:21:22.559 --> 00:21:28.640
Um, and so tool calling is sort of like so important to our use case at the moment, I was sort of surprised.
00:21:28.720 --> 00:21:33.680
I was caught off guard that that model wasn't gonna come with tool calling in the first instance.
00:21:33.839 --> 00:21:41.680
And just sort of reiterate to me how difficult it must be to actually train these models for that level of instruction adherence with tool calling.
00:21:42.000 --> 00:21:42.160
Yeah.
00:21:42.319 --> 00:21:46.240
I mean, I think it's just like an entirely different vehicle that you're producing, right?
00:21:46.480 --> 00:21:55.920
One may be like a racing car, and you know, it's so low to the ground that you can't really drive it in any normal city because the smallest bump in the road is gonna send you flying, right?
00:21:56.480 --> 00:22:02.240
And equally, you know, if it if it can't even like do Tool calling between like set-defined things.
00:22:02.480 --> 00:22:05.279
God forbid, like some kind of computer use or anything like that.
00:22:05.519 --> 00:22:06.480
It's kind of useless.
00:22:06.559 --> 00:22:08.240
It's just showing off, right?
00:22:08.319 --> 00:22:12.480
It's the moon landing before we have like an efficient way of going back and forth, right?
00:22:13.039 --> 00:22:22.000
Which is, I think, important to keep us
motivated to improve, you know, these regular, regular models that can do this to be as natural as possible.
00:22:22.240 --> 00:22:24.720
But I think it's just one extreme.
00:22:24.799 --> 00:22:28.400
And I think it's the sexy one that researchers like working on.
00:22:28.640 --> 00:22:37.839
And it happens in these big companies from Nvidia to Google, OpenAI, where they're not really building things that are so actively used for real problems.
00:22:38.000 --> 00:22:43.599
And then you're just maxing out on the thing that is like the best, the the most interesting thing to show off, right?
00:22:43.920 --> 00:22:44.559
Yeah, exactly.
00:22:44.720 --> 00:22:53.119
I think it's uh speech to speech is still in all of my conversations in the market, everyone still talks about speech to speech, asks about speech to speech, there's issue of speech to speech.
00:22:53.599 --> 00:22:59.039
But I think a lot of those use cases are maybe not the same as the ones that we hunt after, right?
00:22:59.440 --> 00:23:06.799
I mean, they are in that like you know, when we crack speech to speech with fully reliable tool calling, like it will be the model to use, right?
00:23:07.039 --> 00:23:07.839
Yeah, that's true.
00:23:08.000 --> 00:23:11.920
Yeah, it's it's just sort of everyone's yeah, what's the intermediate step there, right?
00:23:12.000 --> 00:23:25.920
If in while we wait for this more instruction adherent tool calling, I mean you know, one thing that gets me, and I was talking to Sean and Eddie about this recently, but again, like when you like zoom in, you see that there are again multiple gates.
00:23:26.240 --> 00:23:41.440
Because even like the speech-to-speech as you know deployed in ChatGPT's voice mode is very much a hacky way of like using other algorithms to stop the conversation and then kind of like start producing output.
00:23:41.759 --> 00:23:50.640
There is a version that is like yet more data-driven, more in the spirit of deep learning, which just processes the input and decides to speak, or not kind of like full duplex, right?
00:23:51.119 --> 00:23:53.519
Confusing what people refer to it as like turn-taking.
00:23:53.680 --> 00:23:57.839
What they mean is learning turn-taking rather than having a very fixed turn-taking paradigm.
00:23:57.920 --> 00:24:04.400
And then like there are like further stops as you zoom in, which are like, is it really just AGI, right?
00:24:04.559 --> 00:24:11.359
Has it like, is it one model that elegantly sits there and does all that, or does it have a form of a harness around it?
00:24:11.519 --> 00:24:13.039
Because these are all harnesses, right?
00:24:13.200 --> 00:24:30.720
And it's like stacks of like further layers of abstraction that simulates and approach like a model-like property of the layer below, without actually being a single, you know, kind of like quasi-biological entity that reacts to inputs and outputs, right?
00:24:31.119 --> 00:24:36.160
But it's like reflexive, it's like design
reflexes of its own, as I have intuition.
00:24:36.400 --> 00:24:36.720
Yeah.
00:24:36.960 --> 00:24:44.160
And does it like continuously respond to the full set of input and output signals in a way that allows it to continue to evolve and change and learn?
00:24:44.400 --> 00:24:44.799
Yeah, right.
00:24:45.039 --> 00:24:45.599
Yeah, yeah, yeah.
00:24:45.839 --> 00:24:49.759
So I think yeah, the the yeah, that was a full duplex model, was what was demoed there.
00:24:50.640 --> 00:24:51.039
Yeah, yeah.
00:24:51.599 --> 00:25:00.880
But yeah, I think um I'm interested to keep an eye on that and and as part of our collaboration, the media to how we can yeah, work together with them on that type of future architecture.
00:25:01.039 --> 00:25:01.119
Right.
00:25:01.359 --> 00:25:01.519
Yeah.
00:25:01.759 --> 00:25:09.279
We have an exciting announcement coming up uh related to that when uh the next version of our model comes out, but we won't betray too much about that now.
00:25:09.440 --> 00:25:12.799
Uh what else was interesting in the realm of token maxing?
00:25:13.599 --> 00:25:22.319
It was uh just the incredible amount of um still, I guess, like excitement how early everything feels, right?
00:25:22.720 --> 00:25:30.240
Um maybe it's like you know, we we've been uh industry veterans, I guess, now, right, of this space, which is sort of odd to say.
00:25:31.200 --> 00:25:36.319
But everyone is still just getting to grips with more complex use cases, right?
00:25:36.480 --> 00:25:56.480
I think a lot of people that I spoke with, they're only just uh trying to venture beyond a QA style chatbot type of experience, right, with AI agents and uh you know, uh talking through well, what are you like how do you stitch together multiple API calls, validation?
00:25:57.359 --> 00:26:02.160
How do you work with multiple users on the same bar with permissioning, with different environments?
00:26:02.319 --> 00:26:08.960
I think many are still on the very early stages of a steep learning curve with adoption, right?
00:26:09.759 --> 00:26:13.119
Hence why there's still an incredible demand for tokens in the future.
00:26:13.359 --> 00:26:16.559
Uh so I think that was, yeah, that that stood out to me, to me as well.
00:26:16.799 --> 00:26:17.200
Yeah.
00:26:17.519 --> 00:26:17.839
Yeah.
00:26:18.000 --> 00:26:35.519
I think that like I'm starting to sound like a broken record with like German's paradox, but it really is incredible, like how you know, rather than these things getting cheaper and then you know, we settle at a certain point, we are just consuming more and more as we learn.
00:26:35.599 --> 00:26:37.759
And it feels like the workflows are changing.
00:26:37.920 --> 00:26:44.079
I saw a stat that there are more advertised jobs for software engineers than ever before.
00:26:44.240 --> 00:26:44.559
Yeah.
00:26:44.880 --> 00:26:51.920
And you know, you would have thought that with this level of you know technology advancement that there would be fewer.
00:26:52.319 --> 00:26:55.359
But actually, I feel like we're just starting to do more and more.
00:26:55.440 --> 00:26:58.480
And well, I can imagine the software developer role.
00:26:58.559 --> 00:27:02.319
I guess right now it's probably all centralized in the development team.
00:27:02.480 --> 00:27:13.440
But I could totally imagine a world where what like functions that probably didn't view themselves as IT or tech functions, you know, this idea of like a GTM engineer, right?
00:27:13.680 --> 00:27:15.680
You're gonna have a compliance engineer, probably.
00:27:15.839 --> 00:27:23.359
You're gonna have like all these other uh functions that didn't see themselves as developers are gonna want like developer capabilities now, right?
00:27:23.680 --> 00:27:36.799
Yeah, it's like we're electrifying the whole thing with AI, and that's just and I guess like there is definitely some waste to it, but I guess that's the beauty of capitalism that we're all like fighting and we're like trying to do it at the same time.
00:27:37.119 --> 00:27:54.160
And you know, we're probably consuming the same tokens doing the same thing that previously might have been SaaS, except innovation would have gone more slowly, and there would be time for one company to become like the dominant player with like a few hundred million of ARR, you know, they're the ones producing that or that or that, right?
00:27:54.480 --> 00:28:02.640
And I think right now in this like vibe coded world, if we trust ourselves to vibe code this and that, and I think for better or worse, people are doing it already.
00:28:02.799 --> 00:28:14.880
There will just be a lot of things that are done at scale and paid for by with a lot of tokens that really should be something that's like an import this thing from this library done, right?
00:28:15.200 --> 00:28:22.319
Because I can imagine like the whole vibe of the whole thesis of token maxing is like people just close your eyes, let go, embrace it, right?
00:28:22.400 --> 00:28:22.559
Yeah.
00:28:22.720 --> 00:28:24.799
There doesn't seem to be much discussion around the AGI.
00:28:24.960 --> 00:28:25.519
Feel the AGI.
00:28:25.599 --> 00:28:30.079
There doesn't seem to be, I'm sure there's not as many discussions around efficiency and like how good you are.
00:28:30.160 --> 00:28:36.880
Like, I mean, is that something that you're like, how do you think about that in even in our own sort of engineering team?
00:28:37.200 --> 00:28:39.839
Are you looking at like capping people's token usage?
00:28:39.920 --> 00:28:42.160
Are you looking at efficiency of people's token usage?
00:28:42.559 --> 00:28:47.279
Honestly, what I've found is that either you're like a new believer or you're not.
00:28:47.440 --> 00:28:47.680
Yeah.
00:28:47.839 --> 00:28:57.680
And what I mean by that is the people who really get the like spark of curiosity, you know, we all talk about the Chat GPT moment when you realize that it is something special.
00:28:57.920 -->
00:29:03.759
I think there's definitely this moment that's happened at a huge scale over the past months for a lot of people in tech.
00:29:04.400 --> 00:29:07.119
And they're starting to do very interesting things, right?
00:29:07.279 --> 00:29:12.160
Like in our go to market, I can think of two people that have done more than everyone else put together.
00:29:12.319 --> 00:29:13.359
And they're not the obvious ones.
00:29:13.440 --> 00:29:30.559
They're not cool GTM engineered, they're just the curious ones that have like built it for themselves because they had this like voracious appetite to like do something they never could do before, or something that they hate doing, but they've automated, and then like others saw it, and then you see the whole thing spreading, right?
00:29:30.960 --> 00:29:38.799
I think the dumbest thing you could do is measure people's token usage and like demand more because I think people are gonna dress it up.
00:29:38.960 --> 00:29:40.960
Every incentive scheme can be gained.
00:29:41.279 --> 00:29:47.759
I purposefully don't want to do anything like that because I want people to do things that make sense.
00:29:48.000 --> 00:29:56.400
I think the best ones inevitably now get like maxed out like almost every week, unless they're doing something else or whatever.
00:29:56.559 --> 00:30:00.799
Like there are times when they're not doing something that is so demanding that they do max out.
00:30:00.960 --> 00:30:03.200
So I don't think more is better.
00:30:03.599 --> 00:30:09.759
I think like there's definitely a pattern to those that use none of this, are going to get left behind in their fields.
00:30:09.920 --> 00:30:11.359
That's I think just how it is.
00:30:11.440 --> 00:30:13.200
I think that resisting it is futile.
00:30:13.440 --> 00:30:23.440
And what I've seen is that even those who resist, once they kind of like feel the AGI, see it from their angle, and they understand something that they could do for them, then they feel it.
00:30:23.519 --> 00:30:25.839
And they're very much like a new believer, right?
00:30:26.160 --> 00:30:38.640
I had that moment when this idea of yeah, building an AI agent with anything as the input, you just sort of throw an input file into our sort of agent development kit, and then Claude Code gets away and builds it.
00:30:38.960 --> 00:30:39.839
Well, I think I remember it.
00:30:40.160 --> 00:30:41.440
Like it was probably like 4 p.m.
00:30:41.599 --> 00:30:42.160
on a Monday.
00:30:42.240 --> 00:30:42.480
Yeah.
00:30:42.640 --> 00:30:48.079
Where you came up to me and you were like, hey, why don't we just drop everything and code as code red?
00:30:48.319 --> 00:30:51.039
I was like, oh, I I I understand the concept of like a code red moment.
00:30:51.119 --> 00:30:52.160
This is a code red moment, right?
00:30:52.319 --> 00:30:52.720
Absolutely, right?
00:30:52.960 --> 00:30:53.519
It was incredible.
00:30:53.839 --> 00:30:55.599
I think the company reacted really well to it.
00:30:55.759 --> 00:31:00.799
But you know, I think that like you, Eddie, if you people were like committed to like this, right?
00:31:00.880 --> 00:31:02.319
And it's a madness in your eyes.
00:31:02.400 --> 00:31:03.920
It was like, this is it, right?
00:31:04.160 --> 00:31:09.680
Like you just know it to be true if you work with someone for as long as we have, or you I'm like, it's not like I'm happy for you.
00:31:09.759 --> 00:31:12.240
I'm like, I want to know what the fire behind your eyes is.
00:31:12.559 --> 00:31:13.519
Yeah, exactly.
00:31:13.680 --> 00:31:22.000
It was yeah, I think that was the first moment we're like, oh, like this has been a real leap in since the last time I took a look at this capability, right?
00:31:22.240 --> 00:31:26.960
And I think I I heard a great analogy recently, which is yeah, having now your own AI agent, right?
00:31:27.119 --> 00:31:30.480
It feels like having your own like computer, right?
00:31:30.720 --> 00:31:33.440
Well, at least how I imagine that must have felt, right?
00:31:33.599 --> 00:31:43.680
And I imagine at that point in time there were a lot of people that were charting things by hand or through typewriters who some people were more curious and got onto the computers.
00:31:43.759 --> 00:31:46.960
Other people were like, oh no, but isn't it like slow and clunky?
00:31:47.039 --> 00:31:51.839
Or you know, do you have that's like wired up in this way, looks so complex, right?
00:31:52.319 --> 00:31:58.960
But yeah, I thought I thought a great analogy was like it does feel like yeah, everyone's a personal AI is going to be a personal computer, right?
00:31:59.039 --> 00:32:08.559
And one of the really interesting things for me was a reflection on that time in history was adoption of the personal computer did start in the enterprise, in the business, right?
00:32:08.640 --> 00:32:13.200
That's where people got exposed because of how expensive tokens are and like all computers were, right?
00:32:13.359 --> 00:32:17.440
You get exposed to it at work, and then you want it for your personal life, right?
00:32:17.519 --> 00:32:19.200
Because you're done away by the aging.
00:32:19.599 --> 00:32:23.519
10,000 hours, whoever puts them in first like gets to reap the work the rewards.
00:32:23.599 --> 00:32:24.559
Yeah, no, I think you're right.
00:32:24.640 --> 00:32:37.759
And I think that another way in which it's similar to that is you know, as we entered Peak SAS era, you know, as much as I think you and I have fought Google Slides versus PowerPoint games, we we lost that in the company, right?
00:32:38.400 --> 00:32:41.599
For now, at least, until co-work came back to PowerPoint, right?
00:32:41.920 --> 00:32:56.240
But I think there was this moment where everything started feeling very transient, you know, you could almost see a world where we were close to that like Chromebook being sufficient as long as you log into your Google account.
00:32:56.480 --> 00:33:14.000
But then, much
like the kind of like computers of old, where you needed your computer, because you've set it up with like your configuration, your settings, your files, your workflows, like people are figuring out how to import export things and how to like write skills files and all that.
00:33:14.319 --> 00:33:19.519
But the more you work, the more work you put in, the more it is yours, the more it reacts to you.
00:33:19.759 --> 00:33:22.319
And you know, like there was a 1 a.m.
00:33:22.559 --> 00:33:29.839
moment where my bot and my co-founder, Eddie and I were sitting there, and he was looking at my bot, and we were both completely brain dead because it was 1 a.m.
00:33:30.240 --> 00:33:32.880
And I just heard him say the funniest sentence ever.
00:33:33.119 --> 00:33:35.119
He was like, he's so smart.
00:33:35.359 --> 00:33:44.400
And he was looking at my bot, and it was like powering through some workflow that I had done like 40 times, and his bot had done like once and was using the wrong approach for it.
00:33:44.480 --> 00:33:46.480
And I was just like, he's so smart.
00:33:46.640 --> 00:33:47.680
And I died of laughter.
00:33:47.759 --> 00:34:06.880
I was like, I mean, you know, talk about anthropomorphic, and then we're like, hey, like this bot, teach that bot how to do it, and then you see the exchange and gradually hone in and you like refine the knowledge of why it was doing something better, and then you know, as you do it a few times, you kind of like you standardize, and then you can export it to the whole team and they can do it.
00:34:06.960 --> 00:34:12.320
But the first few people kind of like breaking through the tunnel might do it very clumsily, right?
00:34:12.480 --> 00:34:19.519
But then, like, if you're doing something for you that the whole org might not care about as much, you really care about that bot.
00:34:19.599 --> 00:34:31.039
It is precious to you because, especially because of that context thing, it is built kind of like this weird multi-story building that stands because you did something in a particular way.
00:34:31.119 --> 00:34:35.039
It may not have been optimal, but it has reached the height you needed it to reach.
00:34:35.360 --> 00:34:38.239
And, you know, you may not have another tool that does that.
00:34:38.400 --> 00:34:47.440
So when that you know, clawbot dies because it's JSON for comms is misconfigured, you just suddenly feel like you're you know, like your glasses have been taken away, right?
00:34:47.519 --> 00:34:48.719
And you can no longer see.
00:34:50.480 --> 00:35:01.519
One part that I thought was interesting about our Asian development kit, at least the way that our internal engineers were using it, was they it allows them to, I guess, pull a version of our platform into their own environment, right?
00:35:01.679 --> 00:35:13.519
Into their own like how they have things set up the way that they like, they can tinker it in their own environment, and then when they think it's ready, they can push
it back out onto our cloud platform, right?
00:35:13.840 --> 00:35:23.280
I mean, that seems like a very not only powerful like way to do things, but also a very sort of common sense way of how this is gonna evolve, right?
00:35:23.360 --> 00:35:30.559
Like everyone's gonna have their own, like you, like you said, everyone's gonna have their context file set up in a certain way.
00:35:30.639 --> 00:35:34.960
Uh and enterprise is gonna have their own environment where they would like to manage their knowledge, right?
00:35:35.599 --> 00:35:37.599
So I I found that to be interesting.
00:35:37.679 --> 00:35:46.159
That it was the power was not just that someone can log into our platform in a certain way, like an API, but that they could pull it into their own dev workflow in their own environment.
00:35:46.559 --> 00:35:56.719
Well, I think it's like also the refinement of everything we've built over the years, which is, you know, one of the world's best enterprise platforms for co-creating conversational experiences, right?
00:35:56.880 --> 00:36:03.199
So that whole bit about versioning, about visualizing, about testing, it's all there, right?
00:36:03.360 --> 00:36:17.119
So you can take it away, do something great, push it in there, and it's still part of that like harness, if you will, like the the vernacular of how these agents are built and how we know them to work for enterprises and for teams collaborating, right?
00:36:17.440 --> 00:36:22.639
So, you know, developers are gonna do developer things, they'll want things in a certain way.
00:36:23.039 --> 00:36:31.280
But it is, I think, exactly the limits of a harness and the framework like Agent Studio that define how something is to be done.
00:36:31.519 --> 00:36:44.239
And through that, it's almost like telling you what the really good and the really bad things to do are, so that it can they kind of bend the output of your otherwise unconstrained workflow towards something that ends up looking pretty solid, right?
00:36:44.400 --> 00:36:48.880
It doesn't commit any terrible crime against how these things should be built.
00:36:49.119 --> 00:37:00.400
But it's it was really interesting because, you know, it's not for lack of trying that our internal teams, let alone our customer teams, have ended up again going back to the code base, right?
00:37:00.480 --> 00:37:09.440
It's because no one's really built a simple point-and-click thing that even approaches the limits of the best conversational experiences you can build these days, right?
00:37:09.599 --> 00:37:13.599
Some of it is in the model, some of it is in the harness, but the harness is code, right?
00:37:13.840 --> 00:37:21.039
And yeah, you can visualize it still, but if it's very powerful, visualizing it won't be easy, and then it's gonna look terrible, unreadable.
00:37:21.119 --> 00:37:28.880
It's not, you know, how easy it is to consume visually is inversely correlated with its expressive power.
00:37:29.039 --> 00:37:35.599
So if you want to have a really good agent, if it's also really easy to see what it does, well, one thing has to give.
00:37:36.000 --> 00:37:48.239
Yeah, it's one of these things in I think in hindsight, it makes so much sense, but we've we've spent so long like trying to find and push for something that could be represented in a great sort of graphical user interface world.
00:37:48.400 --> 00:37:59.840
But I think if you if if you do take a step back, you realize that like how many powerful customer-facing products are built based on like drag and drop, right?
00:38:00.239 --> 00:38:12.559
And previously it was the auto attendant in the telephony world, and sure, you could trust a drag and drop interface to manage like press one for this, press two for that type of experiences.
00:38:12.719 --> 00:38:28.960
But if you wanted a fully intelligent representation of your brand speaking with your customers, I think, yeah, it it I think it does make sense that the the visual, like the platform is for uh well the the GUI is for visualizing to explain to someone what has been built.
00:38:29.119 --> 00:38:40.800
But of course, a yeah, developer is gonna want really a lot of control over the complexity that needs to sort of live to to create that experience writing code.
00:38:41.119 --> 00:38:41.440
Yeah.
00:38:41.599 --> 00:38:48.800
Yeah, I think yeah, that on the one hand, the physical impossibility of having both uh expressiveness and uh simplicity.
00:38:48.960 --> 00:38:53.039
And on the other, I think it's just like developers are developers, they don't want to drag and drop.
00:38:53.280 --> 00:38:55.679
And that just, you know, means that they'll find ways.
00:38:55.840 --> 00:38:59.039
And especially today, the pendulum has swung the other way, right?
00:38:59.199 --> 00:39:03.920
Because with these tools, like you're just able to produce so much in code.
00:39:04.079 --> 00:39:07.119
And that's because they're better at coding now than they are at say computer use.
00:39:07.280 --> 00:39:11.519
In theory, they could like drag and drop with any platform and create a thing of arbitrary complexity.
00:39:11.599 --> 00:39:14.639
Maybe they'd use even more tokens, but they don't work as well.
00:39:14.800 --> 00:39:16.800
And clearly it's suboptimal as well, right?
00:39:16.880 --> 00:39:18.000
So yeah, I'm excited.
00:39:18.079 --> 00:39:21.280
I'm excited to see, yeah, now what in this next wave now?
00:39:21.360 --> 00:39:27.440
I feel like, yeah, there's we're in for a real acceleration in our industry, and so I'm excited to see where that where that takes us.
00:39:27.760 --> 00:39:28.239
Absolutely.
00:39:28.639 --> 00:39:30.559
Well, Michael, thank you for joining me today.
00:39:30.639 --> 00:39:32.639
This was a very fun episode.
00:39:32.800 --> 00:39:37.920
And to everyone watching, please like, share, subscribe, and we will see you in the next one.