अनुलिपि 🔗
WEBVTT
00:00:00.160 --> 00:00:05.919
As a race, we are adapting towards how to speak to AIs, but we're trying to meet people where they are right now.
00:00:06.080 --> 00:00:11.119
That's the great thing about working here for me is that we have so much data that we can train on.
00:00:11.279 --> 00:00:16.960
And it's all anonymized and everything, but it ha how people talk over the phone.
00:00:17.199 --> 00:00:26.719
And the model knows that this is uh speech and uh it might be interrupted or it might need to recover from bad interruptions and stuff like that.
00:00:26.800 --> 00:00:27.679
We just train that in.
00:00:27.839 --> 00:00:32.159
The general models are not so specializing towards speech yet.
00:00:42.159 --> 00:00:46.399
Hi everyone, and welcome to another episode of Deep Learning with Poly AI.
00:00:46.799 --> 00:00:49.200
Today with me is our VP of Research, Matt Henderson.
00:00:49.520 --> 00:00:54.880
Um, and uh we're here to talk about our uh new model, our new LLM, Raven 3.5.
00:00:55.039 --> 00:00:56.479
Matt, welcome to the podcast.
00:00:56.640 --> 00:01:00.479
Before we start, everyone, please like, share, subscribe.
00:01:00.640 --> 00:01:02.719
Um, and uh Matt, over to you.
00:01:02.960 --> 00:01:04.640
Yeah, hey, thanks for having me on again.
00:01:04.799 --> 00:01:06.959
I enjoyed uh uh my first appearance.
00:01:07.040 --> 00:01:08.159
This is my second time on.
00:01:08.319 --> 00:01:11.120
Yeah, we're talking about things 3.5 today, right?
00:01:11.359 --> 00:01:12.079
Yeah, absolutely.
00:01:12.239 --> 00:01:21.680
I mean, like, you know, I think we are very proud of you know our continued uh investment in RD and just like how much, you know, we are a research-led company.
00:01:21.840 --> 00:01:27.599
Um, I think your PhD was the first of all uh of the other people of the company still.
00:01:27.760 --> 00:01:37.120
Uh and I think that you know, we've had a thread from well, 2012 to today of just working on this stuff, right?
00:01:37.519 --> 00:01:43.040
Um and um I think it'd be great to just tell the world like why is this special?
00:01:43.200 --> 00:01:44.000
Why is it different?
00:01:44.159 --> 00:01:45.120
Why is this not easy?
00:01:45.200 --> 00:01:46.319
What's special about it?
00:01:46.480 --> 00:01:47.920
I think there's more confusion than ever.
00:01:48.000 --> 00:01:55.760
And I think a lot of people have just decided that they're gonna either pick up the model from like OpenAI or Anthropic or you know, Mistral or Quen.
00:01:56.000 --> 00:01:57.280
Um, why are we doing this?
00:01:57.519 --> 00:02:00.879
Yeah, so I guess this is a question, why train your own model, right?
00:02:00.959 --> 00:02:09.199
Like why go, why specialize and as you mentioned, competing models would be the generalist, the big um models on the cloud.
00:02:09.360 --> 00:02:12.800
Um, of course, a big reason for us is speed.
00:02:13.039 --> 00:02:18.800
We target and achieve a sort of sub 300 milliseconds uh response uh time.
00:02:19.039 --> 00:02:23.360
And uh that just puts us like on the edge of optimizing.
00:02:23.599 --> 00:02:32.960
Um and I think at least for now, specialized models are always going to be a generalist, especially when you're optimizing against so many things.
00:02:33.120 --> 00:02:33.280
Yeah.
00:02:33.439 --> 00:02:36.719
So we're optimizing for latency cost.
00:02:37.039 --> 00:02:38.240
Those are easy, right?
00:02:38.479 --> 00:02:43.280
But you also have to optimize for uh instruction following, then it becomes hard.
00:02:43.360 --> 00:02:54.319
And we can really maybe double-click on that for like the, you know, I think we've got like two clusters of audience, one being technical and they'll know what you mean, and the others who um might not know what that means.
00:02:54.400 --> 00:02:58.560
I think there's like the whole like, how do I make a system not hallucinate, right?
00:02:58.800 --> 00:03:03.520
Uh piece, and you know, like what what are we doing to that end, right?
00:03:03.840 --> 00:03:16.240
Yeah, well, we train Raven to do things that are quite specific and specialized for the use case of you know customer support over the phone or over web chat.
00:03:16.719 --> 00:03:24.719
One of those things is being grounded in the prompt or being grounded in the documents that it has access to.
00:03:25.280 --> 00:03:38.639
So it's specifically trained to respond to you and then to cite its the source of information for uh its uh what it said, and optionally say, well, actually this was an out-of-domain request.
00:03:38.879 --> 00:03:51.919
So you get that like built-in LLM powered uh flagging of what topics are being cited, what pieces of content and documents are being cited, and what are people asking that the system just can't answer yet?
00:03:52.400 --> 00:03:53.280
Super useful.
00:03:53.520 --> 00:04:02.400
Not just for keeping the model from hallucinating, that helps as a sort of additional training signal, but also for your logging and just tracking all that stuff.
00:04:02.639 --> 00:04:09.759
Um you wouldn't get that in a general model without sort of extra harnessing and framework around like prompting it.
00:04:10.000 --> 00:04:17.199
You just got something that's built into the model, you don't need to prompt for it specially, and you sort of you get it for free.
00:04:17.439 --> 00:04:17.600
Yeah.
00:04:17.680 --> 00:04:23.680
And I mean, I think when you look at like, you know, people increasingly are playing with these models and with a genetic systems.
00:04:23.839 --> 00:04:29.040
And you know, it does feel like you know, in the matrix, when you see the machine just building a more and more complicated thing, right?
00:04:29.279 --> 00:04:41.680
It if you're using a general model and prompting your way around both stylistic things and then like the right tool calling, it ends up being quite, well, frankly, impossible to read what you've done or to know what you've done.
00:04:41.839 --> 00:04:50.319
And
Load full transcript
then it's kind of like just another layer of paint, just kind of like saying, hey, please don't do this, or in this case do this, and in this case do that, and it starts getting contradictory.
00:04:50.480 --> 00:04:59.680
So think like a model that just inherently doesn't force you to do that, kind of like layers of paint gets you to a point where you build a system that's much better and less complex.
00:04:59.839 --> 00:05:02.720
And less complex is always good because it means that it's more interpretable, right?
00:05:02.959 --> 00:05:10.959
Yeah, so we basically put all of these kind of constraints that we know ahead of time into the model weights rather than into this into a dynamic prompt.
00:05:11.120 --> 00:05:16.079
So you know, another example would be sort of consistency with what language it speaks.
00:05:16.720 --> 00:05:21.680
So we want Raven to be instructable in whatever language you're comfortable with.
00:05:21.839 --> 00:05:23.600
Yeah, that's typically English.
00:05:23.759 --> 00:05:30.079
And then with a flick of a flip of a switch, then it it speaks in Spanish or Portuguese or Japanese or whatever.
00:05:30.399 --> 00:05:32.879
That's super confusing for the general models.
00:05:33.199 --> 00:05:49.920
Because you know they you ask them uh the user asks them a question in Spanish and it retrieves a document that's in English, all the all the prompting is in English, and it says, you know, there's a maybe a reminder respond using this document or something, and then it's forgotten it's supposed to speak Spanish.
00:05:50.160 --> 00:06:00.319
And uh it just replied in English, it's a super bad error to make as one of those things that we can just put into our reward signal and and just be tighter the model.
00:06:00.639 --> 00:06:01.199
Yeah, yeah, no.
00:06:01.279 --> 00:06:06.240
I mean, I think and I think like when when you look at specialization, you kind of like you are what you eat, right?
00:06:06.319 --> 00:06:13.600
Where like if you're working on problems that require this level of precision and like multilinguality, like you kind of have to like get way better at it.
00:06:13.680 --> 00:06:16.480
And tell us a bit more about like the 3.5 aspect.
00:06:16.560 --> 00:06:19.040
Like what's special relative to to Raven 3?
00:06:19.199 --> 00:06:21.360
Like auto-reasoning would be one of one of the things, right?
00:06:21.600 --> 00:06:25.199
Yeah, autoreasoning is is the is the coolest new feature, I think.
00:06:25.439 --> 00:06:32.160
Um, apart from well, maybe we'll dive into that in a bit, but I mean across the board it's just sort of improvements on everything.
00:06:32.399 --> 00:06:40.000
It's we have uh we start from a better pre-trained model that's that has uh better multilingual capabilities.
00:06:40.160 --> 00:06:45.120
So we're just a big lift in in non-English languages as well as English.
00:06:45.519 --> 00:06:48.000
We have these new features like out-of-domain detection.
00:06:48.240 --> 00:06:55.199
We work in web chat very well now, a few improvements in things like custom style following.
00:06:55.360 --> 00:07:06.879
So one of the other things we sort of build into Raven is a good output style for conversations that are happening via voice or no, optionally web chat, right?
00:07:07.199 --> 00:07:21.360
Um, there's a lot of kind of LLM Ease style outputs that the models like GPT will you'll start to recognize and get really frustrated with if you're a caller, you know, been transferred three times and you get through to this.
00:07:21.519 --> 00:07:25.199
Like, I'd be really happy to drill down and get to the bottom of this for you.
00:07:25.519 --> 00:07:27.600
And then ask a question at the end, right?
00:07:27.759 --> 00:07:37.439
I mean, like I think I think what what's like one thing that I've been like uh trying to explain to a lot of people is latency is an obvious kind of like property of a system that makes it better, right?
00:07:37.519 --> 00:07:44.560
If it like spends less time thinking and responds quickly without interrupting you when you've just made a pause, that's obviously a great thing, right?
00:07:44.800 --> 00:08:02.000
I think like the bargin feature is one that is quite interesting, where you know, obviously it's a great thing to show off, and in some places it's incredible, in others it can be quite disruptive if people accidentally you know interrupt the thing or another person speaking in the background does something.
00:08:02.399 --> 00:08:26.160
It is essential for a lot of the rappers to have good margin because their systems built on GPT models just won't shut up, and they're kind of building this like UX assumption of like to anyone who's used the voice mode in Chat GPT, it becomes quite obvious that you have to be very authoritative and frankly, in human terms, quite rude when you speak to it and that you just have to cut it off and direct it where you want it to go.
00:08:26.319 --> 00:08:31.439
And if you do that, it's a bit of a different modality of conversation, but it is very powerful.
00:08:31.759 --> 00:08:43.840
But we what we see on the phone often is that people don't know how to do that, and there's this like discrepancy between, you know, how the power users of voice mode and chat GPT, which are probably not more than 0.1% of the planet, right?
00:08:44.080 --> 00:08:51.759
Uh, behave with it versus everyone else who's just kind of like following the like rules of behavior from a human conversation.
00:08:52.000 --> 00:08:57.279
A lot of it is just like you don't want it to like speak forever and then ask you a question at the end of the whole thing.
00:08:57.600 --> 00:09:03.519
As a race, we are adapting towards how to speak to AIs, but we're trying to meet people where they are right now.
00:09:03.679 --> 00:09:10.000
And um I guess we kind of get that a bunch because of all our training is on you know the data that we collect.
00:09:10.159 --> 00:09:23.679
That's the great
thing about working here for me, is that we have so much data that we can train on, and it's all um you know anonymized and um uh everything, but it has it how people talk over the phone.
00:09:23.919 --> 00:09:33.440
And the model knows that this is uh speech and uh it might be interrupted or it might need to recover from bad interruptions and stuff like that.
00:09:33.519 --> 00:09:34.639
We just train that in.
00:09:34.799 --> 00:09:35.039
Yeah.
00:09:35.440 --> 00:09:40.000
General the general models are not so specialized in towards speech yet.
00:09:40.240 --> 00:09:41.440
I mean, yeah.
00:09:41.600 --> 00:09:48.159
So GPT real time is like a very good model that we compete with and and benchmark against.
00:09:48.480 --> 00:09:50.159
I mean, like I've heard a lot of customers.
00:09:50.240 --> 00:09:54.480
In fact, it's generating a lot of demand where people speak to it and they're like, I want that, right?
00:09:54.559 --> 00:09:57.840
And then it's like, well, you know, that's a certain kind of car, right?
00:09:58.000 --> 00:09:59.679
It's it might be a hypercar.
00:10:00.159 --> 00:10:02.399
Like it's equally difficult, you know.
00:10:02.559 --> 00:10:17.440
You can't really drive it around London because you'll hit a curb wherever you go, or in the case of our model, it will hallucinate things and you know, won't do like tool calling, but it's definitely showing us what's possible in terms of speed and the naturalness of dialogue, right?
00:10:17.600 --> 00:10:18.799
I mean, I find it very impressive.
00:10:18.879 --> 00:10:23.440
We don't want to talk about Raven 4 here, but I think so excited about that one, right?
00:10:23.600 --> 00:10:27.679
Yeah, Raven 4, Raven Omni, we're starting to do audio.
00:10:27.840 --> 00:10:40.159
Yeah, so we're we um don't need speech recognition anymore, and we can sort of hear the user, a native understanding of the speech or the audio, but yeah, I guess that's the next podcast I'll appear on.
00:10:40.399 --> 00:10:42.159
Maybe we maybe stop on that thread.
00:10:42.399 --> 00:10:44.799
Tell me a bit about you know, just like for the audience, right?
00:10:45.039 --> 00:10:57.279
I mean, you know, I remember inheriting your code base, and you know, it was the wonderful library of Tiano, and you couldn't even do like, you know, TensorFlow with its optimizers was revolutionary in terms of how easy it was to like run an experiment.
00:10:57.440 --> 00:11:09.759
Like, why is this still hard when people, you know, live in a world where like a come you know, open claw can set up like an entire operating system and create like development pipelines that you needed a DevOps team for before?
00:11:10.000 --> 00:11:14.000
My latest nation, but like um, why is this still hard?
00:11:14.320 --> 00:11:17.600
I think yeah, we're try we're training this generative model, right?
00:11:17.679 --> 00:11:19.759
That that can that you can speak to.
00:11:19.919 --> 00:11:28.480
And then we can train and and and get
a loss and think, okay, this looks fine, even run it on a benchmark and and get some average number.
00:11:28.639 --> 00:11:39.600
But it really takes us to start talking to the model, getting in front of our um agent deployment teams or in in in front of people to see what are the issues.
00:11:39.759 --> 00:11:41.679
You know, is it calling tools robustly?
00:11:41.759 --> 00:11:43.360
Does it have some sort of weird style?
00:11:43.519 --> 00:11:46.720
When you start talking to a multi-turn conversation, what shows up?
00:11:46.879 --> 00:11:53.360
Maybe it's repetitive, it says great after every call, uh, after uh you know everything the user says.
00:11:53.600 --> 00:12:02.320
Um we it's just the kinds of things that you only come across if you've been building conversational systems for like the length of time that that you and I have.
00:12:02.639 --> 00:12:06.720
We have a list of behaviors we want to put into the model and then we test for.
00:12:06.960 --> 00:12:17.440
And everyone has every one of those behaviors might have a sort of individual creative solution for like more data, more preferences, a different reward signal, something like that.
00:12:17.840 --> 00:12:21.200
When you talk about those kind of like different metrics that you're optimizing for, right?
00:12:21.360 --> 00:12:25.840
Like I guess we talked about style, we talked about kind of like instruction following.
00:12:26.080 --> 00:12:36.240
Then there's the whole like balancing, reasoning, and latency, and how you decide to do that, and then the naturalness of voice versus like the precision of answer and like the length of the answer.
00:12:36.399 --> 00:12:38.559
How do you like optimize for all that in training?
00:12:38.720 --> 00:12:42.240
And like post-training setup, like how does it work, right?
00:12:42.399 --> 00:12:43.759
And I'll add to your list as well.
00:12:43.840 --> 00:12:51.360
So there's also like uh tool calling, and then there's all of those, but in all the different languages and modalities, right?
00:12:51.519 --> 00:12:54.480
Yeah, and different modalities and what some of web chat.
00:12:54.639 --> 00:13:00.960
Um, we also want to check that those things like about hallucination and citing your topics and stuff work well.
00:13:01.200 --> 00:13:06.000
Uh languages is particularly interesting because we would have a whole different style guide for each language.
00:13:06.240 --> 00:13:13.200
You know, that you might want to have different rules about politeness when it comes to an English conversation versus a Japanese conversation, might be more formal.
00:13:13.360 --> 00:13:18.960
And in some languages, you can't sort of avoid but gender the person you're talking to to the pronouns you're using.
00:13:19.039 --> 00:13:22.240
And so we have certain kind of uh rules there.
00:13:22.480 --> 00:13:28.399
So the by the way, what the the question was how do we uh balance all of those things?
00:13:28.720 --> 00:13:38.799
We track all of them in all of our evaluations, uh, and I guess making them measurable is the is the first important thing we do on the research team.
00:13:39.039 --> 00:13:52.639
So we have this internal benchmark, and you'll see that in our uh blog post announcing Raven 2.5 that we we beat um the big public nodals, G5, uh the latest cloud sonnets on those benchmarks.
00:13:52.879 --> 00:13:57.600
And then each has an individual um uh solution for making it work.
00:13:57.840 --> 00:14:11.200
And I guess I want to sort of paint the picture of this isn't just launching a you know run DPO script, getting a model at the end and saying there we train 2.5, it has more data, blah, blah, blah.
00:14:11.519 --> 00:14:15.759
If you were to sort of plot the lineage of the final model checkpoint, right?
00:14:15.840 --> 00:14:30.320
It's it's the average of a bunch of runs of the reasoning tuning stage, which comes from a base model of an average of all these different one-off DPO and GRPO and combined GR GPO, DR, DPO uh runs.
00:14:30.720 --> 00:14:38.480
Um it's become this kind of like messy law like art of figuring out how to post-train for all of those things at the same time.
00:14:39.360 --> 00:14:43.759
The more we talk about the data being like a moat, and it absolutely is a moat.
00:14:44.000 --> 00:14:59.679
Like, you know, one way that Sean, our um uh our CTO described it, and that I thought was really interesting, was like you're basically kind of like just applying generations of different it's like you go to different grades as a model, right?
00:14:59.840 --> 00:15:09.279
And like how you learn chemistry in grade five could very well inform whether you go for like physics or chemistry in like grade seven, right?
00:15:09.600 --> 00:15:28.480
And then like your whole learning path takes a different like evolution, where really like what you did at what part of the training and how like you know you got to rewrite like the the rewards and stuff, kind of like it's not really a linear like training phase in the way that people used to think about it, right?
00:15:28.559 --> 00:15:35.840
But it's really like a multi-stage like building of a system where it's more of a dark art than ever, in a way, right?
00:15:36.080 --> 00:15:49.840
It's pretty sort of surgical and targeted in that like yeah, before you and I are doing our PhDs, we usually you know launch an experiment that trains on this data set and it finishes and that you know it was randomly initialized usually.
00:15:50.000 --> 00:15:56.320
I think you did work on bringing in pre-trained word embeddings and stuff, which is first steps to you know where we are now.
00:15:56.559 --> 00:15:59.840
But now we have these massive models, lots of data.
00:16:00.000 --> 00:16:04.000
We want to sometimes we want to reuse this old model, it's it's really good.
00:16:04.080 --> 00:16:07.519
It's just not quite in this certain situation, it doesn't use the right tool.
00:16:07.679 --> 00:16:09.919
So then we we we fix it
surgically.
00:16:10.240 --> 00:16:13.679
You know, I think about this a lot when people are like, what are we going to be doing in the future?
00:16:13.759 --> 00:16:25.200
And you know, if we were explaining what we're doing, even to like a mathematician in like the 30s or 40s, they'd probably think we lost our minds, or that we're doing something very trivial and that it makes no sense, right?
00:16:25.600 --> 00:16:27.919
Uh I mean they'd understand everything.
00:16:28.000 --> 00:16:42.799
It's just like, you know, you think of like people building like handcrafted products, you know, like leather goods or furniture or whatever, and you kind of see them like you know, polishing this one thing, shaping this other thing, leaving for a few days.
00:16:42.960 --> 00:16:59.679
And you know, I can't help but like draw the comparison where it's really like it's really not that different to this, where your intuition is like the meta-level kind of like sequencer, and like it informs like how much even money and resources and time you want to invest in this like whole like sausage making thing.
00:16:59.759 --> 00:17:03.759
And yeah, we're sort of artisanal model developers.
00:17:04.319 --> 00:17:22.799
Um and we become you know reward function engineers and um you know GRPO loss mixing engineers, but at least we're not prompt engineers, so stuck with a generalist model, and all we have at our disposal is playing around with a with a prompt.
00:17:23.359 --> 00:17:30.000
We we we can backpropagate and uh adjust uh model to exactly what we want.
00:17:30.319 --> 00:17:34.960
The auto-reasoning stuff was like surprisingly difficult to train, I guess.
00:17:35.119 --> 00:17:41.440
And uh when it comes to reward engineering, you know, the model's always going to try to um reward heck.
00:17:42.160 --> 00:17:50.480
And uh well, so with auto-reasoning, what that means is you're you want to get the benefits of reasoning, which means that the model takes some time before responding, right?
00:17:50.559 --> 00:17:53.440
So it does some little generation and thinks thinks to itself.
00:17:53.759 --> 00:18:02.240
But in our case, we don't that would that would that adds latency necess necessarily to the to the call because it's doing its own, it could in theory respond immediately.
00:18:02.319 --> 00:18:05.440
So we only want to do that in cases where it's gonna help.
00:18:05.680 --> 00:18:09.759
Um so how do we teach the model when it should reason when it shouldn't?
00:18:10.000 --> 00:18:13.680
So the naive thing to do is that you kind of just generate with and without reasoning.
00:18:13.759 --> 00:18:19.519
And if the if the one where it reasons is better, then but that then you train like that.
00:18:19.920 --> 00:18:24.559
How do you factor like how long it reasons for into the balancing?
00:18:24.880 --> 00:18:26.480
Yeah, that's the that's the other tricky.
00:18:26.799 --> 00:18:38.799
That's one of our GRPU losses, is making the reasoning short, because all of these um well, um the typical sort of base model that you would get doesn't have any kind of constraint like that.
00:18:38.880 --> 00:18:52.640
And sort of original, if we look back to when reasoning first came in with Deep Seek, so we have some crazy long tra repetitive traces, and you really benchmark performance go way up, but sure, you have to wait like five minutes to for it to actually reply to you.
00:18:52.880 --> 00:18:55.519
So we do that, we do that in this artisanal way, right?
00:18:55.599 --> 00:19:00.079
So we warm it up in a in SFT stage with some traces that we think are good.
00:19:00.319 --> 00:19:11.599
And then when it comes to later stages, we've got specific loss function to say if you can get to the same quality of response with shorter reasoning, that's you know, prefer the shorter reasoning reward.
00:19:12.079 --> 00:19:17.680
But I think one of the core difficulties here is we're trying to teach the model to know what it doesn't know, right?
00:19:17.839 --> 00:19:23.279
So the these models are sort of famous for being confident bullshitters sometimes, right?
00:19:24.880 --> 00:19:27.200
So they're they're not well calibrated inside.
00:19:27.359 --> 00:19:31.200
So how can we teach it to know when it needs to stop and think?
00:19:31.599 --> 00:19:33.839
Uh so that was particularly hard.
00:19:34.160 --> 00:19:38.880
With this like shortening the reasoning thing, those sort of degenerate reward hacking cases.
00:19:39.119 --> 00:19:43.839
Well, I seem to be doing better and better every time I reason less, so I'm just never going to reason.
00:19:44.240 --> 00:19:47.119
So there's these all these things we're bouncing.
00:19:47.279 --> 00:19:54.079
And you know, the trick with when to reason, well, we just warmed it up with some SFT, basically.
00:19:54.240 --> 00:19:56.880
Some good you should reason here.
00:19:56.960 --> 00:20:06.559
When it comes to this, we know this is a very difficult, you know, date time request rather than asking the model to figure out that it can be unreliable without reasoning in those cases.
00:20:06.640 --> 00:20:15.839
We just like a s of adding a bit of structure that helps it like make that decision or learn to make that decision rather than letting it just infer from like data.
00:20:16.160 --> 00:20:16.480
That's right.
00:20:16.559 --> 00:20:24.480
So the beginning of training, it's it has something to latch on to, and then it and then it can kind of like figure out the other things that where reasoning helps.
00:20:24.640 --> 00:20:31.200
It's not obvious to why should it be obvious to the model that needs to reason to figure out how many R's are in the word Raspberry?
00:20:31.440 --> 00:20:31.680
Yeah.
00:20:31.920 --> 00:20:50.079
That doesn't sound like Yeah, because it uh the like the the overall like um signal, whether it was good or not, is one thing, but then like predicting the next word versus predicting it faster is quite a and you know how much faster
it becomes quite a it really is some kind of like racing, right?
00:20:50.559 --> 00:21:02.400
Yeah, but is it but is it you know this is a nice feature, so it's kind of if you want just switch on auto-reasoning, Raven will think when it uh when it thinks it should help and it wouldn't add uh latency.
00:21:02.799 --> 00:21:10.640
How much better would would like one do in these benchmarks if it reasoned, if time wasn't an issue, and you know, like we weren't optimizing?
00:21:10.720 --> 00:21:12.640
Like, how much better are reasoning models?
00:21:12.720 --> 00:21:28.559
You know, when we compare to like you know, Sonic and stuff, if we took Opus, for instance, because I can't detach myself from my like you know uh OpenClaw and what it does with the Opus model, I see a very distinct difference in like coding ability with one versus the other.
00:21:28.799 --> 00:21:31.359
How well would it do like relative to these models?
00:21:31.519 --> 00:21:32.960
Or would it not make much of a difference?
00:21:33.279 --> 00:21:37.200
Those are going to do very well in our benchmarks.
00:21:37.519 --> 00:21:47.039
Uh large reasoning models where latency is not a doesn't matter, then you can you can certainly top all of our our benchmarks.
00:21:47.119 --> 00:21:53.359
So we want to guard a bit against the feature where they do become uh instantaneous uh and fast.
00:21:53.599 --> 00:21:59.759
And you know, when when you talk about using your your claw or clawed butt or whatever, your open claw.
00:22:00.400 --> 00:22:06.880
There's a whole lot of like agent harnessing and stuff that sort of makes it respond very well but adds on latency.
00:22:07.440 --> 00:22:07.759
Yeah.
00:22:08.319 --> 00:22:20.079
So yeah, they you know those and I guess our our approach is to benefit from these types of models, our open models where we can run reasoning and do stuff that's inefficient.
00:22:20.240 --> 00:22:25.279
We used to do that offline and during training and sort of distill it down into these uh smaller models.
00:22:25.599 --> 00:22:25.759
Yeah.
00:22:25.920 --> 00:22:30.640
Because you could think of it as just like we're getting it ready for a race, it has to run a race.
00:22:30.880 --> 00:22:36.559
And at the end of the day, like are we able to get most of the benefits in a faster model then?
00:22:36.640 --> 00:22:40.000
Of you know how much better a bigger model would do?
00:22:40.319 --> 00:22:55.680
Yeah, I think we basically we we can uh get we benefit from all that all that teaching and achieve something which you just uh you you wouldn't be able to do with the latency budget and uh obviously it much, much, much cheaper.
00:22:56.079 --> 00:22:57.039
Okay, cool.
00:22:57.279 --> 00:23:08.000
Well, um I think like with that, this was really just a teaser for what's gonna come in a few weeks' time with the release of the next model, but I think we've probably already said a bit too much about that.
00:23:08.240 --> 00:23:12.079
Thank you so much for joining to everyone in to everyone listening.
00:23:12.160 --> 00:23:14.640
Like, check out the link, check out Raven 3.5.
00:23:14.880 --> 00:23:20.720
Um, there are very exciting releases coming about the platform and all the models included with it over the coming few weeks.
00:23:20.960 --> 00:23:25.519
So, you know, subscribe, like, share, and we'll see you in the next one.
00:23:25.680 --> 00:23:27.680
Matt, thank you for for joining me today.
00:23:27.920 --> 00:23:29.119
Great, thanks for having me on.
00:23:29.279 --> 00:23:30.480
Great to chat to you, Nicola.