WEBVTT
00:00:00.239 --> 00:00:07.919
You know, I think a year ago, uh, there's a point where if you asked Gemini how to hold the cheese on your pizza, it would say use glue.
00:00:08.160 --> 00:00:09.039
Technically accurate.
00:00:09.279 --> 00:00:13.359
Technically accurate, but would not result in a good meal, right?
00:00:13.519 --> 00:00:22.879
And and the good news is that there's a person between what Gemini said to do and actually producing the pizza, and they go, ha ha, that's silly.
00:00:23.039 --> 00:00:24.320
Let you know, try again.
00:00:24.640 --> 00:00:29.600
When the trucks move down the freeway at 70 miles an hour, we just don't have the ability to let that happen.
00:00:30.079 --> 00:00:31.600
This is unsupervised.
00:00:31.679 --> 00:00:33.920
I'm Dave Ferguson, co-founder of Neuro.
00:00:34.320 --> 00:00:39.359
Join me as we turn over the big questions in autonomy and physical AI.
00:00:40.640 --> 00:00:42.240
Chris and I have known each other.
00:00:42.399 --> 00:00:48.320
I was figuring this out since 2002 when I came to visit Carnegie Mellon and I crashed on your couch.
00:00:48.560 --> 00:00:49.759
I was just thinking about that.
00:00:50.079 --> 00:00:50.799
Yeah, it's been a while.
00:00:51.039 --> 00:00:53.200
You had to burn it afterwards, probably.
00:00:53.840 --> 00:00:54.799
No comment.
00:00:55.759 --> 00:00:59.359
But that weekend literally changed the trajectory of my life.
00:00:59.439 --> 00:01:01.759
You know, I decided to join CMU.
00:01:01.920 --> 00:01:05.599
The two of us then worked together on a bunch of projects.
00:01:05.840 --> 00:01:08.480
Uh the DARPA DARPA Urban Challenge that you led.
00:01:08.719 --> 00:01:12.719
And then later you pulled me to Google when you led the Google self-driving car.
00:01:13.040 --> 00:01:14.400
Glutton Punishment that you are.
00:01:14.480 --> 00:01:14.560
Yeah.
00:01:14.879 --> 00:01:16.239
Before it became Waymo.
00:01:16.480 --> 00:01:23.519
Uh and now for the last nine years, uh, you've you've founded and have been leading Aurora working on A V trucking.
00:01:23.680 --> 00:01:27.439
So, Chris, it's it's a huge privilege uh and treat to have you here today.
00:01:27.599 --> 00:01:28.239
Thank you so much.
00:01:28.560 --> 00:01:29.439
Thanks for having me on.
00:01:29.599 --> 00:01:35.200
Uh, it's always great to hang out with you, even you know, if we've got microphones instead of beers, but you know, it'll work.
00:01:35.359 --> 00:01:36.799
It's been a hell of a ride.
00:01:36.879 --> 00:01:43.359
You know, I remember us out and uh at water testing for the first uh for the urban challenge.
00:01:43.599 --> 00:01:52.400
Yeah, we're in we're in that old uh aircraft hangar, and I remember in the side there was like uh an old rusty pull-up bar, which was where we're getting our exercise.
00:01:52.560 --> 00:01:53.760
I felt like you were getting your exercise.
00:01:54.000 --> 00:01:57.359
I felt like it was the guy from Footloose, you know, like in the background.
00:01:57.920 --> 00:02:02.879
Trying to avoid um catching tetanus and winning the diaper urban channel.
00:02:03.680 --> 00:02:05.359
Yeah, so it was some good times.
00:02:05.760 --> 00:02:07.439
Tell us a little bit about trucking.
00:02:07.599 --> 00:02:08.879
Why AV trucking?
00:02:08.960 --> 00:02:11.039
You know, you've worked so long in this field.
00:02:11.120 --> 00:02:13.840
Uh, why is this the right application to be pursuing?
00:02:14.319 --> 00:02:18.960
We've been building Aurora for going on 10 years now, and we had this vision.
00:02:19.039 --> 00:02:21.520
We kind of understood that trucking would be a really interesting application.
00:02:21.599 --> 00:02:22.319
And why?
00:02:22.560 --> 00:02:24.240
Because it's it's really hard.
00:02:24.479 --> 00:02:27.199
Anyone that you know is a truck driver, you should thank them.
00:02:27.439 --> 00:02:30.479
But it's absolutely essential that goods move on trucks.
00:02:30.639 --> 00:02:36.240
If you look around the room we're in, look around any room, literally everything in that room at some point moved on a truck.
00:02:36.479 --> 00:02:46.479
So that combination of competence, technical features and capability, market need, economic opportunity, we're like, okay, this is where we should go put our energy.
00:02:46.719 --> 00:02:47.680
So where are you guys at?
00:02:48.000 --> 00:02:55.680
We are focused on building the core neurodriver, the foundational technology that can be a universal autonomy platform.
00:02:55.840 --> 00:02:58.639
And so for us, we used to work on these delivery vehicles.
00:02:58.800 --> 00:03:04.479
We thought it was very exciting, this entire vertical from manufacturing them, owning and operating them.
00:03:04.879 --> 00:03:09.360
And then we realized, well, we knew for a long time uh that it required a lot of capital.
00:03:09.520 --> 00:03:13.759
I think what happens is the market changed and getting that capital became very challenging.
00:03:13.919 --> 00:03:22.000
So we decided look, we should find a path that is more capital efficient for us to get to market and scale and realize a huge impact.
00:03:22.080 --> 00:03:28.639
And so what that meant for us was licensing the technology, working with partners to get it on as many different platforms as possible.
00:03:28.800 --> 00:03:35.599
So we have this big robotaxi push with Uber, and we have some other stuff in the works on the logistics side that we're very excited about.
00:03:35.759 --> 00:03:45.599
And really, it's about building that core AI AV technology and then trying to universally apply it to a ton of different platforms.
00:03:45.840 --> 00:04:02.879
And for us, it's just made a huge amount of sense that we work with partners and have them do what they are literally best in the world at doing, building cars, running rideshare uh services, and we get to focus on what we hope and believe that we are sort of best in class at being able to do, which is the A V side.
00:04:03.199 --> 00:04:05.919
Yeah, that that's like you said, that's been our approach from from day one.
00:04:06.000 --> 00:04:11.759
It's like we're good at this, we can be proud of that, but we should be humble about the things that we aren't, right?
00:04:11.919 --> 00:04:14.080
And make sure we go find great people to work with.
00:04:16.959 --> 00:04:32.160
There's a lot of buzz around uh and a lot of folks that like to talk about the nomenclature of A V 1.0, AV 2.0, end-to-end models, all of the foundational model technology we're seeing across the OpenAI, Anthropics, Gemini's of the world.
00:04:32.560 --> 00:04:35.439
How are you or not leveraging all of this technology?
00:04:35.519 --> 00:04:36.399
What what does that look like?
00:04:36.800 --> 00:04:39.759
There are places where algorithmic solutions work.
00:04:40.079 --> 00:04:40.240
Right.
00:04:40.399 --> 00:04:43.120
I take, for example, processing a GPS signal.
00:04:43.519 --> 00:04:47.920
We probably don't need to learn how to do correlation across the signal cover much of satellites.
00:04:48.079 --> 00:04:49.759
We can just write down the math and do it.
00:04:50.160 --> 00:04:52.240
And so we should do that where we can.
00:04:52.399 --> 00:04:59.519
But you can't deny the impact that machine learning and AI has on the technology we're developing and building.
00:04:59.680 --> 00:05:02.160
You know, the approach we've taken is what we call verifiable AI.
00:05:02.399 --> 00:05:15.040
And this is really about yes, we absolutely need to be using modern techniques, but we have to make sure that we are able to kind of understand how they perform and put constraints around them.
00:05:15.600 --> 00:05:21.920
And I I think about where the enthusiasm around end-to-end and large models come from.
00:05:22.000 --> 00:05:22.800
And it's obvious.
00:05:23.040 --> 00:05:25.519
If you'd asked me five years ago, would we be where we are today?
00:05:25.600 --> 00:05:27.439
The answer would have been I couldn't have seen it.
00:05:27.680 --> 00:05:28.160
Absolutely not, yeah.
00:05:28.319 --> 00:05:28.639
You know.
00:05:29.120 --> 00:05:31.439
And so you absolutely have to be paying attention to that.
00:05:31.519 --> 00:05:36.879
But you also have to understand how to use the technology in the domain that it's applied to.
00:05:37.120 --> 00:05:43.439
A year ago, there's a point where if you asked Gemini how to hold the cheese on your pizza, it would say use glue.
00:05:43.680 --> 00:05:44.399
Technically accurate.
00:05:44.720 --> 00:05:48.639
Technically accurate, but would not result in a good meal, right?
00:05:48.800 --> 00:05:58.160
And and the good news is that there's a person between what Gemini said to do and actually producing the pizza, and they go, ha ha, that's silly.
00:05:58.319 --> 00:05:59.439
Let you know, try again.
00:05:59.600 --> 00:06:04.079
When the trucks move down the freeway at 70 miles an hour, we just don't have the ability to let that happen, right?
00:06:04.160 --> 00:06:09.279
Because that's the equivalent of it deciding to jerk the wheel and turn off the freeway.
00:06:09.519 --> 00:06:13.839
We've thought about, okay, how can we take this kind of technique and apply it in our space?
00:06:14.000 --> 00:06:16.639
It's important to have defined interfaces.
00:06:16.879 --> 00:06:21.839
So we you know think of it broadly as two systems, seeing the world and then reacting to the world.
00:06:22.240 --> 00:06:28.399
Each of these is you know, kind of an exponentially hard problem, but very difficult, very complicated.
00:06:28.879 --> 00:06:40.000
But to try to make sure we can understand how the systems worked, we've actually created interfaces that say this is how we express what a person is, what a car is, you know, what the various features of the world that we care about are.
00:06:40.399 --> 00:06:46.560
And then we basically decompose that model and separate it from the model that reacts to the world.
00:06:46.879 --> 00:06:49.920
The downside of this is that you're potentially throwing away information, right?
00:06:50.000 --> 00:07:00.560
If I were to take the the bull argument for end-to-end, it's that the model may find a more expressive way to describe a person, which would provide more subtle cues, which then you could react to.
00:07:00.720 --> 00:07:04.399
Whereas if I say this is how you describe a person, the model is only that expressive.
00:07:04.959 --> 00:07:13.360
The problem though is that we're not building a system that we want to hope works or that we want to kind of guess that it seems to work.
00:07:13.680 --> 00:07:15.759
We want a system where we know that it works.
00:07:16.079 --> 00:07:18.879
And to do that, we have to do verification or validation.
00:07:19.040 --> 00:07:21.600
We have to test it and we have to look at the tests and make sure they work.
00:07:22.079 --> 00:07:31.680
And if you take two exponential systems and you basically test given inputs here and outputs there, the complexity of testing those two systems is the product of two exponentials.
00:07:32.079 --> 00:07:43.439
Uh if we're able to decompose them into two separate systems and be able to kind of test them to the boundaries, then we now have two independent exponential systems.
00:07:43.680 --> 00:07:51.199
And validating those is very hard, but is literally exponentially easier than validating as if it was one system.
00:07:51.439 --> 00:07:58.240
And then the other part of the verifiable AI that we use is the way we kind of uh constrain the output that's possible.
00:07:58.480 --> 00:08:07.680
So in modern LLM applications, you have something that's proposing the sentence that comes out, and then you have a thing that's going through and ranking them.
00:08:07.839 --> 00:08:14.240
In our case, what we've done is we've taken the proposal part, and that is actually algorithmic and deterministic.
00:08:14.639 --> 00:08:22.560
And that means that when we're evaluating what actions to take, all of the actions we know are plausible, right?
00:08:22.800 --> 00:08:29.120
It can't suddenly decide to take an action that is kind of veering off the road because that isn't in the set of things that it's ranking.
00:08:29.199 --> 00:08:29.360
Trevor Burrus, Jr.
00:08:29.600 --> 00:08:32.559
The approach that NERO has taken, you know, it's all related.
00:08:32.639 --> 00:08:46.480
I think you've you've laid out exactly the challenges of trying to get the guaranteed verification of intermediate outputs versus trying to capture the full ceiling of potential by not having any information loss.
00:08:46.879 --> 00:08:54.639
The way we do it is we have the intermediate, we do have intermediate sort of task head outputs, not at the end of the net, sort of in the middle of it.
00:08:54.720 --> 00:09:00.000
And so you can think of it as a little bit like understanding the world and then reasoning about it.
00:09:00.240 --> 00:09:04.240
However, we also pass through a very high-dimensional embedding.
00:09:04.480 --> 00:09:05.919
So we sort of have both.
00:09:06.159 --> 00:09:22.879
But one of the additional safeguards that we get from doing that is that when we do our final, we call it motion selection stage, like the ranked list and checking that what's coming out of the net at the end is valid, we can use those intermediate outputs to do that validation.
00:09:23.200 --> 00:09:40.639
And so we have found that that combination has allowed us to do a really nice job of capturing some of the really difficult, uh nuanced behavior that you get from having the full embedding, like the person standing on the side of the road, and you know, it's a person with a stroller and a dog and a bike.
00:09:40.720 --> 00:09:44.879
And are you classifying the what are you classifying it as or what's the intent and so on?
00:09:45.120 --> 00:09:50.639
But like as a field, we've seen this incredible evolution in how all of this works.
00:09:50.799 --> 00:09:58.320
Like, I mean, what you're explaining around detecting what's out there in the world and then reasoning about it like at a high level, we could say that's what we did in 2007.
00:09:58.639 --> 00:09:58.879
Yeah, yeah.
00:09:59.200 --> 00:10:00.159
Um, but in reality.
00:10:00.720 --> 00:10:01.759
Just think act, maybe.
00:10:02.000 --> 00:10:02.559
That's right.
00:10:02.720 --> 00:10:07.200
And in reality, it's it's wildly different in terms of of the performance.
00:10:08.000 --> 00:10:09.519
It's it's night and day, right?
00:10:09.600 --> 00:10:12.720
That you know, we've been both been working at this for 20-something years at this point.
00:10:12.879 --> 00:10:15.039
And it was incredible back then.
00:10:15.120 --> 00:10:24.799
Like uh, you know, when we did the urban challenge, the idea that you could have a vehicle driving around and not bump into much stuff and interact with traffic, it was truly a grand challenge.
00:10:25.039 --> 00:10:29.919
Um but when you look back at it, it was kind of a toy problem.
00:10:30.240 --> 00:10:40.559
And as we get into what we're dealing with today, the biggest change is we've gone through this kind of prototype phase and we're now into truly industrializing production phase, right?
00:10:40.799 --> 00:10:48.799
This is one of the things that I think a lot of people don't really understand is that you know it probably took, I don't know, 10, 15 people to prototype the iPhone.
00:10:48.879 --> 00:10:53.039
And it literally takes thousands of people to kind of industrialize it and produce it.
00:10:53.200 --> 00:11:07.120
And it's pretty incredible to have had the opportunity to kind of see this industry move from the 15 of us doing the late night drives at Google or back in the desert and into now where you know this is really, it's it's real.
00:11:07.440 --> 00:11:08.399
And it is hard though.
00:11:08.559 --> 00:11:14.240
You mentioned this massive gulf between like a demo versus getting it to production.
00:11:14.320 --> 00:11:22.799
I remember in 2011 when when you enticed me to Google, which wasn't that hard, and I came to visit and I went for a ride with you and Dimitri in the process.
00:11:23.519 --> 00:11:23.840
Exactly.
00:11:24.320 --> 00:11:25.440
Is there anything for me to do even?
00:11:25.840 --> 00:11:26.080
That's right.
00:11:26.320 --> 00:11:28.639
I think my exact question was this this seems pretty good, Chris.
00:11:28.720 --> 00:11:30.080
Like, is there anything interesting left?
00:11:30.159 --> 00:11:32.000
Like, why do you want me to join?
00:11:32.240 --> 00:11:34.559
And that was what like 15, 15 years ago.
00:11:35.039 --> 00:11:35.919
Yeah, a little bit ago, yeah.
00:11:36.080 --> 00:11:38.799
And even back then, like we had incredible demos.
00:11:39.120 --> 00:11:39.440
Oh, yeah.
00:11:39.600 --> 00:11:39.840
Like back.
00:11:40.000 --> 00:11:44.240
You guys had an incredible demo in 2011 when I joined, like before I was even on the program.
00:11:44.480 --> 00:11:56.559
And I think that has been one of the really challenging parts of this entire industry, particularly from like a financing investor perspective, because investors will come and they'll get in a car and they'll this is the same as Waymo, right?
00:11:56.639 --> 00:11:56.720
Right?
00:11:56.879 --> 00:12:18.320
Like this, this, this feels just just like you know, a great demo, like FSD, like all of these, all of these uh examples out there of what seems very compelling, yeah, but there can still be this enormous gap between that very compelling demo and actually getting it to a place where we can launch and scale and save lives and make an impact.
00:12:18.639 --> 00:12:23.440
I used to joke, you know, you give me three graduate students in six months and I'll give you a self-driving car.
00:12:23.679 --> 00:12:25.360
You don't necessarily want to trust your life to it.
00:12:25.519 --> 00:12:29.840
And I think now, you know, you give me uh a summer and two interns and I'll give you a self-driving car.
00:12:30.159 --> 00:12:34.480
When you talk about a safety critical system, you have to be right.
00:12:35.120 --> 00:12:50.000
Yeah, we often say in Neuro that we see the tech and the investment and the IP behind our safety case and the validation that we do to put these vehicles on roads as as valuable as the core sort of foundational um autonomy tech.
00:12:50.240 --> 00:12:52.399
And that that tends to blow people's minds a little bit.
00:12:52.879 --> 00:12:55.279
Because that is not true in almost any other space.
00:12:55.519 --> 00:13:08.480
Like the safety criticality of of the space that we're in is just it's completely different from you know, you mentioned the standard sort of LLM example of incredible technology, but it's okay if it tells you to put glue on your pizza once in a while.
00:13:08.720 --> 00:13:08.879
Yeah.
00:13:09.600 --> 00:13:10.799
So how do we solve that?
00:13:10.960 --> 00:13:25.039
Like you're you're working on trucking for now, and it's huge freeway miles where it's a huge number of deaths, and yet when you break it down in terms of the actual performance, like how many miles between accidents, it's incredibly rare.
00:13:25.279 --> 00:13:25.519
Right.
00:13:25.679 --> 00:13:35.759
And the number of miles that you would need to get to statistically demonstrate just from a naive raw mileage accumulation perspective, it's it's not feasible, right?
00:13:35.840 --> 00:13:39.279
It's in sort of the billions, depending on what confidence interval you want to have.
00:13:39.519 --> 00:13:40.559
How do you solve for that?
00:13:41.039 --> 00:13:42.879
You mentioned you know, safety case.
00:13:43.279 --> 00:13:52.240
We use a combination of drawing from the the in-field experience that we have, which gets us all the common stuff, the rare events that occur, things like near misses.
00:13:52.480 --> 00:13:58.879
We also go and mine the NHTSA databases and you know the NIXA taxonomy for how crashes occur.
00:13:59.279 --> 00:14:01.519
And we create synthetic versions of those tests.
00:14:01.679 --> 00:14:07.679
And that results in, you know, today we run literally millions of tests to get conviction that the thing is ready to go.
00:14:08.159 --> 00:14:15.200
And then it's important that for those synthetic tests that we actually ground the simulation to reality.
00:14:15.840 --> 00:14:22.000
There's a lot of noise around um using generative AI for simulation, which is great, and we do that.
00:14:22.559 --> 00:14:27.120
But at the end of the day, if you just have a simulator, it's a video game.
00:14:27.519 --> 00:14:32.159
Um, and our trucks don't drive in a video game, they drive in the real world.
00:14:32.399 --> 00:14:41.279
And so we have to go and take whatever results we see in the simulation and build conviction that that simulation is actually representative of what happens in the real world.
00:14:41.600 --> 00:14:43.519
Yeah, I think that part is absolutely critical.
00:14:43.759 --> 00:14:54.399
I mean, you remember in the very early days when we were working on safety cases and trying to approach it, I would say from the more traditional automotive perspective, there's not any flexibility.
00:14:54.879 --> 00:15:00.080
Is that when you say you see a pedestrian, it's if a finger is sticking out, you need to take that as a pedestrian.
00:15:00.159 --> 00:15:05.039
And that just does, it just ends up not working for exactly those sorts of scenarios you mentioned.
00:15:07.039 --> 00:15:10.159
I can't wait to see uh more Aurora trucks.
00:15:10.240 --> 00:15:17.279
I think we had this uh we had this handshake earlier this year that I'm gonna get a ride in in one of the trucks, and you're gonna come get a ride in one of our Aurora taxis soon.
00:15:17.600 --> 00:15:18.000
100%.
00:15:18.320 --> 00:15:19.279
We're looking forward to it.
00:15:19.440 --> 00:15:29.679
I I was you know looking around and seeing the the gravities with the the kid on them, and it's the second best looking AV out there right behind right behind the Aurora truck.
00:00:00.239 --> 00:00:07.919
You know, I think a year ago, uh, there's a point where if you asked Gemini how to hold the cheese on your pizza, it would say use glue.
00:00:08.160 --> 00:00:09.039
Technically accurate.
00:00:09.279 --> 00:00:13.359
Technically accurate, but would not result in a good meal, right?
00:00:13.519 --> 00:00:22.879
And and the good news is that there's a person between what Gemini said to do and actually producing the pizza, and they go, ha ha, that's silly.
00:00:23.039 --> 00:00:24.320
Let you know, try again.
00:00:24.640 --> 00:00:29.600
When the trucks move down the freeway at 70 miles an hour, we just don't have the ability to let that happen.
00:00:30.079 --> 00:00:31.600
This is unsupervised.
00:00:31.679 --> 00:00:33.920
I'm Dave Ferguson, co-founder of Neuro.
00:00:34.320 --> 00:00:39.359
Join me as we turn over the big questions in autonomy and physical AI.
00:00:40.640 --> 00:00:42.240
Chris and I have known each other.
00:00:42.399 --> 00:00:48.320
I was figuring this out since 2002 when I came to visit Carnegie Mellon and I crashed on your couch.
00:00:48.560 --> 00:00:49.759
I was just thinking about that.
00:00:50.079 --> 00:00:50.799
Yeah, it's been a while.
00:00:51.039 --> 00:00:53.200
You had to burn it afterwards, probably.
00:00:53.840 --> 00:00:54.799
No comment.
00:00:55.759 --> 00:00:59.359
But that weekend literally changed the trajectory of my life.
00:00:59.439 --> 00:01:01.759
You know, I decided to join CMU.
00:01:01.920 --> 00:01:05.599
The two of us then worked together on a bunch of projects.
00:01:05.840 --> 00:01:08.480
Uh the DARPA DARPA Urban Challenge that you led.
00:01:08.719 --> 00:01:12.719
And then later you pulled me to Google when you led the Google self-driving car.
00:01:13.040 --> 00:01:14.400
Glutton Punishment that you are.
00:01:14.480 --> 00:01:14.560
Yeah.
00:01:14.879 --> 00:01:16.239
Before it became Waymo.
00:01:16.480 --> 00:01:23.519
Uh and now for the last nine years, uh, you've you've founded and have been leading Aurora working on A V trucking.
00:01:23.680 --> 00:01:27.439
So, Chris, it's it's a huge privilege uh and treat to have you here today.
00:01:27.599 --> 00:01:28.239
Thank you so much.
00:01:28.560 --> 00:01:29.439
Thanks for having me on.
00:01:29.599 --> 00:01:35.200
Uh, it's always great to hang out with you, even you know, if we've got microphones instead of beers, but you know, it'll work.
00:01:35.359 --> 00:01:36.799
It's been a hell of a ride.
00:01:36.879 --> 00:01:43.359
You know, I remember us out and uh at water testing for the first uh for the urban challenge.
00:01:43.599 --> 00:01:52.400
Yeah, we're in we're in that old uh aircraft hangar, and I remember in the side there was like uh an old rusty pull-up bar, which was where we're getting our exercise.
00:01:52.560 --> 00:01:53.760
I felt like you were getting your exercise.
00:01:54.000 --> 00:01:57.359
I felt like it was the guy from Footloose, you know, like in the background.
00:01:57.920 --> 00:02:02.879
Trying to avoid um catching tetanus and winning the diaper urban channel.
00:02:03.680 --> 00:02:05.359
Yeah, so it was some good times.
00:02:05.760 --> 00:02:07.439
Tell us a little bit about trucking.
00:02:07.599 --> 00:02:08.879
Why AV trucking?
00:02:08.960 --> 00:02:11.039
You know, you've worked so long in this field.
00:02:11.120 --> 00:02:13.840
Uh, why is this the right application to be pursuing?
00:02:14.319 --> 00:02:18.960
We've been building Aurora for going on 10 years now, and we had this vision.
00:02:19.039 --> 00:02:21.520
We kind of understood that trucking would be a really interesting application.
00:02:21.599 --> 00:02:22.319
And why?
00:02:22.560 --> 00:02:24.240
Because it's it's really hard.
00:02:24.479 --> 00:02:27.199
Anyone that you know is a truck driver, you should thank them.
00:02:27.439 --> 00:02:30.479
But it's absolutely essential that goods move on trucks.
00:02:30.639 --> 00:02:36.240
If you look around the room we're in, look around any room, literally everything in that room at some point moved on a truck.
00:02:36.479 --> 00:02:46.479
So that combination of competence, technical features and capability, market need, economic opportunity, we're like, okay, this is where we should go put our energy.
00:02:46.719 --> 00:02:47.680
So where are you guys at?
00:02:48.000 --> 00:02:55.680
We are focused on building the core neurodriver, the foundational technology that can be a universal autonomy platform.
00:02:55.840 --> 00:02:58.639
And so for us, we used to work on these delivery vehicles.
00:02:58.800 --> 00:03:04.479
We thought it was very exciting, this entire vertical from manufacturing them, owning and operating them.
00:03:04.879 --> 00:03:09.360
And then we realized, well, we knew for a long time uh that it required a lot of capital.
00:03:09.520 --> 00:03:13.759
I think what happens is the market changed and getting that capital became very challenging.
00:03:13.919 --> 00:03:22.000
So we decided look, we should find a path that is more capital efficient for us to get to market and scale and realize a huge impact.
00:03:22.080 --> 00:03:28.639
And so what that meant for us was licensing the technology, working with partners to get it on as many different platforms as possible.
00:03:28.800 --> 00:03:35.599
So we have this big robotaxi push with Uber, and we have some other stuff in the works on the logistics side that we're very excited about.
00:03:35.759 --> 00:03:45.599
And really, it's about building that core AI AV technology and then trying to universally apply it to a ton of different platforms.
00:03:45.840 --> 00:04:02.879
And for us, it's just made a huge amount of sense that we work with partners and have them do what they are literally best in the world at doing, building cars, running rideshare uh services, and we get to focus on what we hope and believe that we are sort of best in class at being able to do, which is the A V side.
00:04:03.199 --> 00:04:05.919
Yeah, that that's like you said, that's been our approach from from day one.
00:04:06.000 --> 00:04:11.759
It's like we're good at this, we can be proud of that, but we should be humble about the things that we aren't, right?
00:04:11.919 --> 00:04:14.080
And make sure we go find great people to work with.
00:04:16.959 --> 00:04:32.160
There's a lot of buzz around uh and a lot of folks that like to talk about the nomenclature of A V 1.0, AV 2.0, end-to-end models, all of the foundational model technology we're seeing across the OpenAI, Anthropics, Gemini's of the world.
00:04:32.560 --> 00:04:35.439
How are you or not leveraging all of this technology?
00:04:35.519 --> 00:04:36.399
What what does that look like?
00:04:36.800 --> 00:04:39.759
There are places where algorithmic solutions work.
00:04:40.079 --> 00:04:40.240
Right.
00:04:40.399 --> 00:04:43.120
I take, for example, processing a GPS signal.
00:04:43.519 --> 00:04:47.920
We probably don't need to learn how to do correlation across the signal cover much of satellites.
00:04:48.079 --> 00:04:49.759
We can just write down the math and do it.
00:04:50.160 --> 00:04:52.240
And so we should do that where we can.
00:04:52.399 --> 00:04:59.519
But you can't deny the impact that machine learning and AI has on the technology we're developing and building.
00:04:59.680 --> 00:05:02.160
You know, the approach we've taken is what we call verifiable AI.
00:05:02.399 --> 00:05:15.040
And this is really about yes, we absolutely need to be using modern techniques, but we have to make sure that we are able to kind of understand how they perform and put constraints around them.
00:05:15.600 --> 00:05:21.920
And I I think about where the enthusiasm around end-to-end and large models come from.
00:05:22.000 --> 00:05:22.800
And it's obvious.
00:05:23.040 --> 00:05:25.519
If you'd asked me five years ago, would we be where we are today?
00:05:25.600 --> 00:05:27.439
The answer would have been I couldn't have seen it.
00:05:27.680 --> 00:05:28.160
Absolutely not, yeah.
00:05:28.319 --> 00:05:28.639
You know.
00:05:29.120 --> 00:05:31.439
And so you absolutely have to be paying attention to that.
00:05:31.519 --> 00:05:36.879
But you also have to understand how to use the technology in the domain that it's applied to.
00:05:37.120 --> 00:05:43.439
A year ago, there's a point where if you asked Gemini how to hold the cheese on your pizza, it would say use glue.
00:05:43.680 --> 00:05:44.399
Technically accurate.
00:05:44.720 --> 00:05:48.639
Technically accurate, but would not result in a good meal, right?
00:05:48.800 --> 00:05:58.160
And and the good news is that there's a person between what Gemini said to do and actually producing the pizza, and they go, ha ha, that's silly.
00:05:58.319 --> 00:05:59.439
Let you know, try again.
00:05:59.600 --> 00:06:04.079
When the trucks move down the freeway at 70 miles an hour, we just don't have the ability to let that happen, right?
00:06:04.160 --> 00:06:09.279
Because that's the equivalent of it deciding to jerk the wheel and turn off the freeway.
00:06:09.519 --> 00:06:13.839
We've thought about, okay, how can we take this kind of technique and apply it in our space?
00:06:14.000 --> 00:06:16.639
It's important to have defined interfaces.
00:06:16.879 --> 00:06:21.839
So we you know think of it broadly as two systems, seeing the world and then reacting to the world.
00:06:22.240 --> 00:06:28.399
Each of these is you know, kind of an exponentially hard problem, but very difficult, very complicated.
00:06:28.879 --> 00:06:40.000
But to try to make sure we can understand how the systems worked, we've actually created interfaces that say this is how we express what a person is, what a car is, you know, what the various features of the world that we care about are.
00:06:40.399 --> 00:06:46.560
And then we basically decompose that model and separate it from the model that reacts to the world.
00:06:46.879 --> 00:06:49.920
The downside of this is that you're potentially throwing away information, right?
00:06:50.000 --> 00:07:00.560
If I were to take the the bull argument for end-to-end, it's that the model may find a more expressive way to describe a person, which would provide more subtle cues, which then you could react to.
00:07:00.720 --> 00:07:04.399
Whereas if I say this is how you describe a person, the model is only that expressive.
00:07:04.959 --> 00:07:13.360
The problem though is that we're not building a system that we want to hope works or that we want to kind of guess that it seems to work.
00:07:13.680 --> 00:07:15.759
We want a system where we know that it works.
00:07:16.079 --> 00:07:18.879
And to do that, we have to do verification or validation.
00:07:19.040 --> 00:07:21.600
We have to test it and we have to look at the tests and make sure they work.
00:07:22.079 --> 00:07:31.680
And if you take two exponential systems and you basically test given inputs here and outputs there, the complexity of testing those two systems is the product of two exponentials.
00:07:32.079 --> 00:07:43.439
Uh if we're able to decompose them into two separate systems and be able to kind of test them to the boundaries, then we now have two independent exponential systems.
00:07:43.680 --> 00:07:51.199
And validating those is very hard, but is literally exponentially easier than validating as if it was one system.
00:07:51.439 --> 00:07:58.240
And then the other part of the verifiable AI that we use is the way we kind of uh constrain the output that's possible.
00:07:58.480 --> 00:08:07.680
So in modern LLM applications, you have something that's proposing the sentence that comes out, and then you have a thing that's going through and ranking them.
00:08:07.839 --> 00:08:14.240
In our case, what we've done is we've taken the proposal part, and that is actually algorithmic and deterministic.
00:08:14.639 --> 00:08:22.560
And that means that when we're evaluating what actions to take, all of the actions we know are plausible, right?
00:08:22.800 --> 00:08:29.120
It can't suddenly decide to take an action that is kind of veering off the road because that isn't in the set of things that it's ranking.
00:08:29.199 --> 00:08:29.360
Trevor Burrus, Jr.
00:08:29.600 --> 00:08:32.559
The approach that NERO has taken, you know, it's all related.
00:08:32.639 --> 00:08:46.480
I think you've you've laid out exactly the challenges of trying to get the guaranteed verification of intermediate outputs versus trying to capture the full ceiling of potential by not having any information loss.
00:08:46.879 --> 00:08:54.639
The way we do it is we have the intermediate, we do have intermediate sort of task head outputs, not at the end of the net, sort of in the middle of it.
00:08:54.720 --> 00:09:00.000
And so you can think of it as a little bit like understanding the world and then reasoning about it.
00:09:00.240 --> 00:09:04.240
However, we also pass through a very high-dimensional embedding.
00:09:04.480 --> 00:09:05.919
So we sort of have both.
00:09:06.159 --> 00:09:22.879
But one of the additional safeguards that we get from doing that is that when we do our final, we call it motion selection stage, like the ranked list and checking that what's coming out of the net at the end is valid, we can use those intermediate outputs to do that validation.
00:09:23.200 --> 00:09:40.639
And so we have found that that combination has allowed us to do a really nice job of capturing some of the really difficult, uh nuanced behavior that you get from having the full embedding, like the person standing on the side of the road, and you know, it's a person with a stroller and a dog and a bike.
00:09:40.720 --> 00:09:44.879
And are you classifying the what are you classifying it as or what's the intent and so on?
00:09:45.120 --> 00:09:50.639
But like as a field, we've seen this incredible evolution in how all of this works.
00:09:50.799 --> 00:09:58.320
Like, I mean, what you're explaining around detecting what's out there in the world and then reasoning about it like at a high level, we could say that's what we did in 2007.
00:09:58.639 --> 00:09:58.879
Yeah, yeah.
00:09:59.200 --> 00:10:00.159
Um, but in reality.
00:10:00.720 --> 00:10:01.759
Just think act, maybe.
00:10:02.000 --> 00:10:02.559
That's right.
00:10:02.720 --> 00:10:07.200
And in reality, it's it's wildly different in terms of of the performance.
00:10:08.000 --> 00:10:09.519
It's it's night and day, right?
00:10:09.600 --> 00:10:12.720
That you know, we've been both been working at this for 20-something years at this point.
00:10:12.879 --> 00:10:15.039
And it was incredible back then.
00:10:15.120 --> 00:10:24.799
Like uh, you know, when we did the urban challenge, the idea that you could have a vehicle driving around and not bump into much stuff and interact with traffic, it was truly a grand challenge.
00:10:25.039 --> 00:10:29.919
Um but when you look back at it, it was kind of a toy problem.
00:10:30.240 --> 00:10:40.559
And as we get into what we're dealing with today, the biggest change is we've gone through this kind of prototype phase and we're now into truly industrializing production phase, right?
00:10:40.799 --> 00:10:48.799
This is one of the things that I think a lot of people don't really understand is that you know it probably took, I don't know, 10, 15 people to prototype the iPhone.
00:10:48.879 --> 00:10:53.039
And it literally takes thousands of people to kind of industrialize it and produce it.
00:10:53.200 --> 00:11:07.120
And it's pretty incredible to have had the opportunity to kind of see this industry move from the 15 of us doing the late night drives at Google or back in the desert and into now where you know this is really, it's it's real.
00:11:07.440 --> 00:11:08.399
And it is hard though.
00:11:08.559 --> 00:11:14.240
You mentioned this massive gulf between like a demo versus getting it to production.
00:11:14.320 --> 00:11:22.799
I remember in 2011 when when you enticed me to Google, which wasn't that hard, and I came to visit and I went for a ride with you and Dimitri in the process.
00:11:23.519 --> 00:11:23.840
Exactly.
00:11:24.320 --> 00:11:25.440
Is there anything for me to do even?
00:11:25.840 --> 00:11:26.080
That's right.
00:11:26.320 --> 00:11:28.639
I think my exact question was this this seems pretty good, Chris.
00:11:28.720 --> 00:11:30.080
Like, is there anything interesting left?
00:11:30.159 --> 00:11:32.000
Like, why do you want me to join?
00:11:32.240 --> 00:11:34.559
And that was what like 15, 15 years ago.
00:11:35.039 --> 00:11:35.919
Yeah, a little bit ago, yeah.
00:11:36.080 --> 00:11:38.799
And even back then, like we had incredible demos.
00:11:39.120 --> 00:11:39.440
Oh, yeah.
00:11:39.600 --> 00:11:39.840
Like back.
00:11:40.000 --> 00:11:44.240
You guys had an incredible demo in 2011 when I joined, like before I was even on the program.
00:11:44.480 --> 00:11:56.559
And I think that has been one of the really challenging parts of this entire industry, particularly from like a financing investor perspective, because investors will come and they'll get in a car and they'll this is the same as Waymo, right?
00:11:56.639 --> 00:11:56.720
Right?
00:11:56.879 --> 00:12:18.320
Like this, this, this feels just just like you know, a great demo, like FSD, like all of these, all of these uh examples out there of what seems very compelling, yeah, but there can still be this enormous gap between that very compelling demo and actually getting it to a place where we can launch and scale and save lives and make an impact.
00:12:18.639 --> 00:12:23.440
I used to joke, you know, you give me three graduate students in six months and I'll give you a self-driving car.
00:12:23.679 --> 00:12:25.360
You don't necessarily want to trust your life to it.
00:12:25.519 --> 00:12:29.840
And I think now, you know, you give me uh a summer and two interns and I'll give you a self-driving car.
00:12:30.159 --> 00:12:34.480
When you talk about a safety critical system, you have to be right.
00:12:35.120 --> 00:12:50.000
Yeah, we often say in Neuro that we see the tech and the investment and the IP behind our safety case and the validation that we do to put these vehicles on roads as as valuable as the core sort of foundational um autonomy tech.
00:12:50.240 --> 00:12:52.399
And that that tends to blow people's minds a little bit.
00:12:52.879 --> 00:12:55.279
Because that is not true in almost any other space.
00:12:55.519 --> 00:13:08.480
Like the safety criticality of of the space that we're in is just it's completely different from you know, you mentioned the standard sort of LLM example of incredible technology, but it's okay if it tells you to put glue on your pizza once in a while.
00:13:08.720 --> 00:13:08.879
Yeah.
00:13:09.600 --> 00:13:10.799
So how do we solve that?
00:13:10.960 --> 00:13:25.039
Like you're you're working on trucking for now, and it's huge freeway miles where it's a huge number of deaths, and yet when you break it down in terms of the actual performance, like how many miles between accidents, it's incredibly rare.
00:13:25.279 --> 00:13:25.519
Right.
00:13:25.679 --> 00:13:35.759
And the number of miles that you would need to get to statistically demonstrate just from a naive raw mileage accumulation perspective, it's it's not feasible, right?
00:13:35.840 --> 00:13:39.279
It's in sort of the billions, depending on what confidence interval you want to have.
00:13:39.519 --> 00:13:40.559
How do you solve for that?
00:13:41.039 --> 00:13:42.879
You mentioned you know, safety case.
00:13:43.279 --> 00:13:52.240
We use a combination of drawing from the the in-field experience that we have, which gets us all the common stuff, the rare events that occur, things like near misses.
00:13:52.480 --> 00:13:58.879
We also go and mine the NHTSA databases and you know the NIXA taxonomy for how crashes occur.
00:13:59.279 --> 00:14:01.519
And we create synthetic versions of those tests.
00:14:01.679 --> 00:14:07.679
And that results in, you know, today we run literally millions of tests to get conviction that the thing is ready to go.
00:14:08.159 --> 00:14:15.200
And then it's important that for those synthetic tests that we actually ground the simulation to reality.
00:14:15.840 --> 00:14:22.000
There's a lot of noise around um using generative AI for simulation, which is great, and we do that.
00:14:22.559 --> 00:14:27.120
But at the end of the day, if you just have a simulator, it's a video game.
00:14:27.519 --> 00:14:32.159
Um, and our trucks don't drive in a video game, they drive in the real world.
00:14:32.399 --> 00:14:41.279
And so we have to go and take whatever results we see in the simulation and build conviction that that simulation is actually representative of what happens in the real world.
00:14:41.600 --> 00:14:43.519
Yeah, I think that part is absolutely critical.
00:14:43.759 --> 00:14:54.399
I mean, you remember in the very early days when we were working on safety cases and trying to approach it, I would say from the more traditional automotive perspective, there's not any flexibility.
00:14:54.879 --> 00:15:00.080
Is that when you say you see a pedestrian, it's if a finger is sticking out, you need to take that as a pedestrian.
00:15:00.159 --> 00:15:05.039
And that just does, it just ends up not working for exactly those sorts of scenarios you mentioned.
00:15:07.039 --> 00:15:10.159
I can't wait to see uh more Aurora trucks.
00:15:10.240 --> 00:15:17.279
I think we had this uh we had this handshake earlier this year that I'm gonna get a ride in in one of the trucks, and you're gonna come get a ride in one of our Aurora taxis soon.
00:15:17.600 --> 00:15:18.000
100%.
00:15:18.320 --> 00:15:19.279
We're looking forward to it.
00:15:19.440 --> 00:15:29.679
I I was you know looking around and seeing the the gravities with the the kid on them, and it's the second best looking AV out there right behind right behind the Aurora truck.
