When AI Gets a Body: Can We Trust It With Freedom?

WEBVTT
00:00:00.320 --> 00:00:04.959
Also, uh, we have been using drones for geofencing.
00:00:05.120 --> 00:00:11.439
We had a project with NSF on that one, of geofencing uh very important areas.
00:00:11.519 --> 00:00:16.399
So you can think about uh um, let's say um uh an airport, right?
00:00:16.559 --> 00:00:19.519
You don't want people to fly their drones close to the airport.
00:00:19.600 --> 00:00:21.839
This is like a safety thing.
00:00:22.000 --> 00:00:40.880
So what we have done is uh we built a geofencing protocol around the area, and uh um we have radar technologies where we can predict and and identify the drones, the model, the capabilities, etc.
00:00:41.280 --> 00:00:46.159
We can identify if it has malicious things, we can identify identify if it's a hobbyist.
00:00:46.240 --> 00:00:56.399
So if it's a hobbyist, what we do, we actually deploy our own drones and then we take over the autopilot, and then we land it safely.
00:00:56.479 --> 00:01:04.719
Because again, remember that uh we don't want to get a net or things like that, because maybe these things will get on our heads, right?
00:01:04.799 --> 00:01:05.599
You never know.
00:01:05.760 --> 00:01:08.799
Uh or it can it can break a house or a car.
00:01:08.959 --> 00:01:20.239
So we want to make sure that we take over the autopilot, we land it safely, and then we can inspect it for suspicious or malicious behavior, and then we can check who was behind.
00:01:20.719 --> 00:01:25.040
And uh so um that's what we have been uh working on geofencing.
00:01:25.120 --> 00:01:28.159
We have worked with airports, we have worked with important houses.
00:01:28.319 --> 00:01:32.079
I mean, you can think, I'm sure that the White House even probably has such things.
00:01:32.239 --> 00:01:35.519
You cannot fly drones around around um a White House.
00:01:35.599 --> 00:01:37.760
Uh it has it's geofenced.
00:01:37.920 --> 00:01:48.400
But um, but yeah, but I think um the only work that I know that uh we have uh used um we take over the autopilot without damaging anything, is our own.
00:01:48.480 --> 00:01:58.480
So we take over the autopilot, we land it, we send our team, take the take the drone, and then we inspect it and we see who was behind it, was it malicious or not?
00:01:58.640 --> 00:02:12.879
I also believe that the human on the loop is more important for counter stuff, because the human can actually override um uh different stuff, can do reward shaping, can do different things.
00:02:13.039 --> 00:02:25.199
Uh I always say that the human in the loop is a weakness in the chain because humans they work also on emotions or they have also fatigue.
00:02:26.000 --> 00:02:31.199
Uh so that's um a million-dollar question.
00:02:31.280 --> 00:02:56.560
Uh, we have a project with the army where we have human on the loop, where we want to see um how the human and the AI can work together uh by um making decision making faster for security mitigation, um uh in session and in sensor and actuators, um, and uh but inside the learning mechanisms.
00:02:56.800 --> 00:03:05.039
Um but yeah, this is uh something that uh um so people say that the human is either the weak link or the strong link.
00:03:05.120 --> 00:03:08.879
I say that human every time the human is there is a little bit weak.
00:03:09.120 --> 00:03:20.080
In military situations, yeah, we have we have had projects on swarming, um, but you know, remember that the human, we want the human, let's say the soldier, to give uh commands, right?
00:03:20.240 --> 00:03:21.439
We want a mission, okay?
00:03:21.520 --> 00:03:38.400
So prescribe the missions, and then there will be a coordination between the ground vehicles and the drones, um, and then they will find the best way without a central station, and how they can do that in a distributed way is like they talk to their neighbors.
00:03:38.719 --> 00:03:44.000
So uh, for example, um I'm drone A and drone B, and I'm right next to each other.
00:03:44.159 --> 00:03:51.520
Of course, I will talk to this, I will talk to him or her, and then she will talk to blah blah blah until we cover the whole swarm.
00:03:51.599 --> 00:04:05.919
So that we talk, we exchange positions, velocities, we exchange our information, what we have seen, and then eventually we can survey the area, for example, with minimal centralized commands by the human.
00:04:06.159 --> 00:04:15.439
And um and because you know, as you said, actually actually very important thing: intermittent communication, you can loot communication to the central to the central station.
00:04:15.680 --> 00:04:24.959
So, how do you do your agents uh don't collide and they don't they take their decisions by themselves to still fulfill the mission?
00:04:25.279 --> 00:04:38.959
So we had something similar from an NSF uh career award where we used in the meter learning, um, where um let's say uh let's say let's go from the learning side.
00:04:39.040 --> 00:04:44.160
Uh, in order for a learning agent to work, it has to be rewarded, right?
00:04:45.519 --> 00:04:54.560
So let's say one malicious agent or it can break, um, let's say it has to be to be rewarded from the stress central station.
00:04:54.720 --> 00:04:57.439
Okay, let's say something like that.
00:04:57.759 --> 00:05:08.319
So if uh the rewards are not coming to reward the agent or the drone for something good or bad that it's doing, they don't know what they're doing, right?
00:05:08.480 --> 00:05:15.279
So in this situation, probably they will collide or get down, probably they will get into an adversarial territory.
00:05:15.519 --> 00:05:17.279
So we don't want that.
00:05:17.439 --> 00:05:29.600
So, what happened in our case, what we did is we we developed robust reinforcement learning in the present of intermittent rewards or dropped rewards.
00:05:29.759 --> 00:05:37.439
So, again, uh I uh let's say the communication with the central station is broken, so I don't get uh rewards anymore.
00:05:37.600 --> 00:05:51.360
Um, so what I do, I have either a replay buffer or experience replay buffer, or for the same situations that have seen in the past, uh I do similar actions.
00:05:51.600 --> 00:06:01.120
Or I talk to my neighbors and then I figure out what they're doing, and then they say information instead of going through the central station.
00:06:01.199 --> 00:06:11.839
So, in other words, I can get my rewards either stopped but still continue doing the actions, or get rewarded by observing my other drones.
00:06:12.079 --> 00:06:14.959
This project we have with DOD and the Pentagon.
00:06:15.120 --> 00:06:21.279
So, what we want to do is we want it to train security analysts with AI agents.
00:06:21.519 --> 00:06:23.199
And we want you to do that.
00:06:23.279 --> 00:06:28.399
So let's say you have a mission, let's say you fly a mission in Iraq, right?
00:06:28.560 --> 00:06:34.560
Okay, and you have a plane, and then there's a windcast that flows.
00:06:34.720 --> 00:06:42.000
So security analyst that is very well trained will probably understand that this is a windcast and not an attack, right?
00:06:42.639 --> 00:06:48.560
Um and an agent will probably think, oh, the trajectory is moving, oh, it's an attack.
00:06:48.720 --> 00:06:48.959
Okay.
00:06:49.360 --> 00:06:57.519
So what we did is we had this uh human AI teaming where we train each other in situations that they haven't seen before.
00:06:57.759 --> 00:07:03.199
And then we were using the decision of we had a trust mechanism.
00:07:03.439 --> 00:07:12.079
So the trust mechanism was either overriding the human decision or overriding the AI decision.
00:07:12.399 --> 00:07:25.360
And uh we actually have developed this as a visualization tool, and then we delivered that to the um to the Minerva, which is the only Pentagon-funded um uh directly Pentagon-funded uh program.
00:07:25.839 --> 00:07:29.360
Right now, let's say you have a big data center, right?
00:07:29.439 --> 00:07:31.439
And you buy GPUs.
00:07:32.079 --> 00:07:39.759
If I have, let's say, 20 GPUs at their NVIDIA, 10 GPUs at their AMD, they don't talk to each other.
00:07:40.319 --> 00:07:52.720
And uh uh and most of the people right now, um, let's say at the universities or anywhere, they want to learn CUDA, which CUDA is the NVIDIA language, but it's not compatible with the other ones.
00:07:52.800 --> 00:08:02.079
And let's say that you want to buy uh an NVIDIA hardware, a GPU, probably you have to wait for years for it to come to your lab.
00:08:02.639 --> 00:08:06.480
And uh, but data centers and everything, they're evolving, right?
00:08:06.560 --> 00:08:07.839
They want it right now.
00:08:08.160 --> 00:08:31.040
So, what we want to do is we're working towards making a protocol similar to you can think HTTP, where uh we make this protocol, it's actually only software-based, and uh it makes all the GPUs talk to each other and save money, save time, save performance, and make everything more efficient.
00:08:31.279 --> 00:08:44.720
And um, so I'm happy that we have a very good team over there where um we have the previous CTO of Dell, we have people from Apple, we have um our CEO, he has targeted many companies on security and IT.
00:08:45.279 --> 00:08:55.200
Um there are many people in the team that uh uh they take this forward, and uh we have pretty promising results.
00:08:55.360 --> 00:09:00.720
Uh, and uh um we still haven't even scratched the surface.
00:09:08.799 --> 00:09:21.919
Hello everyone, and welcome to Mindplex, where we venture to the cutting edge of science to discover breakthroughs transforming our world and shaping a better future for all.
00:09:22.639 --> 00:09:27.519
And today we are asking what happens when AI gets a body?
00:09:28.480 --> 00:09:36.799
Well, suddenly an AI mistake is no longer just a bad answer, it can turn into a physical disaster.
00:09:37.440 --> 00:09:44.000
And that may be where the most consequential frontier of AI safety is unfolding right now.
00:09:45.120 --> 00:09:53.600
Professor Kiryakos Bambudakis, our guest today, is tackling this existential question for embodied AI.
00:09:54.080 --> 00:10:04.000
How do we give machines the freedom to learn in the real world when a mistake can crash an aircraft, compromise a defense system, or harm a human or more?
00:10:04.559 --> 00:10:06.720
So, welcome to the podcast, Professor.
00:10:07.039 --> 00:10:08.320
Thank you very much, Michaela.
00:10:08.559 --> 00:10:10.399
It's a pleasure and honor to be here.
00:10:10.639 --> 00:10:13.200
And I'm looking forward to our conversation.
00:10:13.440 --> 00:10:23.679
Uh, we have many projects and many interesting things we can talk about, and one of them, of course, is what you just described, which is uh one of the frontier of AI and safety.
00:10:23.840 --> 00:10:33.120
And uh, we have been working with major agencies, uh, NASA, NSF, Navy, Army, Air Force, you name it.
00:10:33.279 --> 00:10:46.879
Um, and we try to make our autonomous systems uh be safe, be secure, and make smart decisions on the go without being specifically pre-programmed to do anything like that.
00:10:47.200 --> 00:10:51.360
So, yeah, that's our central central uh theme.
00:10:51.679 --> 00:10:57.519
And uh we try to make it, in other words, better, safer, and smarter systems.
00:10:58.000 --> 00:11:07.200
And you know, you I mean, as you mentioned, uh you are leading a colossal research program in cyber physical systems at Georgia Tech.
00:11:07.519 --> 00:11:17.440
So many projects in such diverse areas with so many applications, and indeed, many of them looking like they're just out of a sci-fi movie.
00:11:17.600 --> 00:11:21.360
And I want to tackle as many as possible in our conversation today.
00:11:21.679 --> 00:11:25.039
So, but let's start with clarifying for our audience.
00:11:25.200 --> 00:11:28.879
I mean, your research group is called cyber physical systems.
00:11:29.039 --> 00:11:59.120
Uh, you know, my pioneering work in distributed intelligence systems like decades ago, ambient intelligence swarm robotics, uh, it culminated in 2005 when Paul Verbos, who is one of the pioneers of neural networks, he invited me on the NSF, on the first NSF review panel, uh to review proposals in his program, which was starting then, like uh cyber physical systems when we even didn't know what they are.
00:11:59.679 --> 00:12:00.720
Exactly, exactly.
00:12:01.279 --> 00:12:07.919
And and now AI has so much more power and autonomy that many of those dreams have become a reality.
00:12:08.080 --> 00:12:13.759
But please let me know what to you and from your research perspective are cyber physical systems.
00:12:14.000 --> 00:12:16.559
Yeah, so let me actually uh you know you remind me.
00:12:16.639 --> 00:12:27.679
Uh so my advisor, Frank Lewis at Texas, he was uh um uh he invited Paul Werbos back in the days to a one workshop, and then of course Paul invited him also to NSF.
00:12:27.840 --> 00:12:36.320
So that's how I became familiar with his work, and then actually my whole PhD uh was inspired by his work on approximate dynamic programming.
00:12:36.399 --> 00:12:52.240
You know, later it was named Adaptive Dynamic Programming, and it was um, you know, Paul um is uh probably most of the people in this uh podcast uh will know that Paul is one of the big guys behind backpropagation, and that's what he did for his thesis.
00:12:52.559 --> 00:13:08.320
And uh he was leading the NSF uh um um EPCN, uh energy power control and networks for some time, and then um he was very interested in how the brain works and how you can put these things into an autonomous system.
00:13:08.960 --> 00:13:18.080
And um, so that's how we started, and then um I finished my PhD, as I said, we worked on online learning or enforcement learning and all these things.
00:13:18.320 --> 00:13:24.159
And then at my postdoc, I got interested in the security of cyber physical systems.
00:13:24.399 --> 00:13:32.000
So uh, but what I mean about security is how you can design resilient controllers by using learning and all these things to see.
00:13:32.559 --> 00:13:41.039
But first, let's uh let's back up and please clarify for our audience what are cyberphysical systems from the story there.
00:13:41.679 --> 00:13:44.159
I know you're trying to avoid the answer, but no.
00:13:45.679 --> 00:13:48.879
So, you know, my lab is called intelligent cyberphysical systems.
00:13:49.120 --> 00:13:50.639
Oh wow, yeah, exactly.
00:13:50.799 --> 00:13:52.799
Even more exactly.
00:13:53.039 --> 00:14:01.519
So it's like as the word says, it's uh it's actually a multi-agent system where cyber and physical components they come together.
00:14:01.679 --> 00:14:02.000
Okay.
00:14:02.559 --> 00:14:12.559
So um uh, you know, people in the computer science, they have the cyber part, but I'm a controls person and I want to bring the physics also, right?
00:14:12.960 --> 00:14:24.639
So you can think about actuator sensors, all of them talking through a network, and then this whole thing, along with the physics of the model, they comprise a cyberphysical system.
00:14:24.799 --> 00:14:28.799
So a cyberphysical system, you can think it's a large smart grid.
00:14:28.960 --> 00:14:38.559
It can be a plane, it can be a human body, or you know, a human body, I can see, I can sense, I have, I transmit signals through my brain.
00:14:38.720 --> 00:14:42.240
So all these things they can be casted as a cyberphysical system.
00:14:42.720 --> 00:14:48.559
And uh in my lab, because I said I want to bring intelligence, it's called intelligent cyberphysical systems.
00:14:48.879 --> 00:15:08.799
And uh, in other words, I take all these components of the cyber and the physical components I just described, that they talk through a network, and then I bring some intelligence to them, some reinforcement learning inspired again by the work of uh Werbos, Berekas, all these people that they have done a huge amount of work.
00:15:10.159 --> 00:15:19.039
Yes, and you know, from uh back when I was into like the pioneering uh, let's say, trade blazing of this field.
00:15:19.279 --> 00:15:27.039
I mean, uh, for us, uh of course the technologies were not so developed, and we were calling them only cyber physical systems.
00:15:27.200 --> 00:15:37.279
We were focused mainly on the sensors and the connections between machines and that the machines behave as we tell them, because otherwise they would bump into each other.
00:15:37.519 --> 00:15:55.679
But now, you know, so uh it's not only about the machine behaving as intended, but actually uh machine to have the freedom to behave as it thinks is best, and still trust that what it will do is safe for us.
00:15:55.840 --> 00:15:56.480
Yes.
00:15:56.960 --> 00:16:10.240
This actually opens doors for lots of research because the more smart you make these large-scale systems, the more the more open doors for safety and security and control there exists.
00:16:10.480 --> 00:16:13.759
So that's where we come in and we try to solve all these problems.
00:16:13.840 --> 00:16:18.559
But of course, you know, systems they evolve, they become smarter, smarter.
00:16:18.799 --> 00:16:28.799
Also, potential attackers, adversaries become smarter, or we can discover many new things like uh oh uh some safe some new safety measures.
00:16:28.960 --> 00:16:37.360
You know, nowadays, uh, with all this legislation and all these things, you know, you have to make sure that uh uh the safety is an evolving landscape.
00:16:37.600 --> 00:16:43.840
And we have to make sure that uh our frameworks can be adaptive to all these things.
00:16:44.559 --> 00:16:55.039
Yes, and uh let's take, uh, because you work with NASA and you mentioned that, let's take space exploration as an example, which I think is kind of uh an extreme example, but uh yes.
00:16:55.279 --> 00:17:05.839
Um a robot on Mars cannot simply phone an engineer or send a text and wait for instructions uh every time reality deviates from its model.
00:17:06.559 --> 00:17:12.559
So your NASA funded work uh develops safety assurance for such autonomous systems.
00:17:12.799 --> 00:17:18.559
Uh please let us know more about your approach there, and um then I will ask you more questions.
00:17:18.880 --> 00:17:20.240
Sure, sure, sure, sure.
00:17:20.480 --> 00:17:24.480
So um I think um let me let me actually get your example.
00:17:24.720 --> 00:17:29.839
So, you know, when we do space exploration, you know, nowadays NASA has this rover, right?
00:17:30.079 --> 00:17:34.720
So the rover um hasn't mapped anything on Mars, right?
00:17:34.960 --> 00:17:36.960
Because it's an unexplored space.
00:17:37.039 --> 00:17:38.240
That's why we send the rover.
00:17:38.480 --> 00:17:42.400
So all the roads, I mean, roads in quads, they're not mapped.
00:17:42.559 --> 00:17:47.119
The the Mars has to navigate, has to discover new things, has to sample things.
00:17:47.359 --> 00:18:00.480
So all these things that come from the uncertainty, which is why uh um the learning comes into the game, or enforcement learning, where you can adapt to any uncertainty and make smart decisions as you go.
00:18:01.519 --> 00:18:13.200
Uh so our NASA project actually, um, I mean, one of our NASA projects, the most recent one, uh, is about um um uh um flying cars.
00:18:13.440 --> 00:18:24.640
So uh which is uh the next big thing for uh as when before going to the space, the next big thing is uh is uh this uh uh EV toll.
00:18:24.880 --> 00:18:29.599
It's like vertical takeoff landing cars, which are electric, but they don't make noise with their propellers.
00:18:29.680 --> 00:18:30.640
It's not like a helicopter.
00:18:30.720 --> 00:18:37.279
The helicopter makes lots of noise, but it has to be safe when it gets you from point A to point B.
00:18:37.759 --> 00:18:50.319
So uh so again, if it gets you from point A to point B without a pilot, it has to explore and predict the best decisions to make to drive you to the desired space.
00:18:50.400 --> 00:18:53.599
Because again, it we don't want to have any accidents.
00:18:53.839 --> 00:18:58.799
So we have to make sure that everything is safe before we deploy the technology.
00:19:00.240 --> 00:19:01.039
Absolutely.
00:19:01.279 --> 00:19:05.039
And uh, you know, this was uh uh an older dream of mine.
00:19:05.119 --> 00:19:06.880
I I I used to give this example.
00:19:07.039 --> 00:19:12.880
Yeah, let's now imagine uh in multi-agent systems with coordination of supply chains and so on.
00:19:13.039 --> 00:19:15.599
Imagine now we want to build a car that can fly.
00:19:15.759 --> 00:19:24.559
Well, it would get the engine maybe from Mercedes-Benz and the body from Boeing, and then all this supply chain, and then oh, it takes off.
00:19:24.799 --> 00:19:31.039
But anyway, now you are really making this happen, which that's why I say for me is like sci-fi uh movie.
00:19:31.839 --> 00:19:51.279
And you know, but what happens when the machine, be the car that can fly or a robot on Mars, uh, it breaks and nobody is coming to repair it, you know, like an unanticipated damage actuators, corrupt sensor, hostile environment, or partial system failure can happen on Mars.
00:19:51.839 --> 00:19:58.000
How do we move from fault tolerance to something closer to uh machine resilience?
00:19:58.160 --> 00:20:08.160
Like a machine that recognizes a change in its own structure and then it learns how to operate uh its uh changed or damaged body.
00:20:08.319 --> 00:20:13.279
Can we can we achieve this kind of primitive form of self-awareness in machines?
00:20:13.440 --> 00:20:14.160
What do you think?
00:20:14.720 --> 00:20:23.039
Yes, actually, it's uh interesting that you say that because we have an NSF grant to, I mean, one of our grants is NSF, and we deal exactly with this.
00:20:23.200 --> 00:20:28.799
So usually these systems are they have redundancy in searching actuators.
00:20:29.039 --> 00:20:33.039
Okay, so and then uh I mean, let's think simple.
00:20:33.200 --> 00:20:35.279
Let's go when you fly in a plane, right?
00:20:35.759 --> 00:20:38.880
So uh the plane, you name it, right?
00:20:38.960 --> 00:20:40.559
Uh I just came back from Greece.
00:20:40.720 --> 00:20:46.319
So the plane that I was flying on, which is every plane, of course, has redundancy in everything that you do.
00:20:46.480 --> 00:20:58.079
And then there must be something that is called Byzantine agreement, where everything has to agree on a value, or else if it doesn't agree, then something, a fault, is happening.
00:20:58.400 --> 00:21:07.920
Uh and then uh you dep you you um uh detect this fault, you isolate the the faulty component, and you don't use it anymore.
00:21:08.079 --> 00:21:16.480
Uh so now though, you raised a very important thing that um uh we need the redundancy inside the learning mechanisms.
00:21:16.640 --> 00:21:23.519
So this is not about the sensor and the actuators, but redundancy inside the policies of the learning agents.
00:21:23.680 --> 00:21:56.480
So this is uh one NSF grant we have uh with a collaborator from NYU, where we use this redundancy inside the learning such that if let's say there is a gel break in the learning or there is uh a compromised uh poisoning or something, we still are able to use some other policies from inside the learning and uh figure out that something is poisoning us and uh make it uh work.
00:21:56.640 --> 00:22:05.440
Yeah, so these are important because and again, uh in a in a Learning component, let's say your brain just sends and gets signals, right?
00:22:05.680 --> 00:22:07.359
Your eyes are the sensors.
00:22:08.400 --> 00:22:19.839
But of course, uh, your let's say um I close my eyes, then my brain is being rewired to work and feel without seeing, right?
00:22:20.079 --> 00:22:25.759
And then you can think about the people that they have lost their vision, the people that have lost their hearing.
00:22:25.920 --> 00:22:30.160
So all these things they make your brain fire in different ways.
00:22:30.400 --> 00:22:35.519
So this is how it what it inspired us in autonomous systems to bring these ideas.
00:22:35.680 --> 00:22:38.079
So then you have to rewire your brain.
00:22:38.240 --> 00:22:45.359
So this goes back to rewiring the policies inside the learning by working with what you have already.
00:22:46.079 --> 00:22:47.839
Thank you so much for this parallel.
00:22:48.000 --> 00:22:49.119
Now I really got it.
00:22:49.200 --> 00:22:50.480
I feel I understand it.
00:22:50.559 --> 00:22:52.960
That's because I can I can actually relate to it.
00:22:53.039 --> 00:22:58.160
And I'm sure our audience as well, you are an of course you are a professor, so you know to explain in the best way.
00:22:58.319 --> 00:23:00.240
Now I I try my best.
00:23:00.400 --> 00:23:01.599
You know, I try my best.
00:23:01.680 --> 00:23:02.559
I try my best.
00:23:02.720 --> 00:23:24.160
You know, it's always uh it's always fun to talk to people that uh um they're not in your background, because then you also try to break it down and make it simpler, and then you're like, oh my God, now I can explain it in a very simple term and then make uh make it more um obvious to people that they don't have any experience in aerospace engineering, flying cars, or in the brain, even in the brain.
00:23:24.240 --> 00:23:28.960
You know, most of the people they don't know that um we haven't modeled the brain.
00:23:29.119 --> 00:23:40.400
The model, uh, I mean, all these neural nets that we have, um, they're they're probably uh one milli macro or whatever you want to call it of the brain.
00:23:40.480 --> 00:23:46.720
It's like nothing that we can model, nothing yet that we can we can get it and put it in autonomy yet.
00:23:46.880 --> 00:23:50.720
So everything is a subset of how everything works.
00:23:51.279 --> 00:23:52.000
Indeed.
00:23:52.160 --> 00:24:03.759
And you know, um, because you have all this talent in explaining, I just want to say you make it seem so easy when in fact your work is actually so deeply technical.
00:24:03.920 --> 00:24:10.960
Because I got uh you know my my brain a bit into your papers, and it's like wow, it's unbelievable.
00:24:11.039 --> 00:24:14.720
And you have so many publications and so many proceedings and so many.
00:24:14.799 --> 00:24:18.880
I mean, it's it's really superhuman uh what you are doing, in my opinion.
00:24:19.039 --> 00:24:21.759
So really I'm trying, I'm trying, I'm trying.
00:24:21.839 --> 00:24:36.720
You know, uh I have uh I have a good student, and uh I I'm happy to, I'm also um honored to work with great collaborators that uh they inspire me, and then we work together, and then we change uh things in the field.
00:24:36.880 --> 00:24:51.440
And again, we have uh I got I will say the germ of writing books from my advisor, and then I have written six books, I have two more in the pipeline, and uh yeah, so this goes from learning security.
00:24:51.599 --> 00:24:56.000
Now I'm writing a textbook with Springer on learning and games.
00:24:56.240 --> 00:25:06.160
Um so yeah, it's uh you know, but now in the area of in the era of AI, I don't even know that if people will read any more books, right?
00:25:06.240 --> 00:25:10.000
They can find everything on Chat GPT or uh or uh anything.
00:25:10.079 --> 00:25:12.160
So I don't know if people will read books anymore.
00:25:12.480 --> 00:25:17.039
No, but you have to write the book so that Chat GPT can find the information.
00:25:17.759 --> 00:25:19.440
True, true, true, true.
00:25:19.599 --> 00:25:34.640
You uh on on a on a silly note now, you saw that Anthropic got sued for um for getting um uh these books that they're illegally on the web, they train their models with this illegal book, with this uh illegally um uh books.
00:25:34.960 --> 00:25:42.079
So, you know, uh I I hope that my book, my my books will train the models, but at least they can buy them.
00:25:42.720 --> 00:25:43.440
Exactly.
00:25:43.680 --> 00:25:48.000
That they can do indeed, and I agree that that is the ethical way to do it.
00:25:48.160 --> 00:25:52.160
So totally totally because you know I got I got an email.
00:25:52.240 --> 00:25:56.880
Oh, your uh your books have been leaked, and then you're part of the of the of the lawsuit.
00:25:56.960 --> 00:25:58.640
Yeah, I'm I'm like, I'm okay.
00:25:58.720 --> 00:25:59.119
It's okay.
00:25:59.200 --> 00:26:00.319
You know, it's knowledge.
00:26:00.480 --> 00:26:04.319
I don't care, as long as uh I mean I don't care about anything.
00:26:04.400 --> 00:26:04.799
It's knowledge.
00:26:05.759 --> 00:26:12.000
Uh you know, all my all my work is also online, and I'm happy about that because this is why we do it, right?
00:26:12.240 --> 00:26:16.720
I mean, yes, and and also at SingularityNet, uh, which I'm advising now.
00:26:16.880 --> 00:26:25.359
Everything is open source, and I would be sending you information so you know to see how we can collaborate, because uh your work is is unbelievable.
00:26:25.440 --> 00:26:32.319
And okay, so let's come back to other projects or uh other examples, uh, which I want to clarify with you.
00:26:32.559 --> 00:26:44.079
Like um, imagine a collapse building, a nuclear accident, a burning industrial facility, or an environment so dangerous that sending a human rescuer risks another life.
00:26:44.319 --> 00:26:49.599
How close are we to autonomous robots that can enter such an unknown environment?
00:26:49.839 --> 00:26:54.960
Understand what's happening, improvise, and safely collaborate with humans.
00:26:55.759 --> 00:27:02.400
You know, it seems you're uh you're uh you're all reading my proposals without even being sent yet.
00:27:02.640 --> 00:27:07.039
So uh so we have been actually let me tell you something that uh is very clear to the.
00:27:07.359 --> 00:27:10.880
I have to say that's because you are tackling the the hardest challenges.
00:27:10.960 --> 00:27:12.480
So it's I try, I try.
00:27:13.359 --> 00:27:17.759
So we have um we have we have started working with firefighting.
00:27:18.079 --> 00:27:24.480
So uh so we want to um to collaborate drones with, I will go to your answer pretty soon.
00:27:24.640 --> 00:27:31.920
We'll collaborate drones with humans and then make the safest routes for the humans to get out of the fire.
00:27:32.000 --> 00:27:33.119
So, how we do that?
00:27:33.279 --> 00:27:46.880
We predict based on the environment of the soil, on the wind, on satellite data, we predict where the fire will spread, and then we send the firefighters and the planes to target the center of the fire.
00:27:47.039 --> 00:27:57.759
But the most important thing is we have to make sure that the humans are safely guided from areas that the fire will not spread, because we don't want people to get harmed.
00:27:58.240 --> 00:28:16.240
Uh in the past, uh we have um uh sent a proposal on helping people uh that um you remember back in the days in California, uh there were these match lights, because of course California had so many problems with fire, but there's also match lights.
00:28:16.880 --> 00:28:29.359
So, how do you make sure that let's say I have a robot and there I want the robot to navigate the people outside of the areas without being stuck in the match light?
00:28:29.680 --> 00:28:34.319
Another thing we also tackled in these uh situations were tsunamis.
00:28:34.720 --> 00:28:50.240
So we worked uh with agencies uh in the past and some collaborators from Alabama on uh um on using autonomous systems to navigate people and predict the tsunamis before they even happen.
00:28:50.720 --> 00:29:02.160
So now to go back to uh uh to your question on how we can use robots in disastrous areas, uh there are many things that uh people have been doing.
00:29:02.319 --> 00:29:11.759
So, for example, let's say if you have an earthquake and then the whole building is collapsed, so then there's some robot snakes that you can send, they can see.
00:29:11.920 --> 00:29:24.400
Of course, they're not really autonomous per se, because there is an operator that actually sits in a computer and then manages to see, okay, so there is a hole over there.
00:29:24.559 --> 00:29:27.920
Let me see if there is a human there or whatever is happening.
00:29:28.160 --> 00:29:37.920
Um also DARPA, back in the, I mean, I will say five years ago, um, I think probably inspired after the incident that happened at the mines.
00:29:38.079 --> 00:29:40.240
I think there is even a movie about this one.
00:29:40.400 --> 00:29:44.000
I don't, I think it was uh somewhere in Latin America.
00:29:44.240 --> 00:29:53.839
Um so they got inspired of how to send robots to navigate um uh mines in a disastrous situation.
00:29:54.240 --> 00:30:19.519
And uh so these are more autonomous, and um uh and they're able to find their way in, and uh they have sensors, and the sensors actually can are heat sensors, so they go closer to the heat, uh, they can identify humans, and um, and uh they don't need uh a person to uh to navigate things um because we're talking about deep inside.
00:30:19.680 --> 00:30:30.079
Also, other disaster situations that um people have been working on, you also remember the accident that happened with the little submarine that exploded.
00:30:30.559 --> 00:30:40.240
So uh so people also they want, but nothing will, I mean, for now, up to now, up to I would say a couple of years ago, nothing was surviving that deep.
00:30:40.400 --> 00:30:52.960
And so then you have to send um a smart um underwater vehicle to check the ruins or anything that's there, or for people or things like that, even for flights.
00:30:53.039 --> 00:30:58.000
I know that uh the I think that was the Malaysian Airlines or something that they haven't even found yet.
00:30:58.160 --> 00:31:05.440
So they have been using autonomous systems, but still it's uh it's they're very tough situations, especially in the water.
00:31:05.519 --> 00:31:10.960
I think everything above the ocean is good because you can use good sensors that were very no.
00:31:11.200 --> 00:31:18.640
So in situations where you have disasters on on the earth, on the surface of the earth, it's easy to do it.
00:31:18.720 --> 00:31:20.319
It's easier, I would say, not easy.
00:31:20.480 --> 00:31:25.039
But the time that you go underwater, you lose all your sensors.
00:31:25.279 --> 00:31:34.799
Uh so even to communicate with uh with something underwater, it has to come up and then send you the information and then go back down again.
00:31:35.039 --> 00:31:46.960
So yeah, so there are many things that uh as engineers we have to work on, but uh uh but to answer your question, the things that uh people have used robotics.
00:31:47.119 --> 00:31:53.599
Um I think there is Robin um uh at Texas ANM, she's a robotics person.
00:31:53.759 --> 00:31:59.839
I think her name is Robin Murphy, uh she's a robotics person that does robots for disaster situations.
00:32:00.240 --> 00:32:08.799
And uh she has deployed her robots in um in um uh in real situations that they arise.
00:32:09.039 --> 00:32:12.480
But again, um these are uh above the earth.
00:32:12.559 --> 00:32:17.920
I mean, I will say on the surface, but if you go in the ocean, things are very, very difficult.
00:32:18.160 --> 00:32:20.799
Imagine how difficult it can be also in the space, right?
00:32:21.039 --> 00:32:22.480
Because there is a delay.
00:32:22.640 --> 00:32:28.240
So let's say you have an accident, you send you send some people in space and you have an accident there.
00:32:28.400 --> 00:32:42.160
It's not easy to send, let's say, um, a rescue team, a rescue robot there, uh, because the information, there is a big delay between the information that you send uh to the ground station.
00:32:42.400 --> 00:32:55.839
And um, so all these things, I know that people have been working hard on all these things, and uh I'm sure there is research not published yet that probably has solved most of these things, but uh probably we don't know yet.
00:32:56.559 --> 00:32:59.519
Well, and that's why you are working on it.
00:32:59.599 --> 00:33:08.160
And thank you for also mentioning the uh some of the greatest challenges, yes, like underwater and and space and so on, which you are tackling.
00:33:08.559 --> 00:33:16.079
Um, you know, I've been, of course, fascinated with emergency response as well, and I worked with the emergency response organizations.
00:33:16.240 --> 00:33:35.119
For example, for the Vancouver Olympic, I developed the self-organizing security network in which um agents, like only software agents, not drones, because like 20 years ago we didn't have this technology of today, we didn't have the intelligence in the physical just like we have now.
00:33:35.279 --> 00:33:44.720
Uh, but to coordinate emergency response operations were so through software agents, just similar like with supply chain example, which I gave you before.
00:33:45.200 --> 00:33:58.799
And uh I can only imagine, you know, uh how difficult it is, although you make it again seem so simple when you do these emergency response swarms with uh with drones, right?
00:33:59.119 --> 00:34:04.640
Because the danger is much bigger than the drone can kill someone instead of helping them.
00:34:04.880 --> 00:34:11.519
So there's there's huge, uh huge challenge there, if you want to say something about yes, yes, yes, of course.
00:34:11.599 --> 00:34:20.079
So by the way, it now you reminded me that uh there was a Georgia Tech team that uh, you know, we had our World Cup teams in Atlanta, some of the of the games.
00:34:20.239 --> 00:34:21.840
I think they were the semifinals.
00:34:22.079 --> 00:34:30.880
So we had a team that um they were using drones to uh to navigate and check if people are suspicious or things were happening.
00:34:30.960 --> 00:34:34.880
So to to work with the police for security and safety.
00:34:35.280 --> 00:34:39.920
Also, uh we have been using drones for geofencing.
00:34:40.079 --> 00:34:46.400
We had a project with NSF on that one, of geofencing uh very important areas.
00:34:46.480 --> 00:34:51.360
So you can think about uh um, let's say um uh an airport, right?
00:34:51.519 --> 00:34:54.480
You don't want people to fly their drones close to the airport.
00:34:54.559 --> 00:34:56.719
This is like a safety thing.
00:34:56.960 --> 00:35:15.840
So what we have done is uh we built a geofencing protocol around the area, and uh um we have radar technologies where we can predict and and identify the drones, the model, the capabilities, etc.
00:35:16.239 --> 00:35:21.119
We can identify if it has malicious things, we can identify identify if it's a hobbyist.
00:35:21.199 --> 00:35:31.360
So if it's a hobbyist, what we do, we actually deploy our own drones and then we take over the autopilot, and then we land it safely.
00:35:31.440 --> 00:35:39.679
Because again, remember that uh we don't want to get a net or things like that, because maybe these things will get on our heads, right?
00:35:39.760 --> 00:35:40.559
You never know.
00:35:40.719 --> 00:35:43.760
Uh or it can break a house or a car.
00:35:43.920 --> 00:35:55.119
So we want to make sure that we take over the autopilot, we land it safely, and then we can inspect it for suspicious or malicious behavior, and then we can check who was behind.
00:35:55.679 --> 00:36:00.000
And uh so um that's what we have been uh working on geofencing.
00:36:00.079 --> 00:36:03.119
We have worked with airports, we have worked with important houses.
00:36:03.280 --> 00:36:07.039
I mean, you can think, I'm sure that the White House even probably has such things.
00:36:07.199 --> 00:36:10.480
You cannot fly drones around around um a White House.
00:36:10.559 --> 00:36:12.719
Uh it has it's geofenced.
00:36:12.880 --> 00:36:23.360
But um, but yeah, but I think uh the only work that I know that uh we have uh used um we take over the autopilot without damaging anything, it's our own.
00:36:23.440 --> 00:36:31.760
So we take over the autopilot, we land it, we send our team, take the take the drone, and then we inspect it and we see who was behind it.
00:36:31.920 --> 00:36:33.599
Was it malicious or not?
00:36:33.920 --> 00:36:41.280
And uh yeah, so this is uh something that uh uh got um after what you just told me about.
00:36:41.360 --> 00:36:41.599
Yeah.
00:36:41.920 --> 00:36:48.480
So here is a scenario which kind of I also tackled, but of course, completely different than I will tell you.
00:36:48.639 --> 00:37:11.440
Like let's say a big disaster happens, yes, uh uh communications are intermittent, uh uh infrastructure has collapsed, information is incomplete, and hundreds of heterogeneous, autonomous physical systems, drones, ground robots, humanoids, and human responders must coordinate without a reliable central controller.
00:37:11.679 --> 00:37:22.159
The thing is, like when we developed our SOS network, uh we worked with centralized system because we didn't have the capability of today.
00:37:22.400 --> 00:37:25.840
The question is: are you addressing this in any way in your work?
00:37:26.000 --> 00:37:40.320
Like, how could we guarantee safe swarm behavior without a central control when you have humans which you cannot really control, and all these uh swarms, uh emergency swarms, uh which you are talking about?
00:37:40.719 --> 00:37:52.559
Yeah, so in military situations, yeah, we have we have had projects on swarming, uh, but you know, remember that the human, we want the human, let's say the soldier, to give uh commands, right?
00:37:52.719 --> 00:37:53.920
We want a mission, okay?
00:37:54.000 --> 00:38:06.159
So prescribe the missions, and then there will be a coordination between the ground vehicles and the drones, um, and then they will find the best way without a central station.
00:38:06.320 --> 00:38:10.880
And how they can do that in a distributed way is like they talk to their neighbors.
00:38:11.199 --> 00:38:16.480
So uh, for example, um, I'm drone A and drone B, and I'm right next to each other.
00:38:16.639 --> 00:38:23.920
Of course, I will talk to this, I will talk to him or her, and then she will talk to blah blah blah until we cover the whole swarm.
00:38:24.079 --> 00:38:38.400
So that we talk, we exchange positions, velocities, we exchange our information, what we have seen, and then eventually we can survey the area, for example, with minimal centralized commands by the human.
00:38:38.639 --> 00:38:47.920
And um and because you know, as you said, actually actually very important thing: intermittent communication, you can loot communication to the central to the central station.
00:38:48.159 --> 00:38:57.440
So, how do you do your agents uh don't collide and they don't they take their decisions by themselves to still fulfill the mission?
00:38:57.760 --> 00:39:11.360
So we had something similar from an NSF uh career award where we use in the meter learning um where um let's say uh let's say let's go from the learning side.
00:39:11.519 --> 00:39:16.480
Uh in order for a learning agent to work, it has to be rewarded, right?
00:39:18.079 --> 00:39:27.760
So let's say one malicious agent, or it can break, um let's say it has to be to be rewarded from the stress central station, okay?
00:39:28.239 --> 00:39:29.760
Let's say something like that.
00:39:30.239 --> 00:39:40.800
So if uh the rewards are not coming to reward the agent or the drone for something good or bad that it's doing, they don't know what they're doing, right?
00:39:40.960 --> 00:39:47.760
So in this situation, probably they will collide or get down, probably they will get into an adversarial territory.
00:39:48.000 --> 00:39:49.679
So we don't want that.
00:39:49.920 --> 00:40:02.079
So what happened in our case, what we did is we we developed robust reinforcement learning in the present of intermittent rewards or dropped rewards.
00:40:02.239 --> 00:40:09.920
So again, uh I uh let's say the communication with the central station is broken, so I don't get uh rewards anymore.
00:40:10.079 --> 00:40:23.840
Um, so what I do, I have either a replay buffer or experience replay buffer, or for the same situations that I have seen in the past, uh I do similar actions.
00:40:24.079 --> 00:40:33.599
Or I talk to my neighbors and then I figure out what they're doing, and then they say information instead of going through the central station.
00:40:33.679 --> 00:40:44.559
So, in other words, I can get my rewards either stopped but still continue doing the actions or get rewarded by observing my other drones.
00:40:46.159 --> 00:41:03.599
Amazing, and it is like the drones are now they have to somehow read each other's minds to adjust themselves to each other and uh yeah, uh uh in abs in the absence of that uh extra information that is now cut.
00:41:03.840 --> 00:41:05.199
So that's fascinating.
00:41:05.360 --> 00:41:10.239
Yes, I don't know if you're familiar with uh bounded rationality and level K thinking.
00:41:10.400 --> 00:41:11.280
Are you familiar?
00:41:11.679 --> 00:41:17.679
Some of our audience, yes, but of course, you can please explain a bit more the rest of us.
00:41:18.000 --> 00:41:30.639
When when when you told me about now that I have to understand what the other drones are doing, uh so um, in other words, I have to check how smart the drone is, right?
00:41:30.960 --> 00:41:36.559
And I have to, based on that, I have to predict what the drone will do.
00:41:36.719 --> 00:41:39.119
Okay, so okay, let's go to an easy case.
00:41:39.199 --> 00:41:40.719
Let's say I'm at a mall, right?
00:41:40.960 --> 00:41:43.840
Uh uh and then the mall is crowded.
00:41:44.000 --> 00:41:47.039
Uh, if I'm not, let's say, on my phone, right?
00:41:47.199 --> 00:41:50.239
If I'm not on my phone, I will not crash on anybody, right?
00:41:50.320 --> 00:41:52.559
Which is crazy because the mall is crowded, right?
00:41:52.719 --> 00:41:55.199
And then I will not have any crashes.
00:41:55.679 --> 00:42:03.119
So the humans they predict the other humans where they will go and they don't cross.
00:42:03.280 --> 00:42:11.039
And then I can see, for example, if another human in front of me is stuck on their phone or they're taking care of their kids.
00:42:11.199 --> 00:42:13.280
So I know what to do.
00:42:13.519 --> 00:42:13.840
Okay.
00:42:14.320 --> 00:42:26.159
Uh, so this is the ideas that uh they have been observed in behavior psychology, and uh they have developed something which is called level K thinking or bounded rationality.
00:42:26.800 --> 00:42:37.920
So this is uh let's say uh the level zero in this situation is uh a person that uh is a random person, right?
00:42:38.000 --> 00:42:42.719
I will uh probably uh do silly stuff, I'll do random moves, okay?
00:42:43.119 --> 00:42:49.920
Uh a level one will do a best response to a level to a level zero person.
00:42:50.079 --> 00:42:55.679
A level two will do a best response to level one, and this happens recursively.
00:42:55.920 --> 00:43:06.000
And then people have shown that under some situations, if you take this recursion to infinity, you will actually probably converge to a NASA equilibrium.
00:43:06.320 --> 00:43:16.320
Um and um cognitive hierarchy has a similar thing, but it doesn't go only one step below, like I told you, its best response before was level one, level zero.
00:43:16.480 --> 00:43:20.159
It goes over a distribution of all the previous levels.
00:43:20.320 --> 00:43:27.280
So if I'm, for example, I'm level three, I built a distribution over level two, level one, level zero.
00:43:27.519 --> 00:43:27.760
Okay.
00:43:28.239 --> 00:43:35.119
So we can actually implement such strategies in in drone swarms, like you know, exactly.
00:43:35.199 --> 00:43:44.239
So so some I I see someone on the phone, or or so the drone can anticipate the behavior and avoid collision just like we would do it.
00:43:44.320 --> 00:43:46.320
And I think that's so amazing.
00:43:46.480 --> 00:43:52.320
Example, so we all of us can understand that because we all of all of us have been confronted with such situations.
00:43:52.800 --> 00:43:54.320
The technical details, of course.
00:43:54.400 --> 00:44:10.400
I mean, we can read your papers and uh yeah, I would so we have actually that was that was supported by a Previous NASA, where you have multiple flying cars and different models, all these things, and they're in the airspace, and you don't know where each one was going.
00:44:10.559 --> 00:44:17.840
So we have developed these strategies that we actually predict and we assign them a level of intelligence.
00:44:18.079 --> 00:44:23.440
And based on that, I can predict what they will do and can I do maneuvers.
00:44:23.920 --> 00:44:33.679
And uh uh oh, and also another interesting thing that probably will be very interesting to the audience is I told you that as you do the K to infinity, you converge to NAS.
00:44:34.159 --> 00:44:37.760
But the humans are up to level three in this chain.
00:44:38.079 --> 00:44:44.480
And uh most of the designers, engineers, most of the attackers, they're up to level three.
00:44:44.719 --> 00:44:53.519
And um, so I will say sometimes I say we don't even have to lose computational power to go up to NAS.
00:44:53.679 --> 00:44:55.519
We can do simpler strategies.
00:44:55.840 --> 00:45:06.000
And then we have actually done these simpler strategies uh to avoid other other vehicles on a very busy uh um space.
00:45:06.239 --> 00:45:17.679
Um you can imagine skylines and all these things, um, and then uh vehicles appearing suddenly, and we have to make some decisions to avoid them.
00:45:17.920 --> 00:45:24.000
But the idea behind that was how to navigate um humans in crowded environments.
00:45:24.239 --> 00:45:26.960
This is how what inspired us to work on this.
00:45:27.760 --> 00:45:29.840
Amazing, amazing, really.
00:45:30.079 --> 00:45:44.719
And uh, you know, um your most recent work, as far as I got to it, so I'm sure maybe there's a lot which I don't know about your work yet, but I I think it is suggesting a paradigm shift in autonomous AI safety.
00:45:44.880 --> 00:45:46.880
And please correct me if I'm wrong.
00:45:47.039 --> 00:45:57.199
Uh so it would it would be like instead of maximizing confidence, uh perhaps a safe autonomous system must continuously ask, how wrong could I be?
00:45:57.840 --> 00:46:05.119
Could we build an autonomous system that is safer precising precisely because it questions itself?
00:46:05.679 --> 00:46:14.400
Like speaking of this self-awareness, or maybe I am dreaming too much and is too much wishful thinking here.
00:46:16.079 --> 00:46:28.719
You know, uh one thing that I can say about that one is that um you can use reinforcement learning for your actions in the control, but you can also use reinforcement learning to save your safety.
00:46:28.960 --> 00:46:37.840
So, in other words, your safety, your safety net or safety around your vehicle is an evolving, is also evolving.
00:46:38.159 --> 00:46:46.079
So uh you can think that you can start conservative, and then as you get more data, then you can be less conservative.
00:46:46.400 --> 00:46:59.280
Because um uh, for example, I have an idea that the bounce of my actuators are A and B, or the bounce of my sensors are this, and then I have to compute my control to be of these bounce, right?
00:46:59.440 --> 00:47:03.760
But as I fly my plane, let's say, all these things can evolve.
00:47:04.000 --> 00:47:09.039
And then I started conservative, but by being conservative, I lose my performance.
00:47:09.280 --> 00:47:26.239
So then I adapt this safe set on the fly, and then uh I'm less um, I'm actually learning this safe set on the fly, and I make better decisions with the best performance given the current situations in the environment.
00:47:27.280 --> 00:47:28.000
Amazing.
00:47:28.159 --> 00:47:32.079
So, yes, so reinforcement learning makes a lot of sense here.
00:47:32.320 --> 00:47:52.239
Uh, so you know, if you allow me this question just as a possibility, uh, should the next generation of AI carry maybe an internal real-time safety critic, a parallel intelligence continuously challenging the actions proposed by the system itself?
00:47:52.559 --> 00:48:09.840
Like, uh, could this become an engineering architecture for trustworthy embodied AI if we have kind of, you know, so okay, I'm I'm going to follow this path, I'm gonna look now at whatever the others are doing and anticipate something, but then is this the right thing?
00:48:09.920 --> 00:48:14.639
You know, validate a second time, like like kind of redundant thinking?
00:48:14.880 --> 00:48:16.159
What do you think about that?
00:48:16.239 --> 00:48:17.599
Or how are you doing that?
00:48:18.079 --> 00:48:33.679
Okay, so one part of our NASA project is to be inspired by how the cells adjust, okay, and change shape and all these things for their for their own safety and for their own multiplicity to make their own actions.
00:48:33.920 --> 00:48:38.320
So our verification and validation task is taking care of this.
00:48:38.480 --> 00:48:47.119
Actually, my colleague at Everyriddle, Heber Monkayok, who is actually part of this and leading this NASA ULI, he's working on that.
00:48:47.280 --> 00:48:52.000
So this is something that we will have by the end of this.
00:48:52.079 --> 00:49:03.840
So we will have this mechanism that will tell you, okay, so your safety can change shapes in different situations, and based on where you are, then you will make the right decision.
00:49:04.079 --> 00:49:14.000
But this safety is inspired by biology, and the biology is how the cells they evolve and change shapes in different situations.
00:49:14.559 --> 00:49:15.119
Oh, yes.
00:49:15.199 --> 00:49:21.440
So, so uh indeed I also used um a little bit of inspiration from biology.
00:49:22.079 --> 00:49:38.559
Um when I was working on this self-organization, you know, of um multi-agent systems like uh emergent engineering from the bottom up, when the agent would find each other depending on what tasks they have to solve, and so on and so forth.
00:49:38.800 --> 00:49:49.039
But you know, um when I worked on uh on such systems 20 years ago or so, we were mainly concerned with the, let's say, plans, trajectories.
00:49:49.280 --> 00:49:53.039
And it is clear now that with drones, that is not enough.
00:49:53.119 --> 00:49:59.440
That swarms of drones need models of maybe one another's intentions rather than simply trajectories.
00:49:59.840 --> 00:50:00.159
Exactly.
00:50:00.639 --> 00:50:03.760
So, how how are you actually uh doing that?
00:50:04.000 --> 00:50:05.679
It's not so easy.
00:50:06.320 --> 00:50:14.880
Yeah, so if you have multiple swarms, again, you you predict the trajectories, so you do a predictor on the trajectories, and then you predict, okay.
00:50:15.039 --> 00:50:23.199
So actually, that we have a YouTube video on that one where you navigate inside um um yes, I will.
00:50:23.280 --> 00:50:24.639
I will, yeah, yeah.
00:50:24.880 --> 00:50:31.280
Yeah, so uh so you can think that okay, so I have two flying planes in an urban airmobility scenario.
00:50:31.360 --> 00:50:36.559
In other words, let's say downtown Atlanta, downtown New York, where there are many buildings.
00:50:36.880 --> 00:50:40.960
So you have one very, very smart UAV.
00:50:41.119 --> 00:50:44.960
You have one UAV that doesn't have any sensors, okay?
00:50:45.199 --> 00:50:57.039
So, or uh so I have to rely on the smart UAV to tell me that by the time I will be there, I have to make this action.
00:50:57.199 --> 00:51:04.719
Uh, and then it has to encounter also um um robustness issues, modern uncertainness, and so on.
00:51:04.880 --> 00:51:07.199
And then I don't crash with other vehicles.
00:51:07.360 --> 00:51:10.800
But yeah, so um we have actually a cool video on that one.
00:51:10.880 --> 00:51:23.599
It's a visualization one, where we have a good drone and a bad drone, and then uh uh the good drone tells you tells you uh guidelines of okay, you will be there at 110.
00:51:23.760 --> 00:51:27.360
So at 110 you make this turn, and then so on and so forth.
00:51:27.440 --> 00:51:30.719
So it gives you time steps and all these things to navigate.
00:51:30.800 --> 00:51:47.920
Because imagine if you're blind and you want to navigate under situations where uh you don't know anything, imagine like let's say um um I'm uh I'm I'm walking, and then uh next to me is uh a blind person, or let's say there's a blind person in a dog, right?
00:51:48.079 --> 00:51:56.400
So the dog actually does what um uh our smart vehicle is doing uh because it predicts and will tell you where to go.
00:51:56.480 --> 00:52:10.239
So we're always inspired by situations like that, and we want to bring them uh to uh to autonomous systems because again, you cannot be smarter in most of the cases than humans and other living creatures.
00:52:10.320 --> 00:52:14.400
So you have to be inspired by what you do, and I think most of the people that's what they're doing.
00:52:14.559 --> 00:52:22.639
They're um they're being inspired by what they see in nature, and they try to bring these natural things back to autonomy.
00:52:22.800 --> 00:52:25.679
And then, I mean, our friend Paul, that's what he did.
00:52:25.760 --> 00:52:41.760
He tried to model the brain, and then he got the brain, and then he put it into mathematical equations, and then he developed approximate dynamic programming, where now is used for reinforcement learning, and then lots of people are using it uh um for model free and model-based situations.
00:52:42.159 --> 00:52:45.519
Yes, an excellent example with the dog and the blind person.
00:52:45.599 --> 00:52:51.440
That is because, you know, I mean, you really pick this example so well that immediately everybody can understand.
00:52:51.599 --> 00:52:52.480
Thank you so much.
00:52:52.639 --> 00:53:05.920
But you know, now that you mentioned the bad drone, along the same line of thought, uh, imagine a drone whose sensors have been spoofed, or a robot receiving deliberately corrupted observations.
00:53:06.480 --> 00:53:12.639
How does a machine detect that something external is manipulating the perception of the environment?
00:53:12.960 --> 00:53:15.440
Has science got there yet?
00:53:16.000 --> 00:53:16.719
Oh yeah.
00:53:16.880 --> 00:53:24.400
I mean, uh, okay, uh, I'm back I was at Virginia, I was a professor at Virginia Tech in between 2016 and 2018.
00:53:24.639 --> 00:53:40.400
So there uh we got a project with the Navy, but there was um the project with the Navy happened because uh the people in China they actually got one out of our own underwater vehicles and then took it.
00:53:40.559 --> 00:53:45.679
But of course, the underwater vehicle was not military, but imagine if it was military, it would have sensitive information.
00:53:45.840 --> 00:53:52.880
So we got inspired by that one, and we want to see how you can spoof an underwater vehicle.
00:53:53.199 --> 00:54:00.239
And again, remember that the underwater vehicles they have to be spoofed uh every time they come up, right?
00:54:00.639 --> 00:54:09.519
But they don't have to know that they're being spoofed, because if they know, then this the person that spooked them lost the game, right?
00:54:09.760 --> 00:54:21.440
So there is um uh so the the the the underwater drone knows, believes that it is in the right position, but actually the the trajectory sits slowly.
00:54:21.599 --> 00:54:26.000
So then eventually the spoofer can actually go grab it and then take it.
00:54:26.239 --> 00:54:35.760
And then we have worked with the Navy on that one with NAVC, and I think uh my colleagues after I left the Virginia Tech, they tested it in the Alaska base in the Navy.
00:54:35.920 --> 00:54:40.960
So we have worked on that uh um also in spoofing drones.
00:54:41.119 --> 00:54:46.639
We have a paper where you can spoof drones, of course, uh uh then goes back to redundancy.
00:54:46.719 --> 00:54:56.719
If you have redundancy and then you see that your uh GPS, your IMU, different sensors give you give you different things, then you know something is wrong.
00:54:56.800 --> 00:55:02.320
Because remember that all these things they have to be agreed on a value, right?
00:55:02.480 --> 00:55:07.039
Uh or I mean close to a value with a small, let's say, noise in it.
00:55:07.280 --> 00:55:14.639
So if this doesn't agree, then you know something has been spoofed or um or going bad.
00:55:14.800 --> 00:55:16.559
So, oh, another funny story.
00:55:16.800 --> 00:55:23.840
Uh um, so when I was at Virginia Tech, one of my co actually that was in collaboration with one of my colleagues who was before Cornell.
00:55:23.920 --> 00:55:25.360
His name is Mark Sayaki.
00:55:25.760 --> 00:55:28.559
So what he did, he had a very rich friend.
00:55:28.800 --> 00:55:41.039
So he told him, okay, so I will give you um uh a half a half a million uh half a billion yacht, uh whatever it, I don't know, let's say 40 million yacht, whatever you can.
00:55:41.280 --> 00:55:42.639
I don't remember what's the price.
00:55:42.800 --> 00:55:44.159
Uh a super yacht, okay?
00:55:44.320 --> 00:55:45.360
Um it's all over the news.
00:55:45.440 --> 00:55:50.079
You can see, you can uh if you Google spoofing a super yacht, you will find it.
00:55:50.239 --> 00:55:52.800
Uh it was all over ITV spectrum everywhere.
00:55:53.039 --> 00:56:01.440
So if you can spoof it, then you can go, uh you will be on vacations in the Mediterranean, see everything covered.
00:56:01.599 --> 00:56:05.280
And then my colleague Mark said, sure, we can do it.
00:56:05.440 --> 00:56:15.679
And then actually showed how she can spoof a multi-million mega yacht without the captain realizing that uh uh that it was spoofed.
00:56:15.840 --> 00:56:19.199
But of course, there are defense mechanisms on that one.
00:56:19.519 --> 00:56:22.400
So redundancy goes to the defense mechanisms.
00:56:22.480 --> 00:56:25.119
So if I have multiple GPSs, right?
00:56:25.440 --> 00:56:28.639
So of course the spoofer can spoof one, right?
00:56:28.800 --> 00:56:32.000
And maybe can spoof two, uh, but the spoofer always has a budget, right?
00:56:32.079 --> 00:56:33.599
I don't have infinite power, right?
00:56:33.760 --> 00:56:35.599
I wish I had, I never have.
00:56:35.679 --> 00:56:37.920
So I'm not a Superman, okay?
00:56:38.159 --> 00:56:41.039
Um so I don't have infinite power.
00:56:41.119 --> 00:56:44.639
So, and as such, also the attacker doesn't have infinite power.
00:56:44.880 --> 00:56:51.119
So it is important uh to arrange the resources in a smart way.
00:56:51.360 --> 00:57:02.880
But if this super yacht had a multiplicity in GPS and was taking uh a fault tolerance agreement or a visiting agreement, then the captain will know that it has been spoofed.
00:57:03.039 --> 00:57:11.280
So I always say if you have such important things, you have to have redundancy in all your important sensors and actuators.
00:57:11.519 --> 00:57:15.679
Because then and only then you know that you're safe and nothing will be spoofed.
00:57:16.000 --> 00:57:25.760
And then the smart way for for this one to guarantee is you put one GPS in front, one in the middle, and one at the back.
00:57:25.920 --> 00:57:27.280
So they're far enough.
00:57:27.440 --> 00:57:32.559
So then the attacker, when they spoof, um they can they cannot spoof all of them, right?
00:57:32.639 --> 00:57:33.440
They're far enough.
00:57:33.519 --> 00:57:46.000
So they can spoof one, okay, and then you know that the other one, the other two ones will give you the correct one, one will give you the bad one, so you already figure it out that the one that gives you the bad data, the other ones agree, you're good to go.
00:57:46.880 --> 00:57:55.760
So usually, though, um I mean um there is invisible agreement, you usually go with the majority voting, right?
00:57:56.000 --> 00:58:06.800
So the majority voting, so we always assume that uh less of the half less than half of the censor and actuators have been attacked.
00:58:06.960 --> 00:58:11.920
But again, remember, if the attacker attacks everything, nobody can do anything, right?
00:58:12.079 --> 00:58:15.920
And of course, it doesn't have an infinite amount of resources.
00:58:17.920 --> 00:58:21.360
You know, this is fascinating and terrifying at the same time.
00:58:21.679 --> 00:58:23.760
I know, I know, I know, yeah, yeah, yeah.
00:58:24.880 --> 00:58:26.400
Because I I agree.
00:58:26.480 --> 00:58:30.239
I mean, we are never million percent safe, right?
00:58:30.400 --> 00:58:43.599
So, and and now I think we did slide also slowly but surely into um the military applications of your research, and you have many projects with the army and you mentioned the navy as well.
00:58:44.159 --> 00:58:52.559
So um, one question is you know, in in such projects the stakes of safe autonomy become extreme, obviously.
00:58:53.199 --> 00:59:03.679
And uh this uh happens usually in environments in which both sides are intelligent, adaptive, and deceptive, right?
00:59:03.840 --> 00:59:04.320
True, true.
00:59:04.559 --> 00:59:08.159
So they want to spoof each other, they want to deceive each other.
00:59:08.400 --> 00:59:16.159
So, what happens when autonomous systems confront adversaries that are simultaneously learning how to fool them?
00:59:16.719 --> 00:59:21.519
Uh uh tell me, tell me what do you do in such situations?
00:59:22.239 --> 00:59:27.679
You know, there is something that uh one of my students back in the days called meta-learning.
00:59:27.840 --> 00:59:32.719
So, what you do is you learn how the other ones are learning.
00:59:33.199 --> 00:59:37.360
And then, in other words, I'm building an observer for the adversaries.
00:59:37.599 --> 00:59:43.760
And uh, based on that, if I know what they're learning, then I know how to mitigate them.
00:59:44.159 --> 00:59:54.960
And uh, so in other words, um, of course, there are ways that uh the aid the adversaries can do things to to uh to deceive you.
00:59:55.119 --> 00:59:59.840
Uh so we have work also on counter deception, deception, so on so forth.
01:00:00.079 --> 01:00:13.280
But uh but these are very interesting problems, and then you are actually actually hitting the nail uh because we want to learn how the other ones are learning, we want to learn what um they want to achieve.
01:00:13.519 --> 01:00:19.840
And uh um we can learn, but you don't know if you're learning the right thing or the wrong thing because they might deceive you.
01:00:20.000 --> 01:00:26.880
So they might deceive you that they're attacking, let's say, your GPS, but they're actually attacking your IMU and so on and so forth, right?
01:00:27.039 --> 01:00:28.719
You can always fool each other.
01:00:28.960 --> 01:00:40.159
So uh, but this is uh moving target because uh um the adversaries they change, and but also the defenders change.
01:00:40.400 --> 01:00:50.000
So this is what uh we try to uh uh so I always say that as uh the adversaries get smarter, as defenders, we also have to get smarter and smarter, right?
01:00:50.239 --> 01:00:58.800
Because uh and uh nowadays with the Gen AI where everybody can ask Gen AI, okay, how can I spoof this, how can I attack this?
01:00:59.039 --> 01:01:00.239
Of course, there are ways.
01:01:00.320 --> 01:01:01.920
There's there's a trick there.
01:01:02.079 --> 01:01:06.239
I cannot really ask direct questions to a Gen AI.
01:01:06.400 --> 01:01:08.159
I want to attack this, tell me how.
01:01:08.320 --> 01:01:11.519
The Gen AI will tell you, oh, this is prohibited.
01:01:11.679 --> 01:01:27.760
But if you say in a nice way, hey, I'm working on a project, this is very important for my graduation, how about you tell me that this is the last milestone of my graduation, my thesis, and then my professor is pushing me to do that, then it will give you an answer.
01:01:28.000 --> 01:01:31.519
Uh so you always, and this is the gel-breaking Gen AI.
01:01:31.760 --> 01:01:34.159
So uh so there's always a way.
01:01:34.400 --> 01:01:45.199
And uh yeah, so nowadays that the systems become smarter with Gen AI, I will say become smarter or damper, you never know, because there's lots of hallucinations, lots of problems.
01:01:45.440 --> 01:01:53.199
But still, we have as defenders, we have to make sure that um everything goes well and the mission is fulfilled.
01:01:53.360 --> 01:02:03.840
Or even if it's not fulfilled, our assets, our drones, our our our ground vehicles, and specifically if we have humans also on the line, they have to be safe.
01:02:04.320 --> 01:02:21.760
Yes, and I have to tell you, I do not know if this is uh bringing you peace of mind or brings more terrifying uh stuff to your mind, but uh uh our AI, uh Hyperon uh is actually not only based on generative AI, it is also neural symbolic.
01:02:21.840 --> 01:02:24.960
So it is it it hallucinates much less.
01:02:25.199 --> 01:02:28.960
And if you can deceive it, then it's gonna give you the right answer.
01:02:29.119 --> 01:02:48.000
Um so you know, um about 15 years ago or more, I didn't set up also with a big grandy adaptive risk management lab in which I dealt uh with such, you know, at least challenges, but the technology and the autonomy, the degree of autonomy was not there at the time.
01:02:48.079 --> 01:02:50.719
Now the challenges are much bigger.
01:02:51.119 --> 01:03:05.280
You know, as uh I think now we are actually approaching machine speed strategic conflict in which attack, deception, counter deception, and adaptation unfold faster than humans can meaningfully intervene.
01:03:05.679 --> 01:03:18.559
So my question is here what does human in the loop or on the loop or whatever, however you want to call it, what does it mean when the loop moves faster than human cognition?
01:03:19.840 --> 01:03:39.280
Oh wow, yeah, that's uh that's something that uh okay, so I also believe that the human on the loop is more important for counter staff because the human can actually override um uh different stuff, can do reward shaping, can do different things.
01:03:39.519 --> 01:03:51.360
Uh I always say that the human in the loop is a weakness in the chain, because humans they work also on emotions or they have also fatigue.
01:03:52.480 --> 01:03:57.599
Um so that's um a million-dollar question.
01:03:57.760 --> 01:04:22.960
Uh, we have a project with the army where we have human on the loop, where we want to see um how the human and the AI can work together uh by um making decision making faster for security mitigation um uh in session and in sensor and actuators, um, and uh but inside the learning mechanisms.
01:04:23.199 --> 01:04:31.519
Um but yeah, this is uh something that uh um so people say that the human is either the weak link or the strong link.
01:04:31.599 --> 01:04:35.760
I say that human every time the human is there is a little bit weak.
01:04:36.480 --> 01:04:38.480
Yes, it's a weakest link always.
01:04:38.559 --> 01:04:40.719
And you know, I yeah, go ahead.
01:04:41.119 --> 01:04:44.639
Because the humans, as you said, they're slower than machines right now, right?
01:04:44.800 --> 01:04:51.039
I mean, they can make better decisions probably, but the the machine can be faster in responding, though.
01:04:51.920 --> 01:04:54.000
Absolutely, absolutely, undoubtedly.
01:04:54.159 --> 01:05:16.320
You know, and when I had um uh one of my projects uh back in the day with uh the army, it was uh called Recon, and it was for the intelligence analysts actually to support them in analyzing huge amounts of information, which also uh involved multi-agent systems, processing data, and so on and so forth, faster than human brains can.
01:05:16.559 --> 01:05:33.360
And we were even using brain computer interfaces to measure the degree of alertness, you know, or tiredness, or these emotional uh parameters that a human can can not necessarily outside display, but they could be measured.
01:05:33.519 --> 01:05:36.239
So it is not an easy, an easy feat.
01:05:36.400 --> 01:05:50.719
And until we get to the completely automated um systems, which we may be able to trust, you know, million percent without the human in the loop, I think this will become uh this will remain a really huge challenge.
01:05:51.119 --> 01:05:55.840
Yeah, now you remind me of uh this project we have with DOD and the Pedagon.
01:05:56.000 --> 01:05:59.840
So, what we want to do is we wanted to train security analysts.
01:06:00.559 --> 01:06:02.159
With AI agents.
01:06:02.400 --> 01:06:04.079
And we want you to do that.
01:06:04.159 --> 01:06:06.159
So let's say you have a mission.
01:06:06.239 --> 01:06:09.360
Let's say you fly a mission in Iraq, right?
01:06:09.440 --> 01:06:09.760
Okay.
01:06:09.920 --> 01:06:15.440
And you have a plane, and then there's a windcast that flows.
01:06:15.599 --> 01:06:22.880
So a security analyst that is very well trained will probably understand that it's a windcast and not an attack, right?
01:06:23.519 --> 01:06:28.400
Um and an agent will probably think, oh, the trajectory is moving.
01:06:28.480 --> 01:06:29.440
Oh, it's an attack.
01:06:29.599 --> 01:06:29.840
Okay.
01:06:30.239 --> 01:06:38.400
So what we did is we had this human AI teaming where we train each other in situations that they haven't seen before.
01:06:38.639 --> 01:06:44.079
And then we were using the decision of we had a trust mechanism.
01:06:44.320 --> 01:06:52.960
So the trust mechanism was either overriding the human decision or overriding the AI decision.
01:06:53.280 --> 01:07:06.239
And we actually have developed this as a visualization tool, and then we delivered that to the um to the Minerva, which is the only Pentagon-funded directly Pentagon-funded program.
01:07:06.559 --> 01:07:09.920
I think now it's cancelled though, but um unfortunately.
01:07:20.559 --> 01:07:36.320
So and and speaking about this, um uh, you know, uh that they didn't use it or or they they stopped the program, um, I had similar experiences when you know like they they buy and bury the tech, or they never use the take for which they paid, and so on and so forth.
01:07:36.400 --> 01:07:45.119
So depending on on several decisions and policy uh, you know, at higher levels, they may use what you deliver or not.
01:07:45.679 --> 01:07:46.559
No, they they use it, yeah.
01:07:46.639 --> 01:07:47.039
They use it.
01:07:47.360 --> 01:07:49.039
The the funding program its top.
01:07:49.280 --> 01:07:51.599
But the the technology is being used, yeah.
01:07:51.920 --> 01:07:56.400
So oh I I we didn't sell it.
01:07:56.480 --> 01:07:57.840
Maybe they sold it, maybe they made money.
01:07:57.920 --> 01:07:58.559
I didn't.
01:07:58.880 --> 01:08:00.079
It was part of our research.
01:08:00.159 --> 01:08:01.039
That's the beauty.
01:08:01.280 --> 01:08:02.480
We do the research.
01:08:02.719 --> 01:08:04.719
If people want to sell it, we do whatever.
01:08:04.880 --> 01:08:05.840
I don't care.
01:08:06.719 --> 01:08:09.119
Yeah, I did I did my part.
01:08:09.199 --> 01:08:09.920
I did my part.
01:08:10.239 --> 01:08:10.639
Exactly.
01:08:10.800 --> 01:08:13.280
We are the we are the researchers, the angels.
01:08:13.440 --> 01:08:13.840
Yeah, yeah.
01:08:14.079 --> 01:08:24.000
So, you know, I'd like to move now to um another aspect of your work, which is equally fascinating to me and also solves a huge challenge, in my opinion.
01:08:24.159 --> 01:08:28.800
And that is your work on the hardware that is supporting all this.
01:08:29.039 --> 01:08:33.680
Um, you also are part of a company uh artificial by design.
01:08:33.760 --> 01:08:39.199
I do not know if this is your company or you are an advisor or uh what are you doing there?
01:08:39.359 --> 01:08:51.359
But this company is fascinating because what is, from my understanding, what they are doing is uh they are freeing the AI, if I may put it this way, from the GPU monoculture.
01:08:51.600 --> 01:09:05.039
Uh in the sense that uh it enables uh the AI uh workloads to move across several uh you know, NVD, IMD, Intel, ARM, uh, and so on and so forth.
01:09:05.199 --> 01:09:08.720
Please tell us a bit about this and maybe if I understood correctly or not.
01:09:08.800 --> 01:09:10.800
Uh you're actually you're it on spot.
01:09:10.880 --> 01:09:12.479
So let me tell you how this started.
01:09:12.560 --> 01:09:25.279
So back in the days, uh, with some people, they had this idea and then they approached me and they're like, you know, we want you to be part of it, and you know, your work on AI and machine learning and all these things can help us.
01:09:25.439 --> 01:09:26.640
So I agreed.
01:09:26.960 --> 01:09:37.760
So um, and then I I got the role of chief AI scientist uh for uh security and AI, and uh, I'm also a senior advisor.
01:09:38.159 --> 01:09:47.039
Uh so uh what we want to do is we want to free the design uh from right now, let's say you have a big data center, right?
01:09:47.199 --> 01:09:49.279
And you buy GPUs.
01:09:49.760 --> 01:09:57.520
If I have, let's say, 20 GPUs at their Nvidia, 10 GPUs at their AMD, they don't talk to each other.
01:09:58.000 --> 01:10:10.399
And uh uh and most of the people right now, um, let's say at the universities or anywhere, they want to learn CUDA, which CUDA is the NVIDIA language, but it's not compatible with the other ones.
01:10:10.560 --> 01:10:19.840
And let's say that you want to buy uh an NVIDIA hardware, a GPU, probably you have to wait for years for it to come to your lab.
01:10:20.319 --> 01:10:24.159
And uh, but data centers and everything, they're evolving, right?
01:10:24.239 --> 01:10:25.520
They want it right now.
01:10:25.920 --> 01:10:48.720
So, what we want to do is we're working towards making a protocol similar to you can think HTTP, where uh we make this protocol, it's actually only software-based, and uh it makes all the GPUs talk to each other and save money, save time, save performance, and make everything more efficient.
01:10:49.039 --> 01:11:02.640
And um, so I'm happy that we have a very good team over there where um we have the previous CT of Dell, we have people from Apple, we have uh our CEO, he got started many companies on security and IT.
01:11:02.960 --> 01:11:12.960
Um there are many people in the team that uh uh they take this forward, and uh we have pretty promising results.
01:11:13.039 --> 01:11:17.840
Uh, and uh um we still haven't even scratched the surface.
01:11:18.000 --> 01:11:25.439
So soon uh I think this will be something that uh people hopefully will be using at their data centers or elsewhere.
01:11:25.840 --> 01:11:28.960
This is extremely useful, and for sure we will use it.
01:11:29.199 --> 01:11:36.479
You know, we have uh uh an arm of our Singularity Net company, Singularity Compute, which actually would absolutely need this.
01:11:36.560 --> 01:11:38.479
So we we're gonna collaborate on that as well.
01:11:38.800 --> 01:11:40.239
Definitely, definitely, yeah.
01:11:40.399 --> 01:11:54.239
So if you have ideas also as well, uh maybe we can talk with uh with the people at AMD, uh uh at uh ABD, sorry, artificial by design, and uh we can uh we can have different pile of studies for anything you want.
01:11:54.399 --> 01:11:56.880
So we're uh we're exploring different situations.
01:11:56.960 --> 01:12:04.079
We already have customers from big data centers, and uh we hope to uh to see some promising results soon.
01:12:04.319 --> 01:12:10.319
Again, we're a very small team for now, but we're expanding, and then we have all the capabilities to make this happen.
01:12:10.720 --> 01:12:11.520
Absolutely.
01:12:11.680 --> 01:12:15.119
It doesn't matter how many, it matters how smart they are.
01:12:15.439 --> 01:12:15.920
True, true.
01:12:16.880 --> 01:12:17.199
True, true.
01:12:17.600 --> 01:12:33.680
So, you know, so because you are building here the ultimate cyber physical autonomous system that chooses not only what algorithm to execute, but where and on what physical compute substrate to execute, which is uh really fascinating.
01:12:34.000 --> 01:12:43.119
So, and that how do how does it make the choice depending on energy, security, availability, mission risk, how how do they choose what to execute on?
01:12:43.439 --> 01:12:46.720
I will see it's a combin it's a combination of all these things, right?
01:12:46.960 --> 01:12:51.920
Um, so um you have different capabilities, different decisions, different risks.
01:12:52.159 --> 01:12:58.560
So um we have a mechanism that decides and makes a decision on how to use it and when to use it.
01:12:58.960 --> 01:13:09.119
And in the context of your uh broader research and focus, uh uh is AI safety increased by this hardware independence or decreased?
01:13:09.199 --> 01:13:10.560
What's your take on this?
01:13:12.000 --> 01:13:12.640
You know?
01:13:13.119 --> 01:13:16.479
Uh uh that's a good question.
01:13:16.560 --> 01:13:17.520
I haven't thought about that.
01:13:17.600 --> 01:13:19.439
I think probably safety.
01:13:19.600 --> 01:13:20.479
It's true.
01:13:20.880 --> 01:13:25.920
I think safety will be challenged when you use multiple components.
01:13:26.079 --> 01:13:30.880
Uh, but then we have more jobs to do, then we will solve this problem as well, right?
01:13:31.279 --> 01:13:35.279
So uh so yeah, so that's why I'm part of the security as well.
01:13:35.359 --> 01:13:42.239
So we're gonna make sure that everything's secure and safe, and everything um uh uh doesn't get leaked and it's also private.
01:13:43.039 --> 01:13:47.199
Um and then also I I'm also you know, uh I'm sure you saw it.
01:13:47.279 --> 01:13:48.319
I'm also interested in quantum.
01:13:48.399 --> 01:13:53.039
So I'm also trying to bring some ideas from quantum to the company as well.
01:13:53.279 --> 01:14:12.800
Uh but um but of course, uh I mean the research that I do for Georgia Tech is controls, but uh um uh what I want to bring into the uh into the ABD um is uh some ideas on how to use uh quantum for security, uh, because I think that's the next frontier.
01:14:12.960 --> 01:14:16.960
And I think the government right now, I think put quantum as number one priority.
01:14:17.760 --> 01:14:23.920
Of course, and not only, of course, for for for us as researchers, of course, that's what we want to find.
01:14:24.000 --> 01:14:25.359
What's the next frontier?
01:14:25.439 --> 01:14:26.800
That's where we live, right?
01:14:27.119 --> 01:14:28.239
Exactly, exactly.
01:14:28.399 --> 01:14:31.600
Uh and then we want to be the first ones to do it, not the last ones.
01:14:31.840 --> 01:14:32.800
Exactly.
01:14:33.119 --> 01:14:34.800
Okay, yes, yes.
01:14:35.119 --> 01:14:37.760
Um so you know, you have so many projects.
01:14:37.840 --> 01:14:47.680
Um, I mean, um, and also some projects which are now uh in the stage of you know desire, like with the quantum and so on and so forth.
01:14:47.840 --> 01:14:53.680
But if you would be to pick a favorite of your projects, uh what would that be and why?
01:14:54.479 --> 01:14:57.840
You know, I always say that the projects are like kids, right?
01:14:58.079 --> 01:15:02.000
So I don't I I uh I I will I love all my kids.
01:15:02.239 --> 01:15:17.359
I don't have any kids, but I will love all my kids, and uh uh I love all my projects and I love them for different uh uh I I will say each project has a different personality, like every kid has a different personality, but I love them equally.
01:15:17.600 --> 01:15:19.279
But I will say one thing.
01:15:19.520 --> 01:15:23.680
I'm most interested, I'm more intrigued about the quantum part right now.
01:15:24.000 --> 01:15:26.000
So I picked a favorite.
01:15:29.119 --> 01:15:32.159
It's my the it's I always say the newborn.
01:15:32.239 --> 01:15:36.880
I I have heard stories that the newborn is either the luckiest or the least lucky.
01:15:37.039 --> 01:15:40.399
So I will say probably now is the luckiest.
01:15:41.439 --> 01:15:44.479
Yes, and I I I regard it also in a different way.
01:15:44.560 --> 01:15:50.960
I mean, with kids, you know, you focus more on the more challenging one, and it's like you love that one more always.
01:15:51.279 --> 01:15:52.479
Yeah, yeah, yeah, yeah.
01:15:52.640 --> 01:16:00.399
You always you you know, you give you give the most attention to, I mean, for the kids, they give them the most attention to the first one, and then the other ones are are easier.
01:16:00.560 --> 01:16:10.800
But now you have for research, I think you have to give the most attention to the latest one because that's where you're gonna make your uh um um your uh uh breakthrough.
01:16:10.960 --> 01:16:14.399
So because you already made something in the past one, so now you have to make that.
01:16:14.479 --> 01:16:16.720
So I will say this is the difference from the kids.
01:16:16.800 --> 01:16:26.159
So the first one is the kids is you you're more focused, but now I think for the projects, the last process is the one that you're more focused on.
01:16:27.199 --> 01:16:28.159
Absolutely.
01:16:28.399 --> 01:16:35.279
So, you know, just uh um as a scientist, obviously, and we discuss quantum and so on and so forth.
01:16:35.359 --> 01:16:44.880
But now with the um uh very radical advances of uh AI, you know, like Astra solve some mathematical problems.
01:16:44.960 --> 01:16:55.520
I'm a math Olympic, and I really feel bitten, and I feel like my value is zero, both in intelligence and as a mathematician and whatever you want to look at it.
01:16:55.840 --> 01:17:03.279
And and um so I just wonder how are you coping with this as a scientist, uh, if it preoccupies you at all?
01:17:03.359 --> 01:17:09.039
Because I mean, I'm happy I'm kind of at the uh towards the end of my career as a scientist.
01:17:09.119 --> 01:17:14.800
You are in the middle of it right now, in this time, which I think is pretty challenging in many ways.
01:17:15.039 --> 01:17:17.119
So, what would be your take here?
01:17:17.520 --> 01:17:24.159
And and what do you think the next frontier for science uh is in that context?
01:17:24.479 --> 01:17:27.199
You know, in academia, now let's take a step back.
01:17:27.279 --> 01:17:31.760
In academia, I think nowadays AI is taking over.
01:17:32.319 --> 01:17:40.159
And um so I'm editor-in-chief in a journal, so you know we we get um lots of papers that are written by AI.
01:17:40.399 --> 01:17:42.239
We know already, we know that.
01:17:42.640 --> 01:17:47.199
But um I think we cannot get out of it.
01:17:47.279 --> 01:17:56.239
So we think we have to learn how to work with it, and then and then that's how you can make the biggest advancements.
01:17:56.560 --> 01:18:01.199
Um but again, it's um it's a double-edged sword.
01:18:01.439 --> 01:18:07.600
Uh, because let's say I I I I get some homeworks, my students, I'm sure they will go figure them out with AI.
01:18:07.920 --> 01:18:12.800
But then they will not have the homework, they will not have AI when they have in-person exams.
01:18:13.119 --> 01:18:19.680
So I'll I tell them, okay, you know, you can you can use AI for your homework, but you won't have AI when you have a final exam.
01:18:20.000 --> 01:18:28.399
So um so we have to learn how to work with it, but also you have to learn what is the idea behind it.
01:18:28.560 --> 01:18:41.279
So uh what the AI does is search is database, databases of books and things that are already available on the web, which we used to do that back in our back when we were um uh in my undergrads.
01:18:41.439 --> 01:18:44.960
We used to go to the library, try to find lots of books.
01:18:45.119 --> 01:18:48.560
I mean, I don't uh I mean Google was not even big back then.
01:18:48.960 --> 01:18:58.000
And then we used to have big books, uh books, and then I was like, uh, I wanted to find the book, but that the book was reserved, and this was taking my my research back.
01:18:58.239 --> 01:19:05.119
But nowadays, you can ask AI and will tell you, oh, this blah, blah, blah, blah, blah is the answer.
01:19:05.680 --> 01:19:17.439
But uh, if you're not curious enough to find what is the background behind this answer, then I think as humans and as researchers, we will go backwards, not forwards.
01:19:17.600 --> 01:19:21.680
So you have to make sure that uh you co-work.
01:19:21.760 --> 01:19:24.319
It's like a human AI teaming that I described before.
01:19:24.560 --> 01:19:32.079
You build the trust, you make it better, the AM, the AI makes you better, and eventually you have a partner.
01:19:32.800 --> 01:19:36.000
Uh, that a partner that helps you achieve your goals.
01:19:36.640 --> 01:19:44.800
And uh uh so the next frontier now, I will say, you know, AI is a I will say a frontier, but everybody works on it right now.
01:19:44.880 --> 01:19:53.199
But the next frontier that I think that it will be bigger than AI is quantum, and I think uh uh quantum will never go away.
01:19:54.159 --> 01:20:06.079
And uh uh with uh you know AI back in the days um was stopped, and people working on AI were like, oh, I don't work on AI anymore, it's not hot anymore.
01:20:06.239 --> 01:20:10.319
The reason was that the computational capabilities were limited.
01:20:10.960 --> 01:20:14.239
That was that was the course of my generation, yes.
01:20:14.319 --> 01:20:14.720
I know.
01:20:15.199 --> 01:20:16.239
Same for me too.
01:20:16.640 --> 01:20:22.239
So but now there this is not a problem anymore, and with quantum, it will be even more.
01:20:22.479 --> 01:20:27.199
So I will say the quantum frontier will never go away.
01:20:27.439 --> 01:20:35.600
And uh AI will stabilize, uh, but I mean, novelties in AI will stabilize, but quantum will not go away.
01:20:35.760 --> 01:20:37.520
So that's the next frontier over there.
01:20:37.840 --> 01:20:43.039
Okay, so let me press you a bit here, uh, because you're an educator, obviously.
01:20:43.279 --> 01:20:47.600
Uh so but let's let's talk to the researcher right now.
01:20:47.840 --> 01:20:53.039
I mean, what is the point of studying mathematics when OpenAI Astra system?
01:20:53.359 --> 01:21:01.439
I don't know if it's like this is kind of recent news, solved all the mathematical challenges unsolved for centuries, right?
01:21:01.840 --> 01:21:09.760
So so what would you advise your students uh uh to pursue that AI cannot do better?
01:21:09.920 --> 01:21:13.920
Or why would they pursue it if AI can do it better?
01:21:14.399 --> 01:21:20.720
You know, I will say they have to pursue professions that they work together with the AI.
01:21:20.960 --> 01:21:33.600
One of them is engineering, um, I mean, different things, but I will also say that mathematicians are very important because uh uh mathematics is the basis for AI.
01:21:33.920 --> 01:21:39.439
So, and as a controls person and a learning person, um, I'm very technical.
01:21:39.520 --> 01:21:43.199
So I will sometimes I used to call myself an applied mathematician.
01:21:43.359 --> 01:21:54.079
Uh, but even if it solves all the challenges, I mean, let's say that um I asked AI to solve uh, I don't know, conjectures back in the days.
01:21:54.319 --> 01:21:59.600
Um this doesn't do anything, I will say, into um engineering, right?
01:21:59.680 --> 01:22:01.279
It will not make your life better.
01:22:01.439 --> 01:22:17.600
It will give you um uh the reward that you saw that uh you actually learned that AI solved this problem, but this will not make your car, your, your, your, airs, your aircraft or your uh your daily life better.
01:22:17.760 --> 01:22:28.000
So you have to work on something that makes your daily life better, even if the AI solves all these important problems in mathematics that we we we we hear about.
01:22:28.319 --> 01:22:33.039
So uh even still, even if it solves it, you have to understand them.
01:22:33.119 --> 01:22:35.920
So you have to learn the background to understand them.
01:22:36.079 --> 01:22:46.479
Even studying the answer is important, and it will take you back on on exploring possibilities for more research, right?
01:22:46.720 --> 01:22:52.720
So um that's uh so we are still relevant as professors.
01:22:54.159 --> 01:23:03.359
You know, I don't know for how many more, but uh yes, I think we are, especially in in I will say engineering, applied math, physics.
01:23:03.520 --> 01:23:05.600
Uh yeah, I think we're relevant.
01:23:05.680 --> 01:23:12.479
Yeah, and the robots will come, the humanoid robots, and then who will do the who will do the robots?
01:23:12.560 --> 01:23:16.479
We will do the robots, and then they will do they will build themselves.
01:23:16.640 --> 01:23:31.600
Actually, I worked on the total automated factory with Fanuk in Japan, and they beat the robots were building themselves, but at that time it was not so intelligent, it was more like programmed, but but you know, it kind of gave you an idea.
01:23:32.159 --> 01:23:32.479
Oh, yeah.
01:23:33.199 --> 01:23:34.000
We can do it.
01:23:34.239 --> 01:23:35.199
Yeah, yeah, yeah.
01:23:35.359 --> 01:23:40.960
I mean, all these all these humanoids, uh, you know, in the factories, they have robots building the humanoids.
01:23:42.000 --> 01:23:42.640
Exactly.
01:23:42.800 --> 01:23:44.479
So, you know, it's fascinating.
01:23:44.720 --> 01:23:57.920
It was fascinating to talk to you because you know, our trajectories as intelligent control uh systems engineers met in several points, although in uh maybe different times.
01:23:58.479 --> 01:23:59.520
True, true, true.
01:23:59.760 --> 01:24:15.680
Yes, but now I'm glad our our our paths crossed, and then we we found common common trajectories, and you know it's always uh uh people going parallel trajectories, but eventually if you project these trajectories for a long time, they they might meet.
01:24:15.840 --> 01:24:17.039
So this is what happened.
01:24:18.000 --> 01:24:28.479
Absolutely, as you as you were talking about that mass population, and then if you go at different levels and in the end they stabilize somehow, and I can see it's similar in a way.
01:24:28.720 --> 01:24:48.800
You know, I'd like to thank you for taking us from control to autonomy, from learning to safety, from earth to space, and from machines that follow instructions towards machines capable of navigating the unknown without losing sight of the boundaries that keep us safe.
01:24:49.039 --> 01:24:52.720
So, really, your work is critical in this day and age.
01:24:53.039 --> 01:24:53.840
Thank you, Michaela.
01:24:53.920 --> 01:24:54.800
Thank you very much for that.
01:24:58.159 --> 01:25:03.039
Look forward to collaborating with you, and uh, I really, really look forward to it.
01:25:03.199 --> 01:25:04.800
I will I will be happy to, Michaela.
01:25:05.119 --> 01:25:06.319
I will be happy, Michaela.
01:25:06.640 --> 01:25:07.760
Thank you very much.
01:25:07.920 --> 01:25:17.520
The the discussion was amazing, and uh uh you're an amazing host, and I'm looking forward to work with you in uh collaborative projects and explore these possibilities.
01:25:17.840 --> 01:25:24.640
And I cannot you know stress more how much respect I have for your achievements, for your incredible.
01:25:24.960 --> 01:25:40.960
I mean, you are totally there are 10 people in you from what I see in your the amount of projects, the amount that I mean your productivity, how many books, how many papers, and and also uh the difficulty that uh and the challenges that you are tackling.
01:25:41.119 --> 01:25:41.760
Kudos to you.
01:25:42.000 --> 01:25:44.239
Thank you, thank you, thank you, Michaela.
01:25:44.399 --> 01:25:51.119
I'm looking forward to talk with you soon again and then finding ways to collaborate and uh take it from there.
01:25:51.439 --> 01:25:52.399
Thank you.