Drones That See: Inside Modern Drone Vision Systems

WEBVTT
00:00:02.008 --> 00:00:07.087
Well, hello and welcome to Visions, a machine vision and automation solutions podcast.
00:00:07.621 --> 00:00:11.307
I'm your host, Jim Tatum, Senior Editor of Vision Systems Design,
00:00:11.580 --> 00:00:16.561
and Visions is an Endeavor business media production from your friends at Vision Systems Design.
00:00:17.205 --> 00:00:21.637
Here you'll find the latest on everything from end-user machine vision solutions
00:00:21.637 --> 00:00:26.673
to trends, developments, and perspectives on all things machine vision and imaging.
00:00:27.573 --> 00:00:30.667
Whether you've been working in the industry for a while, or you're just starting
00:00:30.667 --> 00:00:34.867
to take a closer look at it, this podcast is designed to grow your knowledge
00:00:34.867 --> 00:00:38.962
and bring greater focus to your understanding of the imaging and machine vision industry.
00:00:39.630 --> 00:00:41.226
And now, onto our show.
00:00:42.811 --> 00:00:47.655
Well, hi, everybody, and welcome to Visions. Today, we're going to take a deeper
00:00:47.655 --> 00:00:52.255
dive into how advances in cameras, sensor fusion, edge computing,
00:00:52.255 --> 00:00:57.624
and AI are making drone vision systems more capable across a wide range of applications.
00:00:58.289 --> 00:01:03.355
In fact, recently, VSD's head of content, Sharon Spielman, had a terrific conversation
00:01:03.355 --> 00:01:06.474
about some of these very issues with Arthur Erickson,
00:01:06.816 --> 00:01:12.365
who's CEO and co-founder of Hylio, a Texas-based company that specializes in
00:01:12.365 --> 00:01:14.119
drone technologies and applications.
00:01:14.825 --> 00:01:16.590
Let's give a listen to what they had to say.
00:01:17.456 --> 00:01:22.025
Welcome. Today we're looking at drone vision systems, an area where advances
00:01:22.025 --> 00:01:27.015
in machine vision, sensor integration, and real-time image processing are changing
00:01:27.015 --> 00:01:31.725
how drones serve the critical roles in a range of industries and applications,
00:01:32.148 --> 00:01:34.732
including emergency response and security.
00:01:35.329 --> 00:01:39.555
For our vision systems design audience, the challenges go beyond just adding
00:01:39.555 --> 00:01:44.720
cameras, success hinges on optimizing sensor performance, flight stability,
00:01:45.074 --> 00:01:48.615
and manufacturing precision to deliver real-time, actionable,
00:01:48.615 --> 00:01:50.890
visual data in challenging environments.
00:01:51.361 --> 00:01:55.615
As automation and AI-driven machine vision advance, these systems are becoming
00:01:55.615 --> 00:01:59.035
more robust, adaptable, and capable of learning on the fly.
00:01:59.760 --> 00:02:04.341
Joining us today is Arthur Erickson, CEO and co-founder at Hylio,
00:02:04.654 --> 00:02:08.195
to share his insights on the technical evolution of drone vision,
00:02:08.468 --> 00:02:12.218
from sensor technology and payload design to AI-driven processing,
00:02:12.531 --> 00:02:16.038
and what it means for advancing situational awareness in the field.
00:02:16.769 --> 00:02:19.820
Thank you for joining us today. Thank you for having me.
00:02:20.081 --> 00:02:23.335
You bet. First, why don't you tell me a little bit about yourself,
00:02:23.335 --> 00:02:24.963
your background in the drone market?
00:02:25.624 --> 00:02:29.860
Sure. Well, it's nothing too exciting. I won't harp on too much about myself.
00:02:29.860 --> 00:02:35.274
I went and studied aerospace engineering at UT Austin, so University of Texas in Austin.
00:02:35.658 --> 00:02:39.563
And that's actually where I started the company with three other fellow UT alum.
00:02:39.871 --> 00:02:44.358
So it's a Longhorn company, classic Longhorn startup dorm room type of story.
00:02:44.938 --> 00:02:48.230
But yeah, it was through my curriculum as an aerospace engineer,
00:02:48.230 --> 00:02:51.010
undergrad only, that I first got into drones.
00:02:51.010 --> 00:02:54.880
And basically for our capstone project, for example, that entire year,
00:02:54.880 --> 00:02:56.800
it was design a drone from scratch.
00:02:56.800 --> 00:03:00.536
In this case, it was a fixed wing platform, so plane style design.
00:03:00.728 --> 00:03:04.600
We were working with the Austin Fire Department to do a simulated mission in
00:03:04.600 --> 00:03:07.830
which we'd locate a target and then drop an aid package, right?
00:03:07.830 --> 00:03:09.528
A simulated aid package on that target.
00:03:09.871 --> 00:03:13.170
And that was how we were graded. So it was kind of a combination of those technical
00:03:13.170 --> 00:03:16.340
skills I picked up in school and then just entrepreneurial spirit.
00:03:16.340 --> 00:03:20.010
And, of course, my great team put that all together. And then we started Hylio.
00:03:20.010 --> 00:03:24.240
It'll be 12 years in January. So all the way back in 2015, again,
00:03:24.240 --> 00:03:25.622
while I was still in school.
00:03:26.098 --> 00:03:30.170
So that's where we come from. Since then, primarily, we have focused on the
00:03:30.170 --> 00:03:34.220
agriculture industry. So meaning that we design and build and sell drones that
00:03:34.220 --> 00:03:37.769
are more or less replacing traditional crop dusting methods.
00:03:38.146 --> 00:03:41.600
That's airplanes and helicopters, but I should also mention that's applications
00:03:41.600 --> 00:03:45.660
done by tractor or self-propelled ground rig as well. So we're offering a more
00:03:45.660 --> 00:03:47.633
precise, environmentally friendly,
00:03:47.923 --> 00:03:51.157
better for your wallet, better for the neighbor's wallet solution.
00:03:51.488 --> 00:03:55.806
So that's typically what we do. But to swing back to this conversation a bit,
00:03:56.068 --> 00:03:59.910
as a company's grown in size, we've become more successful, had more engineers
00:03:59.910 --> 00:04:03.544
on staff, had more mature products that we could bring to bear.
00:04:03.736 --> 00:04:06.650
We've branched out to other industries such as emergency response.
00:04:07.004 --> 00:04:10.910
So a bit what we're talking about here with the wildfire detection and suppression
00:04:10.910 --> 00:04:13.190
and other things along that vein as well.
00:04:14.307 --> 00:04:19.820
Great. Yeah, so 12 years. So I imagine that you've seen quite a bit of change.
00:04:19.820 --> 00:04:24.760
So from a machine vision integration perspective, what key advancements have
00:04:24.760 --> 00:04:26.919
you seen in drone vision systems recently?
00:04:28.776 --> 00:04:33.460
Probably one of the main it's not maybe super exciting but one of the main things
00:04:33.460 --> 00:04:37.437
is just unit cost right so bringing the unit cost of relatively high resolution
00:04:37.605 --> 00:04:42.240
cameras down and making them lightweight as the drone industry explodes making them,
00:04:42.862 --> 00:04:47.310
cheaper it's just obviously going to be a force for more use out in the field
00:04:47.310 --> 00:04:53.830
so i think personally a lot of that has come from the EV market right so basically all these cars now
00:04:54.270 --> 00:04:59.348
obviously auto is a massive industry as they've integrated more digital vision systems.
00:04:59.714 --> 00:05:03.360
We've, as a drone industry, reaped those benefits. Camera technology as well,
00:05:03.360 --> 00:05:07.240
right? So every cell phone has a camera. So yeah, I think just price point is
00:05:07.240 --> 00:05:11.196
a big one there. So just packing more punch in a smaller and cheaper package.
00:05:11.602 --> 00:05:16.044
And then on the AI side or on the compute side, edge compute as a concept
00:05:16.445 --> 00:05:20.610
and instantiated in more efficient, again, cheaper, but more efficient edge
00:05:20.610 --> 00:05:25.140
processing power is key, of course, to actually having these practically come
00:05:25.140 --> 00:05:29.300
into play in a scenario that's very time sensitive, such as emergency response, right?
00:05:29.300 --> 00:05:33.130
So you can't be deeming stuff back to the cloud, waiting 24 hours to have some
00:05:33.130 --> 00:05:37.068
server do it, and then acting on the information. It's got to be done right there in the field.
00:05:37.717 --> 00:05:43.140
Right. So let's talk about the types of cameras and sensors that are incorporated
00:05:43.140 --> 00:05:48.120
to optimize the image capture across those harsh operational environments that
00:05:48.120 --> 00:05:51.029
you're talking about. Yeah, so it really depends.
00:05:51.925 --> 00:05:58.988
If we're talking about something like identifying a fire in a wildfire scenario,
00:05:59.418 --> 00:06:05.138
you don't actually need that much vision to, you don't need that much resolution, I should say.
00:06:05.138 --> 00:06:08.118
It doesn't have to be like a high fidelity feed that you're reading because
00:06:08.118 --> 00:06:11.388
a fire is pretty obvious, right? So especially if you have a thermal sensitive
00:06:11.388 --> 00:06:14.982
sensor, a fire is pretty dang different than its surroundings.
00:06:15.301 --> 00:06:18.238
So in a case like that, it's fairly black and white. You don't need a super
00:06:18.238 --> 00:06:21.868
high-powered thermal sensor, or you probably don't even need thermal in general.
00:06:21.868 --> 00:06:23.129
You could get away with RGB.
00:06:23.465 --> 00:06:28.548
So that's pretty easy. But in a scenario like if you go to our traditional industry,
00:06:28.548 --> 00:06:31.552
which is, again, precision agriculture, you,
00:06:32.228 --> 00:06:36.778
again, depending on your context, might need some pretty high-resolution camera
00:06:36.778 --> 00:06:41.298
work to be able to identify a somewhat different-looking green thing versus
00:06:41.298 --> 00:06:42.888
another green thing, right?
00:06:42.888 --> 00:06:45.585
So the plant you want to save versus the plant you want to kill, like a weed.
00:06:45.945 --> 00:06:51.018
So typically, I mean, to ground it, most of our customers like on our sensing
00:06:51.018 --> 00:06:52.218
platform we have, which we call
00:06:52.218 --> 00:06:57.142
the Photon, a lot of the customers are going with a 4K on 20 megapixel
00:06:57.571 --> 00:07:01.726
RGB sensor with optical zoom up to 5 to 7x.
00:07:02.022 --> 00:07:07.158
And this is typically good enough for them to be pretty speedy with some crop
00:07:07.158 --> 00:07:12.148
scans where you can fly at 300 to 400 feet above the crop and identify literal.
00:07:12.828 --> 00:07:16.108
Individual weeds or individual leaves even on plants.
00:07:16.108 --> 00:07:21.208
You could see stuff like fungal outbreaks, little white spots or black spots,
00:07:21.208 --> 00:07:24.968
for example, on the leaves of corn or something to that nature.
00:07:24.968 --> 00:07:27.948
So it really comes down to, again, your use case and like what you need your
00:07:27.948 --> 00:07:29.281
ground sampling distance to be.
00:07:29.559 --> 00:07:34.808
But people are operating anywhere from half a centimeter per pixel to two or
00:07:34.808 --> 00:07:37.936
three centimeters per pixel based, again, on ultimate use case.
00:07:38.814 --> 00:07:42.908
Right. So how do the drone vision systems,
00:07:43.622 --> 00:07:48.743
deal with the varying lighting conditions and the weather interference and the
00:07:48.743 --> 00:07:53.653
motion blurring to maintain the image fidelity for the downstream processing?
00:07:54.563 --> 00:07:57.779
How do you select, how do you specify that for your drones?
00:07:58.527 --> 00:08:02.237
Again, this is like designing your system. Not to go on a tangent here,
00:08:02.237 --> 00:08:06.187
but like our Photon, which I had just mentioned, is designed intentionally to
00:08:06.187 --> 00:08:08.937
be very easy for you to swap payloads out, right?
00:08:08.937 --> 00:08:13.697
So you can get really any third-party sensor, like a Sony ILX.
00:08:14.037 --> 00:08:17.217
You know, it's basically like their DSLR, but shrunk a little bit.
00:08:17.217 --> 00:08:18.699
And then you can stick it on for like a drone.
00:08:18.989 --> 00:08:23.099
You could use like multi-spectral or hyper-spectral cameras, thermals.
00:08:23.413 --> 00:08:25.987
You could put lidars on there which i know it's not a camera but or a vision
00:08:25.987 --> 00:08:29.087
system but i mean you could put anything on there right because it's going to
00:08:29.087 --> 00:08:33.966
change so like our typical camera that we would sell to an agricultural customer
00:08:34.390 --> 00:08:37.884
would be somewhat of a one size fits all right so like,
00:08:38.534 --> 00:08:42.597
decent at least like HD so 1080 you know video streaming capabilities but if
00:08:42.617 --> 00:08:44.397
they're not using it for precise weed id,
00:08:44.908 --> 00:08:49.187
just like a navigational tool that's all they need hdr right so it can actually
00:08:49.187 --> 00:08:53.207
perform really well in low light settings but it's not true night vision right
00:08:53.207 --> 00:08:54.775
so we do have some customers that actually,
00:08:55.251 --> 00:08:58.837
strictly operate at night or need to operate at night for some application whether
00:08:58.837 --> 00:09:02.027
that be ag or something else and in which case you know they would we'd have
00:09:02.027 --> 00:09:07.277
a straight up thermal based system or infrared near infrared perhaps and we'd
00:09:07.277 --> 00:09:10.024
have an illuminator coupled with the sensor so you're
00:09:10.418 --> 00:09:13.048
illuminating it and then receiving that data back so it.
00:09:13.772 --> 00:09:17.557
That sorry to give you a bad answer but just like it really we do all sorts
00:09:17.557 --> 00:09:19.617
stuff based on the customer.
00:09:19.617 --> 00:09:23.387
So we can integrate by design. It's a very extensible system.
00:09:23.387 --> 00:09:25.767
So if they need night vision, we can do that. If they need thermal,
00:09:25.767 --> 00:09:28.476
we can do that. If they need high resolution RGB, we could do that.
00:09:29.254 --> 00:09:37.857
Okay. So in terms of the processing architecture, what onboard compute AI capabilities
00:09:37.857 --> 00:09:41.037
enable real-time interpretation of visual data?
00:09:41.037 --> 00:09:46.579
Is that that photon that you're talking about for obstacle detection and terrain mapping.
00:09:47.417 --> 00:09:51.837
And then that also kind of makes me think about, I know you said you cover ag,
00:09:52.317 --> 00:09:57.148
you cover emergency services, and I believe surveillance do cover,
00:09:58.028 --> 00:10:02.174
aerospace and defense. I know you were starting out with agriculture,
00:10:02.174 --> 00:10:07.937
but are you branching out into all of the industries or can we talk about that a little bit?
00:10:08.483 --> 00:10:11.017
Yeah. So we are basically doing all of the above.
00:10:11.734 --> 00:10:15.914
We view our drones as a platform technology. And so not to belabor it,
00:10:16.214 --> 00:10:20.724
but what I was just getting at, we make it very easy for any third-party provider
00:10:20.724 --> 00:10:23.894
or company to put sensors or modules onto it.
00:10:23.894 --> 00:10:28.254
So ag is one thing, but yeah when you talk to the and i keep bringing up the
00:10:28.254 --> 00:10:31.094
fire people but like it's just something that's top of mind because we're doing
00:10:31.094 --> 00:10:33.961
that right now but fire suppression people or the police department,
00:10:34.448 --> 00:10:37.476
right they're going to have like in the case of fire you might need to drop
00:10:37.761 --> 00:10:41.574
a flaming tennis ball essentially and this is to do a controlled burn or you
00:10:41.574 --> 00:10:45.928
might need to spray out a special compound that like literally like directly suppresses the fire
00:10:46.335 --> 00:10:50.184
the police might have uh even something like a loudspeaker on the drill which
00:10:50.184 --> 00:10:51.224
is just a different thing to
00:10:51.224 --> 00:10:54.363
do crowd control right to speak to crowd like you know disperse disperse
00:10:54.589 --> 00:10:57.454
perhaps like a tear gas canister right that's something we haven't done yet
00:10:57.454 --> 00:11:00.784
but we're in conversations about so things of that nature i mean there's all
00:11:00.784 --> 00:11:02.374
sorts of different things you can stick on these things.
00:11:03.256 --> 00:11:04.725
Bringing it back though it's just,
00:11:06.051 --> 00:11:11.324
yeah can you build base that's like 80 there and just like the french fries
00:11:11.324 --> 00:11:13.525
that go with the hamburger are like relatively easy
00:11:13.896 --> 00:11:17.304
and maybe like you know a few weeks worth of integration work for our team and
00:11:17.304 --> 00:11:21.504
then the customer's team to actually implement so that's kind of what we see
00:11:21.504 --> 00:11:25.957
generally for a lot of these enterprise-level customer interactions right now is like a little bit,
00:11:26.439 --> 00:11:30.706
mostly like the stock unit we sell, and then like a little bit of customization on top.
00:11:31.344 --> 00:11:34.414
Okay. So why don't you talk a little bit about that?
00:11:34.414 --> 00:11:38.844
So the payload design, the weight distribution, and the vibration damping,
00:11:39.233 --> 00:11:44.209
how do all those impact the optical sensor performance and flight stability,
00:11:44.708 --> 00:11:45.874
in the drones themselves?
00:11:45.874 --> 00:11:47.534
I mean, because, you know, a
00:11:47.534 --> 00:11:51.561
speaker as opposed to having to carry fire suppression are very different.
00:11:52.218 --> 00:11:54.163
Yeah, no, that's a good point. Yeah.
00:11:55.092 --> 00:11:58.041
Yeah, I should elaborate more on your last question about kind of how we do
00:11:58.041 --> 00:12:04.402
some of the processing, but it's a very fair point that vibrations are the enemy of clear visual data.
00:12:04.831 --> 00:12:09.021
You have to solve that first mechanically, of course. So we've designed a brushless
00:12:09.021 --> 00:12:13.761
motor, actuated three-axis gimbals, and that's typically what our cameras are
00:12:13.761 --> 00:12:15.001
mounted on our drone platform.
00:12:15.001 --> 00:12:18.380
So that gives you control where the camera is pointing while you're flying,
00:12:18.682 --> 00:12:22.869
but it is actively stabilizing the camera feed the entire time.
00:12:23.159 --> 00:12:26.741
That to get a little bit even nittier, grittier about it, like you've got this
00:12:26.741 --> 00:12:31.239
gimbal, but it itself is mounted on vibration dampening structures, right?
00:12:31.355 --> 00:12:35.241
Rubber type structures that are placed in such a way through analysis we've
00:12:35.241 --> 00:12:38.438
done and testing to really minimize that vibration even further,
00:12:38.821 --> 00:12:40.921
even beyond that, you still have vibration.
00:12:40.921 --> 00:12:45.671
So then you apply like a digital stabilization layer, which there's a fair amount
00:12:45.671 --> 00:12:49.211
of open source tools out there that we play around with and stuff that we've
00:12:49.211 --> 00:12:50.419
developed in-house as well.
00:12:50.750 --> 00:12:54.371
So that's like edge detection. And, you know, we'll get into it.
00:12:54.371 --> 00:12:58.911
But the basic things that camera companies are applying to stabilize moving
00:12:58.911 --> 00:13:03.081
frames, we're not doing anything like we're not a camera company at the end
00:13:03.081 --> 00:13:06.320
of the day. Right. So we're not doing anything bleeding edge or anything in that regard.
00:13:06.633 --> 00:13:10.331
But really just trying to cross our T's and dot our I's and make sure that you
00:13:10.331 --> 00:13:12.822
mechanically eliminate the vibrations as much as possible.
00:13:13.136 --> 00:13:16.881
Do as little digital stabilization as you can, because ultimately,
00:13:16.881 --> 00:13:19.601
obviously, that's going to lead to artifacts and stuff like that that you don't
00:13:19.601 --> 00:13:21.472
want if you do too much digital correcting.
00:13:21.809 --> 00:13:24.827
So we try to do it the right way mechanically.
00:13:25.598 --> 00:13:30.029
What we employ, object recognition and whatnot, is a variety of things.
00:13:30.029 --> 00:13:33.708
Again, a lot of it is leveraging open source stuff out there like YOLO,
00:13:34.033 --> 00:13:36.750
machine learning, detection libraries.
00:13:37.098 --> 00:13:42.160
So it's like basic stuff like, can you identify a human versus a cow versus a car?
00:13:42.467 --> 00:13:45.890
There's a lot of stuff out there that's available to everybody already for that.
00:13:46.320 --> 00:13:49.949
But it really comes down to using the right tools or algorithms for the right
00:13:49.949 --> 00:13:53.599
problem and designing the software in the interface in such a way where the
00:13:53.599 --> 00:13:57.319
customer or the operator could switch between the stuff when needed, right?
00:13:57.319 --> 00:14:00.979
So, yeah, you might use a different model when you're doing detection of a car
00:14:00.979 --> 00:14:05.239
or a person, you know, maybe in a law enforcement sense versus the model you
00:14:05.239 --> 00:14:09.439
use, of course, for identifying what we call volunteer cotton in a cornfield
00:14:09.439 --> 00:14:11.967
or vice versa or Johnson grass in a cornfield.
00:14:12.251 --> 00:14:16.989
So it's more of like an interface thing. These tools all exist under the hood,
00:14:16.989 --> 00:14:22.039
of course, but can you seamlessly switch between them and give like the customer
00:14:22.039 --> 00:14:25.869
that power without overcomplicating it to use the right model at the right time?
00:14:26.681 --> 00:14:32.772
So how do you go about when you're sitting down with your customers?
00:14:33.248 --> 00:14:40.245
Are you selecting off the shelf? Are you using customized components?
00:14:40.524 --> 00:14:45.528
How does that work? And I'm speaking specifically for the vision systems on the drone.
00:14:46.401 --> 00:14:51.781
I don't know how. I guess I'm kind of looking when you're doing your drones,
00:14:51.781 --> 00:14:54.004
when you're making them, i would imagine,
00:14:54.696 --> 00:14:59.611
you have your own components that you're putting together for the drone itself
00:14:59.611 --> 00:15:03.241
but then you're going to a third party for the vision systems
00:15:03.566 --> 00:15:07.731
is that yeah that's that's usually right at least for now we are actually developing
00:15:07.731 --> 00:15:11.241
like from the board level up like our own camera but even then i mean of course
00:15:11.241 --> 00:15:15.246
you're still buying the sensor from Sony or whomever so like usually,
00:15:15.849 --> 00:15:18.701
like right there's a company we utilize called e-con Systems they're actually
00:15:18.701 --> 00:15:21.941
based in India Yeah, but they make great products. A lot of them are designed for drones.
00:15:22.711 --> 00:15:25.861
To our points earlier, they're cheap, they're lightweight, good stabilization, etc.
00:15:26.716 --> 00:15:32.220
So we use their cameras, but we build a gimbal around it. And then we have an onboard compute.
00:15:32.661 --> 00:15:36.684
Typically, we're using like Jetson or in Nanos right now. So little computer,
00:15:37.206 --> 00:15:38.601
NVIDIA powered, of course.
00:15:38.601 --> 00:15:42.831
And this is doing a lot of this post-processing we're talking about in real time.
00:15:42.831 --> 00:15:46.711
So segmentation or just a obstacle detection or object detection,
00:15:46.711 --> 00:15:50.871
target detection, and then spitting whatever's needed into the flight controls,
00:15:50.871 --> 00:15:54.003
which are they going to turn that into like an actionable maneuverer?
00:15:54.823 --> 00:15:58.509
And you're providing the hardware and the software to your customers?
00:15:58.956 --> 00:16:04.270
Yes, that's correct. So we are full stack in that sense, which not a lot of
00:16:04.270 --> 00:16:06.810
people do what we do, I should say. There's a lot of companies out there that
00:16:06.810 --> 00:16:09.750
outsource the software or vice versa, outsource the hardware.
00:16:10.052 --> 00:16:14.458
Drones, it's an orchestra, right? It's all these things working in perfect harmony.
00:16:14.800 --> 00:16:18.210
And we've just found, because we've done a mix of both, right?
00:16:18.210 --> 00:16:21.203
When we were a younger company, just four kids out of UT Austin,
00:16:21.667 --> 00:16:23.350
there was a lot of system integrating we had to do.
00:16:23.350 --> 00:16:27.680
Where we were going out and buying hardware that we didn't design, trying to make it work.
00:16:27.680 --> 00:16:31.130
But ultimately, there's a lot of limitations there, especially at the edges.
00:16:31.130 --> 00:16:35.150
But eventually for the entire system, like you're looking at some pretty serious
00:16:35.150 --> 00:16:40.200
inefficiencies or sometimes a downright inability to do a desired outcome because
00:16:40.200 --> 00:16:43.490
you ultimately didn't control that supply chain, that, you know,
00:16:43.490 --> 00:16:44.650
the entire design of the system.
00:16:44.650 --> 00:16:48.690
So again, with the orchestra that is a drone, we think it's imperative that
00:16:48.690 --> 00:16:51.083
you do both the hardware and the software for ultimate outcomes.
00:16:52.058 --> 00:16:56.100
So, speaking of the orchestra, then the listener of the orchestra,
00:16:56.100 --> 00:17:00.950
which would be the people who buy the drones, are there any vetting processes?
00:17:00.950 --> 00:17:06.470
Okay, so can you talk a little bit about sensor fusion and how it is evolving
00:17:06.470 --> 00:17:11.754
in these systems to improve the situational intelligence accuracy and the robustness?
00:17:12.409 --> 00:17:15.480
Yeah, that's pretty important. When the drone is flying, I mean,
00:17:15.480 --> 00:17:19.090
this is especially important for obstacle detection and avoidance.
00:17:19.462 --> 00:17:24.763
These drones, like in the agriculture context, are often flying quite fast, 40 miles per hour.
00:17:25.216 --> 00:17:30.370
They're flying low. So yes, they're flying over depopulated cornfields.
00:17:30.370 --> 00:17:33.930
So it's not really a risk to humans, but the drone is at risk because it's flying
00:17:33.930 --> 00:17:36.960
low, fast, and there's trees and there's guide wires and whatnot.
00:17:36.960 --> 00:17:41.706
So you need a pretty robust vision system. So it's got to be able to actually see everything.
00:17:41.909 --> 00:17:45.230
And it's quite a noisy environment in terms of obstacles that these systems
00:17:45.230 --> 00:17:47.401
are detecting, right? Because you have all this vegetation.
00:17:47.761 --> 00:17:52.150
Again, you have all these trees, power lines, whatever. You need to identify
00:17:52.150 --> 00:17:54.442
everything, even as small as a half-inch power line.
00:17:54.901 --> 00:17:58.280
You need to identify everything, but then you need to compute on board to say,
00:17:58.280 --> 00:18:01.060
is that actually something I need to worry about, right? And throw away the
00:18:01.060 --> 00:18:03.770
stuff that is just noise, like actual noise.
00:18:04.240 --> 00:18:08.321
Also ignore the stuff that is a real obstacle but is it in your path for example
00:18:08.582 --> 00:18:12.410
so you're doing all this like calculation on board in real time what's the heading
00:18:12.410 --> 00:18:17.745
of the drone what's the speed where are these obstacles in you know proportion to my heading or.
00:18:18.320 --> 00:18:22.630
In relation to my heading i should say and so it takes a lot and like to bring
00:18:22.630 --> 00:18:23.956
it back to your point about sensor fusion,
00:18:24.467 --> 00:18:29.740
like there's no single sensor that does everything perfectly right and so for
00:18:29.740 --> 00:18:33.633
example i'll ground it again we use radars a lot for obstacle detection,
00:18:34.150 --> 00:18:39.390
because they're a pretty good overall fit because in the ag drone world you're
00:18:39.390 --> 00:18:43.150
spraying products so there's a lot of moisture in the air like mist and fog
00:18:43.150 --> 00:18:47.820
and there's sunlight reflecting off this moisture there's dust that you're kicking up
00:18:48.226 --> 00:18:51.770
so all these things radar can relatively easily penetrate through and give you
00:18:51.770 --> 00:18:55.640
still a pretty good picture of what's on the other side but versus like a camera
00:18:55.640 --> 00:18:57.320
which might be obscured by dust or,
00:18:57.780 --> 00:19:01.665
or liquid or a lidar system which would just bounce off the first thing it sees.
00:19:02.112 --> 00:19:04.084
So you have to play this,
00:19:05.089 --> 00:19:09.593
compromise game with what's the best fit for most of the situations or,
00:19:10.247 --> 00:19:13.202
what you can do is what we're starting to do now which is like it's going to
00:19:13.202 --> 00:19:15.681
increase the cost a bit but add more
00:19:15.959 --> 00:19:19.032
more different sensors and fuse them together so then you are actually truly
00:19:19.032 --> 00:19:22.632
covering like all the bases right so i i think the big push for us right now
00:19:22.632 --> 00:19:25.944
is okay we have radars which i just mentioned we're trying to incorporate,
00:19:26.536 --> 00:19:28.837
binocular vision not as,
00:19:29.476 --> 00:19:33.042
you know replacement to the radars in conjunction again along the theme of fusing
00:19:33.042 --> 00:19:37.975
it because what the binocular vision could do better than a radar is make let's say.
00:19:39.135 --> 00:19:44.202
More practical planned decisions or help you make more practical plan decisions
00:19:44.202 --> 00:19:45.863
with how to go around an obstacle
00:19:46.205 --> 00:19:50.572
so proactively avoiding like a tree line or something versus a radar which just
00:19:50.572 --> 00:19:55.946
sees it and then you're you're reacting to it and and kind of losing a lot of energy there right and,
00:19:56.631 --> 00:20:00.172
you know seeing the tree line stopping and then doing a bit of a Roomba approach
00:20:00.172 --> 00:20:03.292
and getting around it, right, versus where you could see it like a human would
00:20:03.292 --> 00:20:09.002
with binocular vision, say, okay, I can kind of see the way around it and then start getting that way.
00:20:09.002 --> 00:20:12.742
Yeah, that's where it gives you more benefit. But yeah, sorry,
00:20:12.742 --> 00:20:16.153
that was kind of a meandering answer. No, that was good.
00:20:16.728 --> 00:20:20.332
Maybe you could talk about some of the system integration challenges that you
00:20:20.332 --> 00:20:25.072
face when combining the cutting-edge vision hardware with the drone flight control
00:20:25.072 --> 00:20:26.756
and communication modules?
00:20:27.395 --> 00:20:31.875
Yeah, I think this is like a mostly a compute challenge, as in you only have so much compute.
00:20:32.072 --> 00:20:35.262
So a lot of these sensors are more than enough in terms of data.
00:20:35.262 --> 00:20:41.103
Like, you know, you could be pulling gigabytes basically off these things per flight, if not more.
00:20:41.963 --> 00:20:46.605
So, yeah, you're getting plenty of information. The challenge is for the engineering
00:20:46.605 --> 00:20:50.825
team is, yeah, to what we were discussing earlier, what do you throw away?
00:20:50.825 --> 00:20:54.525
Like, what do you use that limited compute for that drone every cramp counts,
00:20:54.525 --> 00:20:58.095
right? So you can't just have a giant server rack of processing power on this
00:20:58.095 --> 00:21:02.544
thing. You have to have, you know, tiny little jets in, which is good, but it's small.
00:21:02.881 --> 00:21:06.115
So just deciding, doing the triage of deciding what's the right information
00:21:06.115 --> 00:21:11.021
to actually store and compute is the challenge there. It's like a budgetary problem almost.
00:21:11.793 --> 00:21:15.755
So do you have, does the software that you provide the customers,
00:21:15.755 --> 00:21:18.635
does it have, say, selection boxes.
00:21:19.670 --> 00:21:26.261
Of what is needed for each particular application so that they cannot be wasting,
00:21:26.783 --> 00:21:30.922
data collection because it's not going to do anything with that data anyway?
00:21:31.224 --> 00:21:33.314
So how does that selection happen?
00:21:33.720 --> 00:21:39.231
You know, we don't really have a lot of that yet, but that is in the works. You would have,
00:21:40.076 --> 00:21:43.763
Like I dropped that menu in our software to your point and it would be,
00:21:44.239 --> 00:21:47.646
okay, here's the model that's best for identifying humans. Here's the one for
00:21:47.646 --> 00:21:49.742
they use for plants, et cetera, et cetera.
00:21:50.044 --> 00:21:52.526
Of course, we'd want to also expose camera settings to people.
00:21:52.526 --> 00:21:54.521
So like, you know, what is your, your data rate?
00:21:54.684 --> 00:21:58.416
You know, are you limiting the resolution or whatever, if it's a software defined camera system.
00:21:58.782 --> 00:22:03.296
So I just try to give the customer more customization without overwhelming them.
00:22:03.296 --> 00:22:07.216
Right. Cause most of these operators aren't like these camera vision nerds.
00:22:07.216 --> 00:22:13.596
Right. So they need to be explained to things in outcome type of language,
00:22:13.596 --> 00:22:17.656
like see human, see plant, see whatever. That's what I was just going to say.
00:22:18.436 --> 00:22:21.636
That's honestly what I was just going to say. If you ask somebody what data
00:22:21.636 --> 00:22:26.466
rate do you need, do you also tell them if you're looking for a plant, you'll need this?
00:22:26.466 --> 00:22:29.586
If you're looking for a human, you'll need this. If you're looking for a fire,
00:22:29.586 --> 00:22:33.476
you'll need this. Exactly. I mean, we're going to hide that behind these buttons
00:22:33.916 --> 00:22:35.865
that are doing that crunching on the back end.
00:22:36.068 --> 00:22:38.626
I mean, there's going to be like a pro selection menu where it's like,
00:22:38.626 --> 00:22:42.413
okay, like you can tweak anything you want. We don't guarantee functionality, right?
00:22:42.709 --> 00:22:45.856
So that for the power users. But generally for most users, it's going to be,
00:22:45.856 --> 00:22:51.608
again, driven by function. And so we'll do all the setting tweaking under the hood. Gotcha. Okay.
00:22:52.113 --> 00:22:57.916
So looking ahead, how do you see advances in AI image processing algorithms
00:22:57.916 --> 00:22:59.596
and new sensor modalities
00:23:00.008 --> 00:23:05.776
influencing the next generation of drone vision systems for OEMs and integrators
00:23:05.776 --> 00:23:07.944
and machine vision industries.
00:23:08.309 --> 00:23:12.524
I'm just thinking of your 12 years that you have in this industry,
00:23:12.814 --> 00:23:15.589
maybe looking at crystal ball for the next 12.
00:23:16.267 --> 00:23:21.816
But yeah, so I guess I'll leave with a bit of a hot take. I think AI is,
00:23:21.816 --> 00:23:24.816
of course, being overused. I think that's obvious to everybody for a lot of
00:23:24.816 --> 00:23:26.193
things that it doesn't need to be used for.
00:23:27.081 --> 00:23:30.888
It's not going to change in the drone world either. So you've already seen this
00:23:30.888 --> 00:23:34.685
over the years, but even more so now, you're going to see companies trying to apply AI
00:23:35.074 --> 00:23:40.228
and relatively expensive in terms of compute models and whatnot to do very mundane
00:23:40.228 --> 00:23:44.958
tasks that could be solved with simple linear regression data analysis principles, right?
00:23:44.958 --> 00:23:49.238
So not AI at all, but just basically input-output, you know, formulaic stuff.
00:23:49.720 --> 00:23:53.288
So I think you're going to see a lot of overuse of AI, which is going to be
00:23:53.288 --> 00:23:55.926
really computationally expensive and inefficient.
00:23:56.204 --> 00:23:59.608
And therefore the ultimate product that these various companies hypothetically
00:23:59.608 --> 00:24:03.118
are selling is not going to work that well because, again, it's a budget.
00:24:03.118 --> 00:24:08.175
It's a triage problem. If you're not blowing all of your compute on like relatively
00:24:08.395 --> 00:24:11.348
dumb use of AI, you're not going to have anything left over and you're not going
00:24:11.348 --> 00:24:15.148
to be a very effective product. So you're going to see this big boom of these
00:24:15.148 --> 00:24:17.688
companies promising the world with AI models on their drones.
00:24:17.708 --> 00:24:20.878
They're not going to do that much. Then there's going to be some sort of contraction
00:24:20.878 --> 00:24:24.908
or consolidation or whatever you want to call it about how they use AI are going
00:24:24.908 --> 00:24:28.728
to rise to the top. And hopefully we're the latter, but like we're humble.
00:24:28.728 --> 00:24:31.168
And of course, we're learning stuff every day. So we'll try to be better,
00:24:31.168 --> 00:24:32.277
just introspect on that.
00:24:32.910 --> 00:24:36.578
Terrific. Well, I can't thank you enough for spending a little bit of time with
00:24:36.578 --> 00:24:38.512
us today, Arthur. Thanks again.
00:24:38.866 --> 00:24:43.431
Until next time, this is Sharon Spielman with Vision Systems Designed. Stay curious.
00:24:44.795 --> 00:24:48.868
Well, that's a wrap for this episode of Visions, produced by Endeavor Business
00:24:48.868 --> 00:24:51.028
Media, a division of Endeavor BTV.
00:24:51.900 --> 00:24:54.898
Thanks very much for tuning in. If you enjoyed today's show,
00:24:54.898 --> 00:24:58.698
be sure to subscribe to the podcast and share this episode with a colleague
00:24:58.698 --> 00:24:59.650
who would find it helpful.
00:25:00.300 --> 00:25:06.188
Until our next episode, you can find us at vision-systems.com or on LinkedIn,
00:25:06.188 --> 00:25:10.848
Facebook, or X for more insights, updates, and breaking news to keep you in the know.
00:25:11.608 --> 00:25:15.306
Thanks for tuning in. Until next time, stay focused on your visions.