TRANSKRIP 🔗
WEBVTT
00:00:06.480 --> 00:00:07.679
Hello everyone.
00:00:08.080 --> 00:00:09.039
Good afternoon.
00:00:09.599 --> 00:00:18.879
Thanks for your time, and uh it's really my pleasure to hear, to be here, and invite all uh guest speakers here.
00:00:19.280 --> 00:00:39.439
Uh today we would like it's a very special special section because uh we invite we invite from Silicon Vendor to application developer from uh from worldwide, I would say, from Korea, from India, and yeah.
00:00:39.600 --> 00:00:42.079
So it's a very interesting for me.
00:00:42.240 --> 00:00:56.880
And uh to discuss about HAI is a really a very complex thing because it's not only uh we cannot tackle the only single angle from Metro Silicon or from system or hardware or software.
00:00:57.119 --> 00:01:05.040
Actually, from our point of view, it's a very complex and a very fragmentized situation uh area.
00:01:05.280 --> 00:01:17.359
And uh we right now everybody is talking about AI, but uh we I think it's a time to deep dive into some challenges and uh where's the opportunities.
00:01:17.599 --> 00:01:23.920
So I would like to bring all of them and share their experience.
00:01:24.079 --> 00:01:35.439
They are very, very professional ones, and I would like to uh have a free conversation with all of you and let's see what we can co-work together.
00:01:35.680 --> 00:01:40.719
All right, so uh today we have uh uh five panelists.
00:01:40.879 --> 00:01:44.879
Well, the first one I would like to introduce uh Hen from Quakam.
00:01:45.200 --> 00:01:48.079
He's uh uh BD director in Taiwan.
00:01:48.319 --> 00:01:48.640
Hen?
00:01:49.439 --> 00:01:50.239
Hey, hello everyone.
00:01:50.319 --> 00:01:53.439
It's the Pleasure to represent the CoCon to join the event.
00:01:53.680 --> 00:01:58.959
It's Hen from Kokong Taiwan, and I'm in charge of the IEOT, the business developer in Taiwan.
00:01:59.040 --> 00:02:00.239
Yeah, good to see you guys.
00:02:00.640 --> 00:02:01.200
Thank you.
00:02:01.359 --> 00:02:05.519
And hence my good friends in Taiwan we work in some projects.
00:02:05.680 --> 00:02:09.599
And the second is uh is uh Edward from Reset.
00:02:09.759 --> 00:02:09.840
Yes.
00:02:10.159 --> 00:02:11.199
Hi, my name is Eldar.
00:02:11.439 --> 00:02:18.479
I'm part of AI COE tech lead for Renaissance, uh, mainly working on the software and the hardware IPs.
00:02:18.639 --> 00:02:20.639
So it's very nice to meet all of you.
00:02:21.599 --> 00:02:22.240
Great.
00:02:22.560 --> 00:02:31.919
And uh then uh Daniel, Daniel from Enerzai, I think uh you know Daniel because he delivered a uh workshop with uh with our team member today morning.
00:02:32.080 --> 00:02:35.039
It's a very sparkling, sparkling workshop.
00:02:35.280 --> 00:02:36.879
I hope you you didn't miss it.
00:02:37.280 --> 00:02:38.319
Okay, Daniel.
00:02:42.400 --> 00:02:46.719
Hi everyone, uh I'm Daniel from Enerzai, and I'm the CEO and co-founder of Energy.
00:02:47.120 --> 00:02:49.520
So Enerzai is an HGI software
Load full transcript
company.
00:02:49.599 --> 00:02:55.360
We build very small uh language models, vision models that could run on edge with minimum power and memory.
00:02:55.520 --> 00:02:59.199
So it's a great honor to be with all uh of the panels here today.
00:02:59.360 --> 00:02:59.840
Thank you.
00:03:00.319 --> 00:03:01.360
Thank you, Daniel.
00:03:01.520 --> 00:03:04.319
And then uh Piush, Piyush is from India.
00:03:04.479 --> 00:03:13.039
They are doing very special things, and he their company uh is uh they develop models to detect the tigers in the forest.
00:03:13.280 --> 00:03:19.759
Yeah, so today very it's pleased to have uh have a Piush here to share his experience.
00:03:20.479 --> 00:03:22.080
Thanks for the kind introduction.
00:03:22.240 --> 00:03:23.280
Very happy to be here.
00:03:23.439 --> 00:03:24.800
Hi everyone, I am Piush.
00:03:24.879 --> 00:03:27.039
I lead this company called Kudrat AI.
00:03:27.199 --> 00:03:30.960
We operate out of India, majorly in the forest areas of India.
00:03:31.120 --> 00:03:40.639
So, in in terms of what we work on, we make these AI-powered cameras that we deploy in forest areas for all sorts of forest management related uh work.
00:03:40.800 --> 00:03:45.520
And again, we focus on the actual application of AGI-based applications.
00:03:45.680 --> 00:03:47.360
So, yeah, very happy to be here.
00:03:47.840 --> 00:03:51.120
Okay, it's a very interesting application, right?
00:03:51.680 --> 00:03:54.800
So uh last but not least, our uh Iung.
00:03:54.960 --> 00:03:58.960
Ihang is uh belongs to our our team, AdventTech YSEH.
00:03:59.439 --> 00:04:02.400
Uh pre-sales, she's in charge of a pre-sales manager.
00:04:02.560 --> 00:04:03.840
So uh take this chance.
00:04:03.919 --> 00:04:07.120
I would like to bring here to introduce uh some of our experience.
00:04:07.439 --> 00:04:07.840
Ihan?
00:04:08.240 --> 00:04:10.800
Okay, hi everyone, and thank you for having me today.
00:04:10.879 --> 00:04:23.199
Uh and I work closely with Hank and with our partners to bring in our hardware expertise optimization centered around use cases so that we have the best fit uh to deliver the solution in an optimum manner.
00:04:23.360 --> 00:04:24.160
Yeah, okay.
00:04:24.319 --> 00:04:24.879
Thank you.
00:04:25.360 --> 00:04:26.399
Okay, thanks.
00:04:26.639 --> 00:04:29.920
So uh from this come back to our topic today.
00:04:30.000 --> 00:04:54.000
Uh I think the first and um most fundamental things that we see that um in in the AI era, uh one of the interest most interesting things is that uh in the past 10 years, if we say cloud, we always think about uh AWS, Microsoft, Oracle, all our software giant.
00:04:54.319 --> 00:05:01.519
But in the AI era, we found that chip vendor is uh plays uh important roles.
00:05:01.839 --> 00:05:08.160
So what's the main difference and why the chip becomes so important?
00:05:08.399 --> 00:05:16.399
Of course, we say if we talk about uh cloud, uh we need to very powerful computing power, but
uh how about age?
00:05:16.800 --> 00:05:29.680
Yeah, so what's the why all the chip vendors are invest huge, especially with if you know that uh in the past one month there are many interesting developments in Qualcomm, right?
00:05:30.000 --> 00:05:53.120
So uh I would like to invite uh Hen and Elder so to explain what's your uh what you are doing and uh what's the main directions from both of your company and what's the main values you would like to provide to the uh developers or to the markets, please.
00:05:53.360 --> 00:05:54.399
Yeah, please, yeah.
00:05:54.720 --> 00:05:57.120
Yeah, thanks for asking the question.
00:05:57.360 --> 00:06:04.720
Uh I I think uh you can see in the especially for the HAI, we know the couple the advantages.
00:06:04.879 --> 00:06:08.639
For example, like low latency and also some privacy.
00:06:08.800 --> 00:06:18.959
But you can see quo-con in the H AI, we offer like, because the AI sum is the far H, far H you probably need a faster response, but not the so heavy AI.
00:06:19.120 --> 00:06:23.519
And also in the far H to the near H, probably need a higher AI.
00:06:23.680 --> 00:06:38.240
So you can see the Qual Kong, especially in the hardway point of view, we can provide like lower AI one tubs to the even to the uh 870 tubs the computing power to over in the portfolio.
00:06:38.399 --> 00:06:45.519
But still, we saw the if you saw the developer journey, even the hard way itself probably not good enough.
00:06:45.839 --> 00:06:52.399
So uh in the past few past years, you can see Kokong do a lot of like acquisition and also the things.
00:06:52.560 --> 00:06:56.399
It's trying to make the developer is more frictionless.
00:06:56.720 --> 00:07:04.240
So that's the major important things, how to the AGI uh to enter the last mile to do the landing.
00:07:04.319 --> 00:07:07.920
So let's watch the developer journey Kokong trying to do.
00:07:08.000 --> 00:07:10.639
So you can see the H impulse, the friends here.
00:07:10.800 --> 00:07:20.160
So the aging post in the March, we work uh a coefficient with the HMP post and also to work in the last mile, how to help the customer to landing.
00:07:20.319 --> 00:07:24.319
In the meantime, in the last month, you can also see the Arduino announcement.
00:07:24.560 --> 00:07:31.839
Kokong also acquired the Arduino, try to uh play the role into the developer ecosystem.
00:07:32.000 --> 00:07:39.199
So from the developer to the real the enterprise product, and in the last mile, we also have the function.io.
00:07:39.759 --> 00:07:44.800
They can do the IOT, uh the OT uh the OTA, the device management.
00:07:44.959 --> 00:07:50.560
So from the Kokom point of view, we talk about the AI, but we're really, really serious about the lending.
00:07:50.720 --> 00:08:03.040
So you can see the all the AGI and all the offering we saw less will help the customer how to fast do the lending and to real realize the AGI in the field.
00:08:03.199 --> 00:08:03.439
Yeah.
00:08:03.759 --> 00:08:04.079
Okay.
00:08:04.639 --> 00:08:05.199
Awesome.
00:08:05.360 --> 00:08:17.920
So uh what I what I hear is that uh Qualcomm right now, when when you uh focus on AGI, you are very, very focused on how to help the developers, communities.
00:08:18.000 --> 00:08:18.720
Yeah right?
00:08:18.959 --> 00:08:24.480
So because those professional developers can help you to expand your whole territories.
00:08:24.639 --> 00:08:31.920
And so how to help them to easier uh having their resource is uh one of your main missions.
00:08:32.240 --> 00:08:32.879
Yes, yes.
00:08:33.200 --> 00:08:34.000
All right, great.
00:08:34.399 --> 00:08:37.519
And you can see we also do a lot of some pre-work.
00:08:37.679 --> 00:08:43.360
For example, like the probably already one year, we we support like the AI hub.
00:08:43.759 --> 00:08:49.759
So the major thing is beyond the Kokon the MPU, we still need to run the integer A.
00:08:49.919 --> 00:08:56.720
So some customer is the field plan, oh, they need to convert the model to optimize in the Kokon chipset.
00:08:56.960 --> 00:08:59.919
So we host the AI hub as open source.
00:09:00.000 --> 00:09:06.799
Then we do the pre-optimation on the Kokong solution about the the couple uh the AI model.
00:09:06.960 --> 00:09:09.840
In the meantime, we also provide like the AI appliance.
00:09:09.919 --> 00:09:18.879
We just announced this year, that's the one uh H AI can run the Lama for the 108, 109, the billion the parameter.
00:09:19.200 --> 00:09:22.720
So we also support all the hacking phase, the the open source.
00:09:23.039 --> 00:09:29.679
So that's the things we we do the front work and those the infar to support the developer to create an application.
00:09:30.000 --> 00:09:30.399
Awesome.
00:09:30.559 --> 00:09:31.759
Sounds great, right?
00:09:32.000 --> 00:09:47.360
So please, uh what's your your opinion and uh please share your realistically, first of all, if we look at the Edge I foundation itself, every year you have more and more people joining and participating.
00:09:47.519 --> 00:09:50.639
This just shows that the Edge I itself is growing.
00:09:51.039 --> 00:10:09.039
Um from a statistical perspective, if you check from McKinsey or you check any Gardner's reports, they say daily there's around 400 petabytes of data generated, and almost 80 to 90 percent of it is not used because realistically sending it all to the cloud is a bit hard.
00:10:09.360 --> 00:10:14.159
Uh from a Renes perspective, we're largely a hardware vendor.
00:10:14.320 --> 00:10:24.399
We have a very scalable portfolio going rather than going very high end, we go very low end from 16-bit all the way to 64-bit uh MPUs.
00:10:25.120 --> 00:10:28.559
Um we look at it from I think three perspectives.
00:10:28.720 --> 00:10:29.840
One is hardware.
00:10:30.000 --> 00:10:40.399
So the hardware itself goes 16-bit MCUs, 32-bit ARM, and 32-bit uh Renesus IP itself and high-end 64-bit MPUs.
00:10:40.559 --> 00:10:46.320
Uh we have a large
variety of AI accelerators as well, from MCUs to MPUs.
00:10:46.480 --> 00:10:57.440
Um while we provide a lot of, let's say, flexibility from a hardware perspective, this adds a challenge of how to support how to support models to be deployed on these hardwares.
00:10:57.600 --> 00:11:02.639
As the MathWorks team mentioned, that they have compilers to run on a variety of hardware.
00:11:02.799 --> 00:11:06.399
Uh, this is very challenging, to be honest, to run, I would say.
00:11:06.639 --> 00:11:09.279
Um so we do have our own compilers.
00:11:09.440 --> 00:11:15.440
Um we run multiple front ends such as TensorFlow Lite, PyTorch, ExecuTorch, and on an X.
00:11:16.000 --> 00:11:25.679
Uh we lower it down and then we deploy it on a variety of our hardwares and we make sure that the APIs are kind of synchronized so that they all run on the same on the variety of hardware.
00:11:25.840 --> 00:11:34.320
And the last thing, this is for like more high-end customers who have their own AI teams, but also we need to enable the developers themselves.
00:11:34.480 --> 00:11:47.039
Uh, we provide a lot of variety of kits that we have a lot of examples starting from a lot of application examples, very simple post-processing and pre-processing that you can use, but also a variety of models that can be very easily deployed.
00:11:47.279 --> 00:11:50.240
And finally, we also have a lot of partners that we work with.
00:11:50.399 --> 00:12:02.799
A lot of them are part of the AGI Foundation, and we provide production ready models for customers who don't really want to invest into developing their own models, but they just want to have ready-made solutions.
00:12:03.200 --> 00:12:03.519
Okay.
00:12:03.919 --> 00:12:19.759
So I can hear some keywords from uh Qualcomm and Renas as to uh that is their focus on one is that uh they would like to focusing on developer community, which are of you, right?
00:12:20.080 --> 00:12:37.360
So uh this is a very interesting translation because in the past, normally uh developer uh I would say uh all the other chip vendor or even advent uh system integration providers like AdvanTech, we are more focusing on B2B business.
00:12:37.519 --> 00:12:47.120
But right now, developers becomes a very important, or I would say it's uh main focus we would like to uh engage with.
00:12:47.679 --> 00:12:50.080
So uh it's a very interesting part.
00:12:50.399 --> 00:13:02.480
But uh another the second the second keyword relevant to the first one is that how to help the developers to solve the problem of complexity, right?
00:13:02.799 --> 00:13:07.440
From one single uh platform to another or to another.
00:13:07.679 --> 00:13:10.960
Yeah, I think that's uh very, very critical issues.
00:13:11.200 --> 00:13:18.960
So uh I will leave this uh question later for for the other uh in the in uh in the other speech uh panelists.
00:13:19.120 --> 00:13:23.600
But uh right now I would like to shift to the our focus to real world.
00:13:24.639 --> 00:13:34.879
So Daniel, what's from your from your experience, because you are you helped uh many customers develop uh different kinds of AI model.
00:13:35.600 --> 00:13:41.360
So why the customers are shifting to age or what's the big buyers?
00:13:42.080 --> 00:13:52.080
What's their most important uh criteria to consider uh H AI or developer AI model?
00:13:52.320 --> 00:13:56.799
Because right now, many of many of the many of our customers said, what's the ROI?
00:13:57.039 --> 00:13:59.840
How could I evaluate a return?
00:14:00.000 --> 00:14:04.720
I think it's uh very important uh questions.
00:14:04.960 --> 00:14:06.720
So what's your your idea?
00:14:07.039 --> 00:14:09.200
Uh okay, thanks for a very good question.
00:14:09.360 --> 00:14:20.000
So I think the real challenge, uh most of the clients who uh consider deploying like migrating the AI workloads from data center to edge uh would be cost, obviously.
00:14:20.080 --> 00:14:22.799
And uh we've been doing this before ChatGPT.
00:14:22.879 --> 00:14:27.519
So uh for your information, uh it's been six years uh since we founded our company.
00:14:27.679 --> 00:14:31.200
So we've been deploying uh AI models on Edge.
00:14:31.360 --> 00:14:34.639
Uh when we first started this, we mainly focused on computer vision models.
00:14:34.720 --> 00:14:37.519
And back then, I think the motivation was different from now.
00:14:37.679 --> 00:14:46.960
So the motivation, the main motivation was to, you know, as Hen mentioned, uh uh was latency or like privacy, security, or network connectivity issues.
00:14:47.120 --> 00:14:53.279
And that was uh obviously a very important factor uh to migrate those workloads on edge.
00:14:53.440 --> 00:14:57.440
However, like you know, the demand is shifting from computer vision models to language models.
00:14:57.600 --> 00:15:01.679
Many, many clients they want to enable language or voice functions on their devices.
00:15:01.919 --> 00:15:07.200
In that case, uh more than latency, they uh face this cost issues a lot.
00:15:07.440 --> 00:15:12.159
So uh I think the real challenge that they face is the cost issue again.
00:15:12.399 --> 00:15:40.159
And uh it's like the AI API cost, it might seem really cheap when we just uh consider that like per token level, but at the end of the day, considering that our clients have to deploy the same function, they connect all the AI APIs to hundreds of millions of their devices, like after a year they find their cost ballooning or like them and and I mentioned during today's workshop, uh the more problematic thing would be those costs are unprojectable.
00:15:40.240 --> 00:15:45.120
So it's really a big threat for your business, like to plan your business or build your business model.
00:15:45.360 --> 00:15:49.840
So uh based on that, we try to disc we help our clients.
00:15:49.919 --> 00:15:56.559
Uh uh, I face our team helping out our clients to disconnect from those expensive AIPIs.
00:15:56.639 --> 00:16:09.279
And the
challenge for us for that would be the gap between the target hardware, like the computing resource of the target hardware and the model size or the complexity of the model, the gap is has become bigger than the computer vision model.
00:16:09.440 --> 00:16:13.039
So uh Energy's approach for that would be that we go full scale.
00:16:13.200 --> 00:16:19.120
So uh as full scale, we define ourselves as providing a company providing a full-stack edge I solution.
00:16:19.200 --> 00:16:32.639
So we start from the model design, we define the project target with our clients, and based on that target or requirements, we do the model design, train it, and compress it, and not only compress it, we like push the to the limits.
00:16:32.799 --> 00:16:36.159
Uh again, I gave it through this uh today's workshop.
00:16:36.240 --> 00:16:43.039
Uh we do like one extreme low-bit quantization, one bit quantization for language models, so we could squeeze the memories uh real small.
00:16:43.120 --> 00:16:49.679
And also we have our own AI compilers so that uh we uh we make sure that it's executed properly.
00:16:49.840 --> 00:16:55.919
So we have to go full scale, we just don't automate a certain portion of this uh AI development or deployment part.
00:16:56.080 --> 00:17:08.079
I think uh this might not seem like sexy or like uh easy to scale, but for now uh we're a team focusing more on providing value and uh uh you know solving the real problems that our clients have.
00:17:08.319 --> 00:17:20.480
So to summarize, uh I think the cost issue is the biggest uh motivation of our clients uh migrating their AI workloads to edge, and I think it will be uh the demand will grow uh as time goes by.
00:17:20.559 --> 00:17:20.720
Yeah.
00:17:21.200 --> 00:17:24.400
It's a really amazing achievement from 4-bit to 1 bit.
00:17:24.640 --> 00:17:24.799
Yeah.
00:17:24.960 --> 00:17:33.039
So could you please just share some any any uh case so that uh can we give us uh some concrete uh example?
00:17:33.359 --> 00:17:34.400
Thanks for mentioning it.
00:17:34.480 --> 00:17:39.759
I might have uh uh missed the most important part or the most attractive part of our team.
00:17:39.839 --> 00:17:42.960
So I could share the most recent uh rollout case.
00:17:43.039 --> 00:17:48.640
Uh actually last Thursday, last week, uh we rolled out our uh uh language model.
00:17:48.799 --> 00:17:59.680
So it would it's a language model for voice control, and it runs on a very uh uh low end, uh not low end, but a very like constraint uh computing resource system on chip.
00:17:59.920 --> 00:18:06.319
Uh and we deployed that to two million set of boxes back in Korea uh for our client called LGU Plus.
00:18:06.720 --> 00:18:08.400
Uh and it worked out really good.
00:18:08.480 --> 00:18:09.759
The performance was really nice.
00:18:09.920 --> 00:18:17.039
Actually, they they uh like compared it with the AI APIs they which they could enjoy connecting through the cloud, but they found it really nice.
00:18:17.200 --> 00:18:23.279
So eventually we commercialized it and rolled out uh actually rolled out so the end users could use it last Thursday.
00:18:23.440 --> 00:18:32.079
So and that model actually was running below 100 megabytes, which I can emphasize because it was quite a constraint that we had when we first kicked off our project.
00:18:32.160 --> 00:18:32.400
Yeah.
00:18:32.799 --> 00:18:34.000
Oh, sounds great.
00:18:34.319 --> 00:18:34.640
Okay.
00:18:35.200 --> 00:18:38.400
So Piush, your turn.
00:18:38.640 --> 00:18:41.519
I would like to hear your story.
00:18:41.680 --> 00:18:41.839
Yeah.
00:18:42.160 --> 00:18:43.039
Um yeah, sure.
00:18:43.200 --> 00:18:49.599
So before I start, maybe how many people have been to forest areas like in this group?
00:18:50.079 --> 00:18:51.519
Probably not a lot, right?
00:18:51.839 --> 00:19:00.160
So like for for us people living in urban areas, it's very tough to understand the challenges that people in in those uh rough terrains face.
00:19:00.319 --> 00:19:09.519
So I can just give you a context that uh just from India's point of view, like there is almost 10,000 kilometers of forest and village boundary that is present.
00:19:09.759 --> 00:19:13.839
And unfortunately, that entire boundary boundary is totally unmonitored.
00:19:14.000 --> 00:19:18.559
And the only way to monitor it till now has been through manual human patrolling.
00:19:18.880 --> 00:19:26.319
But now you can understand, right, how biased or how ineffective such patrolling would be to like do it 24 by 7, right?
00:19:26.559 --> 00:19:34.160
So the next layer of like solving that problem would be to deploy cameras or existing old hardware-based solutions.
00:19:34.319 --> 00:19:43.119
Now the problem with those solutions comes is like since there nothing is happening at the edge, they are essentially transmitting all sorts of data, whatever they are getting.
00:19:43.200 --> 00:19:49.599
And unfortunately, due to doing that, a lot of their battery gets quickly depleted, let's say within a month.
00:19:49.839 --> 00:19:53.519
Now, imagine you have to cover one 10,000 kilometers of length.
00:19:53.599 --> 00:19:56.799
You might need hundreds of thousands of such sensors in field.
00:19:57.119 --> 00:20:01.680
And if you have to replace the battery every few weeks, it's not going to scale, right?
00:20:01.839 --> 00:20:08.799
So that's where I think edge-based solutions can be very, very important, especially for the work we are doing.
00:20:08.960 --> 00:20:13.680
So I can just give you context like what we are doing in India right now.
00:20:13.920 --> 00:20:19.359
Battery life has expanded from less than a month to like six months or more than that.
00:20:19.440 --> 00:20:23.839
So few cameras are running which have been running on a single charge for more than six months.
00:20:24.000 --> 00:20:27.759
So the way we are achieving that is using AGI.
00:20:28.000 --> 00:20:35.440
Like we are able to segregate all sorts of noise and only send selective data to the end users who then react based on that data.
00:20:35.599 --> 00:20:51.759
So now the advantage of AJ has been since the battery life has increased so much, they can easily deploy more and more such solutions in the field, and they are also more comfortable in scaling such solutions because it ultimately has become more reliable for them to scale it up.
00:20:51.920 --> 00:20:57.200
So, yeah, I mean Edge has played a major role in what we are targeting right now.
00:20:57.359 --> 00:21:06.559
Currently, it has been limited to like increase in the battery life, but the future potential is limitless because data privacy and these kind of concerns we always get, right?
00:21:06.640 --> 00:21:15.920
Like if we can eliminate, for example, if we have a use case where the end user does not allow human images to be sent to the server, they only need certain wildlife species.
00:21:16.079 --> 00:21:19.839
So we all now also have the capabilities to only send wildlife images.
00:21:20.079 --> 00:21:23.599
So that from that point of view, data privacy problem is also solved.
00:21:23.680 --> 00:21:27.839
And again, like it's a very like rapidly developing technology.
00:21:27.920 --> 00:21:31.680
I was going around uh through the all the stalls that are present out there.
00:21:31.920 --> 00:21:34.799
There are so many edge-based solutions that are coming out.
00:21:34.960 --> 00:21:45.759
I think the future potential is going to be limitless once we are able to achieve the rise price point of the uh chip itself and also the power consumption as it comes down.
00:21:45.839 --> 00:21:52.480
I think this industry is going to explode, and I see clearly a lot of use cases, especially in remote areas.
00:21:52.720 --> 00:21:56.799
I think that's where edge-based technologies can play a significant role.
00:21:58.559 --> 00:21:59.680
Great story, huh?
00:21:59.920 --> 00:22:00.240
Yeah.
00:22:00.480 --> 00:22:17.039
So uh from chip vendor and from the application vendors, we can we can catch another another uh keyword that hardware and software, we could not separate, we could not no longer separate them apart.
00:22:17.519 --> 00:22:23.680
We have to achieve the optimization, we have to get the cost uh uh reduced cost.
00:22:23.839 --> 00:22:30.559
So software have to consider the hardware, hardware, hardware side vendor side, they are investing in software.
00:22:30.720 --> 00:22:33.839
So the integration part becomes a key issue.
00:22:34.160 --> 00:22:38.480
And I think uh this issue is uh getting more and more important.
00:22:38.640 --> 00:22:55.440
So uh Ihan, could you please to explain why we are developing a new software stack from Advantage View and uh what's the main challenge we are like to solve and uh yeah, what's the value we could deliver?
00:22:55.920 --> 00:22:56.079
Sure.
00:22:56.240 --> 00:22:57.599
Thanks for raising this, Hank.
00:22:57.839 --> 00:23:06.480
So um there are plenty of uh toolkits
around uh doing AI optimization, uh ML ops, uh device management, and vice versa.
00:23:06.640 --> 00:23:12.240
But the problem is right now, once uh you've got your AI solution, you want to move it off the cloud.
00:23:12.319 --> 00:23:13.920
It's people gluing the pieces together.
00:23:14.160 --> 00:23:21.119
So we move from one tool to the other until you're able to pop that model in a container on a device somewhere in the field.
00:23:21.359 --> 00:23:32.160
Now imagine if you have 100 devices, maybe a thousand, maybe five thousand, we're not gonna be able to go there and maybe stick in a USB stick and then upgrade that.
00:23:32.319 --> 00:23:34.240
We probably have to do some kind of OTA.
00:23:34.480 --> 00:23:39.920
And what happens if the OTA is on a different tool chain from your AML ops?
00:23:40.160 --> 00:23:46.640
Then that means a bunch of different people, different teams trying to orchestrate stuff and it it really quickly becomes a mess.
00:23:46.960 --> 00:23:56.240
So what we see here is the opportunity to streamline this pipeline from orchestrating the different tool chains, which we've tried started doing with uh our own hardware.
00:23:56.559 --> 00:24:11.519
So we have access to the low-level kernels and the toolkits because we're designing those, uh supporting that optimized against the different uh chipsets, and then with our partners' applications, we uh land it on specific selected hardware.
00:24:11.599 --> 00:24:20.480
So we actually tune it, we verify it works, and then that gives everybody a bit of uh easier job trying to piece stuff together.
00:24:20.720 --> 00:24:21.039
Yeah.
00:24:21.440 --> 00:24:21.759
Okay.
00:24:22.079 --> 00:24:22.400
Okay.
00:24:22.880 --> 00:24:30.720
Yeah, so uh I think it's a very uh very exciting shift now right now.
00:24:30.960 --> 00:24:37.519
Software, software partners they are engaging hardware, hardware from to software.
00:24:37.680 --> 00:24:52.640
So I would like to ask you to because it's a very good chance, from like uh Daniel and PU, do you have any request or requirement to uh to hand or to Elder or to Advantage?
00:24:52.720 --> 00:24:55.599
Do you have any any any any questions or requirements?
00:24:55.839 --> 00:24:58.720
Yeah, I think uh it's a very interesting conversation.
00:24:59.200 --> 00:25:13.680
So uh require but before like talking about the requirements, you know, actually for this event, like this uh the workshop that I had with my good friend Benson, uh uh I had a chance to use the WIDA container.
00:25:13.839 --> 00:25:19.200
And it was really nice because we had we didn't have much time to prepare for the demo.
00:25:19.279 --> 00:25:31.119
Uh and what I what we wanted to do was to uh you know make a pipeline of like like like uh converting my speech into a text and having an LLM take the text and come out with an answer and stuff.
00:25:31.279 --> 00:25:40.400
So like uh in terms like when when if we had to deal with the entire pipeline, it must have took us a long time.
00:25:40.559 --> 00:25:43.839
So it would it would have been impossible for us to demo in this event.
00:25:44.160 --> 00:25:53.440
But thankfully, uh currently uh Widack container, one of the Wida containers that uh that AdventTech provides has an LLM running on Adrenal GPU.
00:25:53.680 --> 00:25:56.799
So for the demo, I was able to leverage that container.
00:25:57.039 --> 00:26:02.079
Uh and and you know uh our team only had to focus on the STT part.
00:26:02.160 --> 00:26:04.640
So we had a cool demo with Vincent, it was really nice.
00:26:04.799 --> 00:26:18.799
And that show I think that shows like it's it was a demo for today, but thinking that the the time to market is really important uh like for all businesses, and it's it and of course uh it's really important for Energy as well.
00:26:18.880 --> 00:26:23.359
We need to deliver the best working at AI models to our clients as fast as possible.
00:26:23.680 --> 00:26:37.359
In that case, if we already have a you know like ready to develop uh container like the LLM that I use today, if it wasn't for the demo, if it wasn't for a project, it would have saved us a lot of time, like uh development cost and everything.
00:26:37.599 --> 00:26:54.720
So uh I think it's uh it shows how software like providers and also hardware platform providers like Eventech should collaborate to uh so that our end users could eventually enjoy the best working AI models as fast as they could because this AI ecosystem is moving so fast.
00:26:55.119 --> 00:27:03.599
And so based on that, my requirement would be uh having more containers, like more coverage of containers, so that we could you know uh further leverage.
00:27:03.759 --> 00:27:04.079
Yeah.
00:27:05.200 --> 00:27:08.000
I did not ask Daniel to promote our weight up.
00:27:10.319 --> 00:27:10.880
Thank you.
00:27:11.119 --> 00:27:20.240
Yeah, but uh actually uh we found that uh what we are doing is uh for everybody uh today uh we have um some main partners.
00:27:20.400 --> 00:27:25.759
Uh when we chatting, uh we found that everybody's doing the same thing.
00:27:26.000 --> 00:27:30.559
We would like to bring the ecosystem just like uh what uh Daniel said.
00:27:30.640 --> 00:27:36.240
And uh I think uh Qualcomm and uh Reness, they are doing the similar thing.
00:27:36.480 --> 00:27:43.680
We want to bring uh the old value trend together and deliver the value to the to the market.
00:27:43.839 --> 00:27:56.559
So yeah, so maybe Pew and uh Hen and uh you can you can you can just uh talk about how do you manage your ecosystem and what would you like to uh do in the future.
00:27:57.119 --> 00:28:08.160
Yeah, so like adding to the point that you just mentioned that all the different vendors are sort of working on the same thing and eventually merging uh in their synergies.
00:28:08.640 --> 00:28:20.880
I think from from my point of view, we as an application or a direct deployment company, like it's very confusing for us to
choose a particular vendor because this industry is changing so so rapidly and so so quickly.
00:28:21.039 --> 00:28:36.480
So getting the right price point and also getting the right uh application behind it to solve our challenges is very critical because like every month some vendor is announcing a new uh chip, which is of course a very good thing for the industry in general.
00:28:36.559 --> 00:28:46.160
But from my point of view as an application company is to figure out what what exactly we need to go in, because in hardware the iterative cycle is is very long.
00:28:46.319 --> 00:28:54.799
So that's a big pain point because once you choose a particular IC to move from that, it's it's a whole like a pain point, right?
00:28:54.880 --> 00:28:56.960
So it's not that easy from our point of view.
00:28:57.119 --> 00:29:05.519
So if we can figure a way out that this is it, and we can stay on it for at least three to four, maybe five years, I think that's good for our business.
00:29:05.759 --> 00:29:11.599
And one more thing is like from my point of view, I I'm totally bullish on this industry, AGI.
00:29:11.680 --> 00:29:21.440
I fundamentally believe every sensor is going to become a smart sensor, urban areas, remote areas, and I think the potential is trillions of sensors being smart enough.
00:29:21.599 --> 00:29:30.160
It's just that what I'm curious, like whether like as a like an in industry player is like how smart can it become?
00:29:30.319 --> 00:29:32.799
Like what can it solve at the edge level?
00:29:32.960 --> 00:29:42.319
Like that's a very uh I mean open point, like in five years from now, how the industry will look like and what sort of pain points can we solve with it.
00:29:42.480 --> 00:29:53.759
So yeah, so uh ecosystem is important and uh but for how we're I I I said uh how we're in the software, we need to start to integrate together.
00:29:53.839 --> 00:30:08.160
But uh actually it's a very important very extremely difficult thing because uh hardware is a long term, it's uh Our path is uh one, two, three, but software is uh it's so how to combine together is a really challenging thing.
00:30:08.400 --> 00:30:14.640
So maybe Han and Elder you can you can share your yeah.
00:30:15.039 --> 00:30:27.599
Yeah, I I I think the you mentioned the ecosystem, but uh how to grow the ecosystem uh Cochrang is very uh focused on to give the ecosystem developer the friction is the experience.
00:30:28.160 --> 00:30:33.519
So that's the major focus we we try to build from the OS level, SDK level.
00:30:34.000 --> 00:30:37.680
Of course, still uh uh still have room to improve.
00:30:37.920 --> 00:30:41.039
We keep doing it and also we're working with like advantage.
00:30:41.279 --> 00:30:44.960
Like they will have the domain know-how can tell us what we can do better.
00:30:45.119 --> 00:30:51.039
So that's the major things we'll try to improve the ecosystem, even the incubated ecosystem.
00:30:51.279 --> 00:31:02.720
In the meantime, we also to build the ISV program uh from the region and try to incubate the the ISV and to develop the ecosystem.
00:31:02.799 --> 00:31:02.880
Yeah.
00:31:03.200 --> 00:31:19.440
We can we can we can feel that uh Qualcomm's uh efforts because uh in uh two years back uh there was a very famous developer, very professional developers, they said, Qualcomm stuff, no, sorry, I can't find anything you know over the net over the internet.
00:31:19.839 --> 00:31:26.960
But uh today I met him, he said, Oh, Qualcomm is a really good, very interesting development.
00:31:27.119 --> 00:31:30.960
So I think it's a very, very eminent progress.
00:31:31.039 --> 00:31:34.000
Yeah, so we expect it to see in the future.
00:31:34.240 --> 00:31:45.200
And I would say two main pain points are the first one is embedded engineering and machine learning engineering are very distinct silos.
00:31:45.440 --> 00:31:53.440
And um finding people with the knowledge of embedded and ML is not that common.
00:31:53.839 --> 00:32:09.759
And lowering the gap for young, newer students to learn both embedded and ML, uh, we should provide, I guess, more documentation and more training material to embrace the newer generation to go embedded ML.
00:32:10.079 --> 00:32:20.240
Because currently there is a lot of, let's say, machine learning engineers and embedded engineers working together, but they still have a separation in how they develop their models.
00:32:20.400 --> 00:32:31.599
Sometimes if you're going for the lower end, the machine learning models that they develop either will not have operators to be supported on the hardware, or they're just too big to fit in.
00:32:31.920 --> 00:32:36.799
So I guess one of the requests would be to have more embedded ML engineers in general.
00:32:37.279 --> 00:32:49.599
And the second one is from the higher end, there's not it's not that big of an issue, but at the lower end of MCUs, there's a very big heterogeneity of hardware and AI accelerators.
00:32:50.160 --> 00:32:58.240
And how do you deploy them differs from vendor to vendor and from IP to IP.
00:32:58.480 --> 00:33:19.039
So if you, for example, develop a model to deploy on specific, let's say, hardware, it may not be deployed on another hardware because some of the um AI IPs will not just support such uh operators and they might go software fallback, and this may cause a lot of degradation in the inference speed.
00:33:19.200 --> 00:33:29.440
So from a silicon vendor perspective, maybe there should be some discussions on how to agree more on more set of operators that everyone should support.
00:33:30.000 --> 00:33:38.079
And also there's a huge variety of front ends to support, such as recently also Executors came into play in version one.
00:33:38.240 --> 00:33:44.799
So now as a silicon vendor, we need to support another frontend because customers are coming up for requests.
00:33:45.440 --> 00:33:46.319
Okay, thank you.
00:33:46.559 --> 00:33:48.160
Yeah, very impressive.
00:33:48.480 --> 00:33:53.359
I would like to pause here and uh ask do you have any questions?
00:33:53.440 --> 00:33:56.400
And uh, would you like to join us for this conversation?
00:33:56.640 --> 00:34:00.559
I think this is a very, very precious opportunity.
00:34:04.160 --> 00:34:07.759
I think uh we have oh we have uh run over time.
00:34:08.000 --> 00:34:16.400
Yeah, but uh if any question, please take a question is uh very, very, very precious time.
00:34:17.840 --> 00:34:22.239
Okay, so uh give me uh last mu last one minute, okay?
00:34:22.400 --> 00:34:23.199
Yeah, okay.
00:34:23.360 --> 00:34:26.320
Yeah, anyway, we have there is no other session.
00:34:26.559 --> 00:34:29.599
So last question to all of you.
00:34:29.840 --> 00:34:33.599
Uh what do you see in the next five years?
00:34:33.840 --> 00:34:48.079
What will be uh maybe HAI or AI or this world, whatever you you would like to say, what will what's the main trend you have seen, right, from now?
00:34:48.960 --> 00:34:49.679
Ihil, maybe.
00:34:49.920 --> 00:34:50.079
Yeah.
00:34:50.320 --> 00:34:58.159
So I think what we see now as edge AI gets closer to where the data is happening, it's going to be a convergence of disciplines.
00:34:58.400 --> 00:35:16.880
So you've got people who's taking care of the data and the devices, you've got people who have the skill sets to optimize the AI, and you definitely need the know-how from the chipset people, because that's what's making the stuff run real quick and fast enough to respond in real time uh to the problems that are discovered in the data.
00:35:17.119 --> 00:35:36.000
Yeah, but this has to be done as together as a um as a uh crowdsourced knowledge together in the ecosystem because I think if I quote my friend Nicholas, who was uh there at Milan, if we're only looking at one single part of that puzzle, it's just going to be a purely science endeavor.
00:35:36.159 --> 00:35:38.800
It's not gonna be engineering, it's not gonna be deliverable.
00:35:41.039 --> 00:35:42.480
Yeah, Han, please.
00:35:45.199 --> 00:35:46.880
Uh I want to give you an example.
00:35:47.039 --> 00:35:48.880
It's actually the five years ago.
00:35:49.039 --> 00:35:55.840
We worked with some factory about the cable assembly, the AGI checking, the SOP checking.
00:35:56.159 --> 00:36:07.199
At that time, uh because we work with the partner, they mainly do the data collection and also to do the model fine-tuning laboring, we take six months.
00:36:08.000 --> 00:36:14.239
Six months, then later on, if the enterprise customer they want to change the SOP, then we do six months again.
00:36:14.480 --> 00:36:16.960
So that's the experience just five years ago.
00:36:17.119 --> 00:36:19.519
But right now we discuss about the next generation.
00:36:19.679 --> 00:36:29.599
So right now, next generation, because some VLM, some technology improvement ecosystem, we have the auto-laboring tool, some ecosystem concrete.
00:36:29.760 --> 00:36:32.159
So the schedule right
now is totally different.
00:36:32.400 --> 00:36:33.280
Total different.
00:36:54.400 --> 00:36:56.719
And the two have the AJI to boom in.
00:36:57.039 --> 00:36:57.679
Yeah.
00:36:58.480 --> 00:36:59.599
Awesome, great.
00:36:59.840 --> 00:37:08.400
Yeah, so Daniel, what's your how will you see you how what's your your vision and uh how you'll see from your field side?
00:37:08.880 --> 00:37:18.000
So uh as an HGI solution provider perspective, obviously like applications like like uh robotics or physical AI is coming.
00:37:18.159 --> 00:37:25.199
And considering that, I think uh we have to define like really look into how to define the role of HGI would be.
00:37:25.360 --> 00:37:31.440
So uh even though we provide the very low memory, very small models to our clients so that they can use it.
00:37:31.519 --> 00:37:34.159
Like some of the clients they use multiple models of ours.
00:37:34.320 --> 00:37:36.559
And as time goes by, it will stack, obviously.
00:37:36.719 --> 00:37:41.920
And we can't just uh like small models won't won't be sufficient to fix their problems.
00:37:42.159 --> 00:37:49.199
So I think we have to we we never go against the cloud, like AI that's provided to through the data center.
00:37:49.280 --> 00:38:07.599
And in that perspective, uh we define our enterized role to come out with a really good classifier or like a function calling function within that runs on edge, so they could really classify uh like seamlessly classify what has to remain on edge and what we need to send to cloud.
00:38:07.920 --> 00:38:09.199
That's more realistic, I guess.
00:38:09.280 --> 00:38:34.880
So I think in five years, considering that robotic, like thinking of robotics, we should we would have many, many models running on that single endpoint, and we will have those kind of uh bunch of robots uh around us, uh not maybe in five years, but I think within five years we would you know at JI's role would be focusing on you know being a really good function caller or a classifier to divide which would go to cloud and which would remain on edge.
00:38:34.960 --> 00:38:39.840
Yeah, so I feel that uh I I found that uh everybody's uh echo to each other.
00:38:40.159 --> 00:38:53.599
From uh, he said uh we need to consider the how how the real-time decision making can bring the revenue, can bring the benefit to the real world.
00:38:53.760 --> 00:39:01.199
And uh we the technology is boosting, and many exciting things are emerging.
00:39:01.440 --> 00:39:08.639
So and same as uh uh Daniel said, uh there's no longer cloud-ge separation.
00:39:08.800 --> 00:39:17.760
In the future, we'll mingle everything, we will restructure the technology, fit to into fit into the Daniel uh uh business purpose, right?
00:39:17.840 --> 00:39:23.760
So I think uh we we have seen the uh this is uh coming coming in the very near future.
00:39:23.920 --> 00:39:27.280
So can you what's your perspective your perspective?
00:39:27.760 --> 00:39:35.679
I think to begin
with, uh what Mateo said in the presentation today earlier was every six years there's a tsunami of a new technology.
00:39:35.840 --> 00:39:45.840
So what I personally am very excited about is integrating Gen AI on the MCU level itself, how smart it can make our edge devices, like five years from now.
00:39:46.159 --> 00:39:50.000
I think it's going to be like like totally going to change everything.
00:39:50.239 --> 00:39:59.440
So I yeah, from my point of view, I can all all I can just say is like how smart our cameras can become, or like how much can we do it at the edge level itself.
00:39:59.519 --> 00:40:05.280
So that is what I'm very interested about, especially with Gen AI based uh technology right at the edge level.
00:40:05.360 --> 00:40:08.719
So use your time.
00:40:09.599 --> 00:40:10.960
I would say two things.
00:40:11.199 --> 00:40:17.920
One is from the reports that say that 80 to 90 percent of data generated right now is not being used.
00:40:18.239 --> 00:40:26.880
I would hope that that number would go down and most more of the data is actually being used for to do more processing on the edge itself.
00:40:27.360 --> 00:40:34.639
Uh and the second thing I would say is Edge I and Tiny ML was a very big thing and very interesting thing.
00:40:34.800 --> 00:40:40.079
Right before ChatGPT, it was the most interesting thing, and then ChatGPT took everything by storm.
00:40:40.400 --> 00:40:52.159
I would expect that within the next five years, that Edge AI will be the most back on top where you would be able to deploy more SLMs on uh actual devices.
00:40:52.400 --> 00:41:02.960
And I would expect that right now when we deploy our partners or customers deploy AI on our silicon or maybe other silicon vendors, silicon, uh it's still very novel.
00:41:03.199 --> 00:41:06.559
It's still very new, it's still very, oh wow, they deployed this AI.
00:41:06.639 --> 00:41:14.320
And if you see any, let's say industrial automation or anything, consumer products, they may mention in the marketing we have AI.
00:41:14.480 --> 00:41:20.639
But I would expect that within the next five years, AI is a thing that everyone just uses rather than it's something novel.
00:41:20.880 --> 00:41:23.360
And yeah, that's yeah, interesting.
00:41:23.599 --> 00:41:41.519
So today from today's uh discussion, I at least I learned one thing is that uh right now we are I think we are not uh in a paradigm shift, not only technology side, but also it's a business value washover.
00:41:41.679 --> 00:41:42.079
Yeah.
00:41:42.320 --> 00:41:46.960
So you can see the Single Venture starting to do the application.
00:41:47.440 --> 00:41:52.639
And uh application vendor, they are starting to engage in the sensor.
00:41:52.719 --> 00:42:01.199
They want to make it more intelligent, make it more uh uh power consumption, so make it more optimized.
00:42:01.360 --> 00:42:08.960
So everybody in the future, maybe there in between there are multiple value chains in between.
00:42:09.039 --> 00:42:11.840
But right now they started to talk to each other directly.
00:42:12.239 --> 00:42:16.079
So I think uh it's it will not uh it's not only technology part.
00:42:16.480 --> 00:42:19.039
It will be a complete mess of.
00:42:19.199 --> 00:42:21.760
But uh I think the opportunity will be there.
00:42:22.639 --> 00:42:23.119
Thank you.
00:42:23.280 --> 00:42:24.400
Thank you for your listening.
00:42:25.039 --> 00:42:25.360
Thank you.