TRANSKRİPT 🔗
WEBVTT
00:00:06.559 --> 00:00:07.519
Morning everyone.
00:00:07.839 --> 00:00:19.359
My name is Sensog and I'm from Seoul, South Korea, and I'm CEO and co-founder of the JTG AI, and we are revolutionizing the on-device AI deployment through the Lepid implementation.
00:00:20.399 --> 00:00:24.480
So basically, the company, the JTG AI's vision is very clear.
00:00:24.640 --> 00:00:34.320
We'd like to bring on-device AI to the new era, and we'd like to bring AI for everyone and anytime and anywhere.
00:00:35.439 --> 00:00:44.240
So let's think about what is on-device AI, as you know already, like edge AI, on-device AI, serverless AI, in those kinds of terms.
00:00:44.479 --> 00:00:52.000
So bring uh the on-device AI and HAI brings AI without GPU cloud server.
00:00:52.159 --> 00:01:04.799
So with uh by removing the GPU cloud server, we can reduce a cost a lot because you know every company spends a lot to maintain their AI services through the GPU cloud server.
00:01:05.120 --> 00:01:24.560
It's not only it costs a lot, it also like the maintenance fee is also like kind of like a huge burden for the companies because you know it's not only like that expensive for establishment, but they have to maintain through the like the huge data center with the like maintenance like human resources.
00:01:24.799 --> 00:01:35.519
So by bringing the AI service without GPU Cloud Server, we can reduce a lot of cost for the data server, but also for the maintenance fee for the human resources and others.
00:01:35.840 --> 00:01:40.400
And plus that, there are other AI service enhancements that on-device AI brings.
00:01:40.560 --> 00:01:49.040
So, first, by removing the like network connection for the AI services, we can bring the AI to be secured the user's data.
00:01:49.200 --> 00:01:54.000
And we can like we can guarantee the user's data is kept in user's devices.
00:01:54.239 --> 00:02:06.239
Plus that it can we can it can service uh through the like any kind of environment, even for the like aircraft, like submarine, even for the like spacecraft.
00:02:06.640 --> 00:02:22.639
So, plus that, you know, every AI services like send the user's data to the GPU cloud server, so they have like network latency through the like network infrastructure, but we can guarantee the real-time AI by running at running the AI through the user's devices.
00:02:22.879 --> 00:02:29.680
So let's think about the on-device AI, how is how is how it is consistent with.
00:02:30.000 --> 00:02:34.879
So I'd like to say there are three key elements that are needed for uh on-device AI.
00:02:35.039 --> 00:02:40.240
The first, the powerful target device that could replace the GPU cloud server.
00:02:40.400 --> 00:02:48.719
So GPU cloud server is like, even though it is very expensive, people are using it because it is quite like powerful to run the AI services.
00:02:48.879 --> 00:02:56.639
So we need the replacement like devices to like run AI through the devices around us.
00:02:56.879 -->
Load full transcript
00:03:05.599
Second, there are like the resources are limited for those kinds of devices, so we need the optimization for the AI uh AI models.
00:03:05.840 --> 00:03:12.000
Plus that we need the like runtime software to run the AI model at the target devices.
00:03:12.240 --> 00:03:14.240
So let's think about the target device first.
00:03:14.319 --> 00:03:20.879
So we do you know any kind of like target device that is powerful that is like changing our daily lives?
00:03:21.199 --> 00:03:26.400
I'd like to say there is a MPU in our like in in near around us.
00:03:26.560 --> 00:03:30.080
So MPU is powerful enough than the CPU and GPU.
00:03:30.159 --> 00:03:38.560
It it uh so MPU like performs up to like 60 times faster than the classic GPU CPU and GPU.
00:03:38.639 --> 00:03:51.840
And the target objective of the like MPU development was the like is the uh is the like reduced energy consumption, so it reduced the energy consumption for the AI services up to 95%.
00:03:52.800 --> 00:03:57.520
So do we need to like do we have to buy a new one for the MPU?
00:03:57.680 --> 00:03:58.960
I'd like to say it's not.
00:03:59.120 --> 00:04:04.319
So we already have MPU, which is AI target processor, in everyone's palm and pocket.
00:04:04.479 --> 00:04:07.120
It's a smart it's inside of the smartphone.
00:04:07.280 --> 00:04:16.480
So Apple and like Qualcomm and those kinds of like mobile chip manufacturers had released the MPU from 2017 and 2018.
00:04:16.639 --> 00:04:24.399
So from iPhone A and Galaxy S10, everyone has the AI processor in their palm and pocket in everywhere.
00:04:24.639 --> 00:04:26.720
So now it's 2025.
00:04:27.040 --> 00:04:34.959
Everyone in here has a AI pro has an AI processor in every in your like near around you.
00:04:35.439 --> 00:04:42.720
So the key is that MPU is like ready to utilize for the AI services near around us.
00:04:43.839 --> 00:04:53.360
So I'd like to say we are building the on-device AI ecosystem with utilizing those kinds of MPU with the first uh with the fastest on-device AI software.
00:04:53.600 --> 00:05:00.800
So first, we are utilizing the MPU to guarantee you the like fastest on-device AI environment.
00:05:01.040 --> 00:05:10.160
Second, even though MPU is quite like fast enough, people don't really know about, people don't really know about how MPU can be utilized.
00:05:10.319 --> 00:05:14.639
So we are providing the fastest way to utilize the MPU all at once.
00:05:14.879 --> 00:05:23.680
Plus that, even though you know everything about on-device AI, there are like tons of tasks to implement your AI model or AI service to the on-device AI.
00:05:23.759 --> 00:05:28.160
So we cover the end-to-end comprehensive support for the on-device AI.
00:05:28.399 --> 00:05:40.399
Plus that we covers every environment in the mobile area, not only for the like specific environments, for the like some kind of A companies like intelligence or like G companies AI.
00:05:40.560 --> 00:05:45.920
So we'd like to have we'd like to cover all like operating system or all environments right around us.
00:05:46.079 --> 00:05:50.720
We cover the Android, iOS, and mobile Linux and everything around us.
00:05:51.439 --> 00:06:03.519
So I I would say that we guarantee the world fastest on-device AI software, and I'd like to show you that how we can guarantee that the software is fastest.
00:06:04.639 --> 00:06:11.199
So, first we get the we get the AI model for the AI services to deploy it in the mobile environment.
00:06:11.279 --> 00:06:19.439
And second, we optimize from the like general neural network optimization, and plus that, we compile the model for the different targets of the devices.
00:06:19.680 --> 00:06:38.240
So we compile the model for the like Apple and Qualcomm and MediaTech and other companies as well in the different types of the format to utilize the MPU uh separately, and then we run the actual benchmarks through the more than 200 devices in the wall to guarantee you the fastest performance.
00:06:38.560 --> 00:06:52.560
So after getting the fastest performance through the CPU and GPU and MPU, we provide you the benchmark report and we deploy the AI model with the fastest runtime, like uh measured fastest runtime performance.
00:06:53.120 --> 00:06:56.639
So here's how it works for the AI users.
00:06:56.959 --> 00:07:05.759
So the development uh we uh we had collected a lot about how do you think about the on-device AI for the developers.
00:07:05.920 --> 00:07:21.199
The reason why, you can like see a lot of on-device AI and at GI demos through the like demo tables and the YouTube and everywhere, but people don't really, the people don't really had like uh experience to deploy their AI services to the target devices.
00:07:21.360 --> 00:07:30.399
So the developers usually say that it is quite hard to learn what is on-device AI or and and how to implement it the on-device AI.
00:07:30.560 --> 00:07:39.120
And even they learn about the on-device AI, they have to try how how their AI model is like worked at the target devices.
00:07:39.360 --> 00:07:49.519
So we'd like to make uh kind of like the easiest way and fastest way to try their AI model to deploy it at the target device in the very first time.
00:07:49.759 --> 00:08:05.600
So people can measure how their AI actually works at the mobile devices and the target environment, and then if if they can like uh satisfy for the performance itself, they can just like deploy their AI models and services to the like target devices all at once.
00:08:05.839 --> 00:08:14.879
So we provide the only one stack for the developers to try and actually deploy the AI model to the target devices.
00:08:15.120 --> 00:08:23.040
So first we after first the developer use developer uh uploads the AI model to the like our like software dashboard.
00:08:23.199 --> 00:08:34.000
We can we run the model optimization and the target compilation, and then we provide the easiest way to try the like act uh try to measure
the actual performance.
00:08:34.159 --> 00:08:47.360
We generate the QR code and if the you if the developer scan the QR code at their uh like their smartphone, we provide them the your uh your model is running on your environment with how like latency with the CPU and MPU.
00:08:47.519 --> 00:08:55.120
And plus that while the developer is doing that, we run the actual benchmarks through the like 200, more than 200 global devices.
00:08:55.279 --> 00:08:59.759
And then we provide the benchmark report, plus that we generate a runtime source code.
00:08:59.919 --> 00:09:09.759
So they can just copy and paste the runtime source code to the like their like Android stream or X code or those kind of like runtime software project itself.
00:09:10.080 --> 00:09:14.639
So by you this is the actual benchmark report that we are providing for the users.
00:09:14.720 --> 00:09:23.440
So this is the we uh as you can see, the first thing we provide a benchmark report for the like CPU and MPU and every target processor's latency and accuracy as well.
00:09:23.759 --> 00:09:40.399
And then user can just like copy and paste the with uh to implement that type of the like benchmark reported like AI model at their uh project by just copying and pasting those kind of like generating source code.
00:09:40.639 --> 00:09:49.360
So we are providing the easiest way to implement the on-device AI for users because there is no target devices needed because everyone is already have.
00:09:49.600 --> 00:09:58.240
Plus, that we generate the like runtime source codes just for the copy and pasting to uh make the AI like project uh itself.
00:09:58.639 --> 00:10:03.440
So this is the reject by uh made by the copy and pasting those kind of runtime sources.
00:10:03.600 --> 00:10:10.720
So, you know, there are like device limited like AI services in the world, like Apple Intelligence or Galaxy AI.
00:10:10.879 --> 00:10:16.399
But you know, AI AI developer doesn't really have to make the limitation for the devices.
00:10:16.559 --> 00:10:24.399
So by making the application by our software, we provide the easiest way to utilize all the devices all the world or around the world.
00:10:24.559 --> 00:10:28.879
Not only for the Apple, but Galaxy and Xiaomi and other devices all at once.
00:10:30.000 --> 00:10:32.080
Plus, let's think about the AI agent.
00:10:32.240 --> 00:10:36.799
Like AI agent AI agents is a little bit different from the like general AI model.
00:10:37.039 --> 00:10:45.519
So AI agents is like they are collecting uh the like, you know, people don't really have the AI agent model, LLM model.
00:10:45.919 --> 00:10:51.039
So open source community for the AI agents is quite open for the in the hugging phase.
00:10:51.200 --> 00:11:06.159
So we collect the AI model from the hugging phase for the AI agent because there are tons of people who'd like to utilize the AI agent at the on-device AI in the AI environment, but they couldn't even know how to which model is best and
how to deploy the model.
00:11:06.320 --> 00:11:08.960
So we collect the AI model from the hugging phase.
00:11:09.120 --> 00:11:12.480
And then we do the similar benchmark for the LLM as well.
00:11:12.799 --> 00:11:23.600
First, after you submit the hugging phase modeling to our dashboard, then you can just try your like LLM model right inside of your devices without network connection.
00:11:23.759 --> 00:11:39.759
Plus, that we provide the like uh neural network optimization for the seven different types of the data type, and we provide the benchmarking report for the LLM scoring, not only for the like general knowledge, but also for the like question answering and mathematical problems all at once.
00:11:40.000 --> 00:11:50.480
And then we also provide the source code for the LLM, like the AI agent, and you can just make your own device AI agent by copying and pasting that kind of source code block.
00:11:51.039 --> 00:11:55.600
So this is the actual benchmarking report for the AI agent at the target devices.
00:11:55.919 --> 00:12:12.159
So first we quantize the AI model for the seven different types of the models, and we get we provide you the information not only for the fire size and uh how your actual AI agent is uh uh shows the different results at the uh optimized version.
00:12:12.399 --> 00:12:36.320
So we are going uh we are going to like expand more for the those kind of like LLM scoring, not only for those kind of four four different types of the like LLM scoring, we'd like to move forward for the uh for you to give the inspiration that your AI agent model or the model you'd like to use from the hugging phase is shows this kind of performance at everyone's devices or around the world.
00:12:36.639 --> 00:12:44.480
So you can see that there is a like performance research is a kind of like latency for the for the LLM, which is TPS token per second.
00:12:44.639 --> 00:12:50.720
It shows the how your AI agent actually runs at the like target devices.
00:12:50.960 --> 00:12:57.120
So for the AI agent model, so there are like shoes models as well, not only for the small size of the language model.
00:12:57.440 --> 00:13:12.639
But so we like to like provide everyone to they can provide uh they can deploy AI model with this kind of performance, not only for the like runnable performance, but there are like uh which models cannot be run about the target devices.
00:13:12.720 --> 00:13:17.759
We provide that your model is like uh too big in uh too big to run at the target devices.
00:13:17.919 --> 00:13:30.720
So we can qu uh quantize and comp uh optimize those kind of AI model and provide you that if it is not runnable at the like original score, we you can your model can be run about the like type of the Q2 or Q4, Q8.
00:13:31.440 --> 00:13:34.559
We are providing at those kind of information all at once.
00:13:36.639 --> 00:13:36.960
Yeah.
00:13:37.440 --> 00:13:47.840
So this is the actual like um implementation part of the um our AI
on device AI agent uh deployment system.
00:13:48.159 --> 00:13:51.759
So from we are providing the Android source code and iOS source code.
00:13:51.840 --> 00:14:01.279
It's a little bit longer than the general AI model, it's because you know there are like uh loop for loop is needed for the like AI agent for keep talking about uh research.
00:14:01.759 --> 00:14:09.919
So you can just copy the Android application source code and iOS application source code to the each to the like Android Studio or uh X code.
00:14:10.080 --> 00:14:19.919
And we are going to uh provide you the Flutter and Lyx native as well to support those kind of all different types of devices all at once with the unified software framework.
00:14:20.240 --> 00:14:38.639
And then you can just make the on-device AI agent through the everyone's devices, not only for the Android, but also for iOS, but but and not only for those kind of like CoCom and specified device uh manufacturers, but also for other manufacturers in the world, all at once that everyone has already had having devices.
00:14:40.399 --> 00:14:43.759
So here's the key of our pro our company.
00:14:44.000 --> 00:15:07.360
So I I bet you already like heard about how like how the companies are like providing the AI services or HAI to you, that they are like uh performing how fast it is, their chip it is, and or or how easy to deploy their service for the uh their chip is needed to service at the HAI.
00:15:07.519 --> 00:15:11.759
But I'd like to say those kind of environments are all separated.
00:15:12.000 --> 00:15:22.639
So hardware companies are making their own environment, but that makes a problem for AI developers or software engineers who'd like to utilize their hardware itself.
00:15:23.279 --> 00:15:31.840
So they have to focus more on AI model, but they have to study more to like deploy the AI model at the each devices.
00:15:32.080 --> 00:15:47.120
And since the whole environment for uh hardware companies all separated, so they have to study more, even they even don't even know how their AI model is actually performs at the target devices.
00:15:47.440 --> 00:16:06.639
So we are making a pipeline for people to utilize AI model and perform uh perform the profiling at the target devices, and plus that from the profiling reject, we are providing the easiest way to deploy the AI model with the actual runtime source code and to make the AI services.
00:16:06.960 --> 00:16:12.080
So I'd like to say there are four uh key steps that are needed to make the AI services.
00:16:12.240 --> 00:16:20.320
So, first, I I I uh the one that I uh talked to you about at the very first time, the AI model should be optimized.
00:16:20.399 --> 00:16:27.120
So there are like tons of optimization tools from the NVDN and Apple and other companies who would like to uh optimize the AI model.
00:16:27.519 --> 00:16:34.639
But the AI model optimization doesn't do anything but the uh reduce the like data size and everything.
00:16:34.799 --> 00:16:40.639
So it has to be deployed at the like target library implementation, but it's a totally different stage.
00:16:40.879 --> 00:17:04.799
So even though they optimize the AI model and perform some kind of how their actual model is like benchmark at the target devices, they have to study more to implement it to the target devices, all different types of the manufacturers, like not only for the like mobile environment, but also for those kind of other companies in like below us, so the the Impinion and AXP other companies, they have their own like chips utilization.
00:17:04.880 --> 00:17:10.559
So they have to study their how to utilize their AI hardware to deploy their AI model.
00:17:10.720 --> 00:17:25.839
So we are taking charge to like deploying those kind of AI models to our company, and we provide the easiest way for AI developer to utilize the target library implementation or M1 by our software SDK.
00:17:26.000 --> 00:17:32.319
And plus that, there's other things needed for the on-device AI, actual service application implementation.
00:17:32.559 --> 00:17:44.640
So it's uh, you know, those kind of benchmarking and and showing off how your AI is running at the uh device with the simulation is not enough to make the actual software.
00:17:44.799 --> 00:17:58.960
So we are focusing more on the actual software implementation, is more like important from not only for the optimization, but for the actual uh making the AI services to innovate the world by the serverless AI.
00:17:59.200 --> 00:18:06.240
So we covers everything from AI model optimization and on-device application until the on-device application implementation.
00:18:06.480 --> 00:18:11.519
So, plus that there are like tons of other companies who are making the AI model optimization tool.
00:18:11.599 --> 00:18:15.440
It could be used before our uh before our software stack.
00:18:15.599 --> 00:18:24.160
So you can utilize the like tools, uh optimization tools or around the world by Nvidia or by Qualcomm and every other company, like PyTorch and others.
00:18:24.319 --> 00:18:31.440
And then you can just like use our software SDK for the like for deploying your AI model at the like target environment.
00:18:31.680 --> 00:18:42.880
So we started from the mobile environment, is because you know everyone has mobile phone and smartphone, even though they don't have like laptop or like other AI processors.
00:18:43.039 --> 00:18:54.240
They always already have like mobile and mobile and mobile devices, and mobile devices are recently have the AI target processors, so it is ready to be utilized for the AI services.
00:18:54.480 --> 00:18:57.920
And then we'd like to expand more to the like microcontroller units.
00:18:58.079 --> 00:19:14.559
So I'd like to say we just started to talk with uh some microcontroller unit company, uh and they have like at least 40 more 40 more chips to utilize the AI things for their environment.
00:19:14.799 --> 00:19:21.119
But
you know, if you are an AI developer, you don't know which chip is best for my AI services.
00:19:21.359 --> 00:19:27.200
Which chip could be like best for because there are like 40 more AI chips and they all have different specs.
00:19:27.359 --> 00:19:45.599
And you can not determine which one is best for us, but if you'd like to try at least try to run your AI model, it cannot be only for the simulation, but they don't provide anything but the like kind of like slides to their how their like tops or how per how their chips is actually performs.
00:19:45.680 --> 00:19:49.039
But you should at least try before you buy the chip actually.
00:19:49.200 --> 00:19:59.440
So we are keep talking with them to build the benchmarking and actual deployment like services with our software SCK to make the innovation in their sales stack.
00:19:59.599 --> 00:20:13.519
So if you are a hardware company, you could utilize, you could like uh make an on-device AI ecosystem with us to make the to enable AI developers to use your chips as much as possible.
00:20:13.759 --> 00:20:19.119
Plus, that we are keep talking with the Samsung with uh who is making the Axnos for the mobile environment.
00:20:19.200 --> 00:20:29.519
They'd like to make an on-device AI ecosystem, but you know, they are making hardware, but the software is uh kind of like a lot far from the hardware itself.
00:20:29.599 --> 00:20:44.160
So we'd like to make a like um on-device AI software ecosystem, we and we'd like to connect the hardware company to us, and we are trying to build AI through without the GPU cloud server and make the ecosystem to write around us.
00:20:45.039 --> 00:20:52.880
So let's think about the on-device AI, uh uh, how on-device AI innovate the world without the GPU cloud server.
00:20:53.039 --> 00:21:01.839
So I was saying I was saying that on device we I'd like to bring AI to the right uh anywhere, anytime, and everywhere around us.
00:21:02.000 --> 00:21:03.599
So anyone around us.
00:21:03.759 --> 00:21:08.799
So let's think about the AI without the network independence functionality.
00:21:09.119 --> 00:21:26.240
So, you know, recently the subways are like supporting the Wi-Fi and things, but you know, still there are like lots of the underground facilities that cannot run the like network network infrastructure for the like serve for mobile and everything.
00:21:26.400 --> 00:21:37.039
So we are bringing AI without those kinds of network connections, and not only for the underground that I said about the underwater and rural area and everywhere.
00:21:37.279 --> 00:21:46.000
Plus, that from doing the real-time AI performance, we can make the AI services like ready to use for every environment.
00:21:46.160 --> 00:21:48.240
So let's think about the medical AI.
00:21:48.400 --> 00:22:02.079
So medical AI and those kind of like vision, like medical vision things are making a kind of AI services and innovation through the like uh like detecting some cancel and things from
the medical vision and medical images.
00:22:02.319 --> 00:22:22.000
But since the their like AI, like AI infrastructure for the AI services far from the hospital, they are sending the like uh image process available like uh data, making by making it to the like video and send the video to the like GPU cloud server for the AI services.
00:22:22.240 --> 00:22:26.400
But what if we can guarantee them the real-time AI performance?
00:22:26.640 --> 00:22:37.839
They don't have to make the image to the video, they can just like do the like AI services for the medical data at the right inside of the surgery area, surgery room or the hospital, everywhere in the hospital.
00:22:38.000 --> 00:22:40.799
And they don't even need any kind of additional device.
00:22:40.960 --> 00:22:47.440
They can just like put some tablet or smartphone for the like for the environment they like to do something for the AI.
00:22:48.000 --> 00:22:59.759
Plus that there are like sound classification and noise cancellation, and plus that those kind of like TTS and ASL and those kinds of all AI services can be runable at without like the turret connection and they perform very nice.
00:22:59.920 --> 00:23:05.119
And you know, everyone already knows that they are quite learnable right around us.
00:23:05.599 --> 00:23:25.759
Plus that, you know, since the like ChatGPT and like cloud and everything is run in the uh GPU cloud server at the company side, everyone sends everything for the like ChatGPT and everything, uh ChatGPT and those kind of AI agents, but you know, we cannot send everything for the company.
00:23:26.079 --> 00:23:29.440
So we have to secure our our data for the AI things.
00:23:29.519 --> 00:23:42.720
So we by uh implementing on-device AI to user devices and smartphone and everything, we can secure your your data, your finance data, your like medical data, and your old kind of data is secured in your environment.
00:23:43.839 --> 00:23:46.799
It doesn't send any data to the like company side at all.
00:23:46.960 --> 00:23:51.920
So they cannot collect because their your AI services is running in your environment.
00:23:52.079 --> 00:23:56.480
And plus that we can make a kind of like personalization for the AI agents.
00:23:56.559 --> 00:24:00.400
So your like AI model is running at your environment.
00:24:01.359 --> 00:24:13.039
So you can we can like collect your like personal data and make your own AI personalized like assistant and everything through the like brand by bringing the subless AI through the on-device AI.
00:24:14.000 --> 00:24:16.480
So it was uh so thanks for the attention.
00:24:16.559 --> 00:24:23.039
And I'd like to show you that we just opened this open source project as last our last guy.
00:24:23.119 --> 00:24:35.680
So we make we are making uh we are trying to make the ecosystem for the like on-device AI, and we'd like we are trying to, you know, there are like tons of things to do for the like AI implementation at the target devices.
00:24:35.839 --> 00:24:57.839
So we are making the on-device AI infrastructure for the like AI runtime, plus that we are expanding more to the like pre-processor, pre-processor and post-processor for and hold together at the pack as a package to make the AI service the easiest way for the like AI developer and like in the world by making an innovation by of the like software of the on-device AI.
00:24:58.960 --> 00:24:59.440
All right.
00:24:59.680 --> 00:25:01.359
So thanks for the attention.
00:25:01.440 --> 00:25:21.200
Um yeah, please try so because our software is totally free right now, and we are collecting uh use cases, and we'd like we are going to move make the model zoo and we are going to open it so you can even though you don't have the AI models on and anything, you can try or you can build your AI application by our software as as a free.
00:25:21.599 --> 00:25:22.160
All right.