Accelerated Computing in Modern Data Centers With Datapelago

WEBVTT
NOTE
Transcription provided by Podhome.fm
Created: 03/08/2025 21:08:04
Duration: 3335.472
Channels: 1
1
00:00:11.280 --> 00:00:15.460
Hello, and welcome to the Data Engineering Podcast, the show about modern data management.
2
00:00:16.155 --> 00:00:22.654
Data migrations are brutal. They drag on for months, sometimes years, burning through resources and crushing team morale.
3
00:00:23.355 --> 00:00:26.654
DataFold's AI powered migration agent changes all that.
4
00:00:26.955 --> 00:00:35.360
Their unique combination of AI code translation and automated data validation has helped companies complete migrations up to 10 times faster than manual approaches.
5
00:00:35.900 --> 00:00:40.239
And they're so confident in their solution, they'll actually guarantee your timeline in writing.
6
00:00:40.700 --> 00:00:46.415
Ready to turn your year long migration into weeks? Visit dataengineeringpodcast.com/datafolds
7
00:00:46.415 --> 00:00:47.555
today for the details.
8
00:00:48.095 --> 00:00:58.995
Your host is Tobias Macy, and today I'm interviewing Rajan Goyal about how to drastically improve efficiencies in data processing by reimagining the system architecture. So, Rajan, can you start by introducing yourself?
9
00:00:59.370 --> 00:01:08.430
Sure. First of all, pleasure to be here, Tobias. Thank you for taking time. This is Rajan Goel. I'm CEO and cofounder at, early stage data company, Datapelago.
10
00:01:09.050 --> 00:01:25.705
I am here in the valley for last thirty plus years. This is my third start up, but we'll talk more about my journey. But happy to be here at Dubai's. And do you remember how you first got started working in data and why it has kept your attention for this long? Yeah. Well, it is a funny,
11
00:01:26.250 --> 00:01:27.710
My journey to data
12
00:01:28.090 --> 00:01:29.070
world has been,
13
00:01:29.930 --> 00:01:36.190
unorthodox, I would say. My previous life, I used to build security systems, like the network security,
14
00:01:36.490 --> 00:01:39.630
and then I went deep into the technology building processors,
15
00:01:40.090 --> 00:01:41.470
domain specific processors,
16
00:01:42.045 --> 00:01:45.585
and or what's today buzzword is called accelerated computing.
17
00:01:45.885 --> 00:01:48.145
So idea was to how do we make,
18
00:01:48.525 --> 00:02:12.370
a specialized processors, which are programmable and so on, to do a specific function, like security, deep packet inspection, and so on. So that was my previous life. We were building both MIPS and ARM based processors, and we were pretty successful, relatively speaking. And just before this we put our hands on the data processing problem, we were working on data moment problem, which is sounds boring,
19
00:02:12.815 --> 00:02:23.795
but all the modern cloud infrastructure to data bias is built on that principle. You have this disaggregated architecture. If I may take one second few seconds to explain that. If you look at AWS
20
00:02:24.095 --> 00:02:28.349
or Google or Azure, you as a user want to put take a server of
21
00:02:28.810 --> 00:02:31.470
you say, give me 32 codes, four drives,
22
00:02:31.849 --> 00:02:41.455
one GPU, 25 gig network, or some other variant. Guess what? There is no physical server that's queue. You are composing a software defined
23
00:02:41.995 --> 00:02:45.455
server on the fly from a disaggregated set of infrastructure.
24
00:02:45.755 --> 00:02:54.290
Alright? So you have your CPU racks, your storage racks, your GPU racks, and drive racks, and then connected over high speed network, and then you're dynamically composing,
25
00:02:55.069 --> 00:03:05.650
this skew that you're looking for. So in my previous life, I was working on technologies to enable this modern infrastructure that we see today. We enjoy today at AWS or any public cloud.
26
00:03:06.055 --> 00:03:15.995
And in fact, AWS Nitro, if you're familiar with that or for your audience, is built on some of the technology that I built in previous slides. So coming back to your question, sir, I have a spending more time
27
00:03:16.375 --> 00:03:17.755
on making the infrastructure
28
00:03:18.135 --> 00:03:19.515
data centers more efficient,
29
00:03:19.820 --> 00:03:48.194
how to move their data efficiently between compute to compute or storage to compute and so on. So what occurred to me was with Mellanox, with Nvidia, with my previous companies, all these companies have more or less solved the problem. Obviously, it has to continue to improve, but fundamentally solved. So I observed that the next next problem bottleneck will be in the competition. Now you're able to move data. Let's actually process it. And there are two hills in the data processing, Tobias. One is taken by the AI
30
00:03:48.570 --> 00:04:10.985
data model training and inference, which is also data processing, but in a different way. And another is the data processing itself to prepare for AI or for our classic analytic. And that, I think, was left untouched to keep up with the shift in the infrastructure that was happening, which I was kind of fortunate to be part of my previous life. And, hence, my attention got to this. Okay. This is a I mean, it's a very well
31
00:04:11.365 --> 00:04:12.885
hundred $80,000,000,000
32
00:04:12.885 --> 00:04:13.785
market today.
33
00:04:14.260 --> 00:04:37.525
So it's a pretty large business. Problem is big. It's relevant, and why don't we apply our self to solve? That's how kind of a more bottom up engineering mind took me to this world of data, and ever since, it is fascinating. It's validating what we started with, but that's how we got in. I personally got into this world of data coming from computing, storage, and networking world. And digging into some of those
34
00:04:37.985 --> 00:04:38.965
hardware level
35
00:04:39.425 --> 00:04:40.965
capabilities and constraints,
36
00:04:41.639 --> 00:04:51.900
most people who are working in this space, they're going to be fairly familiar with the system architecture of their laptops or their desktops, so the Von Neumann architecture,
37
00:04:52.280 --> 00:05:03.705
which has been prevalent for decades at this point. And as you mentioned, because of the requirements and the high throughput and the high scale of these data centers and the, cloud providers,
38
00:05:04.085 --> 00:05:06.824
you mentioned that they have a slightly different
39
00:05:07.285 --> 00:05:19.550
way of composing those different pieces together to be able to give that flexibility. And I'm wondering from your perspective and from the understanding that you've gathered from working in that space, how that fundamentally changes
40
00:05:19.930 --> 00:05:32.745
the ways that people need to be aware of the compute capabilities and how they link together versus the model that they're likely working from with that Von Neumann approach that of the collocated
41
00:05:33.045 --> 00:05:41.610
CPU, RAM, disk, and just how that factors into the capabilities of actual data processing and data movement in those environments?
42
00:05:42.070 --> 00:05:49.455
Yeah. Yeah. So let's break your question into maybe three topics, if I may, okay, to give the proper context. The first
43
00:05:49.914 --> 00:05:54.895
is the classic idea of I think Nutanix invented this world called hyperconverged.
44
00:05:55.275 --> 00:05:59.294
Idea was you put it in a single pizza box, if I may use that example,
45
00:05:59.675 --> 00:06:38.250
co located your CPU, DRAM, storage, networking, and that's your becomes your cookie, and you kind of replicate it thousand times or 10,000 times, a hundred times to get your scale up. Right? That was the then came the model of disaggregated, not hyper but disaggregated but composable infrastructure that we just talked about that. Right? Now in that regard, now things are far apart. Obviously, networking has solved the problem to make it appear, like, low latency and so on. How do we not spend CPU cycles in that? But it is still not a co located world. So now you have storage tracks, compute racks on separate over network. The third shift, if I may, or next subsequent to that is the compute itself is no longer homogeneous.
46
00:06:38.630 --> 00:06:42.010
The last decade, we, as an industry, spent time to
47
00:06:42.485 --> 00:07:05.400
everybody standardized on Intel or AMD x 86 architecture, and you'd write software once. And Intel is supposed to do every eighteen to twenty four months doubling the performance, and life is good. Right? But that train has stopped. Moore's law has flattened, and you're not doubling the CPU performance every generation to generation. So what's next? Next step function is accelerated computing. Now you have this heterogeneous
48
00:07:05.700 --> 00:07:09.160
elements, CPUs, GPUs, FPGAs, PPUs,
49
00:07:09.775 --> 00:07:13.835
you know, AWS has their own chips. Now you you're not you're not finding a homogeneous
50
00:07:14.135 --> 00:07:22.715
server. You're finding an heterogeneous server. So that's the reality in the infrastructure which is happening. So if you look observe this and ask a question,
51
00:07:23.700 --> 00:07:35.480
if you are a data now coming back to your overall team, if this is the world we are born in, now if you are a data practitioner writing Python code or a PySpark code or writing SQL queries,
52
00:07:35.865 --> 00:08:10.255
you cannot be bothered by GPU kernels or FPGA programming. Right? So how how those people who are building this platform bridge that gap? That that's where companies like Datapelago play that role to build intermediate layers between the Python programmer versus the underneath underneath heterogeneous infrastructure that we are providing. But that system in data centers are built around what that stack should look like. That's kind of maybe high level view of how it should appear. So there are two ways. Now one is the users of the data platform. One are the builders of the data platform. Like, we are the at in our world, we are the building the data platforms using this infrastructure.
53
00:08:10.635 --> 00:08:30.340
But our goal or goal should be that how do we make the same abstraction that people are used to today? If I have to tell a Python programmer how to write a CUDA kernel, it's not gonna work. Right? So how do we abstract it out a level that whatever the today's data scientist or business analyst or the actual people who are using writing applications,
54
00:08:30.720 --> 00:08:43.045
that abstraction remains same, and we can hide this complexity with software modular layer is the new problem statement. Right? And, hence, the Datapelago or other companies like us would want to bridge that gap that has emerged
55
00:08:43.345 --> 00:08:47.410
because of this shift in underlying infrastructure device. Another division
56
00:08:47.710 --> 00:08:49.250
in the data ecosystem
57
00:08:49.790 --> 00:08:56.370
is well, there are several, but one is open versus closed where you have these vertically integrated
58
00:08:56.750 --> 00:09:00.610
closed proprietary stacks in the form of your Redshift,
59
00:09:01.095 --> 00:09:02.315
Snowflake, BigQuery,
60
00:09:02.695 --> 00:09:18.210
systems like that. And so they're built specifically for a specific problem, and they're optimized because of the fact that they control the end to end use case versus these open and composable stacks that have been gaining ground, especially in the past three to four years
61
00:09:18.590 --> 00:09:36.944
of the data lake house architecture, which is an outgrowth of the all of the work that came from Hadoop and a lot of the processing that happened there and then these different offshoot projects. And now they've largely oriented around a handful of table formats that make it easier for the different compute engines to be able
62
00:09:37.404 --> 00:09:45.319
to execute to a specific target, so iceberg, Hudi, Delta Lake. And because of the fact that those are all evolving
63
00:09:45.620 --> 00:09:49.399
separately from each other, they're all developed by different teams. Coordination
64
00:09:49.779 --> 00:09:56.680
is done generally by loose consensus or, you know, very lengthy RFC and proposal processes.
65
00:09:57.065 --> 00:10:15.060
And I'm curious how you see the variance in terms of the capabilities and the evolution of those two ecosystems of these proprietary vertically integrated systems versus these open and composable stacks and how that affects the overall efficiency and capabilities of those systems.
66
00:10:15.440 --> 00:10:18.019
First, let me answer this more fundamentally, Tobias.
67
00:10:18.320 --> 00:10:26.285
You know, I told you just now that in my previous life, we did this desegregation at infrastructure level. Right? From hyperconverse
68
00:10:26.745 --> 00:10:58.165
to disaggregated architecture was exactly same paradigm if you observe it, which is that instead of building a one hyperconverged server where everything is integrated, store it, your drives, your processor, your memory, and network card, and whatever into one server. And then to take a disaggregated very of storage separated from compute and compute separated from GPUs and so on. Right? It's a very similar paradigm shift is happening here from a using now your example from Snowflake or a Redshift with a monolithic con storage and execution engine to a disaggregated
69
00:10:58.625 --> 00:11:02.084
where you have your open table format based data lake versus
70
00:11:02.385 --> 00:11:04.565
an execution engine. I think that's a fundamental
71
00:11:04.945 --> 00:11:06.165
shift has to happen
72
00:11:06.480 --> 00:11:17.140
because one box does not fit for every op. Right? So people need different kind of data formats, different data, different execution capability. So this desegregation has to happen. Right? And that's what industry
73
00:11:17.680 --> 00:11:29.074
kind of went and, did all that. So that's already acceptable norm. Even the let's call them the converge architecture or, like, Redshift or Snowflake, they're also embracing iceberg
74
00:11:29.454 --> 00:11:35.555
or using table external tables and using open table format. So that's already happening there. Right? Now what it enables
75
00:11:36.240 --> 00:11:36.980
is that
76
00:11:37.360 --> 00:11:55.355
now once you are in this new paradigm, now the best engine should win because you as an enterprise leave your data once into your data lake wrapped around your choice of open table format or otherwise. Now you want to have your choice of engine. So there is no vendor lock in. Right? Whether it's a proprietary or whether it's an open source. That's a fundamental
77
00:11:55.894 --> 00:12:25.765
architectural enabling, which is it's enabling is that you can have your own engine. Whether it's an open source or proprietary is different part, but at least architecture allows us to you as an enterprise pick the engine of your choice. That's one part. The second part is, obviously, even in this lake house paradigm, Tobias, you will see some are open source engines like Spark or Trino. And there are some closed source or proprietary even Databricks. Right? They are they are not pushing back their improvements in performance back to open source. So so you will have a,
78
00:12:26.205 --> 00:12:40.780
open source and closed source choices with rightfully so. Some are better performance and have premium, so that's a business decision. But, actually, you would have those. So in both cases. But where where open source plays a role is that standard, open table format,
79
00:12:41.160 --> 00:12:41.660
connectors,
80
00:12:42.200 --> 00:13:14.819
file format, your how what kind of, connectors you use in that SQL side or whatever. Right? So those, if you open it up, then this paradigm has long legs because so that's kind of one point. If I may extend my argument one more further, is that as the data, like Snowflake or Redshift, they were designed for structured and table data when they were designed, right, for some other problem. But now data is shifting. So you want a different engine. So you don't want to have be enforced or forced to ingest the data into existing data warehouse and then only process it. You have to have this external
81
00:13:15.199 --> 00:13:34.055
legs or external connectors to the data in the data lake. So it it is bound to happen that you maybe build a specialized engine for processing unstructured data. Now processing unstructured data requires totally different it's like either you do use models. It's no longer just joins and and, and sort and and,
82
00:13:38.430 --> 00:13:46.690
metadata extraction to process unstructured data. So the engine of that has to marry those two pre past world and the new world. Right? So, hence, I'm a full supporter,
83
00:13:46.990 --> 00:13:49.810
and I think it's gonna enable a lot of innovation
84
00:13:50.334 --> 00:13:51.795
on the data format,
85
00:13:52.495 --> 00:14:02.190
data engines, and the choice of the infrastructure required for each engine, rather than having a monolithic system. That's why I kind of expanded your question, Tobias, but I think I give you enough,
86
00:14:02.830 --> 00:14:17.010
to cover what the question is. Yeah. Absolutely. And on the point too of processing these unstructured data sources and the fact that we can't necessarily rely on these SQL engines that have been the
87
00:14:17.385 --> 00:14:46.250
heavy lifters for several years now. It makes me think that we're almost starting to drift back towards the MapReduce era of you have to write your custom code to turn through all of the data and hope that you got it right because otherwise, you're gonna have to go back and start over, which, thankfully we've gotten better at parallelizing that with systems like Ray and Spark and being able to get better efficiencies out of it. But we're no longer in the case where we have a single language target to be able to process the majority of data. We have these bespoke
88
00:14:46.785 --> 00:14:51.925
and fractal use cases and tool chains for being able to work with these unstructured data sources.
89
00:14:52.385 --> 00:15:07.400
That's true. And I think there's a push and pull here, Tobias. There is a value in preserving the existing language because you have large users of that, and they are familiar with that, and it it is fine. So there is one dimension of this is to extend
90
00:15:08.100 --> 00:15:08.680
the language
91
00:15:09.060 --> 00:15:26.675
semantics as well as syntax to support this new operators. Right? So that's kind of one dimension doing. Other is, obviously, people are using or want to use Python or some other just general purpose language to express your intent and do that. So I think it is still to be proven and seen what what would where would we land once it's all settled.
92
00:15:27.270 --> 00:16:00.720
But I do see initiatives as as value enabling existing user who's familiar with the, SQL, for example, to able to even have in the because if you buy this argument of lake house with a single engine with a unified data store and you say that you want to have a unstructured data semi there's a value in preserving the northbound API. Right? So that from that user, it sees, like, two layers, then I can use it. But, obviously, it it will it will take time to get there. But the other the thing you mentioned, yes, for the ray, having a flattened, if I may call it, distributed framework,
93
00:16:01.340 --> 00:16:21.485
where you as a user can control what your app is doing and then then use the power of ray to just distribute it is is a wonderful idea and that but that's where you more power from the user to how how to write. But, like, SQL world gives you the power of planner and optimizer and so on, which is kind of maybe missing there that we are familiar in the data world. But, yes, I think it's still to be proven,
94
00:16:22.265 --> 00:16:24.899
where language we will we will we will settle over
95
00:16:25.200 --> 00:16:26.660
time. On the point of
96
00:16:27.520 --> 00:16:28.660
the overall ecosystem,
97
00:16:29.520 --> 00:16:31.140
particularly for these disaggregated
98
00:16:31.520 --> 00:16:34.180
stacks having to do a lot of
99
00:16:34.880 --> 00:16:36.100
process and coordination
100
00:16:36.480 --> 00:16:39.955
to be able to make any step changes in capabilities.
101
00:16:40.654 --> 00:16:45.154
For instance, if you were to introduce a new table format today, it would probably
102
00:16:45.855 --> 00:16:57.680
be two to five years before it was actually usable in all of the different processing engines that someone might want to work with. So for instance, Iceberg support for that landed in Spark initially, and then there was some,
103
00:16:58.140 --> 00:17:20.240
initial support in Trino for a while as well as a lot of investment in Dremio, etcetera. If you were to, so the Lance table format is another example where that is a newcomer, and it's starting to be worked on to incorporate it into Trino and some of these other different engines. And because of the fact that it can take so long for these new capabilities to be broadly
104
00:17:20.700 --> 00:17:21.200
available
105
00:17:21.660 --> 00:17:39.385
to different people without necessarily having to bring in a completely different tool chain or rearchitect their data stack, I'm curious what you see as the points of highest leverage in that ecosystem of disaggregated systems to be able to have the broadest change and the fastest
106
00:17:39.685 --> 00:17:42.425
capability of adoption without having to do that rearchitecting
107
00:17:42.725 --> 00:17:43.385
and reimplementation?
108
00:17:44.005 --> 00:17:45.385
I think there are fundamentally,
109
00:17:46.245 --> 00:17:52.800
which is what we just talked in the previous question, is that there is a marriage to standardize the interfaces.
110
00:17:53.420 --> 00:18:02.560
An open table format is one of that interface. How do you deal with your data? Right? So there is a the more we standardize on it and agree on one format,
111
00:18:02.935 --> 00:18:11.755
better it is for the engines to take advantage. That's one part. But I guess the there are two dimensions in which in this context, the status quo will be challenged.
112
00:18:12.055 --> 00:18:13.995
One is the efficiency
113
00:18:14.295 --> 00:18:27.990
part, If whether that that current format is better or efficient. And the second is the for the applications. Right? For the different app application data patterns access patterns for the data. And second is as the data is changing,
114
00:18:28.375 --> 00:18:31.995
because today's date file form the table formats are more
115
00:18:32.375 --> 00:18:44.140
for structured data. But as the data is changing to other multimodal unstructured data, what is that abstraction? Right? How do we make that as a more common format is to be evolved. So it will evolve, but I don't think, fundamentally,
116
00:18:44.919 --> 00:18:45.740
it's gonna
117
00:18:46.200 --> 00:18:52.700
disturb the the relationship between the engine and the and the data. Right? So that's how we see it. But from
118
00:18:53.000 --> 00:19:01.095
my point of view personally and the the where my expertise lies is that the format plays certain role in the overall
119
00:19:01.475 --> 00:19:01.975
systems
120
00:19:02.355 --> 00:19:03.654
performance or efficiency.
121
00:19:03.955 --> 00:19:05.335
That at certain point,
122
00:19:05.875 --> 00:19:12.534
it's good enough. Now you would want to focus on the execution side and the data processing and take advantage of new infrastructure.
123
00:19:12.890 --> 00:19:29.785
That's where we are focused on. Right? So that we leverage whatever the industry will lead us to from the file format point of view. We will support it, but let's focus on the execution side. Because if you think of look at the enterprise bill cost of goods that you are paying 60 to 80% of the compute.
124
00:19:30.245 --> 00:19:33.625
Right? Not so much on the storage or not so much on the networking.
125
00:19:33.925 --> 00:19:53.500
So the compute is where your bulk of the spending is from energy and the and the and the cost point of view. And as the data is getting more and more complex, it is gonna be more and more difficult or more acute problem to optimize. And that's where we are focused on, is how do we make the compute part of the data processing more efficient while still
126
00:19:53.855 --> 00:19:58.035
leveraging all these initiatives in the open table formats and so on. One of the
127
00:19:58.415 --> 00:20:01.795
outgrowths of the past three to four years
128
00:20:02.095 --> 00:20:13.910
in the broader data ecosystem that I think gives a much better point of integration to have a broad impact as well is the broad adoption of the Arrow ecosystem
129
00:20:14.210 --> 00:20:15.830
for data interoperability.
130
00:20:16.450 --> 00:20:32.725
And when I was looking through the documentation for your site, I also saw that you hook into the or early support the Apache Gluten project, which to my understanding is a marriage of the Spark execution and orchestration system with the high performance
131
00:20:33.025 --> 00:20:33.525
libraries
132
00:20:33.990 --> 00:20:39.050
from, I believe, ClickHouse and similar systems. And so digging now into
133
00:20:39.430 --> 00:20:40.730
what you're building at Datapalago,
134
00:20:41.030 --> 00:20:47.510
I'm just wondering if you can talk to what was the point at which you decided that this was the problem that you wanted to solve and,
135
00:20:48.215 --> 00:20:52.155
maybe some of the strategy that you've developed around how to actually tackle that problem?
136
00:20:52.695 --> 00:20:53.995
Yeah. First is
137
00:20:54.295 --> 00:20:55.355
the in memory
138
00:20:55.895 --> 00:21:00.635
format, like arrow, data structure or or now the corresponding engine
139
00:21:01.159 --> 00:21:11.659
has its, its its own place for efficiency point of view. But now we are in a world where it's a this as I said, heterogeneous computing, you use GPUs and FPGAs
140
00:21:12.039 --> 00:21:29.485
with the same in memory format sufficient to extend hardware software code design. We have a data file go. We can we have relooked it and and and extended that. So that's kinda one one part of the problem. And the second one is that, you know, regarding gluten or, or other technologies,
141
00:21:29.809 --> 00:21:33.510
you know, Spark is a wonderful and extensible distributed framework,
142
00:21:33.890 --> 00:21:36.150
but it has it was designed with its
143
00:21:36.530 --> 00:21:43.485
own engine, which was written in Java, Java virtual machine execution engine. So what Gluten allows you to do is that
144
00:21:44.045 --> 00:21:45.745
using my previous life,
145
00:21:46.205 --> 00:21:46.705
analogy,
146
00:21:47.325 --> 00:22:17.174
from networking world where you had people used to build monolithic routers to buy us. Right? You would have a control pin and a data pin monolithic. Right? So it's a very similar analogy in my mind that how do you separate the control pin of Spark from its data plane, which is execution engine? And Gluten plays that role to be able to punch the holes in both sides so that you can plug in your choice of execution engine as a back end. Right? Think of like that. So that's a value that Gluten now given those two clarification
147
00:22:17.554 --> 00:22:19.735
or the context about what it does,
148
00:22:20.355 --> 00:22:37.220
our strategy as with the vision has been is that the what's the gap in the industry? The gap in the industry is that the data engines or the data platform that we're talking about were written ten years back when infrastructure looked different. Now infrastructure is changing underneath.
149
00:22:37.595 --> 00:22:45.295
So what do you do? Either you say, oh, I'm gonna reinvent new Spark, which is for this new world, but you have this whole existing
150
00:22:45.835 --> 00:22:46.335
applications
151
00:22:46.715 --> 00:22:51.950
and workloads written for Spark. What about that? Right? So then you have to figure out a way to
152
00:22:52.330 --> 00:23:21.120
insert the benefits of this, accelerated commuting to this existing ecosystem. Right? That's the view DataPalgo took. And, hence, many of the techniques I have mentioned, some we have not actually publicly mentioned, a lot is to to service that. There, our view is that you build a if I now met tie all the points we have talked, so the world in our mind looks like following. You have a unified data lake. You store all your structured, unstructured, semi structured data there. You have a
153
00:23:21.580 --> 00:23:24.400
universal data processing engine, which is what we are building,
154
00:23:24.860 --> 00:23:29.040
and it runs on the modern infrastructure. It run it handles any data.
155
00:23:29.355 --> 00:23:36.095
And on top, which is more important, it enables any engine, any data engine written in the past to take benefits of,
156
00:23:36.395 --> 00:23:50.390
new hardware and the new data. Right? That's the view we take. And then Gluten and many other technologies that we have internally developed as well as using open source enables us to fulfill that vision of that so that we are not inventing Spark. We are not reinventing Trino.
157
00:23:50.690 --> 00:24:07.325
We want to give the benefit of accelerated computing to existing frameworks. So that's what hence, we have carved out this what we call universal data processing engine to give benefits. So that's our view, Tobias, how we look at it. There have been attempts in the past where people built their own SQL engine, their own,
158
00:24:07.865 --> 00:24:14.580
GPU powered or FPGA powered. Right? So my personal view is that the success of those attempts
159
00:24:14.960 --> 00:24:15.460
were
160
00:24:15.760 --> 00:24:21.460
not because GPU was not good enough or FPGA was not good enough. I think this last one that I just mentioned
161
00:24:21.919 --> 00:24:24.340
was one of the reason why adoption
162
00:24:24.799 --> 00:24:26.805
was not there or or was there.
163
00:24:27.365 --> 00:24:29.785
Because if you don't have a friction to insertion,
164
00:24:30.405 --> 00:24:47.940
then, people can embrace the technology. And given how enterprise are already spending millions of dollar with thousands of users writing code, it will be kind of a nonstarter to suggest, okay. Now I have another engine. Can you learn again and let's stand it up. Right? So that transition, that migration
165
00:24:48.240 --> 00:24:53.059
is is not easy in that case. Right? So, hence, we the way we are doing a data parallel,
166
00:24:53.520 --> 00:24:54.180
is to
167
00:24:54.495 --> 00:24:55.715
not disrupt the
168
00:24:56.015 --> 00:25:10.115
north APIs, if I may use that term. So the users are using whatever they are comfortable with, but still give the benefit of accelerated commuting them. And and lakehouse paradigm has a big role to play because if that separation had not happened
169
00:25:10.470 --> 00:25:16.010
by separating the table or excuse me, data and data lake with the execution engine of your choice,
170
00:25:16.390 --> 00:25:20.169
we would not have been able to to innovate the way. So there are a lot of,
171
00:25:20.470 --> 00:25:27.225
past wonderful work on which we are stand you know, you are leveraging and standing on that. For people who are
172
00:25:27.525 --> 00:25:28.025
looking
173
00:25:28.645 --> 00:25:30.585
to bring in the Datapelago
174
00:25:31.125 --> 00:25:41.900
capabilities, they're already running a Trino or a Spark, what does that integration path look like, particularly if they're running those systems via some sort of vendor like a Databricks or a Starburst?
175
00:25:42.600 --> 00:26:01.065
Oh, so let me, yeah. So, I mean, we have not announced the product yet, but let me generalize it. Right? Obviously, we have customers who are who are using it. But the idea is that let's first stay in the open source world even though technology is applicable to any any engine built with this paradigm. But idea is that
176
00:26:01.365 --> 00:26:02.024
can you
177
00:26:02.485 --> 00:26:03.865
think about a
178
00:26:04.325 --> 00:26:04.825
sidecar
179
00:26:05.460 --> 00:26:06.679
or a training wheel
180
00:26:07.140 --> 00:26:08.200
attached to existing
181
00:26:08.500 --> 00:26:13.799
existing car or existing whatever analogy you want to put in. Right? So that's what think of, like, DataPelgo,
182
00:26:14.340 --> 00:26:16.200
UDPE is that sidecar
183
00:26:16.500 --> 00:26:22.035
or that accelerated wheels which you attach to existing clusters. Now if I may use the word existing,
184
00:26:22.575 --> 00:26:25.955
spark, you attach that. And now all of a sudden,
185
00:26:26.255 --> 00:26:27.395
your same workloads
186
00:26:27.935 --> 00:26:28.755
run faster.
187
00:26:29.055 --> 00:26:31.555
And they underneath, it consumes the
188
00:26:31.960 --> 00:26:38.540
newer breed of servers available in the cloud of the it's there's no need for hardware introduction or whatever.
189
00:26:38.920 --> 00:26:39.420
AWS,
190
00:26:39.880 --> 00:26:40.380
GCP,
191
00:26:40.680 --> 00:26:42.460
Azure, and many other clouds
192
00:26:42.840 --> 00:26:53.405
have lot of choices of servers with a very good price performance points, which have those accelerated computing instances, I call. When I say accelerated computing instances, I meant GPUs
193
00:26:53.785 --> 00:26:59.565
or from AMD or Nvidia, from Tesla t four to to higher end and so on, and or FPGAs
194
00:27:00.080 --> 00:27:16.365
and going forward, TPUs. There are plenty of choices there. User doesn't need to worry about it. Right? So our software, our technology handles it. What is the right computing element to use? That's one part of the problem. But most importantly, insertion is you just when you create a cluster or something, you point to
195
00:27:17.384 --> 00:27:31.740
our binary or whatever you want to call it, and then magically, the system just shrinks size wise but runs faster. Right? So that's the that's how we insert. Let me pause here, Tobias. You should ask question if we need more clarity than this one. No. That that's definitely very useful.
196
00:27:32.200 --> 00:28:30.000
And to your point, the selection of which compute unit to use is definitely one that as somebody who is writing a SQL query or writing some sort of processing logic for the unstructured data that I want to do some entity extraction on. I don't wanna have to think about that level of detail. And so being able to just say, I want this to work, and then it gets federated out to the appropriate compute unit. And then, obviously, there are also questions around data gravity, making sure that those compute units are situated appropriately closely to the actual storage media. And particularly in that space of structured versus semi structured versus unstructured data, where you do have those differences in language interface. I'm wondering how the work that you're doing at the point that it gets to your layer, is it largely just an intermediate representation because you've already been abstracted away from the specifics of the language runtime and, you know, you're you're you're looking at some sort of, intermediate representation
197
00:28:30.539 --> 00:28:55.100
at that point? Or is there some other translation that you're doing from the query language and the execution plan into being able to say, okay. I'm dealing with unstructured data over here, so I need to do these different compute operations versus I'm running a SQL query, so I know I'm gonna be dealing with parquet files or JSON data for over here. Just curious about some of the ways that you're thinking about that problem as well of being able to, to the end user,
198
00:28:55.480 --> 00:28:56.779
unify the
199
00:28:57.159 --> 00:29:15.055
differentiation between those data sources so that they don't have to think about where they're integrating your product. Yeah. Good question. So yes. Absolutely. First, it's in if you don't do the way I am gonna describe, I don't think it it's gonna fly. You would want to keep it away this complexity of
200
00:29:15.435 --> 00:29:37.205
where to run which workload, right, or what computing element to use. So you still if you're writing in PyStone or a PySpark or SQL, you just continue to write as is, and it's a job of the underlying stack to figure out which computing element is the right element to do that. And you did you did mention couple of the points. One is a data affinity, and the second is the right,
201
00:29:37.845 --> 00:29:48.909
performance per dollar or performance per what whatever metric is the right processor to execute that. And many more things go into the mix for that algorithm to decide. And let me run it here. Because,
202
00:29:49.309 --> 00:29:50.450
you know, moving data,
203
00:29:50.750 --> 00:30:06.035
there are high level things you want to move compute closer to data rather than to data to the compute. So there are high level principles you want to address, keep in mind, while doing this kind of scheduling and and run time management of the resources to the operator. That that that's for sure required.
204
00:30:06.575 --> 00:30:08.675
And the the second part is that
205
00:30:08.975 --> 00:30:13.955
when it comes to the handling the data types, as you asked, yes. Absolutely correct.
206
00:30:14.655 --> 00:30:21.500
Let me call it the chaotic data. Like, the when it is more chaotic data than a structured data, you need lot more energy
207
00:30:22.039 --> 00:30:25.179
to extract what you're looking out of that. And that hence,
208
00:30:25.720 --> 00:30:27.419
the accelerated commuting instances
209
00:30:27.799 --> 00:30:30.220
play a better role in terms of efficiency
210
00:30:30.544 --> 00:30:37.044
than a general purpose CPU for to process that chaotic data. That's one point. The second is that that's where
211
00:30:37.345 --> 00:30:38.404
all of these processors,
212
00:30:38.705 --> 00:30:50.810
Tobias, are Turing complete. So you can run anything on a CPU as well. It's gonna functionally correct. But if you want to break the problem where it is right, the computing element, maybe you can influence the gates into some FPGA
213
00:30:51.110 --> 00:30:54.490
to process JSON efficiently, to parse JSON efficiently
214
00:30:54.870 --> 00:30:59.515
than doing it in the GPU. Maybe that's the right thing to do. Or if you're doing a PDF extraction,
215
00:30:59.895 --> 00:31:08.554
nowadays, it's all done in the models, so there is no need to pass the PDF to that. You can just run a model. So now that is better than a GPU. So depending upon your algorithm,
216
00:31:08.934 --> 00:31:19.440
depending upon your data type, that your choice of the engine or the computing element may change. Right? And, obviously, data affinity and all the stuff that you also mentioned in your question are also into consideration.
217
00:31:19.820 --> 00:31:30.645
So, hence, it is a it is and if it's well served for the industry and the user, if all that complexity is hidden from the end user. Otherwise, we will be
218
00:31:31.445 --> 00:31:37.465
the adoption will be too slow, and it it will it will break the system. Right? So that's what companies like us
219
00:31:37.850 --> 00:31:41.950
are working hard to solve that problem that how do I reexpress,
220
00:31:42.330 --> 00:31:43.950
if I may use the word, algorithm
221
00:31:44.330 --> 00:31:50.670
for this new machine, for the new animal that is handed to us. Right? That's what we do. And, if I may quote Jensen Huang,
222
00:31:51.050 --> 00:31:56.645
NVIDIA's CEO, he called it insanely hard problem. Right? Because we are not in a world where
223
00:31:57.025 --> 00:32:03.125
you are handed over a hundred gigahertz processor and recompile the software for that. Right? Now you have to rethink about,
224
00:32:03.505 --> 00:32:04.965
okay. I'm doing this
225
00:32:05.345 --> 00:32:10.810
new new sort or I need a new JSON parsing or a new PDF extraction.
226
00:32:11.190 --> 00:32:16.890
How do I reexpress that algorithm for the new target machine is the is the problem. And all that is
227
00:32:17.190 --> 00:32:34.750
buried and deeply integrated into the underlying stack, like, in case of Datapalago we build. But I'm sure many other companies are also working to solve that problem. And so if I can be a little bit reductive to summarize sort of what you're doing, the way I'm thinking about it is that it's analogous to being the LLVM
228
00:32:35.130 --> 00:32:36.429
for data compute.
229
00:32:36.809 --> 00:32:37.530
Yeah. I mean,
230
00:32:38.169 --> 00:32:40.169
that's a that's a,
231
00:32:40.650 --> 00:32:41.309
a big,
232
00:32:41.770 --> 00:32:52.445
high bar if if you may get there. But, yes, I would put it in a way that yeah. I mean, that's a that's a good way to to put it also, but maybe there's a,
233
00:32:52.985 --> 00:33:21.715
simpler analogy I can mention. But, yeah, we can leave it at that device. That makes sense. As you mentioned, LLVM, that is a high bar. It has had a lot of engineering time that has gone into it. It's being used in a number of different use cases. And as you mentioned, the problem that you're tackling is also, to quote you, insanely hard. And given that, I'm curious what are some of those engineering challenges that you're tackling, some of the foundational computing elements that you're having to reconsider
234
00:33:22.255 --> 00:33:22.755
or
235
00:33:23.055 --> 00:33:27.955
try to look at in different angles to understand how best to actually approach the problem efficiently?
236
00:33:28.335 --> 00:33:46.365
And also to tack on one last question to this, run on question, because of the fact that it is a very engineering heavy effort, I imagine that there is a lot of work that you have to do about how to think about marketing it to people, both for consumers as well as potential investors
237
00:33:46.665 --> 00:33:54.365
to make sure that you're given the appropriate time and space to actually invest in that engineering effort. Yeah. Yeah. Makes sense. So
238
00:33:54.850 --> 00:34:07.350
let's tackle the last one first. Right? Yes. It's like I'm using your light to shine, like, where to go. So you have taken me pretty deep into the stack, and hence, the technology details comes out. Okay? So but, otherwise,
239
00:34:07.715 --> 00:34:08.455
you're right.
240
00:34:08.835 --> 00:34:10.375
From from user perspective,
241
00:34:10.995 --> 00:34:12.055
investors perspective,
242
00:34:12.675 --> 00:34:14.855
we would want to make it as simple
243
00:34:15.395 --> 00:34:16.615
as frictionless
244
00:34:17.395 --> 00:34:18.775
to insert and understand.
245
00:34:19.075 --> 00:34:21.440
And that's what is the whole UDP
246
00:34:21.740 --> 00:34:29.760
idea, accelerating Spark, Coutinho without changing the application with the the things we just talked enables us to do so. So the all these complexities
247
00:34:30.140 --> 00:34:33.119
and the the what we just talked about are
248
00:34:33.645 --> 00:34:46.385
inside our stack, inside the software, not visible to the user. Right? So whether it's a choice of the hardware, whether it is a choice of the size of the cluster, whether where to schedule work and how to break algorithms, this is all
249
00:34:46.790 --> 00:35:02.090
done by our software. Right? So other no. Nothing is easy to to the user. The second part is that what you meant, in the big first part of the question was, what are those complexities, the challenges we are solving? One we just touched about is, reexpressing your algorithm
250
00:35:02.545 --> 00:35:17.470
for this new primitives that computing element we are offered. So that's one part. The second part is also that it is a large distributed systems problem twice, and there's something called Amdahl's law. Right? If you focus on a one function to accelerate
251
00:35:18.010 --> 00:35:23.950
and if that function is only used fraction of the time, you will only get the portion of that fraction accelerated
252
00:35:24.250 --> 00:35:55.310
from overall impact of the from the value proposition point of view. Right? So you need to be very, very careful about the Amdahl's law that you're not spending energy on a small portion of the workload. Right? So in the distributed systems, it's the IO, it's the data movement from storage to compute, compute to compute, and even within a node, data movement from your memory or drives to the CPU. And then now when you're talking about CPU and GPU, you're moving data from CPU over PCI to the GPU and back. Right? So those are all the flash points or hotspots.
253
00:35:55.745 --> 00:35:57.205
You have to be very, very careful
254
00:35:57.505 --> 00:36:12.059
and design a system which is impotence match between compute as well as IO. Right? It's a pretty complicated sophisticated distributed systems. Not, again, to scare the audience or the user, but this is all done underneath by the software to,
255
00:36:13.000 --> 00:36:16.940
a, pick the right computing element, b, pick the right algorithm
256
00:36:17.319 --> 00:36:22.859
to write on on run on that algorithm, and, c, to build a impedance match distributed system
257
00:36:23.160 --> 00:36:43.119
so that the user is able to get the value out of it. That's what makes this problem insanely hard. Right? This all these three things that we just talked. And the fourth thing is is to make it appear to the user as if nothing has changed. That makes it even more exponentially more harder. But as we were talking about this meta concepts of lake house,
258
00:36:43.660 --> 00:36:47.359
open table formats, you know, separation of compute from storage,
259
00:36:47.660 --> 00:36:50.000
gluten, all those things have enabled
260
00:36:50.619 --> 00:37:18.410
us to realize our vision. Right? But to make it seamless from a user point of view. But there is no escape from solving the three things that I just told you fundamentally or to solve it. Otherwise, it it it then it's not gonna give the value that this the the the potential that this new architecture has. Right? I hope this is clear to me. Absolutely. And as you're talking as well about the distributed systems nature of the problem, the fact that you have to impedance match the
261
00:37:18.795 --> 00:37:30.734
compute with the IO capabilities. It also brings in the question of what are those low level storage primitives that you're working with, especially because of the fact that it sounds like you aren't in a position to be able to
262
00:37:31.089 --> 00:37:40.309
standardize that or enforce that because you're just processing on already existing data. So it's not like you're building a full database system, and you can decide between an, LSTM
263
00:37:40.849 --> 00:37:48.835
or, you know, copy on write versus merge on read, So you have to work with the data as it already exists across that
264
00:37:49.135 --> 00:37:51.635
wide variety of different storage approaches
265
00:37:52.175 --> 00:37:58.675
and be able to process them as efficiently as possible. That's correct. Yes. I think the industry has already,
266
00:37:59.010 --> 00:37:59.910
let me say
267
00:38:00.210 --> 00:38:03.670
this way, willing to pay the cost price of desegregating
268
00:38:03.970 --> 00:38:06.070
and getting away with this proprietary
269
00:38:06.690 --> 00:38:13.910
control structures to wrap the data around. Right? So there is a penalty you pay for that. But the trade off is now you have separated and you're desegregated,
270
00:38:14.210 --> 00:38:35.310
and that that's a more architectural knob, which is have more benefits. So that is already realized. People are willing to pay that cost. Where where we come into the picture is, okay. You are already in deficit because of that departure from that monolithic architecture to this disaggregated architecture. How do we bridge that gap and even make it better? And hence, what we just went over, the discussion,
271
00:38:36.010 --> 00:38:37.630
this accelerated computing,
272
00:38:38.170 --> 00:38:55.575
having impedance match, and all those things enables us to not only cover the deficit, but also make it future proof as the data size grows, as the computing capability improves. So that's the that that's the how I look at it. Right? But you're right that the starting point is that we don't have luxury to
273
00:38:55.955 --> 00:39:07.770
store data which is more friendlier to us, and we don't want to do that. That's when the vendor lock in comes in. Enterprises don't want that. Right? They want to have a freedom of store once and use as many times. And if I'm storing
274
00:39:08.150 --> 00:39:14.330
pro a a proprietary format suitable for one engine, now you're locked in by definition to that engine. Right? That is a okay
275
00:39:14.765 --> 00:39:20.705
compromise as long as you're able to deliver on the promise of this thing. Right? Hence, I think this accelerated computing
276
00:39:21.165 --> 00:39:26.225
plays a big role in realizing the overall vision of leak house, this separation.
277
00:39:26.619 --> 00:39:49.665
Otherwise, that deficit will become so huge that people would ignore the future, but at least solve to this. And you may pendulum may shift back to the monolithic because it's not delivering on the promise. Right? So I think we may have a some role in enabling this vision, and making it more future proof, meaning in the using accelerated computing and the things that we just talked about, you know, in a humble way. Yeah. And
278
00:39:50.045 --> 00:39:51.825
in terms of the impact
279
00:39:52.205 --> 00:39:53.105
on behavior
280
00:39:53.565 --> 00:39:54.065
and
281
00:39:54.770 --> 00:39:55.270
prioritization
282
00:39:55.730 --> 00:40:05.270
for teams who are working with these data lake house systems where today, they maybe are trying to run a Trino cluster, they execute a query,
283
00:40:05.570 --> 00:40:12.115
it takes some number of seconds to execute because it's executing across hundreds of gigabytes or terabytes of data.
284
00:40:12.575 --> 00:40:15.075
So then they bring in Datapelago,
285
00:40:15.615 --> 00:40:21.235
and that drops from multiple seconds down to single seconds or subseconds, what whatever
286
00:40:21.615 --> 00:40:25.235
the order of magnitude improvement is. I'm curious how that
287
00:40:25.640 --> 00:40:29.260
shift in speed and efficiency and capabilities
288
00:40:29.720 --> 00:40:33.580
changes the ways that they think about what data to use for which problems.
289
00:40:34.200 --> 00:40:35.100
Yeah. I think,
290
00:40:35.720 --> 00:40:37.180
you know, there is a
291
00:40:37.560 --> 00:40:40.140
a certain angle to this. Right? Right now,
292
00:40:40.440 --> 00:40:41.020
the way
293
00:40:41.535 --> 00:40:50.515
business model is, you're always paying as a consumption of something. So you are paying as a consumption of compute resources. If you make shrink that,
294
00:40:50.815 --> 00:40:57.234
that directly translates to your TCO savings. Right? So that's what the unless you're charging your premium, which many companies do,
295
00:40:57.670 --> 00:40:58.890
but for the performance
296
00:40:59.270 --> 00:41:01.610
improvements and so on. But, otherwise, fundamentally,
297
00:41:02.630 --> 00:41:04.970
the performance has a benefit of
298
00:41:05.270 --> 00:41:13.215
reducing your TCOs. That's one part I wanted to just address. I missed your second part, Tobias. Can you repeat the second half of the question? Yes.
299
00:41:13.695 --> 00:41:15.635
So just wondering how the
300
00:41:15.935 --> 00:41:17.715
increase in speed and efficiency
301
00:41:18.255 --> 00:41:30.400
of that data processing changes the ways that teams think about what data to use for which problems and how it maybe unlocks greater potential for bringing larger volumes of data to bear on the organizational
302
00:41:30.780 --> 00:41:31.280
challenges.
303
00:41:31.900 --> 00:41:40.640
Right. So two parts to that. One is the the the business answer, and then there's a technical answer here. The business answer I hear from many CIOs is that
304
00:41:41.145 --> 00:41:43.965
your TCO savings that I have in my current workload
305
00:41:44.505 --> 00:41:51.165
and freeze up my budget to do exploratory work that I would would have not done otherwise. Okay? Because they want to do
306
00:41:51.465 --> 00:42:00.360
AI processing or Genia or fine tuning or direct processing. Just today's whatever is in the thing. So that is a business advantage of,
307
00:42:00.740 --> 00:42:05.960
reducing your TCO on the current spend. That's one part. The second part is also that
308
00:42:06.340 --> 00:42:09.320
there are many use cases which we make practical,
309
00:42:09.620 --> 00:42:15.325
which were not viable earlier. Right? Because now we are able to meet SLAs with this new architecture
310
00:42:16.025 --> 00:42:23.865
without needing to put it in a, let's say, proprietary data warehouse. If you can do that, then it enables many, many new inter new,
311
00:42:24.265 --> 00:42:30.190
use cases. So for example, in a telco world, we are involved with a company which they have to meet certain,
312
00:42:30.970 --> 00:42:46.505
by morning or something. All the data ingested overnight need to be processed and be ready for the next time when it starts. Right? Earlier, given the budget, they would have to cut short either on the quality of the processing or the amount of data they do. Right? So that that affects their
313
00:42:46.805 --> 00:42:55.465
relevance or be staying competitive in the next day. So, hence, the speed up and this extra volume of data or more deeper analysis you can do
314
00:42:55.820 --> 00:43:03.440
within the same time frame enables them to be more sharper and competitive next stage. So that's a real use case that we are enabling that.
315
00:43:03.740 --> 00:43:06.320
But right now, given the current macroeconomics
316
00:43:06.700 --> 00:43:08.000
and otherwise, everybody's
317
00:43:08.460 --> 00:43:08.960
under
318
00:43:09.435 --> 00:43:12.415
stress, I would say, to reduce their TCO because,
319
00:43:13.115 --> 00:43:13.935
I mean, all
320
00:43:14.315 --> 00:43:15.135
said and done,
321
00:43:15.435 --> 00:43:18.735
this pay as you go model looks okay in the beginning.
322
00:43:19.435 --> 00:43:21.535
But moment you start using at scale,
323
00:43:21.849 --> 00:43:44.835
then the reality starts sinking in on the cost and and so on. Right? So that's kind of where many people are grappling, and our technologies that that we built at building a data panel enables them to keep that fundamentally. Right? We are not here playing a margin game that, okay, if they are paying, you know, 10% savings here and but we are fundamentally changing the price and the performance assumption.
324
00:43:45.855 --> 00:43:49.875
And in that case, there's a we enable them to reduce their TCO
325
00:43:50.500 --> 00:43:59.800
and help them scale, in the as a telco example I gave you, or even free up the budget to do things which, they want to do, but there is no budget, practically speaking.
326
00:44:00.100 --> 00:44:04.395
In the work that you have done with some of your early customers and design partners,
327
00:44:11.915 --> 00:44:14.495
mocks applied to their business problems?
328
00:44:16.155 --> 00:44:16.815
I think,
329
00:44:17.275 --> 00:44:20.839
to be honest, we are I will not have that many,
330
00:44:22.420 --> 00:44:27.480
data points to generalize that. Right? But what I so I will just caution with that,
331
00:44:28.020 --> 00:44:29.079
my next answer.
332
00:44:29.460 --> 00:44:30.359
So but,
333
00:44:31.704 --> 00:44:32.365
you know,
334
00:44:33.145 --> 00:44:36.684
there are many and we have not touched many verticals yet. Manufacturing,
335
00:44:38.184 --> 00:44:40.285
you know, whether it is, in
336
00:44:40.744 --> 00:44:42.845
cars or in in the semiconductor,
337
00:44:44.040 --> 00:44:45.900
in shop, you know, factory,
338
00:44:46.360 --> 00:44:50.700
when there's so many sensors that are generating data. So there are lots of use cases where
339
00:44:51.320 --> 00:44:51.820
efficiency,
340
00:44:52.120 --> 00:44:56.620
performance, latency reduction is important, and we will enable that. Right?
341
00:44:57.095 --> 00:45:01.275
But right now, more or less, I think I can generalize that
342
00:45:01.575 --> 00:45:03.194
either the TCO savings,
343
00:45:03.654 --> 00:45:05.035
right, is is important.
344
00:45:06.135 --> 00:45:09.115
Using that savings to increase their
345
00:45:11.369 --> 00:45:16.670
amount of data they process and how that it helps. That's one part. The second part
346
00:45:17.050 --> 00:45:19.950
is which is also obvious now in the
347
00:45:20.329 --> 00:45:22.109
machine learning or the data
348
00:45:22.455 --> 00:45:22.955
development
349
00:45:23.255 --> 00:45:25.515
excuse me, the model development and model,
350
00:45:25.975 --> 00:45:26.475
training,
351
00:45:27.495 --> 00:45:28.475
workload specific.
352
00:45:28.935 --> 00:45:30.875
Data scientist time is very important,
353
00:45:31.415 --> 00:45:40.560
and how much time they spend on spend on processing data, preparing data is important because it comes in the way of how iterative and how fast you can train models.
354
00:45:40.860 --> 00:45:44.080
So that's so that's another area where we we help
355
00:45:44.460 --> 00:45:46.400
and speed up the data preprocessing
356
00:45:46.780 --> 00:45:52.155
for model training, which translates to saving the data scientist time and and so on. Right? So there are
357
00:45:52.775 --> 00:45:55.095
few higher level values we see, but,
358
00:45:55.815 --> 00:45:58.155
next time we talk, I can give you a little bit more
359
00:45:58.935 --> 00:46:00.480
better insights than that. Yeah.
360
00:46:01.440 --> 00:46:16.835
And in your work of investing in this problem space and working at this layer of the technology stack, which sounds like is maybe even a little bit higher up than some of your previous companies, What are some of the most interesting or unexpected or challenging lessons that you've learned personally?
361
00:46:18.575 --> 00:46:19.075
Well,
362
00:46:19.775 --> 00:46:21.375
you learn a lot, Tobias,
363
00:46:21.775 --> 00:46:22.275
especially
364
00:46:22.975 --> 00:46:29.520
if I may I don't know if this is the angle. I mean, first is as a entrepreneur, as a CEO, or as a founder,
365
00:46:30.940 --> 00:46:31.440
it's
366
00:46:31.900 --> 00:46:33.520
a learning every day. Right?
367
00:46:34.380 --> 00:46:36.240
You know, it's I say that
368
00:46:36.700 --> 00:46:37.200
you
369
00:46:37.580 --> 00:46:38.560
building a feature
370
00:46:39.100 --> 00:46:40.640
versus building a product
371
00:46:40.985 --> 00:46:42.685
and versus building a company
372
00:46:43.145 --> 00:46:45.965
is exponential scale complexity. Right? So,
373
00:46:47.785 --> 00:46:48.425
so hence,
374
00:46:49.065 --> 00:46:51.405
it is a lot of learning technically.
375
00:46:52.585 --> 00:46:54.685
We are kind of breaking new ground,
376
00:46:55.145 --> 00:46:55.885
every day,
377
00:46:56.640 --> 00:47:04.020
because many of the areas that we are we have to solve are some unsolved problems. You know, how do you move data or how do you process
378
00:47:04.480 --> 00:47:16.505
your data sizes, terabytes? Your GPU memory is only 16 gigabyte. How do you manage that? Right? So that's a technical problem to solve and and many more such things. So there are many technical problems to solve,
379
00:47:17.125 --> 00:47:23.465
to do what we do. Then there are many business model related problems to solve. How do we insert into user
380
00:47:23.845 --> 00:47:24.745
space without,
381
00:47:26.010 --> 00:47:28.910
disrupting the ecosystem? That's another way to innovate.
382
00:47:29.290 --> 00:47:34.750
And then that, obviously, third is, you know, it's a fast evolving market, a problem set itself.
383
00:47:35.050 --> 00:47:41.345
Data types are changing and so on. How do you build an architecture which is has legs for future proofing? So there are a lot of areas,
384
00:47:41.745 --> 00:47:42.885
which we are learning,
385
00:47:43.345 --> 00:47:47.125
and I'm personally learning daily, while working on this problem.
386
00:47:48.225 --> 00:47:54.990
And for people who are looking for ways to increase the efficiency and speed and flexibility
387
00:47:55.290 --> 00:47:56.030
of their
388
00:47:56.650 --> 00:48:02.990
data systems, what are the cases where Datapelago is the wrong choice and won't have the impact that they're hoping for?
389
00:48:03.865 --> 00:48:04.445
I think,
390
00:48:05.385 --> 00:48:09.165
one is a fundamental answer. Right? One is that if you are
391
00:48:11.225 --> 00:48:14.285
as I said that we are our first attempt
392
00:48:14.825 --> 00:48:15.725
is to do
393
00:48:16.450 --> 00:48:19.430
an impedance match between data moment and compute.
394
00:48:19.730 --> 00:48:22.550
And if compute computation is the bottleneck,
395
00:48:22.850 --> 00:48:36.865
we will free it up and make that's a little bit fundamental answer I'm giving you to to Tobias. Right? So if if you find a workload where you're processing very less, but you're just moving data or moving things here and there, probably there's not much for
396
00:48:37.165 --> 00:48:42.530
a GPU to play a role in that. Right? So that's kind of a fundamental answer to that. But I guess,
397
00:48:43.150 --> 00:48:44.610
it I would not
398
00:48:44.990 --> 00:48:45.490
give
399
00:48:45.950 --> 00:48:46.850
up, like, a,
400
00:48:47.230 --> 00:48:51.550
too early in that journey because we are still evolving and and,
401
00:48:51.950 --> 00:48:55.410
and and solving. So but right now, yes. If I were to put it today's,
402
00:48:56.484 --> 00:48:57.945
state of the art from DataPelago,
403
00:48:59.205 --> 00:49:00.025
we look
404
00:49:00.405 --> 00:49:06.345
for when you're processing complex data, you are doing spending lot of compute cycles to do it. And,
405
00:49:07.365 --> 00:49:11.580
there is a data movement from your storage to thing, but relative to that, you're spending lot
406
00:49:12.300 --> 00:49:13.760
more on the compute. That's a higher level,
407
00:49:15.100 --> 00:49:17.600
highest order bit, I would say, to resolve,
408
00:49:18.140 --> 00:49:27.120
to see where data per go will be applicable. Right? Because you are attacking the competition part of the problem, more more more than that. We are trying to be or we are efficient
409
00:49:27.714 --> 00:49:28.295
in selecting
410
00:49:29.234 --> 00:49:35.895
how to move data efficiently. Sure. How to I mean, there are many data techniques to do dynamic filtering or,
411
00:49:36.435 --> 00:49:42.535
row group based pruning in parquet or whatnot. Right? Like, so that you're very, very selective in only reading data that you need,
412
00:49:43.050 --> 00:49:46.830
and those are complementary techniques to us. But once that is done,
413
00:49:47.130 --> 00:49:53.310
and even then if you are reading too much data, then, and the ratio of the IO to compute is disturbed,
414
00:49:53.690 --> 00:49:57.630
the and then there is not much probably data that will go to add value right now. Right?
415
00:49:58.235 --> 00:50:03.935
In my previous slide, we have solved the data moment problem, but now in context data it's a computational problem you're solving.
416
00:50:04.955 --> 00:50:08.255
As you continue to build and invest in and
417
00:50:08.635 --> 00:50:10.335
move towards general availability
418
00:50:10.715 --> 00:50:14.750
of the technology that you're building at Datapelago? What are some of the capabilities
419
00:50:15.210 --> 00:50:20.589
or specific projects that you're excited to dig into or any of the features that you're building towards?
420
00:50:21.369 --> 00:50:22.740
Yeah. It it is a
421
00:50:23.875 --> 00:50:32.615
the the good news or the bad news, however you want to take it, it's a endless world. Right? So why is it endless world? So let's let me share few dimensions,
422
00:50:32.995 --> 00:50:35.895
right, in which we want to and we will
423
00:50:36.309 --> 00:50:45.930
go. One is the world of data platforms itself. You know, you handle Spark. You handle Trino. Then there are many more which can benefit from from the this,
424
00:50:46.309 --> 00:50:48.809
accelerated computing. So that's kinda one one dimension.
425
00:50:49.315 --> 00:51:05.760
The second is the data type itself. Right? So data types are changing from multimodal and and unstructured, semi structured data. And the kind of computation you do, kind of models you want to bring in there is another dimension where the platform will will evolve. Right? And the third is application itself.
426
00:51:06.619 --> 00:51:08.880
You know, because now different
427
00:51:09.180 --> 00:51:12.640
what is your distributed framework? Is it a ray or is it a spark?
428
00:51:13.180 --> 00:51:16.880
What kind of so that is another dimension to go. Fourth could be,
429
00:51:17.465 --> 00:51:22.765
multi cloud, which is obvious from hybrid on prem to that. But now there is this new
430
00:51:23.305 --> 00:51:29.325
wave of GPU clouds emerging. Right? How about we address data processing there? So there are many, many dimensions
431
00:51:29.930 --> 00:51:36.350
ahead of us, Tobias, to to go and innovate, and many of them are in our road map, and we are actively working.
432
00:51:37.610 --> 00:51:41.070
Are there any other aspects of the work that you're doing at Datapelago,
433
00:51:41.705 --> 00:51:44.125
the specific problem of efficiencies
434
00:51:44.665 --> 00:51:46.125
in data lake architectures
435
00:51:46.585 --> 00:51:48.445
and the overall evolution
436
00:51:48.825 --> 00:51:51.005
of data processing, particularly
437
00:51:51.385 --> 00:51:59.109
as generative AI brings unstructured data more into the forefront that we didn't discuss yet that you would like to cover before we close out the show?
438
00:51:59.890 --> 00:52:02.069
Not really. I think you covered pretty broad,
439
00:52:02.450 --> 00:52:06.470
areas. I hope this is useful. But, yes, we we have not covered
440
00:52:07.170 --> 00:52:17.865
the how the data processing for generative AI world is evolving and so on, but we can save it for the processing for generative AI world is evolving and so on, but we can save it for the next time. But, yes, there is a very fascinating world emerging of in front of us.
441
00:52:18.244 --> 00:52:23.880
Like, obviously, data is changing, but the kind of data processing, which is a combination of, actually,
442
00:52:25.079 --> 00:52:26.140
actually, I will leave this
443
00:52:26.920 --> 00:52:31.500
for you and your audience, which is that the two pipelines are also converging. People want to apply,
444
00:52:32.119 --> 00:52:33.259
analytics primitives
445
00:52:33.880 --> 00:52:38.940
along with the unstructured data processing together. Right? How how does that work look like?
446
00:52:39.545 --> 00:52:40.045
Where,
447
00:52:40.425 --> 00:52:46.365
you want to do joins as well as extract metadata from PDF. Right? So that's a problem to be solved,
448
00:52:46.665 --> 00:52:48.365
and that's where I think the direction,
449
00:52:49.065 --> 00:52:50.285
platforms will take.
450
00:52:51.305 --> 00:53:01.619
Like, on a single platform, how do I do unified processing, processing both for structured and unstructured data? Right? That's that's the next frontier, I would call it, for for us as well as an industry.
451
00:53:02.640 --> 00:53:18.015
For anybody who wants to get in touch with you and follow along with the work that you and your team are doing, I'll have you add your preferred contact information to the show show notes. And as the final question, I'd like to get your perspective on what you see as being the biggest gap in the tooling or technology that's available for data management today.
452
00:53:19.195 --> 00:53:20.975
What I can answer is that
453
00:53:21.435 --> 00:53:23.215
with the generative AI,
454
00:53:23.760 --> 00:53:25.220
there's lot of innovation
455
00:53:25.680 --> 00:53:27.380
have to happen and already happening.
456
00:53:27.840 --> 00:53:31.140
How do you even use natural languages, for example,
457
00:53:31.680 --> 00:53:32.180
to
458
00:53:32.880 --> 00:53:33.940
express your intent?
459
00:53:34.240 --> 00:53:42.984
Right? From natural language to SQL or natural language to Python or whatever. Right? How do you and there are many initiatives in that. I think that is probably one area
460
00:53:43.525 --> 00:53:44.025
where
461
00:53:44.405 --> 00:53:44.905
industry
462
00:53:45.285 --> 00:53:47.545
will evolve and learn and improve,
463
00:53:48.244 --> 00:53:50.424
to even make the interface easier,
464
00:53:50.964 --> 00:53:57.240
from for so that even people who are not familiar with SQL, can you use your data platform for your other? So that's one area,
465
00:53:57.859 --> 00:53:58.680
but I'm sure,
466
00:54:00.180 --> 00:54:05.240
world is full of smart people. They will think of something better than what I can contribute right now on that problem.
467
00:54:05.780 --> 00:54:06.280
Absolutely.
468
00:54:06.815 --> 00:54:20.915
And everybody's always filling in different gaps where which then expose other ones. So it's a it's a very fractal problem space. So thank you very much for taking the time today to join me and share the work that you're doing at Datapelago and your insights into the
469
00:54:21.300 --> 00:54:25.560
fundamental challenges of data efficiency in these disaggregated architectures,
470
00:54:26.580 --> 00:54:33.960
both in the software and, the hardware layers as we work in these cloud environments. So I appreciate all the time and energy that you're putting into
471
00:54:34.425 --> 00:54:37.645
improving some of those efficiencies, and I hope you enjoy the rest of your day.
472
00:54:38.025 --> 00:54:42.925
Thank you, Tobias. It was wonderful and a pleasure, and many of their questions were insightful.
473
00:54:43.385 --> 00:54:47.725
And I hope that it is useful for for the audiences. Thank you for doing this.
474
00:54:55.500 --> 00:54:58.560
Thank you for listening, and don't forget to check out our other shows.
475
00:54:58.860 --> 00:54:59.900
Podcast.net
476
00:54:59.900 --> 00:55:08.755
covers the Python language, its community, and the innovative ways it is being used. And the AI Engineering Podcast is your guide to the fast moving world of building AI systems.
477
00:55:09.535 --> 00:55:19.590
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email hosts@dataengineeringpodcast.com
478
00:55:19.590 --> 00:55:25.770
with your story. Just to help other people find the show, please leave a review on Apple Podcasts and tell your friends and coworkers.