Why Metadata Still Matters: From Data Catalogs to AI-Ready Semantic Hubs

WEBVTT
NOTE
Transcription provided by Podhome.fm
Created: 10/07/2026 00:40:42
Duration: 5091.767
Channels: 1
1
00:00:11.440 --> 00:00:15.440
Hello, and welcome to the Data Engineering Podcast, the show about modern data management.
2
00:00:16.075 --> 00:00:20.074
Today's episode is sponsored by Parallel, where agents find answers.
3
00:00:20.475 --> 00:00:27.195
Most engineers today closely follow new model releases, but don't pay attention to their agent's most important tool, web search.
4
00:00:27.675 --> 00:00:40.140
Parallel develops enterprise grade infrastructure for agents to retrieve high quality context from the web. Their core products are a suite of APIs for retrieving high quality information from the web with Pareto optimal quality, cost, and speed.
5
00:00:40.780 --> 00:00:53.325
Whether you work on voice agents that need two hundred millisecond latency, chatbots that balance speed, depth, and quality, or long horizon agents to do thorough overnight research for you, Parallel is a single platform for all of your agentic research.
6
00:00:53.565 --> 00:00:57.805
Get started for free today at dataengineeringpodcast.com/parallel.
7
00:00:59.370 --> 00:01:11.130
Your host is Tobias Maci, and today I'm interviewing Christian Brameau and Simon Dinan about their long history of dealing with metadata at Medicarda and how they're adapting to the age of AI. So, Christian, can you start by introducing yourself?
8
00:01:11.885 --> 00:01:17.325
Sure. This is Christian Bremmaud. I'm the founder of Meta Integration Technology,
9
00:01:17.325 --> 00:01:19.085
which is the name of a company,
10
00:01:19.645 --> 00:01:21.405
now offering direct,
11
00:01:21.565 --> 00:01:23.645
to customers under the brand Metacarta.
12
00:01:24.060 --> 00:01:28.140
So we'll tell you more about it and how all this came together.
13
00:01:28.780 --> 00:01:30.299
And how about you, Simon?
14
00:01:31.500 --> 00:01:38.955
Yeah, Christian and I have been doing it for probably too long. We're kind of partners in crime. I'm taking care of technical
15
00:01:39.355 --> 00:01:41.835
side of the house. I'm CTO of the company.
16
00:01:42.635 --> 00:01:44.555
And yes, it's been
17
00:01:45.195 --> 00:01:46.315
almost like
18
00:01:46.635 --> 00:01:49.035
one, two, pushing three decades.
19
00:01:50.940 --> 00:01:59.659
Going back to you, Christian, do you remember how you first got started working in the overall space of data and some of the highlights of your career working in metadata?
20
00:02:00.460 --> 00:02:12.595
Right. So that's going to be a long story to tell a little bit. That first question is going to take quite some time. But I think it's important to understand where we're coming from, to see where we're going on that. So
21
00:02:13.315 --> 00:02:18.834
I could have done many things around in my careers, but no, just after college,
22
00:02:19.250 --> 00:02:21.730
my first job was about metadata.
23
00:02:22.130 --> 00:02:25.090
And that was, you know, over thirty years ago,
24
00:02:25.330 --> 00:02:26.290
as it happens.
25
00:02:26.610 --> 00:02:33.730
And I was in Europe at the time in France and working on a lot of European project about metadata standards.
26
00:02:34.315 --> 00:02:36.635
Basically, that was all about exchanging
27
00:02:36.635 --> 00:02:37.515
metadata
28
00:02:37.834 --> 00:02:39.355
between different tools
29
00:02:39.595 --> 00:02:43.035
and all about having a common metadata repository
30
00:02:43.035 --> 00:02:46.075
in Europe. And that involves standards such as ECMA
31
00:02:46.610 --> 00:02:49.810
in Europe and obviously ECMA works with ISO
32
00:02:50.049 --> 00:02:53.010
and ISO standards on metadata.
33
00:02:53.650 --> 00:02:56.450
That got us to work a lot with The United States
34
00:02:56.849 --> 00:02:59.569
and I came in US in '92
35
00:02:59.845 --> 00:03:02.405
and worked again on standards.
36
00:03:02.405 --> 00:03:03.845
And that's exactly,
37
00:03:04.405 --> 00:03:05.525
the bootstrap
38
00:03:05.525 --> 00:03:07.845
of the metadata with the company.
39
00:03:08.084 --> 00:03:10.805
So the company Meta Integration
40
00:03:10.805 --> 00:03:13.285
Technology was created in '97.
41
00:03:13.820 --> 00:03:17.820
So as Simon was pointing out, it's going to be soon three decades
42
00:03:17.900 --> 00:03:19.340
we've been doing this.
43
00:03:19.820 --> 00:03:23.260
And the name Meta integration is definitely
44
00:03:23.340 --> 00:03:24.380
metadata
45
00:03:24.460 --> 00:03:25.340
integration,
46
00:03:25.420 --> 00:03:34.775
and that's the spirit behind the name. I know meta took another connotation these days, but we were first using the name meta integration
47
00:03:34.775 --> 00:03:35.575
out there
48
00:03:35.815 --> 00:03:36.615
and
49
00:03:36.935 --> 00:03:41.735
really it went on on standards initially and we got heavily involved
50
00:03:42.150 --> 00:03:46.790
with the OMG group, the object management group, but had important standards
51
00:03:46.870 --> 00:03:52.630
known as UML for object modelling, unified modelling language at the time, XMI,
52
00:03:52.950 --> 00:03:54.230
and CWM,
53
00:03:54.230 --> 00:03:59.025
the common warehouse meta model. And here already you see and feel the spirit
54
00:03:59.265 --> 00:04:02.545
of having a common meta model between vendors.
55
00:04:02.545 --> 00:04:08.945
And that involved at the times key companies like Oracle, the Oracle Analytic Group, SAS, IBM,
56
00:04:08.945 --> 00:04:13.750
all these key vendors wanted to have a common meta model to share metadata
57
00:04:14.069 --> 00:04:15.510
between vendors.
58
00:04:15.989 --> 00:04:19.110
But as you can imagine, they each had their own twist
59
00:04:19.270 --> 00:04:32.235
and having a real common meta model was very hard to achieve. Not in some areas. You know, we can all agree what a common meta model is for relational database. All good. But as soon as you dive into analytics,
60
00:04:32.475 --> 00:04:38.875
everybody gets into their own little twist and flavors, you know. So that really
61
00:04:39.330 --> 00:04:42.610
got into the thinking that the metadata standards
62
00:04:42.610 --> 00:04:52.530
didn't really work at the time. And one could question it in the future and we'll get on that a little bit later because that's a common thread happening.
63
00:04:53.330 --> 00:04:59.065
So we created metadata integration and part of our first line of products
64
00:04:59.305 --> 00:05:02.425
was all about what we call model bridges.
65
00:05:02.505 --> 00:05:04.105
So basically metadata
66
00:05:04.345 --> 00:05:06.905
converters between tools and
67
00:05:07.065 --> 00:05:10.720
that really is how we started. We started actually
68
00:05:11.040 --> 00:05:18.400
on government contracts also, and that keeps the value of the company because we never took any venture capital.
69
00:05:18.480 --> 00:05:21.440
This is a 100% employee owned company today.
70
00:05:21.680 --> 00:05:22.960
And that is very important,
71
00:05:23.575 --> 00:05:34.855
as some call us the Switzerland of metadata because we're not controlled by any external body. We don't have any bank accounts in Switzerland. If anybody is thinking this way, that's not the case.
72
00:05:35.335 --> 00:05:37.735
But that's the neutrality of the ecosystem.
73
00:05:38.280 --> 00:05:46.120
So what happened at that point? Metadata don't work. We have this model bridge software, meta integration model bridge, MIMB,
74
00:05:46.280 --> 00:05:52.600
and it can basically move metadata between tools because it can convert between different meta models.
75
00:05:53.705 --> 00:06:01.065
And in those days, the BI tools were business objects like business objects at something called a universe.
76
00:06:01.465 --> 00:06:12.870
Very important in what we're going to come to a bit later because they had some sort of a semantic, common semantic layers so that the reports on top were sharing common metrics,
77
00:06:13.030 --> 00:06:14.230
common dimensions
78
00:06:14.230 --> 00:06:15.190
and everything.
79
00:06:15.830 --> 00:06:21.510
And in those days was another competitor close to you in Canada, Tobias, was Cognos.
80
00:06:22.125 --> 00:06:27.725
And Cognos created an equivalent tool called Cognos Framework Manager. Same idea.
81
00:06:27.965 --> 00:06:32.525
A common metadata layer in the middle. So that all reports used on that.
82
00:06:32.925 --> 00:06:42.510
And these guys changed us. We do owe to the Cognos company in Canada. They came to us and they said metadata is important.
83
00:06:42.910 --> 00:06:47.390
This new tool we're going to have with a good metadata layer
84
00:06:47.390 --> 00:06:49.870
needs to be able to import export
85
00:06:49.870 --> 00:06:51.390
from any tool around.
86
00:06:52.134 --> 00:06:57.735
And that's how it started. They say we would like to put your model bridges OEM inside
87
00:06:58.294 --> 00:06:59.414
our software.
88
00:06:59.574 --> 00:07:04.134
So when when you use Cognos Framework Manager as a BI,
89
00:07:04.375 --> 00:07:12.830
you know, semantic modelling tool, if you wish, at the beginning, we need to be able to import from any metadata and export from any metadata.
90
00:07:13.150 --> 00:07:15.870
For example, a nice model from Erwin,
91
00:07:16.030 --> 00:07:27.555
you know, nice data model with the dimensions, facts, the business names, bring it in. And we need to also be able to forward engineer not just to our reports. We want all this knowledge to go outside.
92
00:07:27.715 --> 00:07:29.395
So metadata integration
93
00:07:29.395 --> 00:07:31.315
took life within
94
00:07:31.395 --> 00:07:32.835
Cognos Framework Manager.
95
00:07:33.300 --> 00:07:35.300
We signed an OEM agreement
96
00:07:35.379 --> 00:07:37.220
with them at the time,
97
00:07:37.699 --> 00:07:43.699
and that was at the very beginning of, you know, twenty five years ago. And it spiraled out of that.
98
00:07:43.940 --> 00:07:47.425
Basically, everybody wanted to have metadata
99
00:07:47.425 --> 00:07:48.385
integration,
100
00:07:48.385 --> 00:07:59.025
the ability to integrate metadata with their tool. So Cognos was the first one, but it follows suit with with others with slightly different needs. The next one was Informatica
101
00:07:59.025 --> 00:08:00.145
Informatica
102
00:08:00.145 --> 00:08:03.270
for the data integration tool, power center,
103
00:08:03.430 --> 00:08:06.870
but also the data catalogue, the ability to bring metadata
104
00:08:07.030 --> 00:08:09.510
and get lineage from any tools.
105
00:08:09.910 --> 00:08:15.350
And then was the first data modelling. I believe it was ER Studio from Unbucketero,
106
00:08:15.350 --> 00:08:25.315
Hidera a little bit late. And these are still extremely active OEM agreements today. You use ER Studio and they have what they call the meta wizard.
107
00:08:25.555 --> 00:08:31.634
When you see import export menu in these tools, what people don't know, it's meta integration inside
108
00:08:31.960 --> 00:08:35.640
giving all the import export from any technology.
109
00:08:36.040 --> 00:08:39.960
And it kept going, you know, business object came in a bit later,
110
00:08:40.279 --> 00:08:41.480
which got
111
00:08:41.720 --> 00:08:45.400
acquired by IBM. So I had an OEM agreement with IBM.
112
00:08:45.560 --> 00:08:49.055
The initial, you know, group within CWM,
113
00:08:49.055 --> 00:08:49.935
so SAS,
114
00:08:50.255 --> 00:08:51.695
also Oracle.
115
00:08:51.935 --> 00:08:57.535
Now Oracle took a different twist. Some vendors wanted to build the data catalogue
116
00:08:57.535 --> 00:09:09.370
and they wanted to have the import export of metadata from us, you know, whether it's a data catalog, a BI design tool or a DI design tool. But some wanted
117
00:09:09.530 --> 00:09:15.210
the entire data catalog from us, you know, the full metadata management solution
118
00:09:15.415 --> 00:09:21.735
rebranded for them. And that was the case of Oracle Enterprise Metadata Management.
119
00:09:21.815 --> 00:09:33.200
This is the entire meta integration metadata management platform rebranded with the Oracle sticker on it. And others did exactly the same. Today, the very popular Click
120
00:09:33.440 --> 00:09:36.720
Talend data catalogue, Talend acquired Click,
121
00:09:36.800 --> 00:09:40.720
is actually our software rebranded with hundreds of customers
122
00:09:40.960 --> 00:09:43.040
using this at extreme large scale.
123
00:09:44.035 --> 00:09:44.755
You know,
124
00:09:45.155 --> 00:09:54.835
I'm not sure I can mention customers in this in this since they are OEM partners, but the largest bank in Canada, largest customers in in EMEA,
125
00:09:54.915 --> 00:09:58.515
in APAC, all over the world as this is internationalized.
126
00:09:58.730 --> 00:10:05.290
And it kept going all the way to Microsoft. So Microsoft Purview metadata import
127
00:10:05.290 --> 00:10:14.250
is our software as well inside. And I mentioned many others I didn't mention, like Erwin, the import export of Erwin from Quest is also our software.
128
00:10:15.815 --> 00:10:21.815
So that kept going and the meta integration brand, meta integration OEM
129
00:10:21.894 --> 00:10:25.175
for just the connectors and the data lineage
130
00:10:25.175 --> 00:10:27.495
or the entire platform rebranded.
131
00:10:28.240 --> 00:10:36.000
And then some companies came to us and said, would like to buy directly from you because we would like a vendor neutral solution.
132
00:10:36.640 --> 00:10:44.395
And that started with some other large companies like Walmart basically came to us and wanted the technology for them.
133
00:10:44.795 --> 00:10:49.515
But also others lately in the insurance business like GAIG
134
00:10:49.595 --> 00:10:51.115
or Canal Insurance.
135
00:10:51.275 --> 00:10:53.835
Same story. You know, we have many, many vendors.
136
00:10:54.260 --> 00:10:58.260
We know what we're using today for BI. We don't know what you're going to use tomorrow.
137
00:10:58.740 --> 00:11:00.340
We need something like this.
138
00:11:00.740 --> 00:11:06.100
And eventually that created the Meta Carter brand, which is our brand directly
139
00:11:06.455 --> 00:11:07.655
to customers
140
00:11:08.055 --> 00:11:10.215
from Meta integration,
141
00:11:10.375 --> 00:11:14.135
basically. So that's the name Meta Carter. So Meta
142
00:11:14.135 --> 00:11:15.495
Carter Carter
143
00:11:15.495 --> 00:11:20.855
is the map. It's the enterprise map of metadata, the enterprise architecture,
144
00:11:21.550 --> 00:11:27.950
any tool, you know, out there. That's the key of what we do is really be vendor neutral.
145
00:11:28.190 --> 00:11:30.510
So that was a long introduction.
146
00:11:30.510 --> 00:11:39.285
I know that. But it's very essential to understand. We breathe and drink metadata from the beginning. We are vendor neutral.
147
00:11:39.445 --> 00:11:42.005
We have to support all these vendors.
148
00:11:42.404 --> 00:11:45.204
And that's basically how we got into that.
149
00:11:47.560 --> 00:11:56.839
That's definitely a long history and metadata has gained a lot of visibility in recent years, particularly starting around 2018
150
00:11:56.839 --> 00:11:58.279
when we had the
151
00:11:58.440 --> 00:11:59.560
fragmentation
152
00:11:59.560 --> 00:12:03.319
of the ecosystem away from a lot of the monolithic
153
00:12:03.135 --> 00:12:06.095
vertically integrated platforms like Informatica,
154
00:12:06.095 --> 00:12:09.455
etcetera, and into a lot of the cloud data warehouses,
155
00:12:09.455 --> 00:12:12.975
the introduction of tools like DBT and ELT workflows,
156
00:12:12.975 --> 00:12:28.120
and a lot of the investment in the various open source components that caused the knowledge of the data estate to be fragmented across all of them. And so then there was a push to try and stitch that back together through these various metadata systems.
157
00:12:28.360 --> 00:12:34.415
And I'm wondering if you can talk through some of the key pieces of friction and fragmentation
158
00:12:34.735 --> 00:12:53.700
that exist in the data ecosystem in particular and some of the ways that your long history maybe shows that everything old is new again or maybe some of the new tensions that are arising up because of the proliferation of new tools and new systems. And I'll I'll pause on the AI piece until later.
159
00:12:54.260 --> 00:12:56.020
Sounds good. Yeah.
160
00:12:56.580 --> 00:13:00.020
The the story has evolved. And, as it evolved,
161
00:13:00.365 --> 00:13:07.885
there are some ups and downs and some keywords that didn't help. One pair of keywords that didn't help is called data catalog.
162
00:13:08.204 --> 00:13:12.524
And I'm going to tell you why, because it's
163
00:13:12.524 --> 00:13:16.310
inevitable that if you are a software vendor, large or small,
164
00:13:16.550 --> 00:13:20.310
you cannot not have a metadata repository
165
00:13:20.310 --> 00:13:26.390
at the heart of your product. Okay, so no matter what, if you take a product from
166
00:13:26.630 --> 00:13:27.510
Informatica,
167
00:13:27.510 --> 00:13:32.805
as you mentioned, or Microsoft or even a smaller vendor, a data modelling tool or
168
00:13:33.365 --> 00:13:35.765
DI vendor must have a
169
00:13:35.925 --> 00:13:36.885
metadata
170
00:13:36.885 --> 00:13:37.685
repository
171
00:13:37.685 --> 00:13:40.725
to share metadata between different tools
172
00:13:40.885 --> 00:13:45.845
of their offering. Right. So a shared metadata repository was essential
173
00:13:46.290 --> 00:13:47.890
for metadata management.
174
00:13:48.290 --> 00:13:52.850
But then the branding and the data catalog twist didn't help
175
00:13:53.170 --> 00:13:57.250
because then, you know, everybody said you need a data catalog.
176
00:13:57.490 --> 00:13:58.290
But then people
177
00:13:58.855 --> 00:14:03.575
say, I already have a data catalogue. Oh, wait, I've got many data catalogue.
178
00:14:03.655 --> 00:14:09.575
Then started the words of catalogue of catalogue and all these kind of activities that were and
179
00:14:09.815 --> 00:14:10.935
it didn't help.
180
00:14:11.335 --> 00:14:13.895
And the answer is it is inevitable.
181
00:14:13.895 --> 00:14:27.269
You must have every tool. If you buy a vendor that doesn't have at least core metadata management to share metadata between its tools, you're not going to be there. So each vendor's got to have its metadata repository,
182
00:14:27.269 --> 00:14:28.709
its own data catalogue.
183
00:14:29.125 --> 00:14:32.964
But then sometimes you need also a higher level of catalogue
184
00:14:32.964 --> 00:14:34.245
that is across
185
00:14:34.245 --> 00:14:36.084
and vendor independent.
186
00:14:36.245 --> 00:14:55.720
Sometimes you don't need it because if you're single vendor, you're fine. You know, you don't need necessary support for all these different BI tools. But the larger your company is, you know, every time we talk to a large company and we ask the question, so what what database do you use? What DI or BI tool do you use? And the usual answer,
187
00:14:56.040 --> 00:15:03.405
name one, we probably got it somewhere, you know, and that's probably what it is. And when it comes to enterprise architecture
188
00:15:03.405 --> 00:15:05.165
and data lineage,
189
00:15:05.165 --> 00:15:10.365
there is no tool left behind. You need to basically support all of them. And there is
190
00:15:10.779 --> 00:15:13.900
a legacy architecture. There is a future architecture.
191
00:15:13.900 --> 00:15:15.820
There is a hybrid architecture.
192
00:15:16.060 --> 00:15:18.380
And that is very, very important.
193
00:15:18.860 --> 00:15:21.580
Now these tools are not created equal.
194
00:15:21.820 --> 00:15:24.300
A lot of data catalogues got produced
195
00:15:24.745 --> 00:15:27.865
as, I would say, a database of metadata.
196
00:15:27.945 --> 00:15:40.360
It's a cheap product to put on the market and you can put a quick UI even faster and better now, AI generated, and it's going to look like a million bucks and it's going to be great. It's going to be fantastic.
197
00:15:40.360 --> 00:15:42.920
But the devil is in the details to do it right.
198
00:15:43.399 --> 00:15:49.160
And then they got a hard time to what I call retrofit the metadata foundation,
199
00:15:49.720 --> 00:15:51.399
you know, on top of a pretty
200
00:15:51.654 --> 00:15:54.695
UI and high level systems.
201
00:15:54.855 --> 00:15:59.015
No, you got to build very strong metadata management foundation
202
00:15:59.255 --> 00:16:00.775
and metadata management
203
00:16:00.775 --> 00:16:04.375
foundations. It's all about version and configuration
204
00:16:04.375 --> 00:16:05.095
management
205
00:16:05.440 --> 00:16:12.720
and putting it all together. Many of these data catalogs do not have a true metadata version and configuration inside.
206
00:16:13.040 --> 00:16:16.960
And that's a big, big deal because it's all about change management.
207
00:16:17.535 --> 00:16:23.774
It's always changing. So first of all, bringing the metadata from the database, the DI, the
208
00:16:24.334 --> 00:16:36.120
the first time it's going to take time. Right. Because, you know, you can remember the first time working with Boeing, their Cognos server was on its knee during the days responding to users.
209
00:16:36.440 --> 00:16:40.440
So we had to harvest metadata at night when he was less busy
210
00:16:40.680 --> 00:16:47.214
and, you know, pass all this. But then there are technique of change management. You don't want to redo that all the time.
211
00:16:47.615 --> 00:16:56.255
You know, maybe only 5% of the reports have changed and a few have been added. So you're not detect change, put it together efficiently
212
00:16:56.430 --> 00:16:59.390
and then what is known as metadata stitching,
213
00:16:59.710 --> 00:17:04.590
which may sound easy if it's a relational database, you know, schema table column,
214
00:17:04.750 --> 00:17:12.804
all good, except true metadata stitching is the type the same. People didn't realize that that column,
215
00:17:12.885 --> 00:17:14.565
they changed the type on you.
216
00:17:15.285 --> 00:17:24.120
There is no errors in UBI report because they don't care. They don't know. But your number is wrong now. And metadata stitching is all about that.
217
00:17:24.360 --> 00:17:28.840
So that's part of the ingredients of true metadata management.
218
00:17:28.840 --> 00:17:32.440
And that basically is the big, big difference
219
00:17:32.760 --> 00:17:38.985
from this perspective. And what also changed and the differentiator between all the vendors
220
00:17:39.145 --> 00:17:41.945
is related to deployment and pricing.
221
00:17:42.505 --> 00:17:45.545
And that's also a huge differentiator.
222
00:17:45.865 --> 00:17:47.145
A lot of the modern
223
00:17:47.225 --> 00:17:57.100
data catalogues or metadata management solutions are cloud only, which is great, which is fine. But on the vendor cloud, we call it, you know, vendor managed
224
00:17:57.100 --> 00:18:07.895
on the vendor cloud. But some companies do not want to work on that particular vendor cloud or sometimes they are highly secure government or banks
225
00:18:07.895 --> 00:18:09.894
or things who really,
226
00:18:10.215 --> 00:18:15.975
besides the big guys like Amazon, Google, and even then they have private secure cloud,
227
00:18:16.340 --> 00:18:19.059
They want their own deployment that we call.
228
00:18:20.020 --> 00:18:25.380
Customer managed versus, you know, managed by by the vendor itself.
229
00:18:25.620 --> 00:18:29.380
And that's a big, big difference. So we differentiate ourselves
230
00:18:29.985 --> 00:18:31.665
by being independent
231
00:18:31.665 --> 00:18:36.625
on the deployment. You can do your self deployment or you can use our cloud.
232
00:18:36.945 --> 00:18:41.024
And it turns out even through our OEM partners like ClickTalent
233
00:18:41.345 --> 00:18:43.985
Data Catalog, 80% of the customers
234
00:18:44.539 --> 00:18:48.059
are basically deploying on their own cloud. I'm
235
00:18:48.059 --> 00:18:53.019
not going to use the word on premise because it's private cloud or it's their own system,
236
00:18:53.340 --> 00:18:56.140
but they are very, very independent.
237
00:18:56.674 --> 00:19:00.034
And that brings to the last factor, which is pricing.
238
00:19:00.515 --> 00:19:02.514
With pure cloud
239
00:19:02.755 --> 00:19:05.554
management, you know, vendor cloud management
240
00:19:05.715 --> 00:19:10.115
comes the side effect known as pricing and capacity pricing.
241
00:19:10.540 --> 00:19:12.940
And that has hurt this market
242
00:19:13.020 --> 00:19:14.539
very, very, very much.
243
00:19:14.780 --> 00:19:18.139
It starts by the notion of named user.
244
00:19:18.380 --> 00:19:19.500
So basically,
245
00:19:19.580 --> 00:19:22.140
you pay by named user with your email address
246
00:19:22.395 --> 00:19:27.034
and you need somebody or somebody is out of a company and you've got to manage all these name users.
247
00:19:27.275 --> 00:19:29.275
It becomes quickly expensive,
248
00:19:29.675 --> 00:19:42.650
but it's everywhere. You have to pay per connector. You add a connector. It's more you pay the volume per objects and it adds up. So what became the first very affordable 50 ks POC,
249
00:19:43.050 --> 00:19:44.169
it's becoming
250
00:19:44.570 --> 00:19:51.254
very hard to go with. So we adopted the exact opposite approach. So not only for deployment,
251
00:19:51.495 --> 00:19:52.934
you can deploy privately,
252
00:19:53.735 --> 00:20:08.499
you know, on your own and and you do not depend on that. You have your storage, you have control and you have a probably more affordable storage if you're a large company than we can provide for you. And again, things are becoming very, very big. We have customers
253
00:20:08.740 --> 00:20:13.700
like the large bank in Canada was mentioning, reaching three terabytes of metadata
254
00:20:14.015 --> 00:20:17.055
underneath with version and configuration management.
255
00:20:17.295 --> 00:20:26.015
It gets big, you know. So what we have is predictable pricing. We don't charge per name users. We have notion of concurrent users
256
00:20:26.335 --> 00:20:27.855
among unlimited users.
257
00:20:28.330 --> 00:20:29.690
So in other words,
258
00:20:30.010 --> 00:20:36.730
you can be on the block and users can come and go. Nobody should be denied access to the catalogue.
259
00:20:36.730 --> 00:20:43.784
Or then you have a lot more users than you thought, but that's good news. This means your metadata management is successful
260
00:20:43.865 --> 00:20:48.105
and then you can buy more concurrent users. We don't charge per connector.
261
00:20:48.185 --> 00:20:52.265
We don't charge per volume. So what predictable pricing leads to,
262
00:20:52.585 --> 00:20:59.610
as much as I said, the majority, maybe two third, are self deployed versus on our cloud.
263
00:21:00.010 --> 00:21:05.210
Most of them buy two years of predictable pricing in advance.
264
00:21:05.450 --> 00:21:08.250
The only reason they might increase in price
265
00:21:08.465 --> 00:21:10.945
is if they have a huge adoption
266
00:21:11.105 --> 00:21:13.265
and have a lot of concurrent users.
267
00:21:13.585 --> 00:21:16.305
And that's really a big, big differentiator
268
00:21:16.305 --> 00:21:17.265
right there.
269
00:21:17.425 --> 00:21:20.945
So again, a long response, but because there is a lot to say
270
00:21:21.265 --> 00:21:23.505
from, you know, a metadata repository,
271
00:21:23.960 --> 00:21:25.399
a data catalog,
272
00:21:25.400 --> 00:21:29.560
a metadata management. Remember Gartner had a metadata management
273
00:21:29.560 --> 00:21:30.360
quadrant,
274
00:21:30.520 --> 00:21:40.304
then it kind of disappeared. Now it's reappearing again because data catalog and then it was AI catalog versus or not, but we'll get into that later probably.
275
00:21:40.865 --> 00:21:42.545
One of the other aspects
276
00:21:43.105 --> 00:21:45.184
of this overall ecosystem,
277
00:21:45.265 --> 00:22:03.820
as you pointed out, there are a lot of Johnny come latelys where they say, yes, we do metadata. We'll bring all of your information about your data estate into one place, but maybe it's lacking some of the more nuanced and detail oriented aspects of true metadata capabilities such as versioning and change management like you were referencing.
278
00:22:04.299 --> 00:22:07.020
Another one of the huge challenges is the
279
00:22:07.534 --> 00:22:10.494
fact that, as you pointed out, every
280
00:22:11.534 --> 00:22:22.014
engine that works with the data needs to have some concept of its own metadata for how its own internal representation works, how it keeps track of state, how it keeps track of data assets.
281
00:22:22.894 --> 00:22:31.590
But you also need to be able to bring that to a more global view in order for things to be able to actually work across those different engines.
282
00:22:33.110 --> 00:22:34.390
I think that's another
283
00:22:35.030 --> 00:22:42.294
constant struggle that we have in the industry of every engine wants to grow more and more of its capability to own more of the metadata,
284
00:22:42.535 --> 00:22:46.135
but it's never going to be complete unless you have
285
00:22:47.095 --> 00:22:55.049
system where that is the sole purpose is to be the cross cutting repository of knowledge of the data estate for an organization.
286
00:22:55.770 --> 00:23:01.049
I'm curious how you are addressing some of the challenges of the
287
00:23:01.290 --> 00:23:03.265
locality versus consistency
288
00:23:03.265 --> 00:23:12.705
that arises with all of those engines needing to own some portion of the metadata, but then also needing to have information about the metadata of the other engines that they don't touch.
289
00:23:13.105 --> 00:23:19.360
Right. Right. So first of all, there is another differentiator that I forgot to mention on the first part, which is
290
00:23:19.680 --> 00:23:23.360
how do you get the data lineage within each of those, you know?
291
00:23:23.760 --> 00:23:25.200
And we
292
00:23:25.200 --> 00:23:30.160
brought a lot of what we ask for is what we call data lineage by design.
293
00:23:30.705 --> 00:23:36.544
So meaning that you have to prove where it comes from, from the design. You know, you can have
294
00:23:37.105 --> 00:23:43.985
an ETL tool, an ELT tool. You can have a script. You can have a very complex BI tool.
295
00:23:44.429 --> 00:23:55.309
And from the source of your data, which can be really on the far left of your architecture, can even come from a mainframe, a COBOL, it can be in a file to its life
296
00:23:55.470 --> 00:24:02.004
in a shot in a BI tool. On the right, there is a lot of data transformation,
297
00:24:02.325 --> 00:24:04.965
each specific in every tool.
298
00:24:04.965 --> 00:24:11.524
And when the number is not right, you have to prove where it's coming from. So back to the previous one first is
299
00:24:11.684 --> 00:24:16.639
one of the key aspect is to prove the lineage on each of these tools.
300
00:24:16.880 --> 00:24:21.840
And what we have developed all these years are pretty much emulators,
301
00:24:21.840 --> 00:24:24.559
debuggers of each of these tools,
302
00:24:24.640 --> 00:24:26.159
you know, like a power center,
303
00:24:26.605 --> 00:24:28.764
like a power BI
304
00:24:28.924 --> 00:24:33.884
on the right side or like a Tableau or like a very complex sparks
305
00:24:34.205 --> 00:24:35.884
for Python script,
306
00:24:35.884 --> 00:24:38.604
you know. So we have compilers, parses,
307
00:24:38.684 --> 00:24:40.044
get the runtime information,
308
00:24:40.580 --> 00:24:48.899
have a full emulation to understand exactly where it goes. So if that number is this, is we can prove the formulas on each of those.
309
00:24:49.140 --> 00:24:58.205
And even the OEM partners we have, people get shocked. You know, why? Why is Informatica Power Center sometimes calling our bridges?
310
00:24:58.285 --> 00:25:02.845
Well, they have a source code of power center over cloud data integration.
311
00:25:03.165 --> 00:25:04.925
But within a transformation,
312
00:25:04.925 --> 00:25:06.525
you can have 10 pages
313
00:25:07.100 --> 00:25:08.220
of SQL
314
00:25:08.300 --> 00:25:10.700
for for for Teradata
315
00:25:10.700 --> 00:25:18.779
or SQL Snowflake SQL or Python script that you need to emulate in order to have the details of a transformation.
316
00:25:18.940 --> 00:25:21.020
So we're not just observing logs.
317
00:25:21.445 --> 00:25:25.525
You know, sometimes we have to look at logs, runtime logs for information.
318
00:25:25.685 --> 00:25:27.445
We have true debuggers,
319
00:25:27.445 --> 00:25:28.965
emulators, parsers
320
00:25:28.965 --> 00:25:30.645
of all this transformation.
321
00:25:30.645 --> 00:25:33.925
And that will be very important later when we get into
322
00:25:34.420 --> 00:25:42.980
our new semantic hub is basically we understand what's going on inside each of these BI transformation to calculate metrics
323
00:25:43.060 --> 00:25:44.980
and what the formulas are.
324
00:25:45.140 --> 00:25:46.419
That is very important.
325
00:25:46.740 --> 00:25:48.500
So now the locality,
326
00:25:48.985 --> 00:25:51.464
the data flows in the enterprise
327
00:25:51.544 --> 00:25:53.544
from many, many levels.
328
00:25:53.945 --> 00:25:57.464
And, you know, there are often three or four different technologies
329
00:25:57.544 --> 00:26:00.504
on the left side of your main data warehouse.
330
00:26:00.585 --> 00:26:07.309
And then it goes into staging areas on the right marts, you know, and then it goes to your reports.
331
00:26:07.470 --> 00:26:13.309
We get all of them. We stage all of them and we take care of the automation.
332
00:26:13.390 --> 00:26:16.350
What happens as an illustration with this locality
333
00:26:16.350 --> 00:26:17.389
of the data
334
00:26:17.550 --> 00:26:18.350
is that
335
00:26:18.895 --> 00:26:21.534
many times it's actually a pass through.
336
00:26:21.775 --> 00:26:28.894
You know, you wouldn't be saying you see that and you're not going to document it on 20 of these systems.
337
00:26:29.375 --> 00:26:33.695
So we have the data lineage and the stitching that puts all these together.
338
00:26:34.220 --> 00:26:39.179
We have automatic detection of this pass through. So if you document,
339
00:26:39.260 --> 00:26:43.020
for example, in the data warehouse, if you document early
340
00:26:43.180 --> 00:26:44.700
in the file system,
341
00:26:45.020 --> 00:26:49.395
something like an SSN has been detected and we get it through or
342
00:26:49.555 --> 00:26:51.795
something very important out there.
343
00:26:52.275 --> 00:26:53.635
We have automatic
344
00:26:53.635 --> 00:26:54.755
documentation
345
00:26:54.755 --> 00:26:58.995
because we detect the pass through and as long as it has not been transformed,
346
00:26:59.155 --> 00:27:01.475
every time you get on these local systems,
347
00:27:01.930 --> 00:27:03.450
you have that definition
348
00:27:03.530 --> 00:27:08.809
that has been done automatically for you. And it turns out, despite all the transformation,
349
00:27:09.290 --> 00:27:21.825
more than half of the data keep passing proof from the, you know, bronze to silver to sometimes staging and goes all the way. And maybe the transformation is at the end, nothing else.
350
00:27:22.225 --> 00:27:24.304
So we have sometimes
351
00:27:24.305 --> 00:27:37.900
this if you have a business glossary term that has been done an automatic pass through on that. Simon, you want to add a little bit on that topic of locality and pass through and automatic documentation?
352
00:27:39.020 --> 00:27:40.460
Yeah, I think you
353
00:27:40.940 --> 00:27:42.380
described it perfectly.
354
00:27:43.180 --> 00:27:45.900
At the end of the day, Tobias can ask about,
355
00:27:46.585 --> 00:27:49.705
yeah, we have a DBT on the scene, we have
356
00:27:50.665 --> 00:27:52.425
cloud, data warehouses,
357
00:27:52.425 --> 00:27:53.305
snowflakes,
358
00:27:53.305 --> 00:27:54.505
data bricks,
359
00:27:54.505 --> 00:27:55.385
you know, they
360
00:27:55.865 --> 00:28:01.705
all bring their own kind of metadata management systems. And yes, we need to put this Humpty Dumpty together.
361
00:28:02.640 --> 00:28:04.320
But you said that
362
00:28:05.040 --> 00:28:07.840
when you look at this tapestry
363
00:28:07.840 --> 00:28:09.280
of relationship,
364
00:28:09.680 --> 00:28:13.600
and yes, you need to have a system that is capable to analyze
365
00:28:14.400 --> 00:28:16.080
kind of this data graph
366
00:28:16.480 --> 00:28:18.000
kind of with the speed of thought.
367
00:28:18.615 --> 00:28:19.975
This
368
00:28:19.975 --> 00:28:21.815
graph has billions
369
00:28:21.815 --> 00:28:22.455
of
370
00:28:22.695 --> 00:28:23.575
notes
371
00:28:24.215 --> 00:28:27.014
and we need to traverse it to give you an answer.
372
00:28:27.575 --> 00:28:29.815
And we're capable of doing it.
373
00:28:30.215 --> 00:28:32.935
But when you find what's wrong,
374
00:28:33.590 --> 00:28:37.750
then you ideally would like to jump back to the source,
375
00:28:38.630 --> 00:28:41.830
to the design document, to some kind of a script
376
00:28:42.150 --> 00:28:44.950
that produced this transformation.
377
00:28:45.510 --> 00:28:51.034
And this is where we differ. This is what's kind of paramount for a lot of organizations.
378
00:28:51.675 --> 00:28:53.595
Instead of kind of tracing
379
00:28:54.795 --> 00:28:55.675
lineage
380
00:28:55.675 --> 00:28:57.274
through logs,
381
00:28:57.515 --> 00:28:58.875
okay, which is very effective,
382
00:28:59.690 --> 00:29:00.809
And we're
383
00:29:00.970 --> 00:29:02.889
doing very well for
384
00:29:04.010 --> 00:29:04.890
outliers,
385
00:29:04.890 --> 00:29:07.769
you know, for things that we cannot get from design.
386
00:29:08.970 --> 00:29:10.570
Because for us design,
387
00:29:10.650 --> 00:29:11.130
it's
388
00:29:11.495 --> 00:29:14.375
kind of initial step. We take parametrization
389
00:29:14.375 --> 00:29:19.895
and build kind of runtime out of it. Runtime
390
00:29:19.895 --> 00:29:20.855
image,
391
00:29:20.855 --> 00:29:22.375
runtime lineage.
392
00:29:23.095 --> 00:29:28.215
Going through logs is fine, but you never know where this log
393
00:29:28.990 --> 00:29:30.190
kind of query,
394
00:29:30.190 --> 00:29:31.870
logged query came from.
395
00:29:32.350 --> 00:29:34.990
And this is a big, big problem.
396
00:29:35.070 --> 00:29:40.190
This problem actually going to get even worse when AI comes to the scene
397
00:29:40.670 --> 00:29:41.390
in spades.
398
00:29:42.674 --> 00:29:46.514
Log can be polluted in many ways because, you know, sometimes
399
00:29:46.755 --> 00:29:48.434
by just observing
400
00:29:48.434 --> 00:29:52.114
log is the only solution versus emulating the design,
401
00:29:52.674 --> 00:29:55.395
is a lot of things are happening in the log and it's not always
402
00:29:55.740 --> 00:30:07.740
easy to trace what happens. So first of all, some jobs are running on regular basis. It's not because they're running every night, but the lineage, the amount of time it runs affects it. Sometimes they're a backup and restore
403
00:30:08.075 --> 00:30:11.434
and the lineage come back, but that is a one time restore
404
00:30:11.434 --> 00:30:17.115
of an external database because of a merger or something. You know, these kind of things happen.
405
00:30:17.115 --> 00:30:25.480
So lineage by design is very important because when there is an audit and you combine all these lineage with version and configuration management
406
00:30:25.640 --> 00:30:28.279
where, you know, the customer gets an audit and
407
00:30:28.679 --> 00:30:32.279
they made a decision on that BI report number.
408
00:30:32.280 --> 00:30:39.735
Okay, but that was at the end of Q3 last year. Okay, where was it coming from? So if you don't have
409
00:30:40.135 --> 00:30:41.335
this lineage
410
00:30:41.335 --> 00:30:45.735
that basically is also combined with version and configuration management
411
00:30:45.815 --> 00:30:47.255
where you can prove
412
00:30:47.335 --> 00:30:52.230
at the time it was coming from this information from the ECTL,
413
00:30:52.230 --> 00:30:53.829
that's where they multiplied
414
00:30:53.830 --> 00:30:55.749
by 1.1%
415
00:30:56.149 --> 00:31:02.950
at that and that changed the number. That is the kind of critical information that I expected in metadata management.
416
00:31:05.475 --> 00:31:06.514
One of the other
417
00:31:06.914 --> 00:31:25.450
ways that metadata has manifested in recent years is the idea of the semantic layer as a distinct product where you have this one technological system that acts as the system of record to say, this is what this number means and maybe a little bit about how it got computed
418
00:31:25.610 --> 00:31:35.585
versus a more holistic approach like what you're doing where you have more of that organizational context by virtue of having the full scope of where the data came from and how it got there.
419
00:31:36.705 --> 00:31:40.705
Also, the idea of the metric system, there are a number of different,
420
00:31:41.185 --> 00:31:43.825
very narrowly scoped aspects that,
421
00:31:44.065 --> 00:31:45.345
to my understanding,
422
00:31:45.345 --> 00:31:45.825
ultimately
423
00:31:46.480 --> 00:31:50.080
boil down to it's a system of record for metadata.
424
00:31:50.080 --> 00:32:01.934
And I'm wondering what are your thoughts on just some of the ways that the industry keeps trying to define these point solutions rather than doing a more holistic approach of tackling all of the metadata
425
00:32:02.095 --> 00:32:04.414
because of the complexities involved?
426
00:32:06.575 --> 00:32:08.414
Right. Basically,
427
00:32:09.774 --> 00:32:11.294
the metadata management
428
00:32:11.695 --> 00:32:16.890
and the metadata were input for managing your DI, managing your BI.
429
00:32:17.290 --> 00:32:21.770
Now, obviously, the most important or equally important to
430
00:32:22.330 --> 00:32:36.234
the driving factor now is the metadata to serve as a context for AI, for LLM. And that is, you know, obviously the semantic out of that. So metadata management is obviously very well positioned for that.
431
00:32:36.794 --> 00:32:47.520
But, you know, a lot of the AI and LLM only were looking at the data. Then obviously everybody realized having all the metadata about the data will help them understand
432
00:32:47.520 --> 00:32:50.640
a lot more what's going on. But as you pointed out,
433
00:32:51.040 --> 00:32:52.800
it's not just the metadata,
434
00:32:52.800 --> 00:33:03.554
you know, the the definitions and everything you know about this table and this column and and all of this. I think the end to end whole story of consumption
435
00:33:03.955 --> 00:33:05.154
and production
436
00:33:05.154 --> 00:33:06.595
of this data
437
00:33:06.620 --> 00:33:10.540
to be used. That provides the true context for AI.
438
00:33:10.860 --> 00:33:15.020
And we are in a great position for that because having the lineage,
439
00:33:15.180 --> 00:33:24.804
we understand all the transformation where it came from, how it got into this data and that was how it's consumed outside. So we have all the joints,
440
00:33:24.885 --> 00:33:27.765
all the lookups, we have all the transformations.
441
00:33:28.085 --> 00:33:31.284
So we can provide a very rich context
442
00:33:31.285 --> 00:33:32.325
to AI
443
00:33:32.485 --> 00:33:35.284
to really answer the other questions
444
00:33:35.285 --> 00:33:36.325
you want to get into.
445
00:33:36.900 --> 00:33:38.740
But let's not forget that.
446
00:33:39.940 --> 00:33:56.835
It was not invented here, you know, as I was mentioning at the beginning of a podcast, you know, having all this semantic layer to be used only by reporting at the time was already the cup of tea of a business object universe or Cognos FM at the time.
447
00:33:57.315 --> 00:33:58.835
So we basically,
448
00:33:59.235 --> 00:34:04.515
in this new version of a product of a meta carta we just released, which is version 12
449
00:34:05.080 --> 00:34:07.959
of our product that shows our history as well.
450
00:34:08.280 --> 00:34:18.440
This version 12 include a very, very includes a very critical product called Semantic Hub and the Semantic Hub as a semantic language inside.
451
00:34:18.984 --> 00:34:20.905
So that's a platform
452
00:34:21.145 --> 00:34:24.505
to collect the semantic models inside
453
00:34:24.665 --> 00:34:27.065
and to drive both AI
454
00:34:27.145 --> 00:34:29.465
and BI on the other side.
455
00:34:29.945 --> 00:34:39.650
Now, because of where we're coming from, we have a very rich way to not start from scratch, to not just start from just, you know, the table and column.
456
00:34:39.810 --> 00:34:43.090
We have the whole context coming in where it's coming from.
457
00:34:43.410 --> 00:34:45.170
And we have the ability
458
00:34:45.490 --> 00:34:47.330
to bring this metadata
459
00:34:47.490 --> 00:34:48.050
from
460
00:34:48.545 --> 00:34:50.065
the problems people
461
00:34:50.225 --> 00:34:54.145
have already suffered with. You know, they have already,
462
00:34:54.865 --> 00:34:58.705
you know, with BI, with whether you use Power BI or Tableau,
463
00:34:58.865 --> 00:35:02.465
you know, it's been a sprawl of reports where everybody duplicated
464
00:35:02.465 --> 00:35:03.105
everything
465
00:35:03.430 --> 00:35:08.470
and all this semantic has been multiple interpretation of the same matrix.
466
00:35:08.630 --> 00:35:12.870
So we can bring these things in the semantic hub to reconcile
467
00:35:12.870 --> 00:35:17.270
them and then push them for a single implementation
468
00:35:17.270 --> 00:35:18.230
on the other side.
469
00:35:18.825 --> 00:35:23.065
So Simon will describe a little bit more of a semantic hub, but the
470
00:35:23.625 --> 00:35:25.465
use of metadata management
471
00:35:25.465 --> 00:35:26.825
is a platform
472
00:35:27.145 --> 00:35:29.625
that's got to drive a semantic
473
00:35:29.625 --> 00:35:38.690
hub out there, but we're not a semantic layer that you have to go through, you know, to get out there. We have a different approach. We bring the metadata
474
00:35:38.850 --> 00:35:43.010
from all these original tools, whether it's a BO universe,
475
00:35:43.170 --> 00:35:47.890
if you have some wonderful metrics out there, whether it's a Cognos or MicroStrategy,
476
00:35:48.375 --> 00:35:52.935
or the latest Tableau and Power BI. We can bring all this in.
477
00:35:53.095 --> 00:35:54.615
You can reconcile
478
00:35:54.775 --> 00:35:56.695
and then we can compile
479
00:35:56.695 --> 00:35:57.255
that
480
00:35:57.815 --> 00:36:03.920
into the semantic layer of your tool, which can be a semantic layer of Snowflake,
481
00:36:03.920 --> 00:36:07.359
that can be metrics views of Databricks,
482
00:36:07.359 --> 00:36:10.079
that can be analytic views of Oracle.
483
00:36:10.079 --> 00:36:15.760
We've been doing it all in what we do. But now you have a metric that is calculated and enforced
484
00:36:16.795 --> 00:36:18.555
in that database layer.
485
00:36:19.035 --> 00:36:21.515
But both the BI tools,
486
00:36:21.515 --> 00:36:28.715
you know, a new BI report, we can produce a semantic model for Power BI. We can also compile it into Tableau
487
00:36:29.060 --> 00:36:30.980
and UAI LLM
488
00:36:31.060 --> 00:36:32.500
are all sharing
489
00:36:32.500 --> 00:36:34.420
that same metric value.
490
00:36:35.060 --> 00:36:38.900
And depending on the context, you can have also different implementation.
491
00:36:39.140 --> 00:36:41.060
But at least you no longer have
492
00:36:41.484 --> 00:36:42.605
the you
493
00:36:42.605 --> 00:36:44.925
resolve two problems. You resolve
494
00:36:45.005 --> 00:36:47.565
the sprawl with different interpretations,
495
00:36:47.565 --> 00:36:59.700
you know, and you resolve also all these precious metrics. Some people are still using business object and Cognos today because they've been incapable of moving these metrics away from others.
496
00:37:00.100 --> 00:37:02.340
And Simon can tell
497
00:37:02.340 --> 00:37:10.900
a little bit more about our use of a semantic hub, but that's definitely the innovation we bring with MetaCarta version 12.
498
00:37:15.295 --> 00:37:16.015
Yes,
499
00:37:16.494 --> 00:37:18.735
this semantic story is exciting,
500
00:37:18.974 --> 00:37:21.295
and it's been kind of exciting
501
00:37:21.375 --> 00:37:22.575
for many years.
502
00:37:23.150 --> 00:37:30.510
Yeah, I came to the scene maybe thirty plus years ago with this business object defining not only
503
00:37:30.830 --> 00:37:35.790
what the real table is, but how to join it, how to describe it,
504
00:37:36.605 --> 00:37:39.244
and end users can start consuming it.
505
00:37:40.525 --> 00:37:45.244
BI kind of took off. Now we have this Power BI beauty, Looker,
506
00:37:45.645 --> 00:37:46.605
ThoughtSpot,
507
00:37:46.684 --> 00:37:47.885
etcetera, etcetera.
508
00:37:48.125 --> 00:37:51.885
But there is plenty of legacy. And at some point, I hope Christian will
509
00:37:52.340 --> 00:37:53.780
tell us that
510
00:37:53.940 --> 00:37:55.540
people who are struggling
511
00:37:55.540 --> 00:37:56.340
with
512
00:37:56.340 --> 00:37:57.300
legacy,
513
00:37:57.460 --> 00:37:58.900
like MicroStrategy,
514
00:37:58.900 --> 00:38:00.340
BO, Cognos,
515
00:38:00.500 --> 00:38:02.260
and they want to reuse
516
00:38:03.940 --> 00:38:04.660
what they have.
517
00:38:05.424 --> 00:38:07.105
Now they have an opportunity.
518
00:38:07.184 --> 00:38:10.464
But what happened probably ten years ago,
519
00:38:10.865 --> 00:38:17.025
plus, yes, semantic layer companies came to the scene. Yes, people allow you to
520
00:38:17.345 --> 00:38:19.984
kind of the translation between
521
00:38:19.984 --> 00:38:21.184
your raw data
522
00:38:21.640 --> 00:38:25.160
and kind of your logical representation of your enterprise
523
00:38:25.240 --> 00:38:30.680
in one place, create a server and expose the servers through API and suddenly,
524
00:38:30.840 --> 00:38:32.440
yeah, sometimes API,
525
00:38:32.440 --> 00:38:33.800
sometimes spreadsheets,
526
00:38:34.200 --> 00:38:36.040
sometimes kind of other applications
527
00:38:36.425 --> 00:38:44.105
can access it. And, but the promise that you will have your kind of semantics in one place.
528
00:38:44.505 --> 00:38:54.510
Like you guys said, metric stores. But it requires you to have this intermediate server and everything goes through it. I think this is a very honorable solution.
529
00:38:54.829 --> 00:39:01.630
If we kind of look what happened lately, this is where kind of cloud data warehouses,
530
00:39:01.950 --> 00:39:06.349
like Snowflake and Databricks. And in the past, even Oracle with analytic
531
00:39:06.585 --> 00:39:12.905
views, Oracle analytic views, let's not forget, they kind of pioneered this, you know, sometime back, but semantic
532
00:39:12.905 --> 00:39:14.905
views from Snowflake
533
00:39:14.984 --> 00:39:15.785
or
534
00:39:16.265 --> 00:39:17.385
Databricks,
535
00:39:18.025 --> 00:39:18.905
metric views,
536
00:39:19.590 --> 00:39:22.470
It's like kind of a new kid on the block.
537
00:39:22.550 --> 00:39:27.830
Know, they try to achieve the same thing. They try to have a place
538
00:39:27.830 --> 00:39:30.630
where you can have logic,
539
00:39:30.710 --> 00:39:37.145
all your business kind of logic in one place. You can define as many as possible kind of KPIs
540
00:39:37.464 --> 00:39:38.505
if you can,
541
00:39:38.905 --> 00:39:42.105
and allowing people just to ask
542
00:39:42.105 --> 00:39:43.545
a question
543
00:39:43.704 --> 00:39:46.424
without being concerned with joins,
544
00:39:46.620 --> 00:39:48.220
you know, basic complexity
545
00:39:48.220 --> 00:39:50.060
of your physical
546
00:39:50.380 --> 00:39:51.660
data representation.
547
00:39:52.460 --> 00:39:54.700
Okay, it's all kind of wonderful,
548
00:39:54.700 --> 00:39:56.540
but what are the alternatives?
549
00:39:56.540 --> 00:39:59.900
And we could have thought about it, you know, on one side,
550
00:40:01.015 --> 00:40:05.015
there are challenges that there is a competition. There is a semantic
551
00:40:05.335 --> 00:40:06.295
modeling
552
00:40:06.375 --> 00:40:07.975
war right now.
553
00:40:08.535 --> 00:40:10.215
On one side, Microsoft
554
00:40:10.295 --> 00:40:11.255
controls
555
00:40:11.494 --> 00:40:12.935
their semantic modeling,
556
00:40:13.460 --> 00:40:16.260
kind of in Power BI type of assets.
557
00:40:16.420 --> 00:40:20.420
On another side, let's say Snowflake would like to be the king of the hill.
558
00:40:21.140 --> 00:40:23.059
Matter of fact, they don't want to work together.
559
00:40:26.775 --> 00:40:27.975
Microsoft
560
00:40:28.454 --> 00:40:31.815
Power BI is not open for business easily to
561
00:40:31.895 --> 00:40:33.575
Snowflake and the same
562
00:40:33.815 --> 00:40:35.335
for Databricks.
563
00:40:35.575 --> 00:40:38.214
Okay, it means there is a crack
564
00:40:38.214 --> 00:40:38.934
in the system,
565
00:40:40.390 --> 00:40:46.390
unfortunately these vendors need to work together, but it's not going to happen. But imagine the world where you can
566
00:40:46.710 --> 00:40:47.590
design
567
00:40:47.590 --> 00:40:52.710
the complexity of your enterprise and we're taking it very seriously this word complexity.
568
00:40:53.430 --> 00:40:58.615
Business is not simple. You can have real world questions
569
00:40:58.615 --> 00:41:00.855
to ask and it requires
570
00:41:01.015 --> 00:41:01.895
you to
571
00:41:02.295 --> 00:41:07.815
have a meta model capable to capture the complexity of the world and this is what we're promising.
572
00:41:08.330 --> 00:41:13.130
Okay, this is quite unique. And again, it comes from our background.
573
00:41:13.210 --> 00:41:15.210
We spent twenty plus years
574
00:41:15.450 --> 00:41:16.730
kind of building
575
00:41:16.730 --> 00:41:24.090
this common meta model of BI, common meta model of data warehousing, common meta model of data integration.
576
00:41:24.225 --> 00:41:29.025
Imagine And if you take this common meta model of BI, this is a kind of a semantic
577
00:41:29.025 --> 00:41:30.785
model. We understand
578
00:41:31.025 --> 00:41:33.265
all types of kind of metrics,
579
00:41:33.265 --> 00:41:35.505
you know, that you can imagine.
580
00:41:35.985 --> 00:41:36.865
And you
581
00:41:37.905 --> 00:41:44.180
have this powerful system, okay, and you have it as a visual development environment. Obviously,
582
00:41:44.580 --> 00:41:48.660
there are kind of agentic skills that you can use to describe
583
00:41:48.740 --> 00:41:59.285
what you want and we will do it as well, but it's for business analyst, you don't need to be kind of SQL guru to use our system. This is very important.
584
00:41:59.445 --> 00:42:08.490
But then when you have your semantic model in place, and by the way, it's much easier to build when you're building it on top of data catalog,
585
00:42:08.490 --> 00:42:10.490
because you already have your
586
00:42:11.210 --> 00:42:15.690
kind of metadata of your database or file system in place.
587
00:42:16.090 --> 00:42:17.210
You already
588
00:42:17.210 --> 00:42:19.050
kind of understand joins
589
00:42:19.050 --> 00:42:20.250
because you
590
00:42:20.545 --> 00:42:22.465
kind of know the lineage
591
00:42:22.545 --> 00:42:24.385
inside of kind of BI
592
00:42:24.385 --> 00:42:25.585
and DI.
593
00:42:25.665 --> 00:42:28.145
And you already have descriptions because
594
00:42:28.465 --> 00:42:31.505
you may be imported them or maybe
595
00:42:31.905 --> 00:42:38.770
you kind of using data catalog as you're supposed to with a glossary, documenting
596
00:42:39.010 --> 00:42:40.050
particular
597
00:42:40.050 --> 00:42:47.570
physical assets like tables and columns. And then you can just drag and drop it into the semantic model and automagically
598
00:42:47.650 --> 00:42:48.770
you're going to build
599
00:42:49.115 --> 00:42:50.795
the core, the foundation.
600
00:42:50.795 --> 00:42:53.035
And then you need to kind of define
601
00:42:53.194 --> 00:42:54.315
your metrics,
602
00:42:54.555 --> 00:42:56.795
or if you're importing it from BI,
603
00:42:56.875 --> 00:43:00.234
you're going to get them already, because we can import,
604
00:43:00.795 --> 00:43:02.315
you know, reports,
605
00:43:02.315 --> 00:43:05.560
whole your kind of estate, understand what's essential,
606
00:43:05.800 --> 00:43:07.640
and bring it into this
607
00:43:08.119 --> 00:43:09.480
kind of semantic
608
00:43:09.480 --> 00:43:10.280
model
609
00:43:10.440 --> 00:43:12.440
that you require.
610
00:43:12.920 --> 00:43:15.000
But the magic comes next.
611
00:43:15.240 --> 00:43:19.795
You have the semantic model that you can query from SQL.
612
00:43:19.875 --> 00:43:23.635
Sorry, you can query from AI agents, okay?
613
00:43:24.035 --> 00:43:26.835
And the difference is kind of very
614
00:43:26.835 --> 00:43:33.670
significant. Remember in semantic layer, okay, you of, you need to ask a question
615
00:43:33.990 --> 00:43:35.670
using their language
616
00:43:35.910 --> 00:43:40.390
type of a thing, and they can translate it into a database,
617
00:43:40.470 --> 00:43:41.910
ask the database,
618
00:43:42.310 --> 00:43:49.155
and send you a result back. In our case, you know, you just need to give us a business
619
00:43:49.234 --> 00:43:50.915
kind of question.
620
00:43:51.474 --> 00:43:54.915
We're going to give you kind of just SQL.
621
00:43:54.994 --> 00:44:05.170
You use the SQL, and again, we are kind of an MCP server. And then you can use another MCP server of the database to reliably kind of ask the question
622
00:44:05.330 --> 00:44:11.650
in terms of SQL directly and do whatever you want. We're not involved kind of in data per se,
623
00:44:12.595 --> 00:44:13.395
we're
624
00:44:13.395 --> 00:44:15.155
still involved in metadata.
625
00:44:15.315 --> 00:44:16.835
And you have full control,
626
00:44:17.155 --> 00:44:17.955
security,
627
00:44:17.955 --> 00:44:23.475
full privacy available to you. Okay, this is a very unique architectural difference,
628
00:44:23.715 --> 00:44:24.275
okay,
629
00:44:24.515 --> 00:44:26.755
that takes us to the next level.
630
00:44:28.130 --> 00:44:29.810
But in
631
00:44:29.810 --> 00:44:30.530
addition
632
00:44:30.690 --> 00:44:31.330
to
633
00:44:32.370 --> 00:44:34.850
being able to provide
634
00:44:35.810 --> 00:44:37.810
SQL information to
635
00:44:38.130 --> 00:44:42.130
agents, and we believe this is the kind of a future of
636
00:44:42.605 --> 00:44:44.525
metadata cataloging,
637
00:44:45.244 --> 00:44:46.845
data cataloging
638
00:44:47.164 --> 00:44:48.605
as they call today.
639
00:44:50.045 --> 00:44:57.244
You need to use kind of wealth of information in your catalog to help agents to produce a deterministic,
640
00:44:57.520 --> 00:44:58.800
sequel.
641
00:44:59.120 --> 00:44:59.680
Because
642
00:45:00.080 --> 00:45:03.600
let me contrast it what's happening in the world today.
643
00:45:03.920 --> 00:45:07.040
A lot of systems, they do have
644
00:45:07.520 --> 00:45:08.720
an understanding
645
00:45:08.720 --> 00:45:10.080
of relationships,
646
00:45:10.080 --> 00:45:12.560
sometimes between tables they have descriptions,
647
00:45:13.115 --> 00:45:16.395
and they give it to you as a context, and then they rely
648
00:45:17.355 --> 00:45:19.435
on the LLM to produce
649
00:45:19.835 --> 00:45:20.635
SQL.
650
00:45:20.635 --> 00:45:22.795
LMs are capable to produce SQL,
651
00:45:23.115 --> 00:45:25.835
but they're very creative. Sometimes they
652
00:45:26.109 --> 00:45:28.430
produce SQL that does not compile,
653
00:45:28.430 --> 00:45:32.990
sometimes they produce SQL that is wrong, does not deliver what's required.
654
00:45:33.310 --> 00:45:34.670
We are guaranteeing
655
00:45:34.670 --> 00:45:36.430
that the SQL we produce
656
00:45:36.510 --> 00:45:41.310
will be done according to the semantic model you created. It will run. And we support
657
00:45:41.645 --> 00:45:46.605
kind of not only kind of modern data warehouses, we support even legacy like Postscript
658
00:45:46.845 --> 00:45:47.805
warehouses,
659
00:45:47.805 --> 00:45:50.925
Oracle, etcetera. Okay, I hope you're
660
00:45:50.925 --> 00:45:51.965
getting the gist.
661
00:45:52.204 --> 00:45:53.485
But in addition,
662
00:45:53.565 --> 00:45:55.725
imagine you're running your
663
00:45:55.805 --> 00:45:56.445
traditional
664
00:45:56.840 --> 00:45:57.800
BI
665
00:45:58.200 --> 00:45:59.320
analytics.
666
00:45:59.400 --> 00:46:05.560
Okay, no problem. We can generate your Tableau Power BI metadata
667
00:46:05.720 --> 00:46:06.520
as well.
668
00:46:06.920 --> 00:46:11.720
And if you attach to a particular database semantic layer, let's say,
669
00:46:12.494 --> 00:46:22.575
know, snowflake semantic views. We can generate them for you as well. And we can... And you can kind of... Because at the end of the day, this is a kind of a true
670
00:46:23.295 --> 00:46:24.414
deterministic
671
00:46:24.414 --> 00:46:24.894
way
672
00:46:25.619 --> 00:46:27.380
to get something from
673
00:46:27.700 --> 00:46:29.460
even Genie
674
00:46:29.539 --> 00:46:30.420
Databricks
675
00:46:30.420 --> 00:46:31.300
or
676
00:46:31.460 --> 00:46:32.900
kind of Snowflake,
677
00:46:33.299 --> 00:46:34.819
agentic environments.
678
00:46:34.819 --> 00:46:40.180
You know, you can ask them a lot of questions. They unfortunately try to get to the database
679
00:46:39.905 --> 00:46:41.265
and make mistakes,
680
00:46:41.425 --> 00:46:43.185
but if they have this
681
00:46:43.585 --> 00:46:45.265
well structured
682
00:46:45.665 --> 00:46:47.345
semantically correct
683
00:46:47.345 --> 00:46:51.345
semantic layer, okay, that we generate, then suddenly
684
00:46:51.425 --> 00:46:57.230
they can give you deterministic answer that will cover 100 of the questions that people are asking.
685
00:46:57.790 --> 00:47:00.830
I hope it gives you kind of a picture overview,
686
00:47:00.830 --> 00:47:03.390
the first kind of taste of what we're cooking.
687
00:47:04.510 --> 00:47:05.150
Absolutely.
688
00:47:05.950 --> 00:47:10.195
And so you've brought in the subject of AI, and that is definitely
689
00:47:11.315 --> 00:47:12.195
unavoidable
690
00:47:12.195 --> 00:47:18.355
in any conversation regarding technology these days. And I'm wondering if you can talk to some of the
691
00:47:18.515 --> 00:47:25.150
new and interesting stressors that the AI and agentic machine speed interaction
692
00:47:25.150 --> 00:47:29.550
with both consuming and producing metadata has on
693
00:47:29.710 --> 00:47:38.735
systems like what you're building and some of the ways that your Medicarda product is oriented towards addressing some of those challenges in the market.
694
00:47:40.655 --> 00:47:42.175
Okay.
695
00:47:43.775 --> 00:47:45.935
Okay. We're jumping into AI.
696
00:47:46.175 --> 00:47:46.895
Loving
697
00:47:47.214 --> 00:47:52.839
it. Okay, let's contrast it with what we kind of used to. Kind of a typical data catalog
698
00:47:52.920 --> 00:47:55.079
many years ago was basically
699
00:47:55.720 --> 00:47:58.599
kind of a town or municipal library.
700
00:47:58.599 --> 00:48:01.160
Kind of you had a dedicated librarian,
701
00:48:01.160 --> 00:48:02.359
let's call her,
702
00:48:03.345 --> 00:48:04.065
Sarah,
703
00:48:04.385 --> 00:48:06.225
and some kind of an elite
704
00:48:06.465 --> 00:48:08.145
IT engineer.
705
00:48:08.305 --> 00:48:10.385
And they catalogue every dataset,
706
00:48:10.385 --> 00:48:11.745
file this
707
00:48:11.985 --> 00:48:13.745
Dewey Decimal cards
708
00:48:13.905 --> 00:48:14.465
and
709
00:48:14.865 --> 00:48:19.520
maybe dusted shelves once a week. It was kind of beautiful, quiet,
710
00:48:19.840 --> 00:48:20.640
deliberate,
711
00:48:20.640 --> 00:48:21.520
predictable.
712
00:48:21.680 --> 00:48:24.480
Now picture kind of changes dramatically.
713
00:48:24.720 --> 00:48:25.360
Instead
714
00:48:26.400 --> 00:48:26.960
of
715
00:48:27.360 --> 00:48:28.560
quiet librarians,
716
00:48:28.560 --> 00:48:30.320
you have thousand
717
00:48:31.215 --> 00:48:32.335
hyperactive
718
00:48:32.415 --> 00:48:35.935
research assistants who are drinking Red Bull, probably,
719
00:48:36.335 --> 00:48:37.695
and never sleep.
720
00:48:39.055 --> 00:48:44.175
This is kind of what autonomous agents and LLMs are doing to metadata management.
721
00:48:44.589 --> 00:48:46.590
And the first kind of headache
722
00:48:47.230 --> 00:48:49.630
is going to be related to
723
00:48:50.190 --> 00:48:54.110
kind of asset sprawl, kind of sorcerer apprentice sprawl.
724
00:48:54.270 --> 00:48:57.310
The human analyst can create probably
725
00:48:57.710 --> 00:48:58.990
scratch pad tables,
726
00:48:59.755 --> 00:49:03.915
you know, maybe a few, maybe a dozen a week. LLM agent can
727
00:49:04.234 --> 00:49:05.675
kind of create,
728
00:49:06.155 --> 00:49:07.355
you know, hundreds
729
00:49:07.355 --> 00:49:08.555
by breakfast,
730
00:49:08.555 --> 00:49:14.075
you know, baseband, sometimes 50 temporary views, synthetic joints
731
00:49:13.770 --> 00:49:16.250
and abandon them after they're
732
00:49:16.250 --> 00:49:18.250
done just with a single
733
00:49:18.730 --> 00:49:20.490
kind of a slack question.
734
00:49:20.890 --> 00:49:23.290
And in this case, if
735
00:49:23.290 --> 00:49:25.210
you keep collecting this information,
736
00:49:26.730 --> 00:49:27.850
you're Sarah, you know,
737
00:49:28.785 --> 00:49:30.465
you would like to clean your...
738
00:49:30.545 --> 00:49:36.545
Have your library kind of in in good shape. Suddenly it looks like,
739
00:49:36.625 --> 00:49:39.905
I don't know, episodes of hoarders or something.
740
00:49:41.720 --> 00:49:44.760
Basically you aren't kind of garbage collecting
741
00:49:44.839 --> 00:49:46.920
at machine speed and
742
00:49:47.480 --> 00:49:52.040
your catalog will choke. This is one of the challenges, stressors that
743
00:49:52.359 --> 00:49:55.000
we are thinking about, you know, and
744
00:49:56.005 --> 00:49:58.565
trying to kind of provide solutions.
745
00:49:59.045 --> 00:49:59.925
Second
746
00:49:59.925 --> 00:50:02.085
issue is that we
747
00:50:02.244 --> 00:50:08.244
lost our kind of train tracks, you know, and I'm talking about kind of non deterministic
748
00:50:08.244 --> 00:50:12.850
of data lineage. Traditional data lineage was kind of very mechanical.
749
00:50:12.930 --> 00:50:19.810
You have a table A plus table B, kind of equal table C with clear, some kind of
750
00:50:20.130 --> 00:50:22.850
ETL or SQL script. You can
751
00:50:23.415 --> 00:50:33.895
trace it backwards with kind of your eyes closed, you know, people knew what they're talking about. And LMs don't just kind of run on these tracks,
752
00:50:34.055 --> 00:50:38.780
they drive kind of off road buggies, you know, because
753
00:50:38.940 --> 00:50:41.020
they kind of think probabilistically
754
00:50:41.020 --> 00:50:43.180
instead of kind of humans.
755
00:50:43.820 --> 00:50:46.620
And when kind of compliance auditor
756
00:50:46.700 --> 00:50:51.215
walks in six months later and asks, why did you merge European
757
00:50:51.215 --> 00:50:53.535
user IDs with marketing
758
00:50:54.095 --> 00:50:55.055
churn
759
00:50:55.455 --> 00:50:56.415
data?
760
00:50:56.895 --> 00:50:58.095
You can point
761
00:50:58.415 --> 00:50:59.135
to
762
00:50:59.215 --> 00:51:04.095
kind of a real DAG from Spark, you know, you have to explain
763
00:51:04.095 --> 00:51:04.975
an algorithm
764
00:51:08.119 --> 00:51:08.920
was
765
00:51:09.799 --> 00:51:12.200
Tuesday morning, a long time ago.
766
00:51:13.400 --> 00:51:19.925
This is another kind of a challenge that we're preparing ourselves. Imagine we will be able to trace
767
00:51:20.565 --> 00:51:25.125
the lineage, understand that it was a prompt and this was a decision,
768
00:51:25.845 --> 00:51:33.359
but still it becomes very messy. It means we're preparing for lineage that will be a web
769
00:51:33.359 --> 00:51:34.320
of
770
00:51:34.559 --> 00:51:35.680
relationships
771
00:51:35.680 --> 00:51:39.680
and we need to untangle it. I think another one is obviously speed,
772
00:51:39.760 --> 00:51:41.520
know. This is what we...
773
00:51:41.920 --> 00:51:43.200
Because traditionally,
774
00:51:43.200 --> 00:51:51.385
you know, we're all kind of involved in software development. We know that LLMs these days they can speed code very quickly. Okay? And
775
00:51:51.785 --> 00:51:54.265
they can as well, you know, SQL,
776
00:51:54.585 --> 00:51:55.785
you know, or
777
00:51:55.945 --> 00:51:57.145
the transformations
778
00:51:57.145 --> 00:52:03.299
very quickly. Means kind of all catalogs that were built like yellow pages updated,
779
00:52:03.299 --> 00:52:04.579
let's say overnight.
780
00:52:04.579 --> 00:52:08.339
Remember Christian was telling kind of stories of Boeing,
781
00:52:08.339 --> 00:52:09.460
yeah, because
782
00:52:09.859 --> 00:52:10.900
resources
783
00:52:11.059 --> 00:52:12.420
not available
784
00:52:12.579 --> 00:52:16.339
during the day. You know, the servers are too busy. But agents
785
00:52:16.339 --> 00:52:19.205
use will need to use catalogs
786
00:52:19.525 --> 00:52:21.365
as kind of their
787
00:52:21.605 --> 00:52:28.325
kind of eyes and ears, kind of in real time. And if an agent has to kind of pause for,
788
00:52:28.565 --> 00:52:31.845
I don't know, three seconds to fetch schema definition
789
00:52:32.079 --> 00:52:33.760
via kind of sluggish
790
00:52:33.839 --> 00:52:40.400
interface of ours, know, the whole pipeline grinds to a halt. It means catalogs has to be nimble.
791
00:52:40.559 --> 00:52:45.760
And again, our experience and our capability to kind of allow people to traverse,
792
00:52:46.265 --> 00:52:48.105
you know, billions of objects,
793
00:52:48.265 --> 00:52:50.665
you know, with the speed of thought,
794
00:52:50.665 --> 00:52:54.905
you know, come very handy in this case. And kind
795
00:52:55.225 --> 00:53:00.990
of... Think the the last thing that kind of that come to mind when we're
796
00:53:00.990 --> 00:53:04.110
talking about this subject is
797
00:53:04.350 --> 00:53:05.150
jigsaw
798
00:53:05.150 --> 00:53:06.350
puzzle problem.
799
00:53:06.590 --> 00:53:09.950
Everybody knows it. Loose lips sink ships.
800
00:53:10.670 --> 00:53:15.895
In data, we always relied on a strict column level access control.
801
00:53:16.055 --> 00:53:19.095
Okay? This is what we used to. But agents
802
00:53:19.095 --> 00:53:20.295
are incredible
803
00:53:20.295 --> 00:53:23.095
at connecting kind of innocent dots.
804
00:53:23.415 --> 00:53:24.935
You know, sometimes they even
805
00:53:25.015 --> 00:53:26.935
can get data
806
00:53:26.935 --> 00:53:30.190
from places where you kind of do not expect.
807
00:53:30.350 --> 00:53:33.790
When you ask agent not to have access to,
808
00:53:34.190 --> 00:53:35.870
I don't know, executive salaries,
809
00:53:35.870 --> 00:53:37.710
but it can read anonymized
810
00:53:37.710 --> 00:53:39.070
travel receipts,
811
00:53:39.310 --> 00:53:41.150
corporate calendar invites,
812
00:53:41.150 --> 00:53:42.590
and batch swipe
813
00:53:42.735 --> 00:53:43.935
kind of logs,
814
00:53:44.095 --> 00:53:52.095
you know. It can kind of put something together, you know, that you don't expect. It means kind of basically secret can be out
815
00:53:52.495 --> 00:53:53.295
easily.
816
00:53:53.295 --> 00:53:59.040
You know, the security kind of will become paramount and data catalogs,
817
00:53:59.120 --> 00:54:03.840
you know, need to kind of help. And I'm here, I'm talking about kind of futures.
818
00:54:03.840 --> 00:54:05.840
I'm talking about problems
819
00:54:05.840 --> 00:54:07.120
that we
820
00:54:07.120 --> 00:54:08.560
kind of thinking about.
821
00:54:08.800 --> 00:54:21.725
But the world is changing. The world is changing kind of every day and we are adapting. And yes, we used to kind of build data catalogs to help people to find things. Now we have to build them to keep machines from running off the rail.
822
00:54:23.819 --> 00:54:29.500
So digging into the Medicarda product itself that you're delivering, I guess,
823
00:54:29.740 --> 00:54:40.984
what are the aspects of the current time in the world that we have that prompted you to go direct to consumer after such a long history of working with these larger vendors.
824
00:54:41.464 --> 00:54:49.464
And I'm wondering if you can explain some of the design and architecture of the Medicarda platform and how it fits into the...
825
00:54:49.464 --> 00:54:55.360
I hate to use the term modern data stack, but the set of technologies that a majority of people are using today.
826
00:54:56.640 --> 00:54:59.840
Sure. I'll address these remaining questions.
827
00:55:00.000 --> 00:55:08.320
Again, the word data catalog is something we don't like to use. It is actually pejorative to the whole purpose of a metadata management
828
00:55:07.895 --> 00:55:11.335
platform. It's just not just about cataloging data.
829
00:55:11.734 --> 00:55:13.095
It's way more than that.
830
00:55:13.494 --> 00:55:14.055
The
831
00:55:14.615 --> 00:55:15.255
reason
832
00:55:15.415 --> 00:55:16.855
we offered Metacarta
833
00:55:16.855 --> 00:55:17.575
in
834
00:55:17.895 --> 00:55:19.895
conjunction with OEM
835
00:55:19.895 --> 00:55:27.340
vendors is because there is a need for both. You know, some people want a solution from one vendor and focusing on a vendor.
836
00:55:27.740 --> 00:55:37.020
And even if they need to support multiple tools, they want to focus on the metadata management platform or data catalog of that vendor. And that's
837
00:55:37.365 --> 00:55:42.405
just fine. And some others want a more vendor neutral solution.
838
00:55:42.805 --> 00:56:01.870
And as I said, we got pushed by quite a few large companies and is, you know, pretty much making the analogies. You provide wonderful Lego bricks to to all these large vendors. But we too are big and we want to have access to your Lego bricks pretty much. So we got pushed in doing that.
839
00:56:02.190 --> 00:56:08.025
We are absolutely dedicated to both. So meta integration as a Switzerland of metadata,
840
00:56:08.025 --> 00:56:15.225
when there is an improvement that we do for for Microsoft and Power BI or for now Salesforce Informatica,
841
00:56:15.225 --> 00:56:19.065
Power Center or we do a new improvement, it's for everybody.
842
00:56:19.630 --> 00:56:20.830
Our ecosystem
843
00:56:20.830 --> 00:56:24.350
for the past thirty years is whatever new connectors,
844
00:56:24.350 --> 00:56:28.510
new lineage or new, it's equally available to everybody,
845
00:56:28.750 --> 00:56:31.150
to all OEM partners. You'll see it within
846
00:56:31.625 --> 00:56:32.505
Informatica,
847
00:56:32.505 --> 00:56:33.704
within Salesforce,
848
00:56:33.704 --> 00:56:38.905
within Erwin, within any tool and so are the direct. So that is the motivation
849
00:56:39.145 --> 00:56:47.580
is whatever you need and what fits you will be there. But there is no version for one versus a version for another.
850
00:56:47.660 --> 00:56:54.460
Whatever latest improvement or news or new lineage or new feature is available to everybody.
851
00:56:54.700 --> 00:56:57.820
And that's what has kept this company going.
852
00:56:58.059 --> 00:57:01.605
It's the, you know, neutrality, the ecosystem
853
00:57:01.845 --> 00:57:05.925
and desire to maintain that for all. So that's what pushed Medacart.
854
00:57:05.925 --> 00:57:23.610
And what what is the architecture and and what has evolved in the latest version of all this? Well, besides the the classic thing you expect on a new version. So first of all, for some reason, we decided to redesign the entire UI. Even UIs have evolved. You know, we have seen colorful
855
00:57:23.690 --> 00:57:36.145
UI and things and now there is a more cleaner look, simplified look, you know, and you see pretty much a similar look when you're looking at Snowflake or Microsoft or any of these products.
856
00:57:36.305 --> 00:57:40.945
And when you look and feel, it's exactly the same. It feels very irated,
857
00:57:40.945 --> 00:57:41.825
you know, focused.
858
00:57:42.290 --> 00:57:46.610
Of course, you upgrade to the latest of for security reason,
859
00:57:46.690 --> 00:57:51.330
you have to keep up with all open source software. You know, it's not just Java, Tomcat,
860
00:57:51.410 --> 00:57:54.690
all these things have to be keeping up. But the real
861
00:57:54.930 --> 00:57:59.705
new aspect of this version is the addition of a semantic hub
862
00:57:59.785 --> 00:58:01.145
as a platform
863
00:58:01.465 --> 00:58:04.425
to drive both AI and BI
864
00:58:04.425 --> 00:58:06.345
to give the same answer.
865
00:58:06.585 --> 00:58:14.520
And our approach to the semantic hub and its language inside is a visual semantic hub with a semantic compiler,
866
00:58:14.520 --> 00:58:28.515
again, that generates the semantic layers, that generate the BI and make sure both AI and BI are looking from the same. So the big difference with the classic people in the semantic space in this new architecture
867
00:58:28.595 --> 00:58:32.435
is we are not a layer. We're not an MDX emulator.
868
00:58:32.675 --> 00:58:36.435
We're not a layer that you call, that you depend upon.
869
00:58:36.595 --> 00:58:45.180
The key aspect of what we provide is at the end, you don't depend on us. We are pretty much the platform for the development of the metadata that we push,
870
00:58:45.500 --> 00:58:47.900
but you're not a real time
871
00:58:47.900 --> 00:58:50.620
bottleneck that you depend upon
872
00:58:50.620 --> 00:58:54.860
to do that. Basically, we're not an MDX layer, we're not a layer.
873
00:58:55.395 --> 00:59:01.715
And also the approach compared to perhaps other data catalog is all about deterministic
874
00:59:01.715 --> 00:59:02.515
SQL.
875
00:59:02.755 --> 00:59:05.875
We don't use AI to generate the SQL.
876
00:59:06.195 --> 00:59:07.315
We compile
877
00:59:07.315 --> 00:59:11.230
the semantic layer into the database. We compile
878
00:59:11.230 --> 00:59:15.390
into the semantic layers of BI tools and we generate
879
00:59:15.550 --> 00:59:16.830
deterministically
880
00:59:17.310 --> 00:59:21.070
the SQL for AI so that they all give the same answer.
881
00:59:21.310 --> 00:59:22.670
Big differentiator
882
00:59:22.670 --> 00:59:32.035
in all this. So yes, that's what's new. And suddenly the semantic hub is where we believe the big future is,
883
00:59:32.435 --> 00:59:34.435
you know, to share a semantic
884
00:59:34.595 --> 00:59:37.635
model and platform between AI and BI.
885
00:59:40.280 --> 00:59:41.080
Another
886
00:59:41.080 --> 00:59:50.600
keyword that you're pushing out is the idea of metadata ops and that metadata management should be a dedicated concern for
887
00:59:50.085 --> 00:59:59.285
organization and not an afterthought or a bolt on to another role. And I'm curious if you can talk to some of the key skills and activities that are associated
888
00:59:59.285 --> 01:00:01.205
with that capability
889
01:00:01.205 --> 01:00:03.605
and some of the ways that organizations
890
01:00:03.605 --> 01:00:07.680
should be thinking about the scope and staffing for
891
01:00:07.840 --> 01:00:10.160
that particular functionality?
892
01:00:11.520 --> 01:00:15.840
Basically, metadata ops is all about the roles and responsibilities
893
01:00:15.840 --> 01:00:16.800
and processes
894
01:00:17.454 --> 01:00:28.095
on top of the tooling, the metadata management tooling and the semantic hub on top of that. You know, you can give the fastest car and the most capable off road vehicle to anyone.
895
01:00:28.494 --> 01:00:37.300
If you don't have the methods and processes, you're most likely going to fail at some point. And I think this is something we now believe is important.
896
01:00:37.620 --> 01:00:42.340
But when we were not direct to customers, we've been kind of ignoring because
897
01:00:42.340 --> 01:00:45.460
we were just supplying the technology and platform
898
01:00:45.540 --> 01:00:49.515
and the tools and everything. But to have a total success,
899
01:00:49.675 --> 01:00:52.555
you also need to have the roles and responsibilities.
900
01:00:53.035 --> 01:00:54.955
And some of them are very key.
901
01:00:55.195 --> 01:01:05.400
You know, part of the success and failures of metadata management and data catalogs in general are often related to that. You know, some people have a very inexpensive
902
01:01:05.400 --> 01:01:15.240
product to start with and then they cannot scale. But part of the other problem they've been facing, they didn't put the right resources and processes on top of that to make it happen.
903
01:01:15.895 --> 01:01:16.535
Now,
904
01:01:17.015 --> 01:01:18.375
what has
905
01:01:18.375 --> 01:01:37.060
happened since then is DevOps and DataOps have been around and understood, especially DevOps, I would say, you know, with the cloud infrastructure. DataOps have been specialized sometimes with some tools. But what metadata ops is really about all these processes around because it's again to support change
906
01:01:37.220 --> 01:01:42.579
in the platform we just explained before. It is to integrate with the existing processes.
907
01:01:43.194 --> 01:01:47.994
Of course, this data warehouse, if you have snowflakes or data breaks or anything, they keep changing.
908
01:01:48.234 --> 01:01:55.115
The tables keep changing, you know, and the ETL and everything keep changing. So we need to integrate in these processes
909
01:01:55.194 --> 01:01:57.595
as they deploy new tables, new views,
910
01:01:58.010 --> 01:01:59.850
we automatically generate
911
01:02:00.010 --> 01:02:02.410
and deploy continuous deployments
912
01:02:02.410 --> 01:02:13.265
of these semantic views, the semantic models on top of that generate so that you can keep up with that. And there are processes you need to integrate with to make it happen.
913
01:02:13.585 --> 01:02:17.985
And that's where metadata ops. You know, it is very important.
914
01:02:18.065 --> 01:02:19.585
And we're now investing
915
01:02:19.585 --> 01:02:21.185
a lot in describing.
916
01:02:21.425 --> 01:02:24.705
And frankly, it helps all these services companies
917
01:02:24.990 --> 01:02:27.950
as well as those who use internal resources
918
01:02:28.030 --> 01:02:33.390
to actually be successful with the tools. Back again, you can have the best tools and technologies
919
01:02:33.470 --> 01:02:36.350
if you don't use it right, if you don't have the processes
920
01:02:36.510 --> 01:02:42.615
to make it happen. And if you do not integrate well with your existing data warehouse processes
921
01:02:42.695 --> 01:02:46.135
because you already have a lot of data integration and deployment process,
922
01:02:46.375 --> 01:02:51.575
you're most likely going to fail and even more at the pace required for AI today. That's definite.
923
01:02:53.380 --> 01:03:00.100
For teams who either already have a substantial data estate and some form of metadata management
924
01:03:00.100 --> 01:03:02.580
and also for teams who are just getting started,
925
01:03:02.900 --> 01:03:09.255
what does the adoption path for the Medicarda product look like and how does it change their day to day work?
926
01:03:10.855 --> 01:03:11.735
Right.
927
01:03:11.735 --> 01:03:16.615
Well, the adoption path, part of our success is our quick time from
928
01:03:17.100 --> 01:03:20.860
install, deploy, harvest and get your first results.
929
01:03:20.940 --> 01:03:25.820
You know, a lot of the data catalogs and metadata management have been given sometimes
930
01:03:25.980 --> 01:03:36.615
the analogy of the data warehouse took forever to install or like the jokes of some, if you manage to install it, you deserve a license and even more to harvest metadata and all of this.
931
01:03:36.935 --> 01:03:40.535
Our key to success from the practice perspective
932
01:03:40.775 --> 01:03:45.255
is it's very easy and quick to install even when you do your self deployment.
933
01:03:45.680 --> 01:03:47.680
And that is very important
934
01:03:47.920 --> 01:03:49.280
because it gets
935
01:03:49.520 --> 01:03:50.080
around
936
01:03:50.320 --> 01:03:56.160
all the red tape of security if you use a third party cloud that you need to get approved.
937
01:03:56.400 --> 01:03:59.120
You know, if you're in your secure environment,
938
01:03:59.120 --> 01:04:00.800
that's already very, very fast.
939
01:04:01.365 --> 01:04:08.885
And then you can harvest, you can stitch and you can get going very quickly. That is a big change.
940
01:04:09.125 --> 01:04:10.325
Some classic,
941
01:04:10.485 --> 01:04:13.445
you know, metadata management solutions compared to
942
01:04:13.685 --> 01:04:16.645
ours is the ability to quickly deploy,
943
01:04:17.079 --> 01:04:18.520
quickly integrate
944
01:04:18.520 --> 01:04:20.440
with the existing processes
945
01:04:20.520 --> 01:04:22.520
and the existing tooling system.
946
01:04:23.720 --> 01:04:36.305
And as you are working with your customers and some of the early adopters of the Medicarda product? What are some of the most interesting or innovative or unexpected ways that you're seeing it used?
947
01:04:37.345 --> 01:04:39.505
Some aspects we didn't
948
01:04:39.505 --> 01:04:41.105
really anticipate
949
01:04:41.105 --> 01:04:43.025
as a top value.
950
01:04:43.425 --> 01:04:45.745
And one of them is to help
951
01:04:46.100 --> 01:04:47.300
the professional
952
01:04:47.300 --> 01:04:48.100
services
953
01:04:48.100 --> 01:04:49.700
and even their own company
954
01:04:49.940 --> 01:04:51.140
managing change.
955
01:04:51.460 --> 01:04:53.300
And that's why metadata management
956
01:04:53.300 --> 01:05:00.500
is very important when you have true version and configuration management. As I was telling before, it's a constantly evolving system.
957
01:05:01.155 --> 01:05:05.955
So people want to split a table in the data warehouse into multiple tables.
958
01:05:06.115 --> 01:05:10.675
They want to change a type. They want to change this. What's the impact of change?
959
01:05:10.915 --> 01:05:20.230
If you have a true version and configuration management, not only you have versions of the past of a configuration and the system, but you can also do predictability
960
01:05:20.230 --> 01:05:21.190
in the future.
961
01:05:21.670 --> 01:05:25.510
So you can say, hey, how about if we change this? What's the impact?
962
01:05:26.065 --> 01:05:28.625
We can clearly say which jobs,
963
01:05:28.785 --> 01:05:41.620
which ETL and DI jobs have to change and where at which line of code. Point in the design of ETL, we even have the opening tool to show exactly where and same in the BI.
964
01:05:41.860 --> 01:05:46.420
So some of things we didn't see is how much valuable this can be used
965
01:05:46.660 --> 01:05:48.100
in cost productivity
966
01:05:48.100 --> 01:05:56.155
and impact of change. How long is it going to take and what's going happen to make it work? You know, that's that's one illustration
967
01:05:56.235 --> 01:06:03.275
that is very important and only true version and configuration management system based metadata management
968
01:06:03.275 --> 01:06:07.035
can solve this kind of thing. Another one is on reporting
969
01:06:07.035 --> 01:06:07.995
and compliance
970
01:06:08.530 --> 01:06:12.450
based as a metadata management repository.
971
01:06:12.450 --> 01:06:14.690
What surprised us is how many people
972
01:06:14.930 --> 01:06:16.930
go straight to the metadata
973
01:06:16.930 --> 01:06:20.050
to the lineage to produce their report, the completeness.
974
01:06:20.465 --> 01:06:21.425
For example,
975
01:06:21.585 --> 01:06:28.465
again, having glossaries of different standards, you know, you have in health care, in finance, many different standards
976
01:06:28.705 --> 01:06:34.880
and they can map and automatically generate the compliance report for health care, for FCC
977
01:06:34.880 --> 01:06:35.760
directly
978
01:06:35.760 --> 01:06:38.240
from the metadata management repository.
979
01:06:38.319 --> 01:06:39.840
These are examples
980
01:06:40.079 --> 01:06:42.480
of usage that comes
981
01:06:42.480 --> 01:06:43.200
into that.
982
01:06:44.975 --> 01:06:46.175
As
983
01:06:46.175 --> 01:06:48.495
you've both been going on this
984
01:06:48.655 --> 01:06:59.055
long and I'm sure very interesting and convoluted journey of tackling metadata wherever it may live and bringing this Medicarda product direct to consumers,
985
01:06:59.430 --> 01:07:01.270
What are some of the most interesting
986
01:07:01.270 --> 01:07:04.870
or unexpected or challenging lessons that you've learned along the way?
987
01:07:05.910 --> 01:07:13.190
Well, how more and more strategic these things have become. Thirty years ago, you had to explain metadata and even metadata management.
988
01:07:13.645 --> 01:07:17.005
And now it's at the heart of any platform
989
01:07:17.005 --> 01:07:20.765
out there. And AI has really materialized
990
01:07:20.765 --> 01:07:23.805
the critical need for metadata management.
991
01:07:23.805 --> 01:07:26.605
And again, it gets back to shared semantic
992
01:07:26.605 --> 01:07:28.125
between all of these systems.
993
01:07:28.680 --> 01:07:31.720
And suddenly nobody saw really AI
994
01:07:31.800 --> 01:07:33.320
taking that importance.
995
01:07:33.480 --> 01:07:39.080
But that became very, very key and a new return on investment
996
01:07:39.240 --> 01:07:44.765
for metadata management because it's been cycles, you know, where people could not find the resources
997
01:07:44.845 --> 01:07:46.525
for data cataloging
998
01:07:46.525 --> 01:07:47.965
and metadata management.
999
01:07:48.125 --> 01:07:54.285
Now you cannot afford to go in in the AI and LLM without proper context and metadata management.
1000
01:07:54.710 --> 01:07:58.470
So it's not really a surprise or unexpected,
1001
01:07:59.029 --> 01:08:03.430
but it's the advance in technology that made it useful, even more.
1002
01:08:04.470 --> 01:08:12.325
And for people who are trying to come to grips with their overall data estate and looking at various metadata solutions,
1003
01:08:12.325 --> 01:08:14.965
what are the cases where Medicarda is the wrong choice?
1004
01:08:16.405 --> 01:08:18.165
Kind of from our experience,
1005
01:08:18.165 --> 01:08:23.910
if you're taking kind of a, I don't know, blood oath to a single vendor, if you run kind of exclusively
1006
01:08:23.910 --> 01:08:26.790
off on one monolithic BI stack,
1007
01:08:26.950 --> 01:08:30.630
is there an intention to change? Yes. Probably
1008
01:08:30.950 --> 01:08:34.230
you can survive with the vendor's ecosystem.
1009
01:08:34.230 --> 01:08:44.735
Just use their native model and accept the kind of wallet garden that you're living in. You know, some people kind of very secretly love managing heavy runtime middleware.
1010
01:08:44.735 --> 01:08:48.495
If you want another kind of live query proxy,
1011
01:08:48.495 --> 01:08:54.920
definitely don't need us. MetaCarta kind of will not kind of wake you up at 02:00
1012
01:08:55.160 --> 01:09:02.440
in the morning. Obviously, if you're really small, kind of you, I know, your data stack is three program,
1013
01:09:02.440 --> 01:09:11.925
you know, Oracle tables in a dream, you know, probably any data catalog could be kind of too much. Yep. I think a lot of people actually
1014
01:09:12.005 --> 01:09:16.965
have kind of this fantasy of thriving on semantic
1015
01:09:17.045 --> 01:09:18.005
anarchy,
1016
01:09:18.005 --> 01:09:21.560
you know, things will work out eventually.
1017
01:09:21.640 --> 01:09:26.440
Yeah, we're going to figure out the gross margin, going to be eventually
1018
01:09:27.240 --> 01:09:30.199
the same kind of into different tools,
1019
01:09:30.200 --> 01:09:37.905
you know, and we can mitigate any problems. Yeah, I don't think we will kind of feed
1020
01:09:37.985 --> 01:09:40.065
kind of this type of company
1021
01:09:40.065 --> 01:09:40.945
mold.
1022
01:09:41.105 --> 01:09:44.385
Christian, anything kind of comes to mind from your side?
1023
01:09:45.185 --> 01:09:50.840
No, that's exactly what you said. I mean, basically single vendor and small businesses,
1024
01:09:51.480 --> 01:09:52.679
not necessarily,
1025
01:09:52.680 --> 01:10:17.130
you know, well suited for this kind of thing. You mentioned a third one, people who really wanna hack the whole thing A to Z, then you go some open source solution or something. What surprised me on the second one on single vendor is that despite that thing, they're single vendor, then they realize there are some other tools out there. You know, I was mentioning a few companies in the insurance business and we work with very big one like GIG and smaller
1026
01:10:17.610 --> 01:10:24.810
one with Canal Insurance. Canal Insurance is a very much a Microsoft job. You know, they're very happy with Power BI and they use SSIS,
1027
01:10:24.810 --> 01:10:25.450
SSRS,
1028
01:10:25.695 --> 01:10:28.735
Power BI, SQL Server, ADLS,
1029
01:10:28.735 --> 01:10:29.374
everything.
1030
01:10:29.614 --> 01:10:35.774
But there are always something outside. So, for example, in the insurance business, there are external application
1031
01:10:35.935 --> 01:10:38.974
that I use like the big, big insurance
1032
01:10:38.974 --> 01:10:39.614
applications.
1033
01:10:40.040 --> 01:10:43.880
And guess what's going on inside? They have BOD,
1034
01:10:44.040 --> 01:10:45.160
that has BODS,
1035
01:10:45.160 --> 01:10:56.264
pardon me, business subject data services from SAP that is doing the data movement in these external applications they use in the industry. And the reporting is done in IBM Cognos.
1036
01:10:56.265 --> 01:11:05.385
So despite the fact they are a smaller, you know, insurance company, despite the fact they are really dedicated to Microsoft and rightfully,
1037
01:11:05.385 --> 01:11:08.480
they needed to have lineage and integration
1038
01:11:08.640 --> 01:11:14.400
with things you would not suspect. And that's how they came to us. But that's that's,
1039
01:11:14.480 --> 01:11:18.400
you know, classic. Unfortunately, if you were controlling 100%
1040
01:11:18.400 --> 01:11:21.680
of your IT stacks, that would be right. But the reality,
1041
01:11:22.215 --> 01:11:31.814
you have to use also external tools out there. You know, that's for sure. Also, if you only interested in the technical aspect or
1042
01:11:31.895 --> 01:11:35.974
the business aspect, you know, part of the key success of our product
1043
01:11:36.469 --> 01:11:37.830
is to integrate
1044
01:11:37.830 --> 01:11:39.829
the business user experience
1045
01:11:39.989 --> 01:11:42.309
with a technical user experience
1046
01:11:42.630 --> 01:11:45.749
because they have really different needs.
1047
01:11:46.070 --> 01:11:47.110
Our approach
1048
01:11:47.350 --> 01:11:48.870
to the business users,
1049
01:11:49.325 --> 01:11:54.685
you don't you do not impose a data catalog or a metadata management tool to a business user.
1050
01:11:55.005 --> 01:12:04.285
They're too busy doing something else. So we took a completely different approach. Whatever BI tool you use, Power BI, whether you use even business object or whether you use,
1051
01:12:04.770 --> 01:12:08.530
you know, Tableau, we have a browser extension in these tools.
1052
01:12:09.010 --> 01:12:12.210
And basically as they are in these BI reports,
1053
01:12:12.450 --> 01:12:16.210
they see on the right from the metadata management the definitions.
1054
01:12:16.685 --> 01:12:22.764
So they are in there say, oh, I didn't know that. Where is it coming from? And they have information
1055
01:12:22.925 --> 01:12:26.204
right in their BI tool, this business user experience.
1056
01:12:26.525 --> 01:12:33.990
And where did they get that from? And then the business user comes straight to the source. They don't understand whether DI or an ETL
1057
01:12:33.990 --> 01:12:36.310
tool or even an enterprise architecture.
1058
01:12:36.470 --> 01:12:38.790
Oh, we do capture this information.
1059
01:12:39.110 --> 01:12:40.950
And then they ask the second question.
1060
01:12:41.270 --> 01:12:43.830
Do we have any of the BI reports?
1061
01:12:44.305 --> 01:12:46.224
Yes, there are five others,
1062
01:12:46.385 --> 01:12:52.385
two in Tableau, three in that, and you can directly open into. That's the kind of business user
1063
01:12:52.785 --> 01:12:53.985
interactions
1064
01:12:54.065 --> 01:12:55.505
that are expected.
1065
01:12:55.745 --> 01:12:59.185
They don't want to hear the word metadata management or data catalogue,
1066
01:12:59.380 --> 01:13:03.700
but then they appreciate what's inside and they come there
1067
01:13:03.940 --> 01:13:06.820
as a vendor neutral portal
1068
01:13:06.900 --> 01:13:17.945
to open the right report at the right place, see where it's coming from. Is there something else, you know, kind of thing. But at the same time, when they don't like the number on that report,
1069
01:13:18.185 --> 01:13:24.345
they want to be able to talk with IT and talking with IT, they can give a link, a URL
1070
01:13:24.345 --> 01:13:26.505
from that same data catalogue
1071
01:13:26.505 --> 01:13:29.409
where the guy in IT can debug
1072
01:13:29.489 --> 01:13:39.809
basically the ETL job or the stored procedures at the right lane line or, you know, the semantic model in Power BI to help. So if you're not interested,
1073
01:13:39.969 --> 01:13:49.955
if you're focusing only on the technical side, only on the business side, we're not necessarily the right choice. But if we want to put both community together, we definitely
1074
01:13:50.115 --> 01:13:54.515
excel at that. And they need to have one repository
1075
01:13:54.515 --> 01:13:58.850
with a different look and feel and different interaction to them.
1076
01:13:59.650 --> 01:14:14.045
And as you continue to build and improve and iterate on the Medicarda product as well as your core business, what are some of the things you have planned for the near to medium term or any particular projects or problem areas that you're excited to explore?
1077
01:14:15.645 --> 01:14:16.685
Well,
1078
01:14:16.685 --> 01:14:19.485
we're just starting on the Symantec hub despite
1079
01:14:19.485 --> 01:14:23.565
the huge thirty years experience of bringing all this together.
1080
01:14:24.190 --> 01:14:29.229
But I think we are planning a lot of evolution on the Semantic Hub Compiler,
1081
01:14:29.630 --> 01:14:32.030
keeping up with the evolution of the technologies.
1082
01:14:32.429 --> 01:14:35.949
It's very important, think, as Simon pointed out a little bit earlier,
1083
01:14:36.110 --> 01:14:38.909
companies like Snowflake and Databricks do understand
1084
01:14:39.335 --> 01:14:46.054
the importance of the semantic layer and they are investing in it. And to the surprise of many,
1085
01:14:46.455 --> 01:14:51.175
you will find out that the Oracle analytic layers are way more capable
1086
01:14:51.335 --> 01:14:57.300
than where, you know, Snowflake or Databricks is. So all these companies are going to evolve
1087
01:14:57.460 --> 01:15:01.539
and we're going to evolve with them in our semantic compiler
1088
01:15:01.620 --> 01:15:06.340
so that you don't have to worry about keeping up with these evolutions
1089
01:15:06.340 --> 01:15:17.315
of all these semantic layers because we generate and we compile into it. We are responsible to keep up for you with the evolution of all these semantic layers,
1090
01:15:17.715 --> 01:15:22.275
both on the input and on the output so that you can focus on something
1091
01:15:22.910 --> 01:15:26.670
in the middle, the semantic app, which is way more powerful
1092
01:15:26.990 --> 01:15:30.909
than, let's say, where the open semantic
1093
01:15:30.990 --> 01:15:55.710
interface standard is right now because we have to capture things from the old systems, what business object or Cognos or many of these BI tools could do, which is not what Snowflake or Databricks could do today. But we got to evolve with them. And by the way, we are supporting the open semantic interface, but it's something that's going to evolve a lot. We had to do a lot more because we do conversions
1094
01:15:55.950 --> 01:15:57.390
between BI tools.
1095
01:15:57.710 --> 01:16:02.910
So what we are going to be working on, it's all about the Semantic Hub compiler
1096
01:16:03.065 --> 01:16:08.104
and all about doing more and more and keeping up as all these platforms
1097
01:16:08.264 --> 01:16:10.104
are going to evolve dramatically
1098
01:16:10.105 --> 01:16:11.224
on top of that.
1099
01:16:11.705 --> 01:16:17.625
I would love to piggyback on kind of Christian statements about our plans
1100
01:16:17.940 --> 01:16:19.300
and reinforce
1101
01:16:19.300 --> 01:16:21.540
them with some examples.
1102
01:16:22.020 --> 01:16:22.740
Definitely
1103
01:16:23.140 --> 01:16:24.180
hyperscalers
1104
01:16:24.180 --> 01:16:26.900
will invent something new tomorrow, okay?
1105
01:16:27.140 --> 01:16:29.460
But you don't want to redo your semantic
1106
01:16:29.940 --> 01:16:32.100
layer, semantic model,
1107
01:16:32.535 --> 01:16:34.135
Okay, because this is a key.
1108
01:16:34.695 --> 01:16:38.454
It means you want to write once and target whatever
1109
01:16:38.455 --> 01:16:39.895
the hyperscalers
1110
01:16:39.895 --> 01:16:41.255
and then tomorrow.
1111
01:16:41.335 --> 01:16:49.690
Okay, this is an engine that we're building. It means it has to be comprehensive. It has to handle anything under the sun and be...
1112
01:16:50.010 --> 01:16:51.770
Could be designed for
1113
01:16:52.010 --> 01:16:54.570
kind of a future targeting. Yes. We
1114
01:16:54.970 --> 01:16:59.130
definitely planning kind of to go from BI dashboards
1115
01:16:59.130 --> 01:17:01.530
to kind of agentic context.
1116
01:17:01.895 --> 01:17:06.054
Yeah. Imagine you've been running kind of legacy micro strategy,
1117
01:17:06.695 --> 01:17:09.574
you know, BO, Cognos, Looker, whatever,
1118
01:17:10.055 --> 01:17:13.815
you know, and it's time kind of to maybe do something new,
1119
01:17:14.380 --> 01:17:24.380
but transition is painful. Imagine you can kind of bring it into our environment easily with a click and then use kind of MCP kind of frontier,
1120
01:17:24.540 --> 01:17:25.580
kind of to
1121
01:17:25.820 --> 01:17:26.380
get
1122
01:17:26.635 --> 01:17:33.195
all the juices left kind of from this system. Okay, you can kind of build a conversational
1123
01:17:33.275 --> 01:17:34.874
AI technology.
1124
01:17:35.355 --> 01:17:37.914
Okay, conversational kind of BI technology.
1125
01:17:37.915 --> 01:17:38.394
You
1126
01:17:38.955 --> 01:17:44.599
can modernize kind of this legacy system and suddenly, you know, you can calculate net
1127
01:17:44.840 --> 01:17:46.199
kind of retention
1128
01:17:46.280 --> 01:17:46.999
and
1129
01:17:47.160 --> 01:17:50.440
it wouldn't kind of guess and create messy sequel.
1130
01:17:50.440 --> 01:17:53.800
It will kind of pull a compiled auditable
1131
01:17:53.960 --> 01:17:55.160
kind of semantic
1132
01:17:55.534 --> 01:17:56.414
contract.
1133
01:17:56.655 --> 01:17:58.735
And again, it's gonna happen
1134
01:17:59.054 --> 01:18:00.255
in milliseconds.
1135
01:18:00.415 --> 01:18:06.094
And I think this is the two major kind of goals for us for our near future.
1136
01:18:06.895 --> 01:18:12.979
Are there any other aspects of the work that you're doing on Medicarda,
1137
01:18:12.980 --> 01:18:21.619
the work that you're doing at Meta Integration Technologies or this overall ecosystem of metadata management that we didn't discuss yet that you'd like to cover before we close out the show?
1138
01:18:23.395 --> 01:18:34.594
I think you raised excellent questions. A lot of topics have been covered. I think we made it pretty clear that the Semantic Hub is our bread and butter. We do believe in
1139
01:18:34.835 --> 01:18:35.795
governed
1140
01:18:36.130 --> 01:18:37.570
semantic models,
1141
01:18:37.650 --> 01:18:39.650
and that's the golden assets.
1142
01:18:39.730 --> 01:18:44.930
And we believe and we lived it for the past thirty years that the technology evolved
1143
01:18:45.250 --> 01:18:45.810
and
1144
01:18:46.050 --> 01:18:48.530
our job is to keep up with it and
1145
01:18:48.835 --> 01:18:53.235
so that people can focus on their assets. Hence, the metadata ops,
1146
01:18:53.474 --> 01:18:54.675
which is basically
1147
01:18:54.915 --> 01:18:57.074
govern and manage your metrics
1148
01:18:57.235 --> 01:18:58.594
and your semantic
1149
01:18:58.755 --> 01:19:00.594
and let the technology evolve.
1150
01:19:00.755 --> 01:19:03.670
But you don't want to have to be recompiling.
1151
01:19:03.670 --> 01:19:04.150
Sorry,
1152
01:19:04.630 --> 01:19:08.310
I didn't mean to say recompiling, but rewriting your code
1153
01:19:08.470 --> 01:19:11.270
and keep up. That is something
1154
01:19:11.670 --> 01:19:16.870
that should not be your job. You should be focusing on the real goal, the real assets,
1155
01:19:17.255 --> 01:19:18.775
which are you metadata,
1156
01:19:19.015 --> 01:19:20.135
you semantic
1157
01:19:20.215 --> 01:19:23.895
and you complex metrics and never underestimate
1158
01:19:23.975 --> 01:19:25.015
the complexity.
1159
01:19:25.015 --> 01:19:28.054
Those who've done very complex BI tools in the past
1160
01:19:28.535 --> 01:19:33.679
understand that well. You know, frankly, for example, the analogy of Oracle I was mentioning before,
1161
01:19:33.920 --> 01:19:35.999
all this started from Oracle
1162
01:19:36.079 --> 01:19:37.359
analytic tools,
1163
01:19:37.519 --> 01:19:45.175
which pushed the analytics into the database. That was like fifteen, twenty years ago, you know, and now, you know, Snowflake Databricks,
1164
01:19:45.175 --> 01:19:47.334
excellent, good evolution on the cloud,
1165
01:19:47.655 --> 01:19:52.454
but it's going to evolve. And we believe that we want to be helping on that.
1166
01:19:55.510 --> 01:20:00.789
I think in another avenue, we considering to pursue and we believe
1167
01:20:00.790 --> 01:20:01.510
that tools
1168
01:20:03.190 --> 01:20:05.909
in our space need to be kind of more
1169
01:20:06.150 --> 01:20:07.190
proactive.
1170
01:20:07.190 --> 01:20:09.750
They need to kind of close the loop with
1171
01:20:10.445 --> 01:20:11.165
what's
1172
01:20:11.325 --> 01:20:13.405
called kind of active compilation.
1173
01:20:13.725 --> 01:20:14.284
You know,
1174
01:20:14.845 --> 01:20:18.445
we kind of create a kind of image enterprise IDE.
1175
01:20:18.445 --> 01:20:19.724
Kind of right now,
1176
01:20:20.205 --> 01:20:22.685
compilers, they take your definitions
1177
01:20:22.685 --> 01:20:23.324
and,
1178
01:20:23.565 --> 01:20:24.925
you know, in output
1179
01:20:25.085 --> 01:20:26.770
kind of native
1180
01:20:26.770 --> 01:20:34.130
SQL kind of warehouse code. Then probably the next step kind of is event driven compilation.
1181
01:20:34.290 --> 01:20:35.090
If an
1182
01:20:35.489 --> 01:20:36.449
engineer,
1183
01:20:36.449 --> 01:20:36.850
you know,
1184
01:20:37.330 --> 01:20:39.534
alters kind of underlying
1185
01:20:39.615 --> 01:20:40.495
physical
1186
01:20:40.655 --> 01:20:44.895
ERP table or kind of some kind of upstream
1187
01:20:44.975 --> 01:20:58.790
DBT model drift happens, you know. Compiler kind of shouldn't wait for an analyst to notice kind of kind of the broken chain on Monday morning. It should not only... It it should kind of flag the logic mismatch,
1188
01:20:58.790 --> 01:21:02.389
try to recover from the system automagically.
1189
01:21:02.870 --> 01:21:04.870
I I think it would be exciting
1190
01:21:05.585 --> 01:21:07.264
to happen and
1191
01:21:07.425 --> 01:21:10.784
I think in general we live in exciting times.
1192
01:21:12.145 --> 01:21:29.810
All right. Well, for anybody who wants to get in touch with the both of you and the work that you're doing. I'll have you add your preferred contact information to the show notes. As the final question, I'd like to get your perspectives on what you see as being the biggest gap in the tooling or technology that's available for data management today.
1193
01:21:31.105 --> 01:21:34.545
The biggest gap in many tools is
1194
01:21:34.864 --> 01:21:36.624
the level of details
1195
01:21:36.784 --> 01:21:41.425
and proof in like in the lineage. You know, a lot of tools
1196
01:21:41.744 --> 01:21:43.585
have high level lineage,
1197
01:21:44.320 --> 01:21:48.560
high level level of functionality for metadata management.
1198
01:21:49.040 --> 01:22:00.415
But the devil is in the details. You know, if you have to go in a high level of depth in this complexity, that's the gap that we see in many, many systems,
1199
01:22:00.895 --> 01:22:03.775
in tools and technology in general.
1200
01:22:04.175 --> 01:22:07.215
Now that's for the metadata management aspect
1201
01:22:07.375 --> 01:22:11.935
of data management. With respect to the gap in pure data management itself,
1202
01:22:12.440 --> 01:22:26.040
I'm not too sure about this because that's our focus is really in the metadata management, to be honest. Data management has other challenges with the pace of changes these days. Maybe Simon, you want to add
1203
01:22:26.165 --> 01:22:28.485
on the data management aspect as well?
1204
01:22:29.364 --> 01:22:30.165
We
1205
01:22:30.165 --> 01:22:36.725
spent decades and billions of dollars kind of perfecting the plumbing to move petabytes
1206
01:22:36.725 --> 01:22:40.645
at lighting speed and only to recognize nobody taught
1207
01:22:40.810 --> 01:22:44.329
the pipes what net revenue means.
1208
01:22:44.810 --> 01:22:47.050
Does it include taxes or not?
1209
01:22:47.290 --> 01:22:53.130
Okay. And the biggest gap is that modern tools are obsessed with syntax,
1210
01:22:53.554 --> 01:22:54.114
and
1211
01:22:54.435 --> 01:22:56.835
they kind of still almost
1212
01:22:56.835 --> 01:22:58.675
clueless about semantics.
1213
01:22:58.835 --> 01:23:03.554
And they will happily kind of tell you where the table lives, who queried it,
1214
01:23:03.955 --> 01:23:06.034
you know, but semantics
1215
01:23:05.700 --> 01:23:09.300
still is a key. I think automating this process,
1216
01:23:09.620 --> 01:23:11.700
it around AI agents,
1217
01:23:12.340 --> 01:23:15.700
kind of the key to high performance kind of sports
1218
01:23:16.180 --> 01:23:20.100
cars that kind of we are building. You know, we're not the engine,
1219
01:23:20.100 --> 01:23:21.435
you know, Medicarta
1220
01:23:21.435 --> 01:23:27.275
and Symantec Hub is gonna full blown sports cars that you can drive
1221
01:23:27.755 --> 01:23:28.875
and have fun.
1222
01:23:29.835 --> 01:23:42.650
All right. Well, thank you both for taking the time today to join me and share the work that you've been doing for a long time now on metadata management and integration and all of the associated
1223
01:23:42.650 --> 01:23:43.530
challenges,
1224
01:23:43.610 --> 01:23:49.595
both technical and organizational that come out of that. So I appreciate all the time and effort that you've put into
1225
01:23:49.755 --> 01:23:52.235
helping all of our ecosystems
1226
01:23:52.235 --> 01:24:02.079
work more effectively with data and now bringing that direct to consumers. So I wish you the best of luck with that latest chapter, and I hope you enjoy the rest of your day. Thank you, Tobias.
1227
01:24:02.400 --> 01:24:04.719
Thank you. It was a pleasure and honor.
1228
01:24:11.840 --> 01:24:16.135
Thank you for listening, and don't forget to check out our other shows. Podcast.net
1229
01:24:16.135 --> 01:24:25.335
covers the Python language, its community, and the innovative ways it is being used. And the AI engineering podcast is your guide to the fast moving world of building AI systems.
1230
01:24:25.895 --> 01:24:35.870
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email hosts@dataengineeringpodcast.com
1231
01:24:35.870 --> 01:24:42.110
with your story. Just to help other people find the show, please leave a review on Apple Podcasts and tell your friends and coworkers.