UTSKRIFT 🔗
WEBVTT
NOTE
Transcription provided by Podhome.fm
Created: 7/6/2024 11:58:46 AM
Duration: 3690.65
Channels: 1
1
00:00:11.334 --> 00:00:15.434
Hello, and welcome to the Data Engineering Podcast, the show about modern data management.
2
00:00:16.440 --> 00:00:18.140
Introducing RudderStack Profiles.
3
00:00:18.680 --> 00:00:28.455
RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable enriched data to every downstream team.
4
00:00:29.074 --> 00:00:35.490
You specify the customer traits, then profiles runs the joins and computations for you to create complete customer profiles.
5
00:00:36.190 --> 00:00:39.410
Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack.
6
00:00:42.365 --> 00:00:50.625
You shouldn't have to throw away the database to build with fast changing data. You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses,
7
00:00:50.990 --> 00:00:57.489
but swap the decades old batch computation model for an efficient incremental engine to get complex queries that are always up to date.
8
00:00:57.870 --> 00:01:04.685
With Materialise, you can. It's the only true SQL streaming database built from the ground up to meet the needs of modern data products.
9
00:01:05.384 --> 00:01:09.310
Whether it's real time dashboarding and analytics, personalization and segmentation,
10
00:01:09.610 --> 00:01:16.910
or automation and alerting, Materialise gives you the ability to work with fresh, correct, and scalable results, all in a familiar SQL interface.
11
00:01:17.295 --> 00:01:18.835
Go to data engineering podcast.com/materialize
12
00:01:20.415 --> 00:01:22.034
today to get 2 weeks free.
13
00:01:22.415 --> 00:01:32.070
Your host is Tobias Macy. And today, I'm interviewing Brian Platts about using JSON LD for building linked data products. So, Brian, can you start by introducing yourself? Yeah. Thanks, Tobias.
14
00:01:32.370 --> 00:01:38.165
Happy to be here. Brian Platts, as you said. I am the CEO and cofounder of Flurry.
15
00:01:38.705 --> 00:01:40.965
I've been building enterprise software
16
00:01:41.345 --> 00:01:47.170
my whole career. Flurry, really, for the past 4 ish years. It's an open source graph, database product.
17
00:01:47.549 --> 00:01:50.530
And, yeah, happy to talk to you about JSON
18
00:01:50.830 --> 00:01:54.655
LD and, which has a strong intersection with what we do. Absolutely.
19
00:01:55.034 --> 00:01:57.615
And do you remember how you first got started working in data?
20
00:01:58.235 --> 00:01:59.594
I have been building,
21
00:02:00.390 --> 00:02:03.049
software companies really my whole career, so,
22
00:02:03.509 --> 00:02:08.650
I'm on, I think, number 7 at this point. And as those companies mature,
23
00:02:09.075 --> 00:02:14.135
you end up, typically acquiring adjacent products and trying to integrate them.
24
00:02:14.435 --> 00:02:27.345
And that's really where, you know, data always ended up becoming the obstacle for me. You know, there's the marketing message of having a product suite and, of course, the idea of integrating apps and thinking about, you know, challenges like UI and
25
00:02:28.365 --> 00:02:30.545
and, experience and and seamlessness
26
00:02:30.925 --> 00:02:32.465
across various applications.
27
00:02:33.700 --> 00:02:40.760
That always seems like it'll be a challenge, but the challenge is always in the data and and mostly because the data doesn't speak the same language.
28
00:02:41.635 --> 00:02:45.095
So that's what's really led me down the path of becoming
29
00:02:45.395 --> 00:02:48.295
very focused on trying to solve that problem.
30
00:02:49.360 --> 00:02:51.940
And in terms of the conversation today,
31
00:02:52.400 --> 00:02:56.565
I mentioned the idea of a linked data product, and I'm wondering if we can just start
32
00:02:57.045 --> 00:03:03.305
by unpacking that term and what it means and some examples of when you might want to build a linked data product.
33
00:03:03.850 --> 00:03:05.290
Yeah. So I think,
34
00:03:05.610 --> 00:03:19.385
everyone's probably heard the buzz around data products by now, so we don't have to, spend too much time on that part of it. I'm excited about data products because I have been seeing and think in inevitable
35
00:03:19.765 --> 00:03:20.905
transition happens
36
00:03:21.300 --> 00:03:23.959
here hopefully in the shorter term than longer term,
37
00:03:24.660 --> 00:03:38.745
where instead of people going out and just buying apps where data is an artifact of an app, we actually flip that completely around where people focus on data first and foremost, and apps can kinda come and go, but you're managing
38
00:03:40.110 --> 00:03:40.930
a a very
39
00:03:41.390 --> 00:03:42.130
high integrity
40
00:03:42.510 --> 00:03:55.334
trusted set of data in the middle that doesn't have to change. And I think, of course, you know, machine learning and AI has been a big driver for that happening, and and the fact that we now have data products, I think, is a great example of that.
41
00:03:55.715 --> 00:04:04.340
The linked part of this, linked data product, linked data really implies, I think, to to most people who are in the space,
42
00:04:04.925 --> 00:04:07.505
kinda 2 things. 1, it implies graph
43
00:04:08.205 --> 00:04:10.785
as opposed to relational or rectangular
44
00:04:11.245 --> 00:04:12.364
kinda shaped data.
45
00:04:12.685 --> 00:04:16.370
We're definitely focusing on graph data here. And then
46
00:04:16.830 --> 00:04:24.050
what it invokes for most people as well is a set of standards put out by the World Wide Web Consortium around linked data
47
00:04:24.365 --> 00:04:30.544
really in an effort to create kind of a similar environment that we have for the web, which uses HTTP
48
00:04:31.085 --> 00:04:31.905
kind of fundamentally,
49
00:04:32.650 --> 00:04:34.590
instead creating a a decentralized
50
Load full transcript
00:04:35.050 --> 00:04:35.949
data network.
51
00:04:36.330 --> 00:04:49.955
And what are the standards that that goes by, and machines would be the, you know, primary consumer of that. There's kind of a third part that the linked data adds in, at least from my perspective and where we have to go with it, and that is
52
00:04:50.335 --> 00:04:54.060
various forms of cryptography that enable the data to be verifiable,
53
00:04:54.440 --> 00:04:54.940
trusted,
54
00:04:55.639 --> 00:05:06.195
very sophisticated permissions embedded within the data. You know, we call this data defending itself. So I think for this to really reach its potential, we have to bring the cryptographic
55
00:05:06.975 --> 00:05:07.955
sort of conversation
56
00:05:08.699 --> 00:05:15.759
into linked data. But, yeah, those are those are kind of what we mean by adding that 1 word in front of data product.
57
00:05:16.724 --> 00:05:17.625
And another
58
00:05:18.085 --> 00:05:21.625
term that folks might have come across or understand
59
00:05:22.245 --> 00:05:38.854
that is at least related to this concept of linked data is the idea of a knowledge graph. And I'm curious if you can explain some of the overlap between those ideas of a knowledge graph and a linked data products and, in particular, explain where they might be divergent from each other.
60
00:05:39.450 --> 00:05:42.830
Yeah. I think of them I I don't know that there's a formal dictionary,
61
00:05:43.370 --> 00:05:46.670
definition of these, but I think of them as roughly synonymous.
62
00:05:47.465 --> 00:05:54.765
So a knowledge graph is a graph database made up of linked data based on these standards.
63
00:05:55.169 --> 00:06:00.229
So this is a knowledge graph from my perspective. I'm I'm sure there's some other people with different definitions.
64
00:06:00.690 --> 00:06:04.389
But, yeah, knowledge graphs really form around the concepts,
65
00:06:04.745 --> 00:06:05.485
the foundational
66
00:06:05.785 --> 00:06:07.324
concept of RDF,
67
00:06:07.705 --> 00:06:19.650
which is a w 3 c standard. And then there's all kinds of things that build up on top of RDF, and I'm sure we'll probably end up talking about that, a little bit more. But when we talk about JSON LD in particular,
68
00:06:20.175 --> 00:06:31.940
I'm excited about it because it does actually use RDF under the hood, but a lot of people don't know that or don't even have to know that. They're just working with JSON. And so I think it ends up making things much more accept
69
00:06:33.060 --> 00:06:34.199
much more approachable
70
00:06:34.555 --> 00:06:42.495
for folks who know JSON, but might not be as familiar with concepts or like RDF or even graph. It's a nice entry point into it.
71
00:06:42.910 --> 00:06:48.130
So that's, that's what excites me in particular about JSON LD as it relates to this.
72
00:06:48.670 --> 00:07:03.669
And now digging into JSON LD for people who haven't come across it before, the LD suffix, it stands for linked data. And I'm wondering if you can talk through some of the applications of JSON LD in this space of RDF,
73
00:07:04.370 --> 00:07:07.669
the semantic web, which is where RDF largely
74
00:07:07.970 --> 00:07:11.990
found its roots, and some of the domains in which JSON LD is typically used.
75
00:07:12.485 --> 00:07:16.185
Yeah. So where we see JSON LD or even RDF
76
00:07:16.805 --> 00:07:18.185
in general is
77
00:07:18.805 --> 00:07:21.449
where there needs to be interoperability
78
00:07:22.470 --> 00:07:25.770
across data. And and this really implies a decentralized
79
00:07:26.150 --> 00:07:27.210
context so
80
00:07:27.634 --> 00:07:38.870
that, you know, people who don't necessarily have to know or have an agreement about a schema or vocabulary upfront are able to actually merge, combine, and link data together, and it's because it it,
81
00:07:39.490 --> 00:07:42.229
handles these sorts of standards. So the interoperability
82
00:07:42.930 --> 00:07:52.635
across domains is the key here. And, of course, I I think as most data scientists know, they spend a huge amount of time on their data just trying to get it into a common format.
83
00:07:52.970 --> 00:08:01.550
So imagine that work's already done. Right? It it's already in a format then machine can automatically figure out what it needs, and, of course, that's gonna speak to the interoperability
84
00:08:01.935 --> 00:08:04.755
of data challenges. Now those exist,
85
00:08:05.135 --> 00:08:06.115
certainly across
86
00:08:06.735 --> 00:08:08.195
boundaries, like organizational,
87
00:08:08.655 --> 00:08:09.155
governmental
88
00:08:09.535 --> 00:08:10.035
boundaries,
89
00:08:10.430 --> 00:08:14.289
but those also exist inside of companies, you know, in a financial services
90
00:08:14.830 --> 00:08:15.330
company,
91
00:08:15.870 --> 00:08:17.009
finance and risk,
92
00:08:17.470 --> 00:08:18.849
probably calls things
93
00:08:19.485 --> 00:08:31.280
which are identical to sales and marketing in the same organization by different vocabulary terms even though it's the same underlying data. So this exists not only across these boundaries, but even inside of organizations.
94
00:08:31.979 --> 00:08:33.200
The JSON serialization,
95
00:08:33.660 --> 00:08:35.040
I think, brings familiarity
96
00:08:35.545 --> 00:08:36.524
for folks. So,
97
00:08:36.985 --> 00:08:43.245
I don't think you could find a developer who hasn't worked with JSON at some point, so people are familiar with that concept.
98
00:08:43.600 --> 00:08:45.220
And in fact, a simple JSON
99
00:08:45.680 --> 00:09:01.890
document can itself just be used as JSON LD. Now it doesn't bring all the special characteristics into it unless you sprinkle it with a few extra things, but you can literally just take a a JSON document, transact it, and use it almost like a document database, but you have full graph indexing
100
00:09:02.270 --> 00:09:04.930
and, query capability. So where
101
00:09:05.470 --> 00:09:13.154
we see it,
the specific domains where probably the most adoption has happened has been the push of Google and Microsoft
102
00:09:13.535 --> 00:09:14.995
to use this across,
103
00:09:16.095 --> 00:09:17.955
embedding this inside of web pages.
104
00:09:18.270 --> 00:09:19.490
And now
105
00:09:19.870 --> 00:09:23.090
almost 50% of the web has JSON LD in it.
106
00:09:23.470 --> 00:09:26.690
So this is, you know, JSON data embedded within,
107
00:09:27.070 --> 00:09:34.235
the actual web page, and it's used, you know, for no surprise by machines to figure out what a web page is talking about, etcetera.
108
00:09:34.589 --> 00:09:39.490
And in fact, my guess is most people in your audience end up using JSON
109
00:09:40.670 --> 00:09:48.995
LD tens, if not hundreds of times a day without ever knowing it. If you've you've posted a tweet or if if they're still called tweets. I don't know what they're called anymore.
110
00:09:49.855 --> 00:10:02.235
But or shared something on LinkedIn and, you know, a nice description and image of that web page you shared comes up. That is because there's JSON LD embedded in the web page that the machine is running, and we have interoperability
111
00:10:02.695 --> 00:10:04.475
there across services without,
112
00:10:04.855 --> 00:10:13.250
you know, the web page provider and LinkedIn or Twitter or anyone else having to collaborate on that ahead of time. We're seeing it more in regulation,
113
00:10:13.790 --> 00:10:18.915
and, we're seeing it more in an area that I would call intelligent agents,
114
00:10:19.295 --> 00:10:26.115
which is really AI being able to go out and find and not only discover data and information, but because of the semantics
115
00:10:26.590 --> 00:10:31.710
and the linked nature of it, be able to automatically figure out what that data means. So it's it's,
116
00:10:32.270 --> 00:10:37.214
significantly more valuable for AI or other agents to be able to understand,
117
00:10:37.754 --> 00:10:38.254
information.
118
00:10:39.115 --> 00:10:46.480
And going back to to your, mention of the usage in Google and a lot of these social media companies of that linked data,
119
00:10:46.940 --> 00:10:56.394
it's, if I'm not mistaken, you're talking about the open graph specification where if you inspect the header of a web page, you'll see an OG colon prefix to identify
120
00:10:57.014 --> 00:11:00.875
what attribute of that open graph you're specifying in the page metadata.
121
00:11:01.574 --> 00:11:04.500
Yeah. Open Graph and increasingly schema.org,
122
00:11:05.040 --> 00:11:08.180
which is actually the, you know, schema that,
123
00:11:08.640 --> 00:11:28.950
schema, if you will, vocabulary, I'd probably refer to it as, that Google and Microsoft put put out. So web pages are described sometimes in both, sometimes in 1 or the other. And 1 of the nice things which we'll probably get to about the space, is that, you know, that it doesn't live in a world where you think that the whole universe will
124
00:11:29.415 --> 00:11:41.040
agree on the same schema or terminology. So there become ways to map schemas together and say this thing over here means the same thing as this thing over here where you don't have agreement on those.
125
00:11:41.340 --> 00:11:49.185
We might get into that a little bit more. But, yeah, definitely Open Graph, schema.org are are heavily used and recognized by the major search engines
126
00:11:49.644 --> 00:11:52.464
and other services that are leveraging web pages.
127
00:11:53.084 --> 00:11:54.225
And so for
128
00:11:54.690 --> 00:11:56.790
application developers and data practitioners
129
00:11:57.170 --> 00:12:05.805
who are thinking along these lines of JSON LD or linked data, in the overall data life cycle, there are usually a few different stages
130
00:12:06.185 --> 00:12:09.325
of the creation and processing of information
131
00:12:09.865 --> 00:12:10.365
where
132
00:12:10.810 --> 00:12:22.435
an application is generally responsible for the either capture or creation of that data. And then there are data pipelines and then data analytics that will extract and make use of that information.
133
00:12:22.975 --> 00:12:40.285
And so I'd like to maybe talk about some of the the ways that JSON LD and these linked data concepts are present in those different stages of the life cycle. So maybe talking first about the application development and some of the concepts of domain modeling within the application, ways that JSON
134
00:12:40.745 --> 00:12:43.245
LD and these schema semantics
135
00:12:43.625 --> 00:12:45.964
come into play in that development process?
136
00:12:46.800 --> 00:12:52.660
Yeah. So there's a quite a bit there in that question, and I think a good place to start is RDF
137
00:12:53.040 --> 00:12:53.540
itself.
138
00:12:53.935 --> 00:13:02.035
And 1 of the nice things so to answer the question, where is this used or how do you think about it? You know, what is RDF? RDFs are are triples
139
00:13:02.540 --> 00:13:03.040
That's,
140
00:13:03.740 --> 00:13:09.839
technically goes by subject, predicate, object, the 3 elements of it. But if you map this to, like, an Excel spreadsheet,
141
00:13:10.305 --> 00:13:21.360
the subject is really like your row number. The predicate is really like your column, and the third part of it is really like the value in that cell. So it's a extraordinarily simple way of representing data.
142
00:13:21.820 --> 00:13:29.745
In fact, any piece of conceivable data can be represented as our EF, which makes it a phenomenal way of just storing any information because
143
00:13:30.125 --> 00:13:33.745
it's it's simple. It can store anything. There's 0 limitations.
144
00:13:34.205 --> 00:13:46.935
So the questions come up on how do you serialize it and what are the best practices for it. And so, I'll mention a couple things there, and we'll kinda build up to the final part of your question. So in JSON,
145
00:13:47.555 --> 00:13:48.615
when you think about
146
00:13:48.995 --> 00:14:11.714
RDF or triples, what you're really doing is, the second part's the predicate, which is really the property, and the value are just your key value pairs in a JSON object. It's as simple as that. You know, it's, first name Brian in the case of me. But how do you uniquely identify a row? And there's a special term in JSON LD where the key is at ID.
147
00:14:12.230 --> 00:14:16.170
So at ID is where you can give something a global identifier.
148
00:14:16.550 --> 00:14:19.370
Now it doesn't have to be global. It can be relative.
149
00:14:19.925 --> 00:14:26.745
But in being global, it means that it's more easy to combine with other data that's using the same global identifiers
150
00:14:27.205 --> 00:14:28.025
for things.
151
00:14:28.970 --> 00:14:42.375
Again, you can sort of map things together after the fact. So you could say this global ID that someone was using over here is the same as this other global ID that I was using over here, and the systems that manage,
152
00:14:42.915 --> 00:14:52.280
knowledge graph data will be able to automatically infer that. So when you query for either 1, the same result will come out. So kind of this magic happens in in the background.
153
00:14:52.740 --> 00:14:54.200
So when you create,
154
00:14:54.795 --> 00:14:58.495
JSON LD, what you're really trying to do is use for your keys
155
00:14:59.675 --> 00:15:03.135
that you're doing, like first name. Instead of just calling it first name,
156
00:15:03.580 --> 00:15:06.800
ideally, you're using some sort of standard vocabulary.
157
00:15:07.340 --> 00:15:14.145
Again, you don't have to because you can always address it after the fact, but we already talked about schema.org. Schema.org
158
00:15:14.525 --> 00:15:19.505
has a very robust way of representing people and companies and other things.
159
00:15:19.850 --> 00:15:25.390
And in fact, it saves you a lot of time as a developer because there's a lot of nuances when you model out data
160
00:15:25.850 --> 00:16:09.515
that a lot of smart people have already modeled out those concepts for you and thought about the edge cases that normally you think about or run into in version 2 of your app or version 3 of your app. But there's nothing stopping you sort of doing the traditional app. If you're developing a new app, application development, I'm just gonna kinda create my data however I want. That's that's still a fine thing to do. You just wanna be able to leverage these common ontologies. And I think that is probably enough for that question, but we can certainly build on it from there. Absolutely. And now moving into the stage of I have the state that exists in an application or in some source system that I want to consume and perform analysis on for data engineers who are working on that process.
161
00:16:09.950 --> 00:16:15.010
I'm curious if you can talk to some of the ways that having the RDF specification
162
00:16:15.470 --> 00:16:19.410
or the JSON LD representation already present in the application
163
00:16:20.095 --> 00:16:37.990
simplifies their work of doing that extraction, in particular, maintaining the context of the information that they're trying to extract. And for cases where there isn't already that representation and it's just a tabular structure like you will rip out of any database or just a a JSON API without these linkages
164
00:16:38.290 --> 00:16:42.055
embedded in it, some of the ways that the engineers might think about
165
00:16:42.515 --> 00:16:47.415
appending this the the additional contextual data in that JSON LD format to
166
00:16:47.740 --> 00:16:49.280
simplify downstream operations?
167
00:16:49.740 --> 00:16:55.440
Yeah. So 1 of the big benefits that you get is that if I have 2 JSON LD files,
168
00:16:55.765 --> 00:17:01.785
say or databases, however I have that data. If 1 file, you know, is for
169
00:17:02.165 --> 00:17:02.665
customers
170
00:17:04.059 --> 00:17:07.679
and another file is or database is for orders,
171
00:17:08.059 --> 00:17:17.255
those can exist independently, but they can actually be queried as though they're the identical database. So you think about all the effort and work that we put in today to
172
00:17:17.590 --> 00:17:21.690
extract our data out and put it into something like a big data warehouse
173
00:17:22.070 --> 00:17:28.015
really with the aim so that we can run queries across it. We're probably doing additional transformations,
174
00:17:28.475 --> 00:17:33.775
you know, in that process to normalize schemas or other things. There's a lot of work involved.
175
00:17:34.160 --> 00:17:45.575
And if the data natively sits in this format, all that work goes away. In fact, you don't even have to have the data physically co resident with each other to query across it as though it's 1 database.
176
00:17:45.955 --> 00:17:49.895
So there's these great advantages to doing that. 1 thing that,
177
00:17:50.435 --> 00:17:53.779
is also great about graph is it's really a superset
178
00:17:54.159 --> 00:17:57.460
of almost every type of other type of way of,
179
00:17:58.559 --> 00:17:59.539
managing data.
180
00:18:00.305 --> 00:18:06.405
Especially with JSON LD, it it retains a lot of the simplicity of a document database like MongoDB
181
00:18:06.785 --> 00:18:08.910
or something like that. It's incredibly
182
00:18:09.290 --> 00:18:13.710
powerful from an analytics standpoint, which is usually why people go to relational databases
183
00:18:14.010 --> 00:18:30.470
because they they're gonna be writing complex queries and be doing analytics on that. This data not only does all of those types of analytics, there's a whole additional set of analytics that become easier once your data's in a graph. And because it's a superset of really these other capabilities,
184
00:18:30.929 --> 00:18:41.065
it means that you can flatten it as well. So even if I have JSON LD data as this native really rich, that's it's it's the most rich way, I believe, you can
even represent information.
185
00:18:41.525 --> 00:18:42.745
If I want to
186
00:18:43.110 --> 00:18:45.130
flatten that data into rectangles
187
00:18:45.430 --> 00:18:53.635
so I can stick it in a SQL database or put it in a CSV file, I can flatten relational. You can't go the other way. You can't take relational data
188
00:18:54.175 --> 00:19:04.040
and without a a lot of transformation work, try to turn it into a graph, but a graph can be easily flattened. So it creates immense flexibility. It creates a lot of reusability
189
00:19:05.140 --> 00:19:15.225
aspects of the data. And the challenge is is that our data usually doesn't originate like this because almost all apps are using relational databases now or some form of that. So
190
00:19:15.525 --> 00:19:24.340
the shift that I believe we'll we'll see that I think we've started to see and will continue to see is that instead of favoring relational databases
191
00:19:24.720 --> 00:19:42.770
as the system of record, we start using graph as the system of record. Traditionally, graph has been used more for analytics. It's more after the fact. Some system of record is maintaining the data, and then, ultimately, I have some transformations where I'm putting it into a knowledge graph or doing other things so I can do more sophisticated
192
00:19:43.390 --> 00:19:55.560
analytics with it. But, if we can just sort of move that up where the graph is the operational data store, then I think it eliminates not eliminates, that's a strong word, but substantially reduces
193
00:19:56.340 --> 00:19:57.480
all of the
194
00:19:57.780 --> 00:20:04.054
data engineering work that have to go into making it usable for data scientists or machines downstream.
195
00:20:06.995 --> 00:20:14.110
This episode is brought to you by HEX. If you're a data person, you probably have to jump between different tools to run queries, build visualizations,
196
00:20:14.570 --> 00:20:18.110
write Python, and send around a lot of spreadsheets and CSV files.
197
00:20:18.865 --> 00:20:20.245
HEX brings everything together.
198
00:20:20.625 --> 00:20:29.060
Its powerful notebook UI lets you analyze data in SQL, Python, or no code in any combination and work together with live multiplayer and version control.
199
00:20:29.760 --> 00:20:38.805
And now, Hex's magical AI tools can generate queries and code, create visualizations, and even kick start a whole analysis for you, all from natural language prompts.
200
00:20:39.265 --> 00:20:43.445
It's like having an analytics copilot built right into where you're already doing your work.
201
00:20:43.899 --> 00:20:50.595
Then when you're ready to share, you can use HEX's drag and drop app builder to configure beautiful reports or dashboards that anyone can use.
202
00:20:51.155 --> 00:20:58.375
Join the hundreds of data teams like Notion, AllTrails, Loom, Mixpanel, and Algolia using HEX everyday to make their work more impactful.
203
00:20:59.140 --> 00:21:00.520
Sign up today at dataengineeringpodcast.com/hex
204
00:21:02.580 --> 00:21:05.320
to get a 30 day free trial of the HEX team plan.
205
00:21:07.015 --> 00:21:29.295
Digging more into this concept of the graph representation of the data, I'm also interested in your perspective as to how necessary it is for that to be stored in a graph native database versus using this flattened representation that you were mentioning or using something like an adjacency list in a relational store for being able to represent that crash structure?
206
00:21:29.995 --> 00:21:32.530
Yeah. I think that it's isn't necessary.
207
00:21:32.910 --> 00:21:41.410
No. I mean, we've been combining datasets and merging datasets together for a long time. I would make the argument that it's not
208
00:21:41.855 --> 00:21:42.355
scalable.
209
00:21:42.655 --> 00:21:55.860
Right? So usually, it's a specific problem that we have to solve that we go about getting the data. We, you know, create our pipelines. We create transformations. We do whatever we need to do to focus on building out that data. And,
210
00:21:56.240 --> 00:22:00.425
you know, we're done. I mean, we'll we'll accomplish the task. But today,
211
00:22:00.805 --> 00:22:03.625
we have a pretty limited set of data
212
00:22:04.005 --> 00:22:07.400
that we're doing this work with. And part of the reason,
213
00:22:07.860 --> 00:22:11.560
there's a lot of reasons, but the 1 I'll focus on here is that
214
00:22:11.940 --> 00:22:25.450
that process is expensive, and it's not super scalable, which limits how much data you're able to do this work with. If we're all more natively in this format and machines can automatically infer what the data means,
215
00:22:25.750 --> 00:22:26.570
can automatically
216
00:22:26.950 --> 00:22:29.929
follow links and and merge data together,
217
00:22:30.470 --> 00:22:34.455
then it makes all that work we have to do to getting to that end state unnecessary,
218
00:22:34.934 --> 00:22:39.034
which means we have a more scalable process. I believe it's estimated that about 96%
219
00:22:39.414 --> 00:22:42.794
of the data that corporations end up producing from their applications,
220
00:22:43.340 --> 00:23:00.049
etcetera, is dark, you know, meaning that it's it's never used. And, we're you know, in some cases, we're paying to store it, but we'll never use it. This is a way of actually making data more natively, much more usable, much more scalable, and be able to not only leverage it inside inside of our organization,
221
00:23:00.350 --> 00:23:01.889
but think about the challenges
222
00:23:02.509 --> 00:23:11.085
and the work we do just to, you know, get our CRM data with our ERP data combined into a single thing that we're running queries against.
223
00:23:11.465 --> 00:23:17.380
Now think about taking parts of that ERP data and sharing it to other partners and external organizations.
224
00:23:17.920 --> 00:23:22.980
A whole another layer of that work has to start again, whereas this format eliminates
225
00:23:23.280 --> 00:23:24.100
all of those,
226
00:23:24.615 --> 00:23:27.035
components. It just makes sense to do this,
227
00:23:27.415 --> 00:23:32.715
but, of course, it's not what we're used to doing. And I think that's 1 of our our biggest challenges right now.
228
00:23:33.290 --> 00:23:35.230
Another aspect of what you were just describing
229
00:23:35.690 --> 00:23:38.030
with these linked data representations
230
00:23:38.730 --> 00:23:40.750
and being able to easily
231
00:23:41.130 --> 00:23:58.890
translate between the vocabularies used in your example between the finance department and the sales department is another conversation that's been happening in the data space recently around this idea of the metrics layer or the semantic layer for the data that has traditionally been,
232
00:23:59.669 --> 00:24:04.955
encapsulated by the business intelligence layer, but is now being moved into a separate
233
00:24:05.255 --> 00:24:12.300
concern so that you can use it in more than just the business intelligence context. And I'm curious how you see
234
00:24:12.760 --> 00:24:18.300
these concepts of the metric store versus data linkages and shared vocabularies
235
00:24:18.840 --> 00:24:19.820
as either
236
00:24:20.155 --> 00:24:24.095
corollaries to or competitors for the same intent?
237
00:24:24.635 --> 00:24:29.855
Yeah. It's a really good question, and and I actually focus on the word you used, which is layers.
238
00:24:30.299 --> 00:24:41.424
And it's a it's a layer. And so this type of data natively embeds semantics in it, and the semantics are highly reusable and can actually be described by other vocabularies
239
00:24:41.804 --> 00:24:43.804
that allow mappings between,
240
00:24:44.284 --> 00:24:44.784
semantics.
241
00:24:45.725 --> 00:24:47.664
To accomplish this with
242
00:24:48.139 --> 00:24:52.240
traditional data or most data that we have today, we have to add a layer,
243
00:24:52.620 --> 00:24:56.095
just like we have to add a layer like a data warehouse to query across datasets,
244
00:24:56.715 --> 00:25:05.060
just like we have to add a layer for, you know, how do we protect data after we move it into the, data warehouse and assign permission?
245
00:25:05.620 --> 00:25:10.280
Our focus, and this is a little bit unique in Flurry, is we allow that data,
246
00:25:10.820 --> 00:25:12.520
to be what we call programmable,
247
00:25:13.024 --> 00:25:26.150
but we can embed security directly into the the data tier, which allows anything that needs access to that data to be able to query for it, but they're only gonna get a subset of of the data that they natively have in it. So
248
00:25:26.610 --> 00:25:27.670
using a traditional
249
00:25:28.050 --> 00:25:32.630
sort of siloed data relational database that's coming off the back end of an application,
250
00:25:33.215 --> 00:25:47.630
All these things you're talking about, adding semantics, adding all of these things can be done, but we're adding layer and layer and layer and layer to accomplish them. This is a way that we can structure data fundamentally that those layers become unnecessary.
251
00:25:48.410 --> 00:26:00.005
Particularly speaking in the world where most people are right now of they have a data warehouse, they have their data pipelines for extracting from source systems, combining it into
252
00:26:00.370 --> 00:26:01.750
a shared query layer.
253
00:26:02.210 --> 00:26:07.670
There's that word layer again. I'm wondering if you can talk to some of the data modeling exercises
254
00:26:08.290 --> 00:26:09.430
that are necessary
255
00:26:09.925 --> 00:26:22.440
to start embedding the contextual awareness into the data versus it being a an after the fact exercise that has to be done or an exercise that has to be part of that extract process
256
00:26:22.900 --> 00:26:23.640
and instead
257
00:26:24.020 --> 00:26:29.845
making it native to the data itself rather than being something that is imposed upon it ex post facto.
258
00:26:30.225 --> 00:26:31.605
Yeah. And I'll focus on
259
00:26:31.905 --> 00:26:36.804
how you do this in JSON LD just to give a concrete representation because there are alternatives
260
00:26:37.105 --> 00:26:40.330
of doing this for other types of data in different serializations.
261
00:26:41.190 --> 00:26:44.490
But in JSON LD, I already talked about 1 special
262
00:26:44.790 --> 00:26:45.290
keyword,
263
00:26:45.670 --> 00:26:49.014
ad ID, and this can give a global identifier for an item
264
00:26:49.394 --> 00:27:00.190
that can allow things to be mapped together. And, again, you can sort of map different ad IDs that you find out later really talk about the same thing. Another very common 1 you're gonna see is at type,
265
00:27:00.570 --> 00:27:05.615
and this is where you're assigning the data a class. And this type of data natively supports
266
00:27:06.075 --> 00:27:07.054
super and subclasses.
267
00:27:07.755 --> 00:27:14.610
So it's also a way of kind of combining data, but it's also a way of adding powerful capability you know, pharmaceutical products. There's
268
00:27:21.184 --> 00:27:23.284
around, you know, pharmaceutical products.
269
00:27:23.585 --> 00:27:25.284
There's Fibo around financial,
270
00:27:25.585 --> 00:27:28.085
and the list goes on and on. But in schema.org,
271
00:27:28.465 --> 00:27:31.450
you know, you have things like movies, books,
272
00:27:31.910 --> 00:27:49.780
and they're both subclasses of creative works. And in a JSON LD database, even though I said this is a book or this is a movie, I can basically do select star from creative work, and all the books and all the movies will come out because it understands the notion of class hierarchy within your data, so it adds very powerful capabilities.
273
00:27:50.160 --> 00:27:52.180
You can actually do the same thing with properties.
274
00:27:52.560 --> 00:27:57.605
You know, you might have a field called mother and a field called father, and they're both subproperties
275
00:27:57.905 --> 00:27:59.205
of a field called parent.
276
00:27:59.585 --> 00:28:01.525
And just natively, you can say selects,
277
00:28:01.825 --> 00:28:04.245
you know, parents for for Tobias,
278
00:28:04.870 --> 00:28:12.970
and it'll return both mother and father for that, for example. So those are 2 kind of common special keys you'll see in JSON LD.
279
00:28:13.275 --> 00:28:18.495
You do not have to use those. So I wanna be clear with that. Just a plain old JSON
280
00:28:18.875 --> 00:28:19.375
object
281
00:28:19.995 --> 00:28:29.530
will be able to, in most systems, transact, but those add value to it. How do you do the mapping specifically was your question, and I wanted to lead to the 3rd
282
00:28:29.909 --> 00:28:30.809
kinda common,
283
00:28:31.510 --> 00:28:32.010
special
284
00:28:32.405 --> 00:28:36.345
key that you'll see in a JSON LD file, which is at context,
285
00:28:36.645 --> 00:28:44.970
so the at symbol in context. And what at context does is it allows you to map the names of the keys that you're using
286
00:28:45.270 --> 00:28:46.809
to a global
287
00:28:47.190 --> 00:28:47.690
identifier.
288
00:28:48.390 --> 00:28:50.570
And so it doesn't really matter
289
00:28:51.075 --> 00:28:54.774
in your data, in your JSON, what key you use for
290
00:28:55.235 --> 00:28:58.215
first name, given name, f name, g name,
291
00:28:58.515 --> 00:29:06.210
just name. You know, you you can call it whatever you want. You can use this at context to say anywhere you see,
292
00:29:06.750 --> 00:29:13.735
g name for given name, it actually is what I mean by this is I'll use schema.org again. Schema.org/gename.
293
00:29:16.279 --> 00:29:19.340
So you're attaching a global name namespace, in this case, schema.org
294
00:29:19.799 --> 00:29:20.700
to that field.
295
00:29:21.000 --> 00:29:25.820
And now anyone receiving that when they see g name in your file and the machines automatically
296
00:29:26.125 --> 00:29:34.620
infer this, by the way, and kind of expand it out. But even as a human, as soon as you see g name, you know that that is actually referring to this vocabulary
297
00:29:35.080 --> 00:29:49.715
that is published and open that's very well vetted, and you have an exact meaning of what that is. You also can learn other things. Like a given name always exists on a person. So now I automatically know this data is a person. And in a database,
298
00:29:50.130 --> 00:30:07.015
a knowledge graph database, you could say select star from person. All of a sudden, all the people show up. Even though you never said that data was from a person, the fact that we said it has a g name, and g name stands for a given name, and a given name's always on a person, we're able to infer
299
00:30:07.399 --> 00:30:12.700
automatically additional data. So a lot of things we're writing custom Python scripts or other things to infer
300
00:30:13.080 --> 00:30:30.080
can just natively be inferred in this data. But your question was, how do we create these mappings? So in JSON LD, you use the at context field, and there can be some additional layers on top of that, but that's a very simple way of of mapping your raw JSON data into something far more useful in a global vocabulary.
301
00:30:30.460 --> 00:30:31.840
Digging into that hierarchical
302
00:30:32.225 --> 00:30:35.685
aspect of the data and being able to build these inferences
303
00:30:36.545 --> 00:30:38.885
brings to mind the question of performance
304
00:30:39.185 --> 00:30:41.285
and some of the ways to address
305
00:30:41.759 --> 00:30:54.565
the lookup cost of all of these different attributes, particularly given the fact that to your point of I wanna find all of the people, but all I have is the g name to indicate that this is a person, which brings to mind the idea of sparse matrices
306
00:30:54.945 --> 00:30:58.325
and some of the performance challenges around sparsity in datasets
307
00:30:59.410 --> 00:31:12.695
and also to your point of making sure that this g name is a context that I apply to these different records or to the specific field in the world where most people are operating, that is typically done through
308
00:31:13.075 --> 00:31:13.815
the transformation
309
00:31:14.275 --> 00:31:19.110
of the data from the raw layer. So, using ELT approaches as an example,
310
00:31:19.410 --> 00:31:30.785
you load the raw data. Now I'm going to coalesce all of these different column names into a single column name to ensure that I'm calling this 1 field the same thing everywhere in downstream use cases
311
00:31:31.085 --> 00:31:39.910
in this linked data approach. Instead, I don't care what the column name is. I'm going to apply this contextual attribute to this column instead, and then I don't have to do those,
312
00:31:40.310 --> 00:31:41.920
transformations. And so it's a,
313
00:31:42.605 --> 00:31:51.990
an emergent schema rather than an enforced schema. And so I'm I'm interested in talking through some of the ways that these emergent properties of the data will impact
314
00:31:52.870 --> 00:31:59.130
the performance characteristics of the system and some of the architectural approaches to mitigate that concern?
315
00:31:59.785 --> 00:32:00.185
Yeah.
316
00:32:00.585 --> 00:32:05.305
Well, I'm I'm glad you asked specifically about architectural approaches because I think that is,
317
00:32:05.785 --> 00:32:20.525
the primary mechanism at least that we think about to mitigate these. But, yeah, if you are so there's there's a number of very simple ways that kinda layer on these standards to do things like subclassing. That's a pretty simple thing. Doesn't have huge performance implications.
318
00:32:20.985 --> 00:32:22.285
There's additional standards
319
00:32:22.665 --> 00:32:26.170
that allow you to develop these immensely rich ontologies.
320
00:32:26.630 --> 00:32:31.770
And 1 of the most common used commonly used ones for those that get to that far
321
00:32:32.070 --> 00:32:59.169
is OWL. And
and I'll say about these standards, the neat thing is that all of this is just represented as rdf data. So, you know, you think about, something like a standard schema in a relational database, and, of course, you set up your schema upfront, and you have a special command for establishing your schema, think of your schema itself just being described as data. But it's not just schema. It's it's so much more beyond that. So OWL can
322
00:32:59.470 --> 00:33:01.250
actually express concepts
323
00:33:01.710 --> 00:33:05.785
that are NP hard problems, meaning that there's, you know, computationally,
324
00:33:06.485 --> 00:33:19.640
you know, certainly quantum computers might make a big impact in this, that it is gonna be immensely difficult for a computer to figure it out. So 1 of the things they do with OWL is they break it up into several subcategories.
325
00:33:20.180 --> 00:33:28.965
Kind of an easy category that computers can figure out really quick, sort of a more sophisticated 1, 1 where it can be figured out entirely with, for example,
326
00:33:29.345 --> 00:33:30.645
forward chaining rules
327
00:33:31.009 --> 00:33:35.830
and then all the way up into, you know, these concepts that, really aren't computationally
328
00:33:36.610 --> 00:33:40.845
scalable, I guess, is what I would say. So 1 thing is is if you are
329
00:33:41.225 --> 00:33:43.965
getting into this area where you're using these sophisticated
330
00:33:44.345 --> 00:33:45.950
rules applies to this data.
331
00:33:46.269 --> 00:34:10.795
And, by the way, 1 of the reasons to use this over, like, custom Python is that it's completely reusable. Once you develop these rules, not only are they data themselves, you query for the rules. You can sort of update them and do these things, but you can just layer these rules on top of any any data that talks about this. And all this inferencing and magic kind of will just automatically happen for it. In any compliance system, it'll all just work in.
332
00:34:11.255 --> 00:34:21.330
But you have to sort of work within the set if you're gonna try and run these in real time, for example. Or maybe I don't need to run them in real time. You know, there's,
333
00:34:22.050 --> 00:34:31.975
you'll see examples, for example, in in financial services where I don't have a bunch of things in the ontology that says, okay. If a person has at least 2 of these 5
334
00:34:32.435 --> 00:34:33.335
types of accounts
335
00:34:33.795 --> 00:34:37.869
and the combined balance of, you know, all the accounts exceeds $1,000,000
336
00:34:38.490 --> 00:34:42.510
and the account is open and this and that, that's a that's an ontology
337
00:34:43.050 --> 00:34:44.349
rule that can be expressed.
338
00:34:44.664 --> 00:35:11.685
Then they're gonna be a high value customer. And you can select star high value customers, and all of a sudden your high value customers show up. But that's a bunch of computation that would normally happen in a Python script or something
like that. And if you have immense volume of data and this is changing all the time and you're trying to run it in real time, it's a consideration to have. Do I need to run it in real time? Can I run it on the side once a day? Is that good enough? Do I have too much volume that if I run it in real time, it's computant
339
00:35:12.710 --> 00:35:25.985
intensive? And do I scale that back? So you have to decide what you use and don't use, and it depends on your use case. As far as how can we address it with architecture, the thing that we're super excited about is that we have broken up the database
340
00:35:26.285 --> 00:35:27.425
into 2 pieces.
341
00:35:27.890 --> 00:35:35.349
So, traditionally, a database is a server you put up, and it does handles all your updates, and it handles all your queries in 1 server.
342
00:35:35.685 --> 00:35:44.665
Can't think of a good reason, except for just a legacy, why that is. We can actually split these 2 functions completely, which means our query servers become ephemeral.
343
00:35:45.180 --> 00:35:52.560
Our database servers, we can spin them up, spin them down. We can put them in all of our data centers right next to our apps if we want.
344
00:35:52.945 --> 00:36:05.440
They cache segments of the data that are hot in memory that keep updated all the time and allows you to push the compute layer to the edge and allows you to have horizontal scalability of that.
345
00:36:05.740 --> 00:36:08.320
So even if you wanna do some of these computationally
346
00:36:08.700 --> 00:36:12.615
intense algorithms, what you can do is actually push them to your consumers,
347
00:36:12.995 --> 00:36:19.095
push them away to the edge, and you don't have to maintain these big centralized servers that are trying to service
348
00:36:19.510 --> 00:36:41.869
unknown load for a whole bunch of users. So there are architectural ways of of making this better as well. For people who have already invested the work of building up these centralized stores of their data, usually in something like a data warehouse or even a data lake or data lake house, whatever you wanna call it, the thing that you have all of your data stuffed into. They've already done a lot of work of trying to normalize the schemas and bring together
349
00:36:42.330 --> 00:36:44.830
all of these records into a common representation
350
00:36:45.530 --> 00:36:56.020
often through a lot of brute force and hard work. What are some of the lessons that they should be thinking about as far as their overall workflow to be able to bring in some of this contextual
351
00:36:56.400 --> 00:36:56.900
knowledge
352
00:36:57.200 --> 00:37:03.060
of their data, embed that into their workflows, and being able to start layering on these capabilities
353
00:37:03.520 --> 00:37:07.035
of the emergent properties of these kind of schema linkages
354
00:37:07.575 --> 00:37:10.795
without necessarily having to do a complete rewrite or rearchitecture
355
00:37:11.095 --> 00:37:11.915
of their system,
356
00:37:12.270 --> 00:37:15.090
and instead being able to add these,
357
00:37:15.550 --> 00:37:18.770
data link linkages and these ontologies into their workflow
358
00:37:19.345 --> 00:37:25.045
so that maybe, eventually, they move to a new architecture, but they're able to actually start taking advantage of these concepts today.
359
00:37:25.665 --> 00:37:31.400
I'll talk about what I think is the most logical and easiest way to get into this as as an ex,
360
00:37:32.040 --> 00:37:39.875
in the scenario you pointed out. You know, we're seeing more and more that there's actually drivers that force you into this scenario. So 1 is regulation.
361
00:37:40.495 --> 00:37:48.410
You know, FDA is looking to speed up vaccine development, and they're looking to push the pharmaceutical companies to be able to submit data
362
00:37:48.895 --> 00:37:53.075
using RDF, using JSON LD because it's decentralized and interoperable.
363
00:37:53.455 --> 00:38:02.900
It's part of that process. You know, Department of Homeland Security in the US is looking to use verifiable credentials, which is just a cryptographic wrapping around JSON LD data
364
00:38:03.200 --> 00:38:03.860
to facilitate,
365
00:38:04.640 --> 00:38:05.700
customs processing.
366
00:38:06.025 --> 00:38:13.779
So they can show it a chain of all the goods that went into a product and make sure there aren't, you know, tariffs in there. If the
367
00:38:14.079 --> 00:38:24.185
in order to ship data across boundaries, if you have to represent it as linked data, then, of course, you're forced to do this. It it sort of doesn't matter the work you did with other things. The regulators are telling you to do this.
368
00:38:24.485 --> 00:38:30.580
We're seeing people wanting to leverage AI as being another driver. If you really want the AI to be able to understand
369
00:38:30.880 --> 00:38:36.580
what the data means natively without having to tell it everything and wanna scale more of your data for consumption,
370
00:38:37.115 --> 00:38:42.015
that's another big driver. But if you don't have 1 of those drivers where you don't have a choice,
371
00:38:42.555 --> 00:38:49.730
as far as representing your data like that, We've talked about some of the benefits that you would get out of having that, but kind of where do you start?
372
00:38:50.190 --> 00:38:53.895
And the most logical place that people start with this
373
00:38:54.535 --> 00:39:03.195
data and the concept of knowledge graph is they create a knowledge graph which has the class hierarchies and everything that represents their domain, their business,
374
00:39:03.670 --> 00:39:12.329
and they map their existing data that they have into that domain. The idea there is that you can introduce other business users to be able to find data.
375
00:39:12.790 --> 00:39:19.135
It effectively effectively accomplishes kinda schema on read. So if you have different parties that are you talked about semantics.
376
00:39:19.675 -->
00:39:27.349
In mapping that, that's a way to automatically map that. A pattern we're seeing is to build that out, catalog your data, and then have,
377
00:39:27.670 --> 00:39:28.410
new datasets
378
00:39:29.030 --> 00:39:33.075
registered in that and have, machine learning or other services
379
00:39:33.375 --> 00:39:47.940
automatically kicked off if they have permission to see it and enrich this catalog with additional metadata to improve the find ability. So that's a that's a real logical place to start. And once you have this kind of, domain described and this knowledge graph described,
380
00:39:48.355 --> 00:39:54.855
you can start to instead of just pointing to all these different datasets and enabling some of this metadata and facilitation,
381
00:39:55.750 --> 00:40:11.815
you can actually start to represent some of that data natively in the knowledge graph format over time, and it sort of just plugs in to the nodes of your overall graph. So that's where we're seeing most people start. When they don't have 1 of these external pressures, it's forcing them to do something.
382
00:40:12.250 --> 00:40:24.335
And as you were talking about the idea of cataloging, it also brings up the question of data catalogs, data discoverability, which is another piece that is a separate layer going back to our earlier conversation
383
00:40:24.635 --> 00:40:25.135
and
384
00:40:25.435 --> 00:40:27.559
another set of exercises that is necessary,
385
00:40:27.859 --> 00:40:34.039
usually bringing in something like a business glossary or bringing in different people to add documentation
386
00:40:34.660 --> 00:40:37.275
on different aspects of the data through wikis,
387
00:40:38.055 --> 00:40:38.555
labeling,
388
00:40:38.935 --> 00:40:44.475
lineage. I'm curious how this idea of the, linked data attributes
389
00:40:44.860 --> 00:40:50.880
as the embedded context of these records and the schema layering and layering on these ontologies
390
00:40:51.500 --> 00:41:00.775
factors into that question of data cataloging and discovery and maybe some of the ways that it will at least accelerate, if not obviate the need for these explicit
391
00:41:01.075 --> 00:41:02.535
data documentation exercises.
392
00:41:03.339 --> 00:41:07.359
Yeah. I think about well, those who have used data catalogs understand the value,
393
00:41:07.819 --> 00:41:12.880
that they potentially bring, although I know, you know, some implementations are more successful than others.
394
00:41:13.325 --> 00:41:20.225
So what I'm referring to when I talk about this knowledge graph acting as a data catalog is sort of like data catalog 2.0.
395
00:41:20.810 --> 00:41:22.910
So similar ideas, similar benefits,
396
00:41:23.290 --> 00:41:27.070
but how do we bring even more capability into that?
397
00:41:27.370 --> 00:41:35.474
We focus on this idea of first modeling out your business, which is an enhanced or sort of a knowledge graph version of a data catalog,
398
00:41:35.934 --> 00:41:42.349
and then being able to use that structure you set up, not just for humans to find out, you know, what datasets have,
399
00:41:42.829 --> 00:41:44.049
store sales information,
400
00:41:44.485 --> 00:41:48.905
but to actually allow machines to natively do that and bring some of this more sophisticated
401
00:41:49.445 --> 00:41:50.345
query capability
402
00:41:51.365 --> 00:41:52.905
to those particular assets.
403
00:41:53.410 --> 00:41:55.430
Once you have that hierarchy established,
404
00:41:55.890 --> 00:41:57.990
you can also start using it for
405
00:41:58.370 --> 00:41:59.430
automatic classification.
406
00:41:59.890 --> 00:42:02.550
So if you're using, you know, natural language
407
00:42:03.285 --> 00:42:12.289
unstructured data, you're sort of running it through a generic model, but your business isn't generic. Your business looks for certain things in that. This hierarchy
408
00:42:12.589 --> 00:42:25.395
is a native way of actually classifying information and giving a lens to machines, machine learning, or other algorithms to do that. In Flurry, we extend this to be able to handle,
409
00:42:25.935 --> 00:42:41.275
permissions and permissions in a a hierarchical way. Once I have this this kind of set up, I can tag nodes. I can establish certain rules, and it's gonna apply to all data assets that sort of flow down from that. So you just start unlocking a lot of capability
410
00:42:41.734 --> 00:42:42.234
to
411
00:42:42.535 --> 00:42:43.035
taking
412
00:42:43.415 --> 00:42:47.595
this concept of this catalog into more of a concept of an ontology.
413
00:42:47.975 --> 00:42:54.080
Solves the same thing the catalog did, but really starts to bring in all of this additional capability as well.
414
00:42:54.460 --> 00:42:58.575
Another thing that comes to mind as we're talking about layering on these ontologies
415
00:42:59.035 --> 00:42:59.935
and the schema
416
00:43:00.555 --> 00:43:01.375
and the
417
00:43:01.675 --> 00:43:03.855
embedding of this contextual information
418
00:43:04.155 --> 00:43:06.494
into the document that has the attributes
419
00:43:06.809 --> 00:43:07.630
is is the idea
420
00:43:08.490 --> 00:43:25.600
of schema enforcement or attribute enforcement. I'm thinking in terms of things like the schema store in things like like Kafka, where you can say, this is the schema for this topic. So any event that comes in has to conform to this structural specification of the data. I'm curious some of the ways that that,
421
00:43:26.140 --> 00:43:26.640
enforcement
422
00:43:27.020 --> 00:43:32.480
of attributes and data types can come into play with these contextual elements that are being
423
00:43:33.164 --> 00:43:34.224
appended to
424
00:43:34.605 --> 00:43:38.704
the documents themselves and that that idea of type and structural enforcement?
425
00:43:39.244 --> 00:43:44.930
So I think most people who've worked with relational data are used to, you know, traditional
426
00:43:45.230 --> 00:43:46.110
SQL like,
427
00:43:46.750 --> 00:43:47.545
schema enforcement.
428
00:43:48.105 --> 00:43:49.964
And there's a couple nuances.
429
00:43:50.424 --> 00:43:55.484
But as it relates to the closest analogy, the thing that we use in our product and and other
430
00:43:56.270 --> 00:44:06.625
knowledge graph products or graph products use is the standard called shackle, which stands for shapes constraint language, and it allows you to establish very, very sophisticated
431
00:44:07.005 --> 00:44:26.670
rules as far as what data is allowed or not. And you can really use this in 2 ways. 1 is you can use it as a reporting engine. The actual reports with the data quality scoring that come out the other side comes out again just like everything in this space as data. So you can easily ingest that into an app, parse it, you can query for it,
432
00:44:27.069 --> 00:44:37.045
whatnot. But, if you're acting as an operational data store as well, it can do enforcement like traditional schema on a relational database would. It'll reject something if it's not a string.
433
00:44:37.425 --> 00:44:41.365
You know, of course, you can do kind of the next level up where you're doing cardinality
434
00:44:41.665 --> 00:44:44.405
min and max. You're doing regular expressions,
435
00:44:44.950 --> 00:44:50.490
but you can take that to entirely new levels where you can actually peer through the graph and say,
436
00:44:50.870 --> 00:44:55.685
this is a valid value if it's connected to this type of entity,
437
00:44:55.985 --> 00:45:02.965
and that entity has a value greater than x. So you can really start linking it through the graph and have validation
438
00:45:03.270 --> 00:45:06.329
happen in ways that you just cannot even
439
00:45:06.790 --> 00:45:07.290
approach
440
00:45:07.750 --> 00:45:09.690
in traditional schema enforcement
441
00:45:10.390 --> 00:45:11.369
in SQL.
442
00:45:12.455 --> 00:45:20.795
The twist that I mentioned is that the whole purpose of this is to end up having fully composable data. You can just layer datasets together.
443
00:45:21.290 --> 00:45:24.990
It's like it was in a data warehouse. It doesn't even need to be in a data warehouse.
444
00:45:25.369 --> 00:45:28.349
You can issue queries across multiple datasets dynamically.
445
00:45:28.650 --> 00:45:34.165
Well, when you do that and you're combining data together, you can't enforce things like cardinality
446
00:45:34.625 --> 00:45:35.125
constraints.
447
00:45:35.585 --> 00:45:37.780
For example, if I had Tobias
448
00:45:38.080 --> 00:45:47.915
in my database and you had it in your database, and in 1, it said your eyes were brown and the other, it said your eyes were blue, even though that would be eye color would be single cardinality,
449
00:45:48.295 --> 00:45:49.434
I now have 2
450
00:45:49.734 --> 00:45:51.755
different values for the same property.
451
00:45:52.080 --> 00:46:00.900
So in each of those datasets, I can enforce single cardinality, but when I merge them together,
I can't necessarily choose a winner. All of a sudden, everything becomes multi cardinality.
452
00:46:01.225 --> 00:46:08.365
And this has benefits and challenges, but it becomes a slightly different way of thinking about composable data. It's a challenge that you just simply cannot get around.
453
00:46:08.850 --> 00:46:37.684
And this is where you might apply something like a shackle rule after the fact where after I merge these datasets together, I wanna see if it still complies with my schema. The nice thing about shackle, again, it's just data. It's like another dataset. It's incredibly reusable. You don't have to recreate it. You can associate it with things and use it in so many different ways. But that would be an example of how you approach schema and how some of this kind of composability does change things beyond what people are used to thinking.
454
00:46:38.385 --> 00:46:40.325
This episode is brought to you by DataFold,
455
00:46:40.625 --> 00:46:47.170
a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production.
456
00:46:47.790 --> 00:46:57.465
DataFold leverages data diffing to compare production and development environments and column level lineage to show you the exact impact of every code change on data, metrics, and BI tools,
457
00:46:57.849 --> 00:47:00.349
keeping your team productive and stakeholders happy.
458
00:47:01.130 --> 00:47:02.670
DataFold integrates with DBT,
459
00:47:02.970 --> 00:47:08.155
the modern data stack, and seamlessly plugs in your DataCI for team wide and automated testing.
460
00:47:08.695 --> 00:47:14.870
If you are migrating to a modern data stack, DataFold can also help you automate data and code validation to speed up the migration.
461
00:47:15.510 --> 00:47:17.770
Learn more about DataFold by visiting dataengineeringpodcast.com/datafold
462
00:47:20.070 --> 00:47:20.570
today.
463
00:47:25.345 --> 00:47:26.484
And in your experience
464
00:47:26.865 --> 00:47:30.085
of working in this space of linked data
465
00:47:30.640 --> 00:47:32.900
using JSON LD as the representation
466
00:47:33.200 --> 00:47:43.724
for being able to embed all of this ontological and contextual information? What What are the most interesting or innovative or unexpected ways that you've seen those concepts applied to data workflows?
467
00:47:44.505 --> 00:47:48.530
I'm I'm really excited, I think, about new things that this,
468
00:47:48.830 --> 00:47:51.730
opens up that would, I think, be extraordinarily
469
00:47:52.190 --> 00:47:57.010
challenging with today's technology, and we've been doing a lot of work in in verifiable
470
00:47:57.625 --> 00:48:16.665
credentials. And this idea, you know, I've talked about embedding permissions and security and trust into the data layer. You know, right now, this always sits in the app layer. And when we suck our data out, we copy it, and we paste it into the data warehouse, we lose all the data security rules. We have to try and recreate them or limit access or whatever.
471
00:48:17.145 --> 00:48:18.445
So, you know, embedding,
472
00:48:19.385 --> 00:48:21.485
security into the data is,
473
00:48:22.025 --> 00:48:23.805
a very, very powerful concept,
474
00:48:24.105 --> 00:48:27.000
but now we can start even extending that further.
475
00:48:27.300 --> 00:48:30.120
And if we think about this data ultimately forming
476
00:48:30.420 --> 00:48:35.875
the world wide web of data, if you will, where these datasets can interconnect, be composed, etcetera.
477
00:48:36.495 --> 00:48:39.795
1 of the coolest concepts is to be able to
478
00:48:40.710 --> 00:48:41.210
cryptographically
479
00:48:42.150 --> 00:48:44.650
have ownership embedded around the data
480
00:48:44.950 --> 00:48:56.115
and allow people to pluck nodes out of the graph, if you will, and store them in a wallet on their phone to prove things about them, have complete control over their data and their data ownership,
481
00:48:56.720 --> 00:49:04.185
so not have to relinquish that data. We are doing a ton of exciting work around this right now with skills and credentials in higher
482
00:49:04.585 --> 00:49:22.235
education. Flurry's powering a big network of a lot of the big names in higher ed to be able to have reusable credentials when you apply for a job. You can prove you have degrees. There's no way of of faking any of this information. You can prove it was issued by MIT or by Arizona State University,
483
00:49:22.855 --> 00:49:25.435
all independently with 0 third parties.
484
00:49:25.735 --> 00:49:27.355
So this concept of decentralization
485
00:49:27.815 --> 00:49:40.645
and trust with 0 third party involved is super powerful. It creates sovereignty over people owning, their own data. Think about your financial records. Think about, you know, proving that you have a job without
486
00:49:40.945 --> 00:49:57.125
the to the mortgage company without them having to call your employer. Think about, being specific when you go from 1 doctor to another, pulling your medical records into your wallet, and be able to very specifically share with a new provider which parts of that record they get,
487
00:49:57.425 --> 00:50:06.180
they can independently verify them without having to even though you're the 1 that gave it to them. So this idea of giving control over data, and then we start to wrap,
488
00:50:06.580 --> 00:50:21.250
a special field of cryptography around 0 knowledge cryptography around it, and it starts to open up being able to do analytics across data where you never physically share a single piece of data. So there's a lot of exciting types of cryptography
489
00:50:21.550 --> 00:50:23.410
that can wrap around these concepts
490
00:50:23.870 --> 00:50:33.255
that can enable data analytics across, you know, competitors without them ever sharing a single row or record of data. They can still do things, across boundaries.
491
00:50:33.795 --> 00:50:35.654
So a
lot of exciting opportunity
492
00:50:36.310 --> 00:50:38.330
once we solve some of these fundamental
493
00:50:38.710 --> 00:50:43.930
rules and problems and store data in this really, really rich native way.
494
00:50:44.385 --> 00:50:46.965
And in your work of building
495
00:50:47.265 --> 00:50:58.010
Flurry and working in this space of linked data products and exploring the capabilities and the challenges, what are the most interesting or unexpected or challenging lessons that you've learned in the process?
496
00:50:59.029 --> 00:51:00.569
I you know, 1 of the
497
00:51:01.424 --> 00:51:09.020
challenges I think that we have faced the most is that we live in a world where we've grown up thinking about data in a rectangular
498
00:51:09.480 --> 00:51:14.140
way, you know, whether it's using Excel, which is rectangular, relational databases,
499
00:51:14.840 --> 00:51:16.780
more rectangles. Everything's rectangles.
500
00:51:17.305 --> 00:51:17.545
And,
501
00:51:18.105 --> 00:51:28.200
you know, the shift to starting to view data as a graph and kind of through this connected cloud, it's a big shift, and it's it's a hard 1 sort of to to go through.
502
00:51:28.840 --> 00:51:32.220
For some people, you need to pick up the concepts, I think, gradually.
503
00:51:32.760 --> 00:51:37.234
And 1 of the things is while all these ideas sound great and it sounds
504
00:51:37.694 --> 00:51:39.555
sounds like it solves the world's problems,
505
00:51:40.015 --> 00:51:47.290
you know, it's still just not the way the world works, so it's not practical to jump completely in. You have to kinda come in gradually
506
00:51:47.590 --> 00:51:51.505
into it. JSON LD excites me a lot because it brings,
507
00:51:51.964 --> 00:51:53.665
it's relatively new serialization
508
00:51:54.125 --> 00:51:56.625
of these concepts that have been around for a while.
509
00:51:56.930 --> 00:52:01.990
But, of course, as I started out saying, everyone knows JSON. So it just brings instant familiarity
510
00:52:02.370 --> 00:52:06.225
for anyone who's worked with data in a JSON format, which is everyone,
511
00:52:06.765 --> 00:52:07.985
to some of these ideas,
512
00:52:08.365 --> 00:52:16.099
and it provides a foundation that you can just start building a familiar foundation. You can start building some of the additional concepts
513
00:52:16.560 --> 00:52:18.420
on top of very gradually.
514
00:52:18.720 --> 00:52:28.805
And so I think that's kinda what we need. And 1 of the reasons we're so excited about JSON LD in particular is that we think it's gonna open up these concepts to more people in,
515
00:52:29.184 --> 00:52:29.924
you know,
516
00:52:30.785 --> 00:52:32.724
in increase adoption more rapidly.
517
00:52:33.490 --> 00:52:38.790
And for people who are listening to this conversation, they're excited about the ideas
518
00:52:39.090 --> 00:52:40.550
of embedding context,
519
00:52:40.895 --> 00:52:41.395
having
520
00:52:41.855 --> 00:52:52.950
these automatic linkages between different records and ideas. What are the cases where either the linked data approach in general or JSON LD in particular are the wrong choice?
521
00:52:53.730 --> 00:52:59.485
Linked data in general. I mean, if data is never gonna be needed to be reused or interoperable,
522
00:52:59.945 --> 00:53:07.210
part of the whole point of this is to make data more interoperable. So, you know, if you're throwing something something up and you know that data is never gonna be used downstream,
523
00:53:07.590 --> 00:53:14.375
it doesn't necessarily make sense to do this. You might if it's what you're familiar with, but if you're more familiar with MongoDB or Elasticsearch
524
00:53:14.675 --> 00:53:15.415
or MySQL
525
00:53:15.715 --> 00:53:19.655
or whatever it is, just use whatever you're familiar with in those scenarios.
526
00:53:20.180 --> 00:53:27.220
So, that would be a a key area of when not to use it. Yes. About JSON LD in particular, you know, JSON is,
527
00:53:27.540 --> 00:53:28.040
obviously
528
00:53:28.420 --> 00:53:30.424
1 of the least efficient
529
00:53:30.724 --> 00:53:31.224
serialization
530
00:53:31.525 --> 00:53:42.020
formats anyone could ever conceive of. So a lot of people who deal with JSON don't like JSON for that reason, but, of course, the counter side of that is it's ubiquitous.
531
00:53:42.640 --> 00:53:43.619
It's just everywhere.
532
00:53:43.920 --> 00:53:46.420
So if you want compact
533
00:53:47.025 --> 00:53:47.525
serializations,
534
00:53:48.625 --> 00:53:51.045
then, you know, there's there's other ways
535
00:53:51.425 --> 00:53:53.765
of getting that in much more compact format.
536
00:53:54.080 --> 00:53:57.780
Now JSON LD is just 1 of many serialization
537
00:53:58.080 --> 00:54:03.815
types of RDF. So if you're still committed to RDF, you know, there's turtle, which,
538
00:54:04.375 --> 00:54:06.555
those files end in dot t t l.
539
00:54:07.015 --> 00:54:10.315
Turtle stands for I don't know what the full acronym is, but terse
540
00:54:10.700 --> 00:54:22.305
is what the t is, meaning it's a very, very compact way of serializing and representing RDF data. So there's many formats of representing it where, you know, if you're looking to,
541
00:54:23.005 --> 00:54:24.944
have things in a very compact
542
00:54:25.325 --> 00:54:32.560
format, you might wanna look at over using JSON as the main mechanism. JSON's really almost more for humans than it is for machines.
543
00:54:33.260 --> 00:54:33.760
And
544
00:54:34.060 --> 00:54:35.680
as you continue to
545
00:54:36.140 --> 00:54:39.345
invest in this space and explore its capabilities
546
00:54:39.885 --> 00:54:44.865
and challenges, what are some of the future directions that you would like to see for
547
00:54:45.190 --> 00:54:47.130
overall adoption or integration
548
00:54:47.430 --> 00:54:53.405
with these linked data concepts in the data ecosystem and areas where there's opportunity
549
00:54:53.785 --> 00:54:57.645
to bring
these ideas more seamlessly to a broader audience.
550
00:54:58.345 --> 00:55:01.805
It just opens up a lot of new capability, and so I'm
551
00:55:02.200 --> 00:55:06.540
about some of those new capabilities. We've talked about a few of them here, this idea of,
552
00:55:06.920 --> 00:55:07.420
actually
553
00:55:07.800 --> 00:55:08.700
owning data,
554
00:55:09.000 --> 00:55:13.674
having complete control over who uses it and how they use it, ideas around programmable
555
00:55:13.974 --> 00:55:16.234
data, being able to embed sophisticated
556
00:55:16.535 --> 00:55:20.714
security or logic directly into the data tier, I think is immensely powerful,
557
00:55:21.170 --> 00:55:24.150
and it's something we spend a lot of time on. The cryptography
558
00:55:24.530 --> 00:55:34.234
around this space, some of it is, you know, still relatively early, but 0 knowledge proves this idea of proving things that data says without ever disclosing the data itself
559
00:55:34.615 --> 00:55:38.230
is, I think just gonna open up a lot of really neat possibilities.
560
00:55:38.610 --> 00:55:43.910
So those are all exciting. 1 of the things that we're working on now is creating a human
561
00:55:44.369 --> 00:55:47.615
layer on top of this sort of global linked network
562
00:55:48.035 --> 00:55:48.535
that,
563
00:55:49.115 --> 00:55:52.095
works a lot like GitHub where every change
564
00:55:52.475 --> 00:55:54.175
is recorded and cryptographically
565
00:55:54.555 --> 00:55:58.619
verifiable. You have complete traceability and provenance of every piece of data.
566
00:55:59.080 --> 00:56:07.535
And an important thing that we have focused on in this is this concept of time travel, efficient time travel, being able to issue a query as of any moment in time
567
00:56:07.914 --> 00:56:08.494
and instantaneously
568
00:56:08.875 --> 00:56:19.319
getting a response. The reason beyond just kind of being great for lineage and provenance sort of tracking, the reason we think this is so critical is when you are doing decentralized
569
00:56:19.859 --> 00:56:20.359
computing
570
00:56:20.914 --> 00:56:23.414
across well, you're doing computing across decentralized
571
00:56:23.795 --> 00:56:24.295
data.
572
00:56:24.835 --> 00:56:38.734
You can't have predictable results unless you have a notion of time and clock in time. Because if all these data sets are constantly updating, I can never reproduce the same results for the same thing. And being able to say, okay. I'm gonna run this computation.
573
00:56:39.115 --> 00:56:42.015
It's gonna involve these, you know, 50 or a 100
574
00:56:42.395 --> 00:56:43.535
decentralized datasets,
575
00:56:44.190 --> 00:56:46.850
but I want this, done as of
576
00:56:47.150 --> 00:56:48.770
this exact moment in time.
577
00:56:49.150 --> 00:56:51.810
Every 1 of those datasets should know how to reproduce
578
00:56:52.155 --> 00:56:54.815
every state of data at every moment in time instantly.
579
00:56:55.115 --> 00:57:02.050
It gives you completely reliable results. You can reproduce any query with the identical results at any future date, etcetera.
580
00:57:02.430 --> 00:57:04.990
So locking in time, this notion of, you know,
581
00:57:05.710 --> 00:57:19.160
a fact really isn't a fact unless you embed time. So time needs to become a core component of all data management to do it. It's something we're really focused on, and and I hope that capability continues to spread.
582
00:57:19.700 --> 00:57:30.234
And with that question of time and evolution and mutation of data in in an immutable way also brings up the question of for you know, what is the horizon
583
00:57:30.535 --> 00:57:34.955
back to which you are going to support? Like, what at what point do you need to start dropping history
584
00:57:35.560 --> 00:57:40.540
for storage or performance reasons? And curious what your, thoughts are around ways to
585
00:57:40.840 --> 00:57:44.255
either expand that horizon or address that need without
586
00:57:44.795 --> 00:57:45.295
necessarily
587
00:57:45.755 --> 00:57:48.175
destroying the ability to do that time travel
588
00:57:48.475 --> 00:57:55.730
up to a certain threshold. Well, you wanna be able to efficiently represent deltas. And as long as you can do that in a very efficient way,
589
00:57:56.030 --> 00:57:57.710
I see no reason to not,
590
00:57:58.030 --> 00:58:02.130
retain complete history. But there's 2 parts of, you know, managing data.
591
00:58:02.605 --> 00:58:05.425
1 is managing sort of the log of changes,
592
00:58:05.885 --> 00:58:19.590
and, we create, you know, cryptographic linkages between every 1 of those changes so you can prove out the entire dataset. You can prove it's never been tampered with. But the other part that's used even more frequently are indexes, and indexes are just optimizations
593
00:58:19.890 --> 00:58:27.944
of that original log. So that's where I see you having the option, and this is what we do, of saying, I don't wanna retain history in my indexes
594
00:58:28.244 --> 00:58:32.900
because indexes are just creating copies and copies optimized in different ways.
595
00:58:33.280 --> 00:58:36.339
And if the queries don't need to leverage that history,
596
00:58:36.640 --> 00:58:42.345
then there's no need to do it. And that's where most of the space in the database comes from. It comes from all the indexes.
597
00:58:42.725 --> 00:58:43.125
So,
598
00:58:43.525 --> 00:58:47.225
I think you can do it forever. I mean, you can get a 2 terabyte
599
00:58:47.850 --> 00:58:49.550
hard drive now for, like, $20.
600
00:58:50.090 --> 00:58:54.670
It's just an immense amount of data. Or not $20, a $100, $120
601
00:58:55.050 --> 00:59:17.845
is what I meant to say. It's just an immense amount of data that you can actually store for virtually nothing. There's no reason to not have that provable history there. And if you actually look at most database patterns, updates don't happen a lot. You know, we create a lot of new data,
new records, new rows, if you will, but, actually, changes to existing rows are not as frequently. Now there's certain workloads where that's not the case, but
602
00:59:18.270 --> 00:59:19.089
probably 98%
603
00:59:19.550 --> 00:59:28.105
of all data has more addition of data than they do churn of existing data. So it it ends up not being a lot. It's super cheap to store.
604
00:59:28.485 --> 00:59:29.625
You can have completely
605
00:59:30.085 --> 00:59:30.585
provable
606
00:59:30.885 --> 00:59:32.745
data if you do have that storage.
607
00:59:33.125 --> 00:59:41.820
Get rid of it in the indexes if you're concerned about this space. Are there any other aspects of this space of linked data products, JSON LD,
608
00:59:42.440 --> 00:59:44.060
the semantic and ontological
609
00:59:44.600 --> 00:59:45.100
aspects
610
00:59:45.915 --> 00:59:51.775
of data attributes and data structures that we didn't discuss yet that you'd like to cover before we close out the show?
611
00:59:52.475 --> 01:00:24.455
Well, I feel like we could, you know, at least I could talk about this for a long time. I we've hit so many things. I'm pretty satisfied with the ground we've covered. Yeah. Definitely lots of opportunities for more deep on a number of different areas. But, yeah, for for today, we'll we'll leave it as is. And so for anybody who wants to get in touch with you and follow along with the work that you and your team are doing, I'll have you add your preferred contact information to the show notes. And as the final question, I'd like to get your perspective on what you see as being the biggest gap in the tooling or technology that's available for data management today? Yeah. I think it's helping people
612
01:00:24.755 --> 01:00:32.380
identify if they're starting from scratch an ontology or a vocabulary that might already be exist, you know, for their specific needs.
613
01:00:32.920 --> 01:00:41.435
Takes a lot of time and usually a lot of iterations to develop a really good vocabulary and to be able to leverage other people who have thought through all those edge cases
614
01:00:41.975 --> 01:00:53.869
is great, and I don't think we have a lot of good tools to be able to find that. And as I said, you can just create your own and kinda do some mappings later. But, ideally, if you can find something that fits your specific use case kinda upfront,
615
01:00:54.570 --> 01:01:04.100
that would be ideal. And and that's something that I hope we can do more of and I hope we see more of out in the industry to help people get going in this. Absolutely. Absolutely. Well, thank you very much for
616
01:01:04.520 --> 01:01:05.020
taking
617
01:01:05.440 --> 01:01:10.100
the time today to join me and share your thoughts and experiences around these concepts of linked data, building products around them, and some of the ways to represent
618
01:01:10.845 --> 01:01:15.425
building products around them, and some of the ways to represent that information
619
01:01:15.725 --> 01:01:16.925
in a way that is,
620
01:01:17.405 --> 01:01:30.650
usable and understandable to all of the different components that need to interact with it. So definitely a very interesting and rich space to explore. So I appreciate you taking the time today, and, hope you enjoy the rest of your day. Thanks, Tobias. It was a pleasure being here.