How Generative AI Is Impacting Data Engineering Teams

WEBVTT
NOTE
Transcription provided by Podhome.fm
Created: 7/21/2024 7:06:58 PM
Duration: 3284.771
Channels: 1
1
00:00:11.264 --> 00:00:15.525
Hello, and welcome to the Data Engineering podcast, the show about modern data management.
2
00:00:16.145 --> 00:00:18.085
Data lakes are notoriously complex.
3
00:00:18.500 --> 00:00:29.075
For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst is an end to end data lake has platform built on Trino, the query engine Apache Iceberg was designed for.
4
00:00:29.775 --> 00:00:34.995
Starburst has complete support for all table formats, including Apache Iceberg, Hive, and Delta Lake.
5
00:00:35.350 --> 00:00:39.530
And Starburst is trusted by teams of all sizes, including Comcast and DoorDash.
6
00:00:40.070 --> 00:00:43.290
Want to see Starburst in action? Go to data engineering podcast.com/starburst
7
00:00:44.835 --> 00:00:50.934
today and get $500 in credits to try Starburst Galaxy, the easiest and fastest way to get started using Trino.
8
00:00:51.930 --> 00:01:00.750
Your host is Tobias Macey. And today, I'm welcoming back Lior Gavish to talk about the impact of AI on data engineers. So, Lior, can you start by introducing yourself?
9
00:01:01.245 --> 00:01:07.025
Hi, Tobias. Thanks for having me today. I'm Lior. I'm the cofounder of of a company called,
10
00:01:07.645 --> 00:01:08.465
Monte Carlo.
11
00:01:09.085 --> 00:01:09.905
We're the
12
00:01:10.220 --> 00:01:16.800
data observability company, which means we we help data teams, data engineers, data analysts, data scientists
13
00:01:17.435 --> 00:01:18.015
create reliable
14
00:01:18.475 --> 00:01:25.134
and and trusted data for for whatever they're using, whether it's analytics or machine learning or increasingly AI.
15
00:01:25.610 --> 00:01:29.869
We've been around for about 5 years. We're now serving over 400
16
00:01:30.329 --> 00:01:31.630
data teams and
17
00:01:32.009 --> 00:01:34.565
and growing quickly. And, you know, we love
18
00:01:35.265 --> 00:01:39.605
everything data, so it's it's always exciting to to be to be on the show.
19
00:01:40.545 --> 00:01:49.870
As I mentioned, you've been on a couple of times before, but for anybody who hasn't heard your past appearances, which I'll link in the show notes, can you just refresh our memories to how you got started in data?
20
00:01:51.475 --> 00:01:56.295
Oh, absolutely. I got started in data, I I wanna say over 15
21
00:01:56.755 --> 00:01:57.575
years ago
22
00:01:58.270 --> 00:02:03.490
as what you would typically call today, a machine learning engineer. I was actually building
23
00:02:03.950 --> 00:02:05.890
NLP models to help
24
00:02:06.445 --> 00:02:08.625
summarize and and and classify,
25
00:02:09.724 --> 00:02:10.625
news articles.
26
00:02:10.925 --> 00:02:12.545
That's how I started. I then
27
00:02:12.845 --> 00:02:16.000
went on to start my own company in in in cybersecurity
28
00:02:16.460 --> 00:02:17.120
and used
29
00:02:17.420 --> 00:02:22.080
analytics and machine learning to, solve certain kinds of of of fraud
30
00:02:22.540 --> 00:02:23.280
use cases.
31
00:02:24.035 --> 00:02:28.775
Company got acquired by a company by by a a bigger, larger than public
32
00:02:29.075 --> 00:02:29.575
cybersecurity
33
00:02:29.875 --> 00:02:31.635
firm, and I went on to to lead the,
34
00:02:32.630 --> 00:02:35.690
the data and engineering teams. There are about a 100 people,
35
00:02:36.070 --> 00:02:41.834
you know, building various kinds of of real time protection from fraud and and cyber attacks
36
00:02:42.455 --> 00:02:46.635
and heavily relied on on analytics and machine learning. So kind of a,
37
00:02:47.189 --> 00:02:49.769
my background is a combination of of engineering
38
00:02:50.310 --> 00:02:51.290
and data,
39
00:02:51.829 --> 00:02:56.305
and therefore also my my interest in in how do you operationalize
40
00:02:56.605 --> 00:02:59.025
data and and how would you make it actually
41
00:02:59.405 --> 00:03:03.840
as reliable and as trusted as it could be when it serves, know, real time applications
42
00:03:04.220 --> 00:03:06.640
and and sometimes millions of users. So
43
00:03:07.260 --> 00:03:09.760
that's that's my, background and gist.
44
00:03:10.625 --> 00:03:12.084
It's always interesting
45
00:03:12.545 --> 00:03:32.605
in this world that we're in now of large language models and generative AI hearing the terms natural language processing and the fact that you were working so hard to be able to summarize those news articles when now people just say, oh, I'll just throw it at chat GPT, and it does it for me. But there are also, of course, the the issues around quality and accuracy in those summarizations.
46
00:03:32.985 --> 00:03:33.485
So
47
00:03:34.345 --> 00:03:36.985
Yeah. Scratches on my back. What I,
48
00:03:37.465 --> 00:03:45.870
spent, you know, months on end trying to to build with very custom and and and and specialized algorithms you can now do with
49
00:03:46.235 --> 00:03:48.235
an API call that that costs,
50
00:03:48.795 --> 00:03:54.175
you know, fractions of of a cent. So, there you go. There's some progress in the world.
51
00:03:55.540 --> 00:03:56.040
And
52
00:03:56.500 --> 00:03:59.160
nowadays, whenever somebody says the word AI,
53
00:03:59.460 --> 00:04:05.155
the assumption is that we're talking about chat TPT and the like. But for purposes of this conversation,
54
00:04:05.935 --> 00:04:14.250
when we're talking about the impact that the current phase of AI is having on data teams and data engineers in particular. I'm wondering if you can
55
00:04:14.630 --> 00:04:20.490
give some clarifying detail about what you mean when we're saying AI for the purposes of this conversation.
56
00:04:21.775 --> 00:04:23.395
It's a great point. I think,
57
00:04:24.015 --> 00:04:31.940
you you know, AI the the debate of AI and ML and what's AI has been going on for for, for as long as I can remember.
58
00:04:32.560 --> 00:04:35.780
And it's always been there's been a lot of opinions on it.
59
00:04:36.160 --> 00:04:47.110
I I I think for to or, you know, for for the sake of this discussion, we could probably think of AI as, generally speaking, the the generative AI models that emerged in the last,
60
00:04:47.509 --> 00:04:48.229
you know,
61
00:04:48.789 --> 00:04:51.610
almost 2 years now. So think, you know, ChatGPT,
62
00:04:52.310 --> 00:04:54.169
OpenAI, and now more recently,
63
00:04:55.405 --> 00:04:56.065
you know,
64
00:04:56.765 --> 00:04:58.785
various kinds of of lamas and
65
00:04:59.165 --> 00:05:02.465
and and, mistral and mistropic and whatever.
66
00:05:02.765 --> 00:05:03.825
I think those
67
00:05:04.240 --> 00:05:09.380
models introduce kind of a a a foundational shift in how we think about using
68
00:05:10.240 --> 00:05:11.380
AI. And
69
00:05:11.805 --> 00:05:17.665
I'm not sure we're super close, but it's it's probably probably the the most promising avenue to
70
00:05:18.605 --> 00:05:18.925
true,
71
00:05:20.360 --> 00:05:21.820
you know, machine intelligence
72
00:05:23.160 --> 00:05:34.315
potentially getting closer and closer to human level intelligence. And so for, you know, for right now, I'm thinking about AI mostly as those large language models that have been introduced pretty recently.
73
00:05:35.590 --> 00:05:36.570
Machine learning
74
00:05:37.030 --> 00:05:39.130
went through a bit of a resurgence
75
00:05:40.550 --> 00:05:42.330
5 years ago thereabouts
76
00:05:42.785 --> 00:05:45.285
with the idea of ML engineers and
77
00:05:45.745 --> 00:05:49.605
the, the work of actually bringing machine learning into the
78
00:05:50.069 --> 00:05:51.610
real time application
79
00:05:52.150 --> 00:05:58.409
experience for end users and MLOps. So there's been a lot of that conversation happening for a little while now.
80
00:05:59.115 --> 00:06:00.335
With the advent
81
00:06:00.875 --> 00:06:01.455
of generative
82
00:06:02.075 --> 00:06:09.330
AI and the new demands that they're placing, particularly thinking in terms of things like prompt engineering, retrieval augmented generation,
83
00:06:10.990 --> 00:06:16.205
the fact of the models themselves being so much bigger. I'm curious if you can start by
84
00:06:16.665 --> 00:06:23.725
giving a bit of an overview of what you're seeing as the new requirements and the new features that are required in data platforms
85
00:06:24.600 --> 00:06:30.060
for being able to support these new categories of model in an operational environment?
86
00:06:31.240 --> 00:06:31.740
Absolutely.
87
00:06:32.455 --> 00:06:38.715
You're right. About 5 or 10 years ago, there's been a a a an explosion of tools to bring,
88
00:06:39.540 --> 00:06:42.680
you know, machine machine learning into production, whether it's,
89
00:06:43.380 --> 00:06:46.680
feature stores and models serving frameworks and
90
00:06:46.980 --> 00:06:48.180
and and model versioning and,
91
00:06:49.375 --> 00:06:50.835
all kinds of different technologies.
92
00:06:51.535 --> 00:06:57.635
Journey of AI has brought in, you know, a new stack and and, I'm happy to go through the different
93
00:06:58.400 --> 00:07:10.375
components of that stack. I think 1st and foremost, and and and and the thing that that probably many of our listeners have have used is or or the model APIs. Right? And OpenAI did something
94
00:07:11.074 --> 00:07:16.375
very cool, which didn't exactly exist in the ML world. It basically made
95
00:07:17.130 --> 00:07:17.630
models
96
00:07:18.090 --> 00:07:20.270
a commodity by serving them,
97
00:07:20.810 --> 00:07:23.550
as an API. Right? You can make a very simple
98
00:07:24.090 --> 00:07:25.790
HTTP request to OpenAI
99
00:07:26.625 --> 00:07:28.405
and and now to many other providers
100
00:07:28.945 --> 00:07:30.485
and get access to the
101
00:07:30.945 --> 00:07:32.085
latest and greatest,
102
00:07:32.785 --> 00:07:34.165
model that's been
103
00:07:34.940 --> 00:07:37.280
trained by, I mean, 500 PhDs
104
00:07:37.580 --> 00:07:39.120
and and a $1,000,000,000
105
00:07:39.420 --> 00:07:41.120
worth of GPUs. Right?
106
00:07:41.420 --> 00:07:44.865
And and and that's a really core part of of the stat
107
00:07:45.245 --> 00:07:51.104
that that everybody's familiar with. Having said that, over the last 2 years, there's there's been a lot of other components,
108
00:07:51.760 --> 00:07:53.060
that kind of emerged,
109
00:07:53.520 --> 00:07:57.300
to help teams build with with generative AI. 1st and foremost,
110
00:07:57.600 --> 00:08:02.325
Rag, you mentioned it. It's this idea of, well, how do we add
111
00:08:03.105 --> 00:08:04.485
long term memory
112
00:08:04.865 --> 00:08:08.805
and other capabilities to to these large language models?
113
00:08:09.560 --> 00:08:10.060
Retrieval,
114
00:08:10.600 --> 00:08:12.860
augmented generation is is the
115
00:08:13.240 --> 00:08:15.180
the probably the most dominant,
116
00:08:15.640 --> 00:08:16.700
way to do so.
117
00:08:17.085 --> 00:08:20.384
And it's this idea of, you know, let's take a lot of,
118
00:08:21.485 --> 00:08:23.585
data, typically unstructured data,
119
00:08:24.044 --> 00:08:29.840
but but sometimes structured data, put it in a database, and make it available to the model,
120
00:08:31.100 --> 00:08:31.600
while,
121
00:08:32.140 --> 00:08:32.880
it is,
122
00:08:33.635 --> 00:08:36.695
responding to to to user prompts.
123
00:08:37.154 --> 00:08:38.615
Right? So for example,
124
00:08:39.075 --> 00:08:40.935
if I a very simple example,
125
00:08:41.279 --> 00:08:43.940
if I want, to use a model to answer,
126
00:08:44.320 --> 00:08:45.700
a lot of questions about,
127
00:08:46.320 --> 00:08:49.620
my, you know, documentation, my developer documentation,
128
00:08:50.135 --> 00:08:54.795
I can put all of these documents that have existed, that they've been built over the years into,
129
00:08:55.815 --> 00:08:56.475
a database.
130
00:08:57.335 --> 00:08:59.435
Specifically, I might use a vector database.
131
00:08:59.779 --> 00:09:00.520
And then,
132
00:09:01.060 --> 00:09:06.120
when the model gets a new question from a user about how to do this or that,
133
00:09:06.500 --> 00:09:08.520
it might use the database to retrieve
134
00:09:09.355 --> 00:09:12.334
relevant documents to the user's prod
135
00:09:12.635 --> 00:09:17.695
and then create, and then, create an answer to the to that question using those documents.
136
00:09:18.490 --> 00:09:19.390
So, you
137
00:09:19.930 --> 00:09:22.270
know, typically by some form of of summarization,
138
00:09:23.050 --> 00:09:23.790
or extraction.
139
00:09:24.490 --> 00:09:26.830
And so RAG is kind of a new
140
00:09:27.370 --> 00:09:28.595
has prompted,
141
00:09:29.375 --> 00:09:30.115
if you will,
142
00:09:30.575 --> 00:09:36.470
a bunch of different technologies, most prominently the the vector databases that that probably many have heard of,
143
00:09:37.110 --> 00:09:37.850
but also,
144
00:09:38.389 --> 00:09:42.569
you know, frameworks and libraries that help build those RAD applications.
145
00:09:43.589 --> 00:09:45.209
Next in line is probably,
146
00:09:46.325 --> 00:09:48.025
various tools for fine tuning.
147
00:09:48.405 --> 00:09:50.745
It's, fine tuning is this idea of,
148
00:09:51.285 --> 00:09:55.545
you know, let's take a large language model that's been trained over maybe the entire
149
00:09:56.230 --> 00:10:10.075
Internet and all of the books ever written and, and whatever it is. And then maybe let's customize it to, you know, to a more specific need. Maybe I want to have Yeah. Specific knowledge about some topic that's,
150
00:10:10.615 --> 00:10:13.275
that's very relevant to to to my business problem.
151
00:10:13.709 --> 00:10:15.329
You can basically use,
152
00:10:16.189 --> 00:10:21.170
a set of of documents as a way to fine tune the model and and add a certain,
153
00:10:22.015 --> 00:10:23.875
a certain specialization to it.
154
00:10:24.334 --> 00:10:27.074
There's a bunch of tools that that will help you
155
00:10:27.454 --> 00:10:28.754
do that effectively.
156
00:10:30.010 --> 00:10:30.910
Most prominently,
157
00:10:31.370 --> 00:10:32.430
you know, APIs
158
00:10:32.730 --> 00:10:35.550
coming, you you know, coming from the model providers
159
00:10:35.930 --> 00:10:38.430
that allow you to to to add a dataset,
160
00:10:38.735 --> 00:10:40.995
train a model, and then serve that model.
161
00:10:41.615 --> 00:10:42.355
But then
162
00:10:42.735 --> 00:10:43.235
also,
163
00:10:43.855 --> 00:10:44.355
again,
164
00:10:45.535 --> 00:10:48.435
other software tools to to make that easier.
165
00:10:49.029 --> 00:10:50.410
The the 4th,
166
00:10:51.350 --> 00:10:56.010
technology that's that's emerging in terms of of generating AI applications are,
167
00:10:57.005 --> 00:10:58.865
or what I broadly call
168
00:10:59.404 --> 00:11:00.625
orchestration technologies.
169
00:11:01.325 --> 00:11:03.425
So think frameworks like langchain,
170
00:11:04.500 --> 00:11:06.600
but there's increasingly more and more frameworks,
171
00:11:07.220 --> 00:11:08.200
agent frameworks,
172
00:11:08.980 --> 00:11:12.920
prop management tools, all kinds of different tools that help you,
173
00:11:14.004 --> 00:11:15.464
basically create an application
174
00:11:15.925 --> 00:11:24.140
using generative AI. It allows you to, to make a series of, or to orchestrate a series of calls to to models,
175
00:11:24.680 --> 00:11:30.855
you know, using the the outputs of a call to, to trigger another call, using,
176
00:11:31.875 --> 00:11:33.575
RAG in in in the mids,
177
00:11:34.275 --> 00:11:38.935
and combining models and and prompts and and and and
178
00:11:39.680 --> 00:11:46.180
and and information from a database and kinda interesting ways to create to create higher level abstractions, if you will,
179
00:11:46.694 --> 00:11:47.915
of of these models.
180
00:11:48.375 --> 00:11:51.035
So that's that's been, pretty exciting too.
181
00:11:51.415 --> 00:11:52.154
And then,
182
00:11:52.855 --> 00:11:53.915
those 4 probably
183
00:11:54.375 --> 00:11:54.875
allow,
184
00:11:55.975 --> 00:11:58.480
from what I've seen, allow users to create pretty
185
00:11:58.860 --> 00:11:59.360
sophisticated
186
00:11:59.819 --> 00:12:01.759
applications using Journey of AI.
187
00:12:02.220 --> 00:12:04.720
But then as you take these things to production,
188
00:12:06.015 --> 00:12:10.035
and expose them to, you know, to a broader set of users, potentially
189
00:12:10.895 --> 00:12:15.235
external users outside of your own company, there's there's a couple more,
190
00:12:16.240 --> 00:12:17.940
sets of tools that come up.
191
00:12:18.720 --> 00:12:19.700
1st and foremost,
192
00:12:20.560 --> 00:12:21.060
security.
193
00:12:22.320 --> 00:12:23.780
Journey of AI opens
194
00:12:24.855 --> 00:12:25.675
a lot of
195
00:12:26.135 --> 00:12:31.755
possibilities in terms of things that could go wrong in terms of security and privacy. And so we're seeing more and more
196
00:12:32.110 --> 00:12:32.930
tools that,
197
00:12:33.390 --> 00:12:35.730
help you manage that in real time.
198
00:12:36.670 --> 00:12:38.690
Generally speaking, AI firewalls,
199
00:12:39.470 --> 00:12:40.610
that's kind of interesting.
200
00:12:41.464 --> 00:12:42.445
And then, of course,
201
00:12:42.825 --> 00:12:46.685
you know, the topic that that's that's near and dear to my heart,
202
00:12:47.305 --> 00:12:49.805
the the reliability and quality tooling,
203
00:12:50.425 --> 00:12:52.189
whether it's, you know, observability
204
00:12:52.569 --> 00:12:56.829
of the type that that we work on in Monte Carlo. It's, you know, the idea of monitoring
205
00:12:57.370 --> 00:13:00.835
the quality of of of the Gen AI system in production,
206
00:13:01.455 --> 00:13:02.195
but also,
207
00:13:02.735 --> 00:13:05.235
obviously, preproduction tools that help with
208
00:13:05.830 --> 00:13:09.850
evaluating new versions of an application or new versions of the model,
209
00:13:10.790 --> 00:13:20.855
etcetera, etcetera. And so all these kinda different tools are are coming up. Lots of companies that are trying to build those. Lots of teams are trying to build those in house.
210
00:13:21.795 --> 00:13:22.615
Very exciting.
211
00:13:23.170 --> 00:13:24.950
I would generally categorize,
212
00:13:26.050 --> 00:13:27.990
you know, I think it's just helpful to remember
213
00:13:28.770 --> 00:13:32.390
how you might get those tools. I think there's kinda 3 big options
214
00:13:33.735 --> 00:13:37.595
here. You can either buy those tools from cloud providers,
215
00:13:38.295 --> 00:13:40.394
like AWS, GCP, Azure,
216
00:13:41.360 --> 00:13:42.899
and and increasingly OpenAI.
217
00:13:44.320 --> 00:13:48.420
And that that's what most most teams do as far as I can tell.
218
00:13:49.605 --> 00:13:50.105
Then,
219
00:13:50.725 --> 00:13:51.225
increasingly,
220
00:13:51.525 --> 00:13:52.345
there's good,
221
00:13:52.965 --> 00:13:58.770
you know, end to end offerings from from the data clouds, from Snowflake and Databricks, both
222
00:13:59.150 --> 00:14:01.810
offer a pretty robust set of tools across,
223
00:14:02.750 --> 00:14:03.730
all these categories,
224
00:14:04.725 --> 00:14:06.745
to help data teams build with AI.
225
00:14:07.205 --> 00:14:13.529
And then you'll, of course, find specialized solutions for different different parts of the stack. So whether it's, you know, Pinecone,
226
00:14:14.790 --> 00:14:20.410
for vector databases on on the Rag side or or Monte Carlo on the observability side.
227
00:14:21.175 --> 00:14:27.274
But of course, when when you need to upgrade from the from the basic version to the to advanced use cases,
228
00:14:27.790 --> 00:14:30.290
you you can definitely find, you know, highly
229
00:14:31.150 --> 00:14:35.490
specialized and and professional tools for for each part of the stack now.
230
00:14:35.955 --> 00:14:37.375
So, yeah, it's,
231
00:14:37.795 --> 00:14:41.175
the the the there's a lot that's coming into the stack to enable,
232
00:14:41.555 --> 00:14:42.615
generative AI,
233
00:14:43.980 --> 00:14:46.560
and and lots of new technologies, lots of new tools.
234
00:14:47.180 --> 00:14:48.240
Having said that,
235
00:14:49.579 --> 00:14:50.079
the,
236
00:14:51.100 --> 00:14:56.625
I do still think the foundation is is, you know, the classic data pipelines. Right?
237
00:14:58.045 --> 00:15:02.385
The the the one thing that that that I think people are realizing is that
238
00:15:03.070 --> 00:15:05.490
no matter how you go about Genov AI,
239
00:15:06.750 --> 00:15:09.090
the the core piece is,
240
00:15:10.350 --> 00:15:12.450
marrying those models with
241
00:15:13.024 --> 00:15:15.285
with the data that you manage
242
00:15:15.665 --> 00:15:19.524
anyways. Right? If you don't combine your own data with the models,
243
00:15:20.230 --> 00:15:21.610
you're basically building
244
00:15:21.990 --> 00:15:25.290
a a commodity. Right? Something that ChatGPT could do.
245
00:15:25.750 --> 00:15:33.035
The whole point is how how do you create a a unique, specialized, personalized experience for your own users that,
246
00:15:33.575 --> 00:15:35.915
that heavily relies on your own data? And then,
247
00:15:36.640 --> 00:15:38.580
what everybody ends up doing is building
248
00:15:39.280 --> 00:15:40.820
lots of data pipelines,
249
00:15:41.680 --> 00:15:43.940
to feed those models with with their own proprietary data and and and
250
00:15:46.045 --> 00:15:51.185
and make those models useful in the context of their own business and and and, you know, their own users.
251
00:15:52.280 --> 00:15:54.060
There are a lot of different pieces
252
00:15:54.440 --> 00:15:55.900
that you talked through there.
253
00:15:56.520 --> 00:16:05.035
Some of them are new infrastructure components. Some of them are just new practices and how to actually manage the model and the end user application.
254
00:16:05.975 --> 00:16:08.954
I'm curious what you have seen as the
255
00:16:09.700 --> 00:16:14.680
ways that teams are thinking about which of those components are the responsibility
256
00:16:15.060 --> 00:16:21.095
of the data engineers, which are the responsibility of ML engineers, if you have them, which of them belong to
257
00:16:21.475 --> 00:16:30.840
the operations and infrastructure teams, which of them belong to application engineers, and just some of the ways that you see the the breakdown of who owns which piece?
258
00:16:32.740 --> 00:16:33.880
Such a good question.
259
00:16:34.715 --> 00:16:36.335
It's a mele right now.
260
00:16:37.115 --> 00:16:40.815
It's so hard. There there isn't I I wouldn't say there's an established
261
00:16:41.860 --> 00:16:44.680
path to building generative AI in every company.
262
00:16:45.700 --> 00:16:48.120
Does it slightly differently based on,
263
00:16:48.500 --> 00:16:49.880
you know, the the specific,
264
00:16:50.925 --> 00:16:54.545
you know, org structure and talents that that exist in that team.
265
00:16:55.085 --> 00:16:56.785
To be clear, you know, there's
266
00:16:57.725 --> 00:16:58.705
you need all,
267
00:16:59.240 --> 00:17:06.860
Right? There's an element of of software engineering here because you are building an application, typically a a web service,
268
00:17:07.575 --> 00:17:10.715
typically with some form of of user facing application.
269
00:17:11.815 --> 00:17:14.475
There's a good element of of data engineering here,
270
00:17:14.775 --> 00:17:17.809
with data pipelines as as we as we discussed.
271
00:17:18.270 --> 00:17:20.610
There's some element of of machine learning engineering,
272
00:17:21.070 --> 00:17:21.570
of
273
00:17:22.270 --> 00:17:23.475
ML engineering and
274
00:17:24.035 --> 00:17:26.375
and and data science here when it comes to
275
00:17:27.075 --> 00:17:30.775
exploring the models and understanding how to use the data. And and and also,
276
00:17:31.554 --> 00:17:33.400
you know, to be honest, generative
277
00:17:34.100 --> 00:17:38.280
AI, at least right now is most effective when it's combined with more,
278
00:17:39.220 --> 00:17:39.720
traditional
279
00:17:40.020 --> 00:17:42.245
ML and and sometimes deterministic approaches.
280
00:17:43.445 --> 00:17:50.265
The the combination is actually very powerful and data scientists are actually good at at at and making this whole thing,
281
00:17:50.880 --> 00:17:58.340
work nicely together. And, of course, there's, you know, product managers and product designers involved because, again, it's it's it's an application.
282
00:17:59.024 --> 00:18:02.225
It needs to work in in a way that makes sense for for its,
283
00:18:02.625 --> 00:18:03.125
consumers.
284
00:18:03.985 --> 00:18:06.164
And so what I typically see is
285
00:18:06.600 --> 00:18:08.620
all of those teams involved,
286
00:18:09.720 --> 00:18:10.940
in various capacities
287
00:18:11.960 --> 00:18:15.340
and and and everybody focusing on their, you know, on their own pieces.
288
00:18:15.815 --> 00:18:16.475
I think
289
00:18:16.855 --> 00:18:17.355
the
290
00:18:18.934 --> 00:18:20.955
and and and and and all those teams also employ
291
00:18:21.335 --> 00:18:25.195
those different pieces of the stack in in different ways. Just to give you an an example,
292
00:18:25.880 --> 00:18:27.900
a a software engineer might use,
293
00:18:28.600 --> 00:18:32.940
you know, a a a a a model API, right, something like OpenAI,
294
00:18:35.245 --> 00:18:35.745
to,
295
00:18:36.684 --> 00:18:37.345
to generate
296
00:18:37.965 --> 00:18:38.465
a
297
00:18:38.765 --> 00:18:42.865
response sometimes maybe in in real time to to a user prompt.
298
00:18:44.420 --> 00:18:47.720
And a data engineer might use that same API to
299
00:18:48.180 --> 00:18:48.680
process
300
00:18:49.140 --> 00:18:57.485
text documents in bulk as part of a pipeline. Right? And a data scientist might might use it in in, you know, in a in a third way. And so
301
00:18:57.945 --> 00:19:01.164
right right now, it's it's a mix. I don't think there's clear
302
00:19:01.870 --> 00:19:07.570
ownership of who does what, and we're definitely saying, you know, software engineers is building data pipelines
303
00:19:08.030 --> 00:19:09.570
and data engineers building
304
00:19:09.925 --> 00:19:11.225
user facing applications.
305
00:19:12.245 --> 00:19:15.705
I think over time, you know, we'll probably get to to some,
306
00:19:16.965 --> 00:19:20.660
you know, some best practice or some understanding of of how
307
00:19:20.960 --> 00:19:23.700
to, you know, how how to split all these different components,
308
00:19:24.400 --> 00:19:29.575
between the between the different teams. And and what I suspect is, you know, we'll definitely see
309
00:19:30.755 --> 00:19:31.495
much more
310
00:19:32.115 --> 00:19:32.695
or multidisciplinary
311
00:19:33.555 --> 00:19:35.260
team tackling this. Right?
312
00:19:35.720 --> 00:19:36.200
And,
313
00:19:36.600 --> 00:19:42.940
you know, and and and this existed for a while now, teams that are made up of software engineers, data engineers, data scientists
314
00:19:43.400 --> 00:19:44.300
working together,
315
00:19:44.975 --> 00:19:47.635
but it was probably the the exception, not the rule.
316
00:19:48.174 --> 00:19:59.390
I think with generative AI, it's increasingly going to be the the rule, if you will, and and we'll we'll see those teams kind of working together to to build, you know, solutions and and full applications.
317
00:20:00.250 --> 00:20:02.830
Another interesting aspect of
318
00:20:03.355 --> 00:20:07.375
the ways that generative AI is turning everything on its head is that
319
00:20:07.835 --> 00:20:14.950
for a long time, it was the case that data engineering was there to support data science, and then it turned into machine learning.
320
00:20:15.809 --> 00:20:16.309
And
321
00:20:16.770 --> 00:20:22.735
now we're seeing it come full circle where these generative AI technologies are also being used in the
322
00:20:23.195 --> 00:20:24.015
data engineering
323
00:20:24.315 --> 00:20:26.335
workflow of pipeline design,
324
00:20:27.030 --> 00:20:29.210
transformation generation, code generation.
325
00:20:29.910 --> 00:20:42.415
And I'm wondering how you're seeing data engineers start to bring generative AI into that development flow and into the work of building and maintaining the data pipelines that then go on to feed the generative AI?
326
00:20:43.909 --> 00:20:45.929
Yeah. Absolutely. I think the
327
00:20:47.830 --> 00:20:56.335
it's kinda what you mentioned, Tobias. Right? Like, it starts from from what a lot of engineers are are doing right now, which is use, you know, various kinds of copilots,
328
00:20:57.675 --> 00:20:58.975
to accelerate
329
00:21:00.060 --> 00:21:02.560
development. And in in in the case of data engineers,
330
00:21:03.820 --> 00:21:04.720
you you can
331
00:21:05.020 --> 00:21:06.515
very effectively use
332
00:21:06.995 --> 00:21:11.575
generative AI to to build, you know, your pipelines, whether using
333
00:21:12.195 --> 00:21:12.695
PySpark,
334
00:21:13.955 --> 00:21:14.775
or SQL,
335
00:21:15.810 --> 00:21:17.270
or or what have you,
336
00:21:18.050 --> 00:21:18.550
generally,
337
00:21:18.850 --> 00:21:22.790
I can probably accelerate some some elements of it.
338
00:21:23.535 --> 00:21:29.155
We're not quite there in terms of, you know, replacing data engineers with AI. We may we would never get there,
339
00:21:29.775 --> 00:21:32.595
but it can certainly make people more more productive.
340
00:21:33.200 --> 00:21:37.220
I think it can even or or the other thing that's happening a little bit,
341
00:21:37.520 --> 00:21:38.580
I think it it
342
00:21:39.360 --> 00:21:40.580
generally, I democratizes
343
00:21:41.825 --> 00:21:44.485
access to to these things. Right? So,
344
00:21:45.105 --> 00:21:46.625
you know, it it the the whole,
345
00:21:47.425 --> 00:21:48.485
text to SQL
346
00:21:49.040 --> 00:21:50.340
thing is working pretty
347
00:21:50.720 --> 00:21:51.860
pretty decently,
348
00:21:52.880 --> 00:21:55.540
which means that maybe certain things that,
349
00:21:55.920 --> 00:21:58.580
used to be delegated to to data engineers,
350
00:21:59.425 --> 00:22:03.045
when there's a need to create an, you know, a new pipeline or an or a new
351
00:22:03.505 --> 00:22:04.005
analysis.
352
00:22:04.785 --> 00:22:16.620
Now maybe someone with less technical skills can do it using generative the ad model. So it's kinda democratizing access to data and to pipelines, and and that actually frees up data engineers to to,
353
00:22:17.165 --> 00:22:22.065
to do the things that that they do best rather than, you know, answering to ad hoc,
354
00:22:22.925 --> 00:22:23.425
requests.
355
00:22:23.970 --> 00:22:24.790
I think the
356
00:22:25.250 --> 00:22:27.110
most exciting thing though for
357
00:22:27.490 --> 00:22:28.550
for data engineers
358
00:22:28.930 --> 00:22:30.070
is actually that,
359
00:22:30.770 --> 00:22:32.390
generative AI unlocks,
360
00:22:33.385 --> 00:22:37.405
access to unstructured data. And what I mean by that is
361
00:22:38.425 --> 00:22:39.945
imagine a lot or,
362
00:22:40.265 --> 00:22:45.820
and and there's plenty of examples, but a lot of enterprises, a lot have a lot of very useful
363
00:22:46.600 --> 00:22:47.100
unstructured
364
00:22:47.400 --> 00:22:47.900
data
365
00:22:48.200 --> 00:22:49.180
documents, basically,
366
00:22:50.120 --> 00:22:51.420
generated in the business,
367
00:22:51.855 --> 00:22:54.195
you know, whether it's legal documents or,
368
00:22:54.575 --> 00:22:56.115
you know, technical documentation
369
00:22:56.815 --> 00:22:57.315
or,
370
00:22:58.174 --> 00:23:01.820
or lots of other corpus of information that that are useful.
371
00:23:02.360 --> 00:23:12.555
If you wanted to to make these things useful for the business in the past, you know, a data engineer can do it. They would need, you know, a data scientist and a machine learning engineer
372
00:23:13.015 --> 00:23:16.155
to come in and and process that data and extract,
373
00:23:16.480 --> 00:23:25.300
generally, extract structured data from it. Right? If you wanted to process all your legal documents and and and extract information from there,
374
00:23:25.965 --> 00:23:31.105
you actually needed to hire, you know, highly specialized data people that would build,
375
00:23:31.405 --> 00:23:33.025
you know, NLP algorithms,
376
00:23:33.645 --> 00:23:35.184
quite like I used to do
377
00:23:35.549 --> 00:23:36.289
15 years
378
00:23:36.669 --> 00:23:42.850
ago in order to do that. And the data engineer would help them kinda string things together and build the pipeline, but but still
379
00:23:43.230 --> 00:23:45.250
a lot of the work would would have to be,
380
00:23:45.925 --> 00:23:47.225
outsourced in a way.
381
00:23:47.525 --> 00:23:51.045
And and now data engineers can actually do that on their own. Right?
382
00:23:51.525 --> 00:23:54.425
Especially with with models being available,
383
00:23:55.640 --> 00:23:58.860
natively in in in tools like Snowflake or Databricks.
384
00:23:59.399 --> 00:24:02.620
A data engineer can take, you know, a body of legal document
385
00:24:03.000 --> 00:24:04.220
documents and extract
386
00:24:04.805 --> 00:24:12.025
information from there without getting any help. It's as easy as as creating a prompt and and applying a
387
00:24:12.325 --> 00:24:12.985
a function
388
00:24:13.450 --> 00:24:15.710
to that dataset. Right? And so
389
00:24:16.250 --> 00:24:20.270
I I think that's a that's a force multiplier. Right? It it opens up
390
00:24:20.650 --> 00:24:22.430
opportunities for data engineers
391
00:24:23.275 --> 00:24:26.255
to use enterprise data much more effectively,
392
00:24:27.275 --> 00:24:29.535
and with much less help from
393
00:24:29.970 --> 00:24:34.870
from from other teams that that they might have depended on in in the past. So that's
394
00:24:35.410 --> 00:24:39.670
that that's a pretty exciting shift and change, you know, in in in my view.
395
00:24:40.495 --> 00:24:42.595
One of the other major ways that
396
00:24:43.455 --> 00:24:48.835
AI and machine learning models have found their way into the data engineering workflow
397
00:24:49.150 --> 00:24:54.450
is in particular in that context of retrieval augmented generation where you need to be able to
398
00:24:54.910 --> 00:25:06.950
generate the vector embeddings of the data that you want to use for that context. And so you have to have some capacity being able to run those embedding models in that pipeline workflow and pipeline environment,
399
00:25:07.490 --> 00:25:11.030
and you also need to be thinking about what are the
400
00:25:11.890 --> 00:25:22.505
considerations around how I want to generate the embeddings, what do I need to to be thinking about as far as chunking of the different sizes of embeddings that we're creating. And I'm wondering if you can talk to
401
00:25:22.860 --> 00:25:24.560
how those new requirements
402
00:25:25.100 --> 00:25:29.840
are being addressed if in data teams and some of the
403
00:25:30.414 --> 00:25:31.075
new skills
404
00:25:31.375 --> 00:25:34.275
and new training that's necessary to be able to
405
00:25:34.735 --> 00:25:37.875
build pipelines to support that RAG use case effectively.
406
00:25:39.260 --> 00:25:40.880
Yeah. Absolutely. I think
407
00:25:41.419 --> 00:25:41.919
the
408
00:25:43.500 --> 00:25:44.000
so
409
00:25:44.539 --> 00:25:45.360
as I said,
410
00:25:45.794 --> 00:25:51.575
incredibly helpful for data teams, right? There's a lot of value in being able to, to process unstructured data.
411
00:25:51.875 --> 00:25:52.775
And, you know,
412
00:25:54.310 --> 00:25:55.050
to be honest,
413
00:25:55.430 --> 00:25:57.530
I would dare say that at this point,
414
00:25:58.230 --> 00:26:02.170
there isn't yet, you know, a best practice. Like, you you can't,
415
00:26:03.095 --> 00:26:09.640
you know, buy a book and understand how to how to build brag. Although I'm sure someone is selling a book.
416
00:26:10.120 --> 00:26:13.260
Nobody really has done that a ton. You can hire,
417
00:26:13.640 --> 00:26:18.414
you know, a rag expert that will tell you how to do it. It's mostly about
418
00:26:18.715 --> 00:26:20.255
getting hands on experience
419
00:26:20.794 --> 00:26:23.934
and being curious and experimenting and finding,
420
00:26:24.690 --> 00:26:29.350
you know, what's what's right for your particular use case and your particular,
421
00:26:30.370 --> 00:26:39.164
need. And so, you know, I'm I I probably couldn't give people advice on on on how to build RAG pipelines at this at this point, and and I've seen
422
00:26:39.530 --> 00:26:40.910
so many different approaches,
423
00:26:41.930 --> 00:26:44.750
oftentimes highly dependent on the background
424
00:26:45.690 --> 00:26:50.735
of the people building the pipelines. Right? Software engineers stack it in a certain way
425
00:26:51.035 --> 00:26:59.060
and then data engineers do it in a completely different way and, you know, and both are very valid right now. I think that the the
426
00:26:59.600 --> 00:27:03.300
having said that, there's there's probably a few things that people need to start
427
00:27:03.920 --> 00:27:04.740
thinking about,
428
00:27:05.295 --> 00:27:07.475
in terms of how to build that effectively.
429
00:27:08.175 --> 00:27:12.995
Right? And, essentially, like, how to go from that prototype phase of, you know,
430
00:27:13.460 --> 00:27:16.520
take a bunch of documents, run them through, lang chain
431
00:27:17.140 --> 00:27:23.615
and and create a and and put them in a vector database. That's, you know, that's something that that we're all learning how to do,
432
00:27:24.075 --> 00:27:25.535
to take it to the next level.
433
00:27:26.075 --> 00:27:28.635
There's kind of a higher level questions that need to be
434
00:27:29.355 --> 00:27:30.335
that that that we're seeing
435
00:27:30.700 --> 00:27:33.040
teams increasingly ask. It's things around
436
00:27:33.420 --> 00:27:34.560
security and privacy,
437
00:27:35.260 --> 00:27:37.360
for example. Like, how do you make sure,
438
00:27:38.460 --> 00:27:39.680
that, whatever
439
00:27:40.745 --> 00:27:41.725
access controls
440
00:27:42.345 --> 00:27:47.085
apply to that data and, you know, whatever sense of information is there
441
00:27:47.669 --> 00:27:49.130
continues to be governed
442
00:27:49.510 --> 00:27:52.010
as it as it goes through data pipelines.
443
00:27:52.549 --> 00:27:55.049
Over the years, we've we've developed various methodologies
444
00:27:55.350 --> 00:27:55.850
around
445
00:27:56.255 --> 00:27:58.995
around that in the in the structured data world, but
446
00:27:59.775 --> 00:28:02.835
applying it with unstructured data is is quite different,
447
00:28:03.830 --> 00:28:09.610
because it's messy, because you take a, you know, a body of documents and you, honestly, you don't even know
448
00:28:09.910 --> 00:28:12.330
what's there and and how it should be protected.
449
00:28:13.015 --> 00:28:19.175
So there's much more potential for leak and for people having access to things they're not supposed to see. So,
450
00:28:20.410 --> 00:28:22.270
that's definitely an area where where
451
00:28:22.809 --> 00:28:23.630
people need
452
00:28:24.090 --> 00:28:25.950
to to to develop, you know,
453
00:28:26.330 --> 00:28:31.125
a a a knowledge and a set of best practices around it again that that apply to their business.
454
00:28:31.425 --> 00:28:32.725
The other one is cost,
455
00:28:33.505 --> 00:28:34.005
efficiency.
456
00:28:34.465 --> 00:28:39.125
You know, again, building a prototype will probably not cost you a lot of money and nobody would care.
457
00:28:39.760 --> 00:28:40.420
But now,
458
00:28:41.120 --> 00:28:42.340
you know, if you process
459
00:28:43.920 --> 00:28:46.740
the the entire corpus of of enterprise data
460
00:28:47.445 --> 00:28:48.745
and and run-in batch
461
00:28:49.125 --> 00:28:52.085
every single day in your pipelines and and, make
462
00:28:52.565 --> 00:28:55.145
I don't know how many API calls per document.
463
00:28:55.940 --> 00:28:59.000
The bill can run pretty high pretty quickly.
464
00:28:59.620 --> 00:29:01.320
And so data teams need to
465
00:29:01.700 --> 00:29:03.240
to figure out how to,
466
00:29:04.284 --> 00:29:10.384
first, how to create visibility and manage that cost, but also how to optimize it because a lot of use cases end up being
467
00:29:10.740 --> 00:29:20.360
exorbitantly expensive. And so you need to think about how to how to minimize the number of calls and then how to potentially use faster, cheaper models where where it applies.
468
00:29:21.044 --> 00:29:23.225
Cost is a thing. And, of course,
469
00:29:23.764 --> 00:29:25.465
reliability and quality. Right?
470
00:29:26.245 --> 00:29:28.664
These these these pipelines tend to,
471
00:29:29.220 --> 00:29:36.920
you know, have have a lot of issues with that. Right? It's first and foremost hallucinations. Right? Everybody that use chat GPT is probably
472
00:29:37.505 --> 00:29:39.925
gone to a restaurant that that never existed.
473
00:29:40.465 --> 00:29:45.765
They asked for for a recommendation for dinner and true story. And
474
00:29:46.130 --> 00:29:50.790
so so you need to figure out how how to deal with those things, how to deal with the fact that it's not deterministic.
475
00:29:51.170 --> 00:29:53.110
Right? Like, how how do you even understand
476
00:29:53.985 --> 00:29:58.245
the the quality of the output? And even if you painstakingly go
477
00:29:58.625 --> 00:30:12.059
and and and look at a lot of results and evaluate them in in various ways, and there are lots techniques around that. How do you make sure that that remains true as you release changes to the pipeline or changes to the underlying model that you're using and and things like that?
478
00:30:12.424 --> 00:30:13.885
And and and so that's,
479
00:30:14.265 --> 00:30:14.765
that's
480
00:30:15.225 --> 00:30:19.165
something that I think, you know, there's a lot of skills to be learned around.
481
00:30:19.540 --> 00:30:23.080
So, yeah, lots of things to learn. I have more questions than answers,
482
00:30:24.180 --> 00:30:25.000
around this,
483
00:30:25.700 --> 00:30:33.875
but but, yeah, it's exciting times. And then I'm sure that that over the next couple of years, we'll, you know, we'll we'll gradually learn how to how to do all these things effectively.
484
00:30:34.735 --> 00:30:35.795
Yeah. The it's
485
00:30:36.360 --> 00:30:47.585
it's funny how much that is the case right now of I have a lot of questions and I have a lot of ideas, but there aren't any established answers yet because all of it is too new. And so everybody's
486
00:30:48.125 --> 00:30:55.580
flailing around in the dark and making their best attempt, and eventually, we'll convene on a set of best practices, but we're not there yet.
487
00:30:56.200 --> 00:30:59.100
Yeah. Too too early to call, I'd say.
488
00:31:00.760 --> 00:31:04.345
One of the other interesting side effects of the current
489
00:31:04.805 --> 00:31:12.025
stage of AI and the the rise of generative AI in particular is the growth and interest of
490
00:31:12.350 --> 00:31:14.210
vector databases, vector indexing.
491
00:31:14.830 --> 00:31:16.690
So we're seeing a new growth
492
00:31:16.990 --> 00:31:23.005
of that particular segment where it seems like every day I hear of a new vector database that's out there
493
00:31:23.465 --> 00:31:25.645
and optimized for some particular use case.
494
00:31:26.105 --> 00:31:26.925
Vector databases
495
00:31:27.385 --> 00:31:28.205
as a technology
496
00:31:28.870 --> 00:31:32.409
predate the current AI craze of generative models
497
00:31:33.110 --> 00:31:36.730
for, in particular, things like semantic and similarity searching.
498
00:31:37.255 --> 00:31:38.235
And I'm wondering
499
00:31:38.935 --> 00:31:41.595
if there are any other aspects of
500
00:31:42.055 --> 00:31:43.275
the ways that
501
00:31:43.655 --> 00:31:44.155
AI
502
00:31:45.110 --> 00:31:47.530
applications and generative models in particular
503
00:31:48.310 --> 00:31:48.810
are
504
00:31:49.270 --> 00:31:51.050
bringing some of these new technologies
505
00:31:51.725 --> 00:31:52.865
or newly created
506
00:31:53.325 --> 00:31:54.625
categories into
507
00:31:55.165 --> 00:31:56.625
the data engineering ecosystem
508
00:31:57.085 --> 00:32:02.870
and how those are being used outside of that context of just supporting generative AI models?
509
00:32:04.370 --> 00:32:05.350
A vector databases
510
00:32:06.289 --> 00:32:11.405
generally use or in addition to being used in kind of rag scenarios,
511
00:32:12.505 --> 00:32:13.325
I think
512
00:32:14.105 --> 00:32:16.285
people are really excited about,
513
00:32:16.745 --> 00:32:18.685
you know, the traditional search.
514
00:32:19.090 --> 00:32:22.390
I've, you know, I've talked to this company that that,
515
00:32:22.930 --> 00:32:25.270
works you know, that's an ad tech,
516
00:32:25.570 --> 00:32:27.110
one of our customers, and
517
00:32:27.434 --> 00:32:28.975
and, you know, there there's certain
518
00:32:30.155 --> 00:32:31.855
workflows there where you help
519
00:32:32.235 --> 00:32:32.735
marketers
520
00:32:33.355 --> 00:32:34.255
find relevant
521
00:32:34.715 --> 00:32:35.775
places to,
522
00:32:36.540 --> 00:32:38.240
to place ads. And,
523
00:32:39.020 --> 00:32:40.000
you know, traditionally
524
00:32:40.300 --> 00:32:46.205
that's been done with basically elastic search, right? Some some form of of keyword
525
00:32:46.664 --> 00:32:51.725
search with with a lot of bells and whistles around it, but and and they've gradually introduced
526
00:32:52.720 --> 00:32:59.860
vector embeddings and and and then, vector database that process. Right? And and not not in the context of rag,
527
00:33:00.480 --> 00:33:00.980
necessarily,
528
00:33:02.284 --> 00:33:06.065
but just, you know, if you wanna find a a place to
529
00:33:06.524 --> 00:33:08.384
to put your ad about, I know,
530
00:33:09.019 --> 00:33:18.639
fit fitness subscription or or what have you finding, you know, places online that that match that with and and and and vector databases actually do that
531
00:33:19.005 --> 00:33:20.145
extremely well,
532
00:33:20.605 --> 00:33:21.105
apparently
533
00:33:21.725 --> 00:33:22.865
much better than
534
00:33:23.485 --> 00:33:25.025
than than traditional search.
535
00:33:25.565 --> 00:33:26.305
And so
536
00:33:26.820 --> 00:33:29.880
there's a a resurgence of of of search
537
00:33:30.420 --> 00:33:35.080
now with with with vector databases available. And and, of course, those powerful
538
00:33:35.505 --> 00:33:37.605
language models that that create the embeddings,
539
00:33:38.465 --> 00:33:44.645
it seems to create, you know, better, higher relevance results with kinda deeper a deeper semantic understanding.
540
00:33:45.580 --> 00:33:49.440
That's, of course, been available, you know, on on Google or, you know, in other,
541
00:33:50.460 --> 00:33:54.400
search engines that that have been highly optimized over the years. But now,
542
00:33:54.825 --> 00:33:57.725
you know, probably, you know, smaller use cases,
543
00:33:58.345 --> 00:33:59.965
can be built quite rapidly,
544
00:34:01.225 --> 00:34:03.930
and have very, very powerful semantic search,
545
00:34:04.490 --> 00:34:06.750
using vector databases and embeddings.
546
00:34:08.010 --> 00:34:09.470
Now bringing it
547
00:34:09.770 --> 00:34:12.430
back around to the question of
548
00:34:13.085 --> 00:34:13.585
reliability,
549
00:34:14.045 --> 00:34:14.545
quality.
550
00:34:15.885 --> 00:34:17.025
We have been
551
00:34:17.485 --> 00:34:21.345
fighting that battle for a few years now in just the pure
552
00:34:21.869 --> 00:34:22.690
business intelligence,
553
00:34:22.990 --> 00:34:24.609
data analyst type
554
00:34:24.910 --> 00:34:28.369
use case of how do we build more reliable systems.
555
00:34:28.994 --> 00:34:33.974
Now with the added stresses and requirements around AI applications and these
556
00:34:34.355 --> 00:34:36.454
very probabilistic and not deterministic
557
00:34:37.180 --> 00:34:41.040
use cases? How are you seeing that impact the way teams are thinking about
558
00:34:41.420 --> 00:34:48.415
reliability, data quality, data observability, and some of the ways that you're thinking about that at Monte Carlo to be able to help support those teams?
559
00:34:49.515 --> 00:34:50.015
Yeah.
560
00:34:50.635 --> 00:34:51.375
Quick question.
561
00:34:51.970 --> 00:34:54.710
First of all, like everything, you know, we're still learning.
562
00:34:55.010 --> 00:34:55.830
There there's,
563
00:34:56.530 --> 00:35:01.750
there there's more question than answers, but I will call out kind of based on on what we're seeing.
564
00:35:02.275 --> 00:35:07.095
First of all, I go it it it all goes back to to basics in a sense. Right?
565
00:35:07.395 --> 00:35:12.920
Pipelines are pipelines, and you have to make sure they're they're working reliably. Right? You have to make sure that
566
00:35:14.580 --> 00:35:16.280
vectors or or tables,
567
00:35:16.980 --> 00:35:19.480
you have to make sure they're getting updated on time.
568
00:35:19.860 --> 00:35:21.240
You wanna make sure that
569
00:35:21.585 --> 00:35:32.470
you have all the dataset there that's that nothing is missing. You also wanna make sure there's no duplications, and duplications are particularly bad in in vector databases because they really
570
00:35:32.930 --> 00:35:36.309
hamper your your ability to get, you know, the the k nearest,
571
00:35:37.170 --> 00:35:46.535
neighbors and and and get the most, you know, the most relevant doc documents there because you you might get 5 copies of the same document if if you have it in the database.
572
00:35:46.915 --> 00:35:52.620
And so you you wanna make sure these things are working. And, of course, all the all the structural stuff. Right?
573
00:35:53.400 --> 00:35:54.300
Do you have the
574
00:35:54.680 --> 00:35:57.660
the right vector dimensions that you're expecting, and
575
00:35:58.085 --> 00:36:02.585
did you use the right embedding model and a consistent embedding model, right? That
576
00:36:03.125 --> 00:36:07.880
that, it's it's very important to use, the same embedding model in the pipeline and
577
00:36:08.839 --> 00:36:10.299
engineering retrieval. Right?
578
00:36:10.759 --> 00:36:16.539
And of course, the metadata around it. Right? Vectors are always accompanied by metadata that helps, you know, trace them
579
00:36:17.075 --> 00:36:17.575
back,
580
00:36:18.115 --> 00:36:32.420
to to where they came from or or associate them with with, you know, with with an account or a user or or other things. And you need to make sure that the metadata is there and it's it's complete and and it's accurate in in the same way that structured data,
581
00:36:32.880 --> 00:36:33.619
was tested.
582
00:36:34.115 --> 00:36:36.055
So so all these things are there,
583
00:36:36.675 --> 00:36:37.795
and and of course,
584
00:36:38.435 --> 00:36:43.655
lineage too. Right? In order to effectively manage the quality and reliability, you need to understand,
585
00:36:44.660 --> 00:36:47.800
where where the date where those vectors came from,
586
00:36:48.420 --> 00:36:49.960
how they're consumed downstream,
587
00:36:50.660 --> 00:36:57.335
all that good stuff in in observability. Right? When you have a problem, you you need to understand what its impact downstream is and and
588
00:36:57.715 --> 00:37:01.655
and what and and and to find the root cause, you you need to understand what's,
589
00:37:02.180 --> 00:37:06.359
what's upstream. So all all these things are kind of classic things that,
590
00:37:07.060 --> 00:37:11.079
translate from from structured data to unstructured data pretty directly.
591
00:37:11.625 --> 00:37:17.805
The one thing that's a little bit different is, well, how you how you measure the the quality of the data itself.
592
00:37:18.745 --> 00:37:21.245
And and in the structured world,
593
00:37:21.770 --> 00:37:24.350
you know, we we we develop a lot of methodologies.
594
00:37:24.650 --> 00:37:26.010
Right? Like, we can,
595
00:37:26.730 --> 00:37:31.194
calculate a lot of quality metrics, like, you know, how many nodes you have and how many duplications
596
00:37:31.974 --> 00:37:32.474
and,
597
00:37:32.934 --> 00:37:34.954
and and all sorts of these things.
598
00:37:35.335 --> 00:37:37.080
And we can have people that
599
00:37:37.640 --> 00:37:46.860
understand the pipeline build, you know, more more precise metrics around it that maybe take into account, you know, a deeper understanding of of the dataset and and the business.
600
00:37:47.224 --> 00:37:47.964
And that
601
00:37:48.585 --> 00:37:49.405
doesn't translate
602
00:37:50.025 --> 00:37:53.964
as well to text or image data that goes into vector databases.
603
00:37:54.480 --> 00:37:55.780
But we're we're seeing,
604
00:37:56.480 --> 00:38:00.260
increasingly methods to deal with that. So as an example,
605
00:38:00.880 --> 00:38:01.520
you know, you
606
00:38:02.080 --> 00:38:04.100
we're we're seeing people doing things like,
607
00:38:04.785 --> 00:38:05.285
well,
608
00:38:05.585 --> 00:38:06.964
I could probably
609
00:38:08.065 --> 00:38:09.845
use generative AI
610
00:38:10.224 --> 00:38:13.204
to calculate quality metrics about
611
00:38:13.780 --> 00:38:14.760
unstructured data.
612
00:38:15.300 --> 00:38:16.040
As an example,
613
00:38:16.340 --> 00:38:18.760
I could take all the texts that I have,
614
00:38:19.380 --> 00:38:20.440
and I could use
615
00:38:20.820 --> 00:38:21.780
a model to,
616
00:38:22.875 --> 00:38:28.655
for example, classified into topics, right, and track what topics I have in the dataset and make sure that's
617
00:38:29.035 --> 00:38:30.495
stable and that's behaving
618
00:38:31.250 --> 00:38:31.910
as expected,
619
00:38:32.690 --> 00:38:36.470
over time as the dataset changes and shifts or goes through the pipeline.
620
00:38:37.170 --> 00:38:42.255
So that's an example. I could also use generative models to determine whether there's
621
00:38:42.635 --> 00:38:44.255
sensitive data or PII
622
00:38:44.795 --> 00:38:51.570
in inside the set, right, and and track it over time and so on and so forth. You can get really creative with with how you
623
00:38:51.950 --> 00:38:58.125
use generative AI to create quality metrics on top of unstructured data. And and I think it's a very promising
624
00:38:59.145 --> 00:38:59.645
avenue,
625
00:39:00.505 --> 00:39:03.245
but of course, a lot to be learned there.
626
00:39:03.705 --> 00:39:04.205
It's
627
00:39:04.825 --> 00:39:07.245
complicated and it's early days.
628
00:39:07.690 --> 00:39:09.150
And, you know, if I'm honest,
629
00:39:09.609 --> 00:39:10.750
if you talk to,
630
00:39:11.450 --> 00:39:14.349
data teams building generative AI pipelines today,
631
00:39:14.924 --> 00:39:17.825
You know, a lot of it still relies on people
632
00:39:18.605 --> 00:39:19.825
eyeballing results,
633
00:39:21.164 --> 00:39:36.035
and trying to make sure that that they make sense. It's it's not always easy to to to scale and and and and automate these things. And so we're gradually learning how to take bites off of that manual process and and turn it into
634
00:39:36.335 --> 00:39:37.315
a more consistent
635
00:39:37.775 --> 00:39:39.155
and automated approach.
636
00:39:40.190 --> 00:39:41.730
As you have been
637
00:39:42.270 --> 00:39:42.770
navigating
638
00:39:43.390 --> 00:39:50.865
the rapidly changing landscape of data in the face of generative AI and the ways that data teams are
639
00:39:51.325 --> 00:40:00.090
working to adapt and ways that you at Monte Carlo are trying to help support them. I'm curious. What are some of the most interesting or innovative or unexpected ways that you're seeing
640
00:40:00.390 --> 00:40:02.890
AI impact those data engineering teams?
641
00:40:04.115 --> 00:40:04.615
Yeah.
642
00:40:05.075 --> 00:40:06.215
So many exciting
643
00:40:06.515 --> 00:40:07.015
examples
644
00:40:07.635 --> 00:40:08.135
here.
645
00:40:08.515 --> 00:40:09.655
I'll name a few.
646
00:40:09.955 --> 00:40:10.615
I think
647
00:40:10.995 --> 00:40:11.495
one,
648
00:40:13.420 --> 00:40:15.360
one really exciting trend is
649
00:40:15.900 --> 00:40:20.880
I think that the fact that we're unlocking structured data is bringing data engineering
650
00:40:21.885 --> 00:40:25.345
closer to the forefront of of decision making.
651
00:40:25.645 --> 00:40:28.944
And and the example I I I I like to quote here is,
652
00:40:29.325 --> 00:40:30.704
you know, talking to this
653
00:40:32.190 --> 00:40:35.490
sports team professional sports team. It's one of our customers.
654
00:40:36.190 --> 00:40:36.690
And,
655
00:40:37.230 --> 00:40:40.444
you know, in the past, you know, data engineers could provide,
656
00:40:41.144 --> 00:40:43.565
some structured data out there about players.
657
00:40:44.585 --> 00:40:51.130
Right, certain statistics about how they performed in matches. But but that data is pretty sparse. Right? And it it's only available
658
00:40:51.750 --> 00:40:53.130
in the in the top
659
00:40:53.589 --> 00:40:54.089
leagues.
660
00:40:55.184 --> 00:40:59.845
But if you're trying to spot the next, you know, the next generation of talent,
661
00:41:00.224 --> 00:41:04.080
the people coming from lower leagues or from high schools, and you're trying to
662
00:41:04.720 --> 00:41:05.220
scout,
663
00:41:05.840 --> 00:41:07.220
you know, the next star,
664
00:41:08.160 --> 00:41:13.460
it's it's really, really hard to get structured data about that, and it can be extremely unreliable.
665
00:41:14.085 --> 00:41:16.025
But there is a lot of unstructured data.
666
00:41:16.325 --> 00:41:21.144
Lots of scouts out there writing reports about players, you know, all around the world.
667
00:41:21.599 --> 00:41:24.500
And so data engineers were actually able to use
668
00:41:25.520 --> 00:41:28.900
AI to parse out that unstructured data
669
00:41:29.885 --> 00:41:30.385
and
670
00:41:31.085 --> 00:41:33.825
in fact, structure it. Right? Measure sentiment there,
671
00:41:34.525 --> 00:41:36.464
and then user use, methodologies
672
00:41:36.765 --> 00:41:38.625
of structured data to extract
673
00:41:39.090 --> 00:41:43.990
intelligence out of it. Right? You can benchmark the same player over time. You can benchmark
674
00:41:44.770 --> 00:41:48.310
the scouter. You can really get a lot of good insight, and
675
00:41:48.925 --> 00:41:55.505
that really brought the the data engineers to the forefront of the, you know, of the of the sporting professionals,
676
00:41:56.690 --> 00:41:58.950
and, you know, made them made them superstars,
677
00:41:59.730 --> 00:42:06.944
essentially. Right? And automated something that was extremely hard to do prior. The you know, and and another example
678
00:42:07.805 --> 00:42:08.305
where
679
00:42:08.765 --> 00:42:12.385
generally they actually have data teams to get in front of the
680
00:42:12.839 --> 00:42:14.220
customers of that organization.
681
00:42:14.520 --> 00:42:16.220
And and I'm thinking here about
682
00:42:16.599 --> 00:42:17.260
a a cybersecurity
683
00:42:17.560 --> 00:42:18.060
company.
684
00:42:18.599 --> 00:42:22.775
And in cybersecurity, there there is a lot of unstructured data, things like
685
00:42:23.474 --> 00:42:23.974
security
686
00:42:24.595 --> 00:42:27.815
policies and various documents in the in the enterprise
687
00:42:28.115 --> 00:42:30.170
and lots of exchanges
688
00:42:30.630 --> 00:42:31.369
of documentation
689
00:42:31.670 --> 00:42:32.170
between
690
00:42:32.630 --> 00:42:37.289
companies trying to deal with each other and and and and buy technology from each other.
691
00:42:37.615 --> 00:42:39.714
And that data team was able to
692
00:42:40.575 --> 00:42:43.875
do a bunch of different cool things with with generative AI, whether it's
693
00:42:44.255 --> 00:42:44.755
responding
694
00:42:45.775 --> 00:42:48.240
to to questions based on,
695
00:42:48.779 --> 00:42:55.119
you know, a a body of of documents that that that describe all security policies or whether it's,
696
00:42:55.645 --> 00:42:57.345
you know, in compliance,
697
00:42:57.725 --> 00:42:58.225
translating
698
00:42:59.085 --> 00:43:00.385
policies to controls
699
00:43:00.685 --> 00:43:01.185
automatically,
700
00:43:02.045 --> 00:43:05.510
things that are really, really high value for that company's
701
00:43:05.890 --> 00:43:06.390
customers
702
00:43:06.930 --> 00:43:10.150
that have been built almost exclusively by data engineers.
703
00:43:11.010 --> 00:43:14.550
So it kinda really brought them to the to the forefront. No no longer,
704
00:43:14.885 --> 00:43:17.704
you know, a platform team working for analysts, but,
705
00:43:18.005 --> 00:43:19.545
rather serving the company's,
706
00:43:20.325 --> 00:43:21.464
customers directly.
707
00:43:22.085 --> 00:43:23.224
That was pretty exciting.
708
00:43:23.930 --> 00:43:24.670
And then
709
00:43:25.130 --> 00:43:32.750
even saw this one data engineering team that was able to actually generate new revenue for the company in a pretty direct manner.
710
00:43:33.345 --> 00:43:35.205
It's an energy company. They
711
00:43:35.905 --> 00:43:40.725
they get, requests for quotes all the time. People asking them like to,
712
00:43:41.079 --> 00:43:41.900
you know, provide
713
00:43:42.680 --> 00:43:44.380
a certain supply of energy
714
00:43:44.760 --> 00:43:47.900
at a certain time and location, and they were able to,
715
00:43:49.085 --> 00:43:52.785
and and historically, they haven't been able to serve all those requests for quotes.
716
00:43:53.485 --> 00:44:03.945
They get a lot of these and and that's potential revenue. Right? If they're able to to respond to a quote that might turn into a customer. Right? And so they basically turn this from this
717
00:44:04.425 --> 00:44:05.405
very manual
718
00:44:05.705 --> 00:44:06.205
process
719
00:44:06.745 --> 00:44:07.565
where humans,
720
00:44:08.025 --> 00:44:17.350
you know, take in, you know, request by request and try to respond to it. They they automated it and and and and using generative AI and data engineers
721
00:44:17.810 --> 00:44:20.630
were able to suddenly generate a whole lot of new revenue,
722
00:44:21.090 --> 00:44:23.615
just from from automating that process and
723
00:44:24.234 --> 00:44:27.775
and and solving the the the manual labor, attached to it. So
724
00:44:28.234 --> 00:44:30.320
I I I think the point is, you know,
725
00:44:30.620 --> 00:44:33.600
data engineering is is is becoming
726
00:44:34.380 --> 00:44:35.440
even more valuable
727
00:44:35.820 --> 00:44:38.000
and and impactful in the business,
728
00:44:38.455 --> 00:44:41.915
you know, more than it's ever been. So and and that's exciting news.
729
00:44:42.855 --> 00:44:59.615
And in your own work of building a product that is supporting people in this space, trying to understand the impact that AI is having on data and the ways that teams are building these systems? What are some of the most interesting or unexpected or challenging lessons that you've learned personally?
730
00:45:00.795 --> 00:45:03.055
I think probably the biggest challenge,
731
00:45:04.250 --> 00:45:06.750
a couple of of big channel. One is understanding
732
00:45:08.410 --> 00:45:12.750
where and how generative AI can be applied, you know, to serve our customers.
733
00:45:13.315 --> 00:45:14.775
You know, we we
734
00:45:15.395 --> 00:45:19.415
we brainstormed, I wanna say, over 70 use cases
735
00:45:19.875 --> 00:45:20.935
in data observability
736
00:45:21.235 --> 00:45:23.260
that the generative AI can support.
737
00:45:23.720 --> 00:45:25.099
And it's it's
738
00:45:25.400 --> 00:45:28.140
it's it's pretty challenging to understand, well,
739
00:45:29.000 --> 00:45:30.940
where does it work well,
740
00:45:31.765 --> 00:45:35.944
where can it be applied to the maximum effect, where can it make the most impact,
741
00:45:36.565 --> 00:45:42.260
and where can it work as reliably as we need it, The which leads to the other challenge.
742
00:45:42.880 --> 00:45:43.380
It's
743
00:45:43.920 --> 00:45:44.420
extremely
744
00:45:44.960 --> 00:45:50.535
or it's it's it's not incredibly hard to get to, like, really cool demos with generative AI.
745
00:45:50.995 --> 00:45:53.255
Right? It's all it's you you can find,
746
00:45:54.035 --> 00:45:56.295
almost any use case you can think of.
747
00:45:56.910 --> 00:45:59.890
You can find a few examples where Genovir really
748
00:46:00.510 --> 00:46:02.930
blows your mind in terms of what you can do,
749
00:46:03.550 --> 00:46:06.424
which I think is why why the world is is so excited about
750
00:46:07.285 --> 00:46:10.904
it. It's a whole other ballgame to bring into production. Right? It's a whole other
751
00:46:11.285 --> 00:46:19.450
ballgame to do it in an environment where you can't expect to you you you can't predict all the inputs or maybe you can't predict them, but,
752
00:46:19.990 --> 00:46:21.690
but models are undeterministic,
753
00:46:22.070 --> 00:46:25.595
so they work only part of the time. And and there's a lot of
754
00:46:25.974 --> 00:46:28.955
art and science that goes into figuring out
755
00:46:29.415 --> 00:46:31.435
how to to make these generative models,
756
00:46:32.055 --> 00:46:32.540
work
757
00:46:33.260 --> 00:46:34.000
well enough
758
00:46:34.460 --> 00:46:36.400
so that they can serve a human
759
00:46:37.020 --> 00:46:46.875
trying to to to to accomplish a, you know, a a a a a task in the in the their day to day life. And I think the biggest learning there was really that
760
00:46:47.255 --> 00:46:47.755
combination
761
00:46:48.215 --> 00:46:48.715
between,
762
00:46:49.309 --> 00:46:51.890
you know, generative AI and and more deterministic
763
00:46:52.190 --> 00:46:53.490
approaches. For example,
764
00:46:53.950 --> 00:46:56.049
you know, using generative AI to
765
00:46:56.545 --> 00:46:58.245
to get an answer, but then
766
00:46:58.945 --> 00:47:03.445
validating it against, you know, heuristics or statistical models
767
00:47:03.940 --> 00:47:09.320
in order to make sure that that, you know, that response makes sense and is and is valid
768
00:47:09.780 --> 00:47:12.255
for a human to use. And so there's
769
00:47:12.875 --> 00:47:14.494
those are probably the the biggest
770
00:47:15.115 --> 00:47:18.620
challenges I've experienced personally while while building with generative AI.
771
00:47:19.980 --> 00:47:23.760
For people who are trying to navigate this
772
00:47:24.220 --> 00:47:26.880
current stage of the data ecosystem,
773
00:47:27.260 --> 00:47:34.785
what are some of the cases where where you're seeing AI as just being the wrong choice and not something that is worthwhile to
774
00:47:35.085 --> 00:47:38.440
invest in or try to incorporate into your data stack?
775
00:47:40.599 --> 00:47:41.339
Good question.
776
00:47:42.760 --> 00:47:46.540
It's not the wrong choice, but I I see a lot of
777
00:47:47.335 --> 00:47:49.095
or or the common pitfall I see is,
778
00:47:50.295 --> 00:47:51.994
is is the Microsoft approach
779
00:47:52.535 --> 00:47:53.035
of,
780
00:47:53.335 --> 00:47:56.234
let's throw a a copilot or a chat interface
781
00:47:56.760 --> 00:47:59.420
on on on every problem. Right? And just
782
00:47:59.960 --> 00:48:06.335
assume that what people want is to interact with in natural language with whatever they were doing before.
783
00:48:06.875 --> 00:48:11.775
And that's sometimes valid, and and it can be useful in certain scenarios. Right? And and,
784
00:48:13.330 --> 00:48:13.990
you know,
785
00:48:14.610 --> 00:48:15.510
in certain cases,
786
00:48:16.130 --> 00:48:19.990
yeah, I'll use an example from data engineer. Text to SQL can be very effective,
787
00:48:21.655 --> 00:48:23.595
but you really have to think about,
788
00:48:24.214 --> 00:48:26.954
about the end user. Right? They may not
789
00:48:27.815 --> 00:48:28.315
or
790
00:48:28.670 --> 00:48:31.810
natural language questions is not the answer to any problem.
791
00:48:32.270 --> 00:48:34.130
And sometimes people would just rather
792
00:48:34.510 --> 00:48:38.285
interact with with with interfaces the way they are. Right?
793
00:48:38.605 --> 00:48:41.665
Sometimes that's easier and simpler and works more reliably.
794
00:48:42.125 --> 00:48:44.305
And so I'd I'd I'd probably say
795
00:48:45.900 --> 00:48:51.599
it's it's important to think about where generative AI really allows people to do something that they've
796
00:48:52.835 --> 00:49:01.150
that they haven't been able to do before and where generative AI has an unfair advantage. Right? And we talked about some of these things, but, yeah, extract extracting
797
00:49:01.450 --> 00:49:02.670
structured information
798
00:49:02.970 --> 00:49:03.470
from
799
00:49:04.010 --> 00:49:05.230
unstructured data
800
00:49:06.170 --> 00:49:09.705
is is really exciting. It's something that people couldn't do before,
801
00:49:10.885 --> 00:49:13.625
but interacting in natural language with a dashboard,
802
00:49:15.300 --> 00:49:16.440
maybe not as
803
00:49:16.740 --> 00:49:17.240
groundbreaking
804
00:49:17.860 --> 00:49:23.535
as as you might think. Right? Sometimes it is, but but oftentimes it's not. And so I would just
805
00:49:23.835 --> 00:49:29.455
think about that that kind of idea of of where where can Generative AI really, really
806
00:49:29.920 --> 00:49:33.540
stand out and and and have a a competitive advantage,
807
00:49:34.160 --> 00:49:34.900
quote, unquote.
808
00:49:35.520 --> 00:49:38.100
And and the answers are not always obvious.
809
00:49:39.835 --> 00:49:41.455
What are some of the
810
00:49:41.915 --> 00:49:49.460
trends that you're keeping an eye on or some of the predictions that you have on the ways that AI is going to impact data engineering
811
00:49:49.840 --> 00:49:51.460
in the medium to long term?
812
00:49:53.440 --> 00:49:56.420
Good question. I I think we'll continue to see
813
00:49:57.335 --> 00:49:59.755
some of the things that we've already talked about.
814
00:50:01.734 --> 00:50:09.500
You know, it'll get easier to build pipelines, it'll get easier to democratize data. It'll get easier to put,
815
00:50:09.960 --> 00:50:12.620
you know, data engineers to work on the
816
00:50:13.005 --> 00:50:15.585
on the hardest, most high value problems.
817
00:50:16.445 --> 00:50:17.185
So that's,
818
00:50:17.724 --> 00:50:20.465
that's definitely happening and will continue to happen.
819
00:50:21.069 --> 00:50:24.690
You know, we'll continue to see more unstructured data being processed
820
00:50:25.230 --> 00:50:26.530
and and being fully
821
00:50:27.150 --> 00:50:28.849
put to use by data engineers.
822
00:50:30.005 --> 00:50:31.865
I think it's probably the most exciting
823
00:50:32.565 --> 00:50:34.185
element of generative AI.
824
00:50:34.565 --> 00:50:35.305
And then
825
00:50:35.925 --> 00:50:36.425
also,
826
00:50:37.180 --> 00:50:45.360
you know, as a result of these two things, I think it'll, you know, bring data engineers closer to the customer, closer to the revenue, closer to decision making
827
00:50:46.085 --> 00:50:48.424
in a way that that data engineer
828
00:50:48.964 --> 00:50:51.944
data engineering teams have have never been before. So
829
00:50:52.484 --> 00:50:54.984
we'll continue to see that in the long term.
830
00:50:55.710 --> 00:50:59.410
Are there any other aspects of this topic of
831
00:51:00.270 --> 00:51:07.825
AI and the impact that it's having on data engineers and data engineering teams that we didn't discuss yet that you'd like to cover before we close out the show?
832
00:51:08.445 --> 00:51:09.585
I'll I'll probably,
833
00:51:11.170 --> 00:51:13.830
again, kind going back to to to the basics.
834
00:51:14.210 --> 00:51:16.790
Right? General AI is cool and shiny,
835
00:51:17.330 --> 00:51:24.595
but in order to make it work in the real world, we have we have to do all the things that we know we have to do. Right?
836
00:51:25.215 --> 00:51:26.275
We need to build
837
00:51:26.600 --> 00:51:28.380
solid pipelines. We need to
838
00:51:28.920 --> 00:51:34.860
make sure they're cost effective, that they're properly governed from a security and compliance perspective, and
839
00:51:35.395 --> 00:51:40.535
and, of course, make sure they're they're reliable and and high quality because at the end of the day,
840
00:51:41.075 --> 00:51:44.215
you know, those models as smart as they are, can't overcome
841
00:51:45.070 --> 00:51:46.290
a security breach.
842
00:51:46.590 --> 00:51:47.570
They can't overcome
843
00:51:48.590 --> 00:51:49.570
garbage data,
844
00:51:50.110 --> 00:51:54.530
and and and they won't be successful if if they're cost prohibitive. Right? And so
845
00:51:54.895 --> 00:51:56.675
we need to make sure all these fundamentals,
846
00:51:57.935 --> 00:52:04.435
work well. And and at Monte Carlo, we're we're excited to tackle the the reliability and quality aspects of it.
847
00:52:04.900 --> 00:52:12.119
And, you know, we're we're also excited to see how how other vendors and and Saks will help with with with the 2 other challenges. So,
848
00:52:12.605 --> 00:52:13.985
yeah, back to the basics.
849
00:52:14.765 --> 00:52:21.950
Alright. Well, for anybody who wants to get in touch with you and follow along with the work that you and your team are doing, I'll have you add your preferred contact information
850
00:52:22.330 --> 00:52:29.685
to the show notes. And as the final question, I'd like to get your perspective on what you see as being the biggest gap in the tooling or
851
00:52:37.579 --> 00:52:38.319
think it's,
852
00:52:38.780 --> 00:52:41.599
it's it's not necessarily one tool, but,
853
00:52:41.900 --> 00:52:45.680
you know, with the theme themes that we've spoken about today in mind,
854
00:52:46.225 --> 00:52:47.445
I think we're seeing
855
00:52:47.825 --> 00:52:48.565
a consolidation
856
00:52:48.865 --> 00:52:51.685
of the the data engineering stack and the
857
00:52:52.145 --> 00:52:53.685
software engineering stack.
858
00:52:54.065 --> 00:52:54.565
And
859
00:52:55.359 --> 00:52:58.900
I'm kinda curious to see how we're going to marry or
860
00:52:59.920 --> 00:53:01.119
like to see more,
861
00:53:01.839 --> 00:53:03.380
easier ways to
862
00:53:04.265 --> 00:53:05.565
marry together those
863
00:53:06.105 --> 00:53:08.365
data pipelines and those customer facing,
864
00:53:09.065 --> 00:53:09.565
applications,
865
00:53:10.105 --> 00:53:13.610
you know, in the same platform working nicely together. I think there's
866
00:53:14.090 --> 00:53:17.870
some interesting opportunities in in stitching those two worlds and
867
00:53:18.330 --> 00:53:19.450
and making the whole,
868
00:53:19.850 --> 00:53:23.535
system work nicely together for, you know, for a team where there's,
869
00:53:23.934 --> 00:53:28.275
you know, a software engineer, a data engineer, and the machine learning engineer, all
870
00:53:28.575 --> 00:53:32.115
all working together to build a single platform, a single application.
871
00:53:33.079 --> 00:53:39.819
Alright. Well, thank you very much for taking the time today to join me and share the ways that you're seeing AI
872
00:53:40.280 --> 00:53:41.500
be an impact
873
00:53:41.905 --> 00:53:54.720
on and for the data engineering teams out there. Appreciate all of the time and energy that you and your team are putting into helping to support those folks. So, thank you again for taking the time today, and I hope you enjoy the rest of your day.
874
00:53:55.180 --> 00:53:56.995
Thank you, Tobias. Super fun.
875
00:54:04.820 --> 00:54:08.600
Thank you for listening, and don't forget to check out our other shows. Podcast.net
876
00:54:09.140 --> 00:54:18.035
covers the Python language, its community, and the innovative ways it is being used, and the AI Engineering podcast is your guide to the fast moving world of building AI systems.
877
00:54:18.735 --> 00:54:28.470
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email host at data engineering podcast.com
878
00:54:28.849 --> 00:54:35.085
with your story. Just to help other people find the show, please leave a review on Apple Podcasts and tell your friends and coworkers.