Declarative Machine Learning For High Performance Deep Learning Models With Predibase

WEBVTT
NOTE
Transcription provided by Podhome.fm
Created: 7/6/2024 5:05:16 PM
Duration: 3562.292
Channels: 1
1
00:00:13.344 --> 00:00:24.640
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it. So check out our friends over at Linode.
2
00:00:25.235 --> 00:00:34.840
With their managed Kubernetes platform, it's easy to get started with the next generation of deployment and scaling powered by the battle tested Linode platform, including simple pricing, node balancers,
3
00:00:35.240 --> 00:00:39.020
40 gigabit networking, and dedicated CPU and GPU instances.
4
00:00:39.720 --> 00:00:48.784
And now you can launch a managed MySQL, Postgres, or Mongo database cluster in minutes to keep your critical data safe with automated backups and failover. Go to python podcast.com/linode
5
00:00:50.204 --> 00:01:02.120
today to get a $100 credit to try out their new database service. And don't forget to thank them for their continued support of this show. Your host as usual is Tobias Macy, and this month, I'm running a series about Python's use in machine learning.
6
00:01:02.425 --> 00:01:09.565
If you enjoy this episode, you can explore further on my new show, The Machine Learning Podcast, that helps you go from idea to production with machine learning.
7
00:01:09.979 --> 00:01:12.799
To find out more, you can go to the machine learning podcast.com.
8
00:01:13.899 --> 00:01:30.330
Your host is Tobias Macy. And today, I'm interviewing Travis Adaire about prediabase, a low code platform for building ML models in a declarative format. So, Travis, can you start by introducing yourself? Thanks for having me on today, Tobias. So I'm Travis. I'm the CTO of prediabase. Prediabase is
9
00:01:30.630 --> 00:01:34.010
a low code platform designed to make machine learning
10
00:01:34.445 --> 00:01:37.505
more accessible and more useful to the enterprise.
11
00:01:38.605 --> 00:01:45.340
Before that, I was a tech lead manager for Uber's machine learning platform, leading a team focused on deep learning training.
12
00:01:45.800 --> 00:01:49.660
1 of the lead maintainers on the Horovod open source project
13
00:01:50.205 --> 00:01:57.265
and also a core contributor to a little project as well, which is 1 of the foundational technologies for prediabase.
14
00:01:58.000 --> 00:02:01.380
And do you remember how you first got started working in the area of machine learning?
15
00:02:01.840 --> 00:02:02.340
Absolutely.
16
00:02:02.640 --> 00:02:07.815
Yeah. So it goes back a bit to 2011 or so. I was working for
17
00:02:08.195 --> 00:02:10.295
Lawrence Livermore National Lab on
18
00:02:11.075 --> 00:02:15.380
processing about 10 terabytes of seismic data. And our goal was
19
00:02:15.840 --> 00:02:20.900
to try to do some analysis of it to detect nuclear weapons tests, interestingly enough.
20
00:02:21.200 --> 00:02:22.980
But what we found was that
21
00:02:23.285 --> 00:02:29.465
there were a lot of over 50% of the data or so was noise, and we had no good way to detect it. So
22
00:02:29.845 --> 00:02:32.900
started pulling out some of my undergrad AI textbooks,
23
00:02:33.280 --> 00:02:37.780
started implementing some support vector machine and ran it on top of Hadoop
24
00:02:38.239 --> 00:02:42.955
and got just really excited about the whole thing. Published an article in Computers and Geosciences
25
00:02:43.415 --> 00:02:43.915
journal
26
00:02:44.455 --> 00:02:49.359
and decided to go to grad school and get my master's in machine learning and
27
00:02:49.739 --> 00:02:51.359
more active in the industry.
28
00:02:51.739 --> 00:02:53.200
And, yeah, that's how it all started.
29
00:02:53.500 --> 00:03:17.295
And now that has brought you to where you are today with the prede base business and the project, and I'm wondering if you can describe a bit more about what it is that you're building there and some of the story behind how you decided that this was a problem space that you wanted to spend your time and focus on. When I was at Uber, I actually started off as a machine learning engineer. So working on kind of the vertical problems of ML.
30
00:03:17.835 --> 00:03:53.025
And what I found was that there were a lot of things I wanted to do. Like, I wanted to, you know, try deep learning and try training on large datasets and multi GPU and all these sorts of things. But there just wasn't a lot of good tooling available at the time to do that. There was TensorFlow and we wanted to run TensorFlow scalably. There was like a whole lot of hoops you had to jump through. And then integrating that with something like Spark that we were using for data processing, it was just like, forget about it. So what I realized was that if I wanted to solve it all problems, I really need to start with the and all tooling and infrastructure.
31
00:03:53.325 --> 00:03:54.705
And so I joined the
32
00:03:55.005 --> 00:03:55.505
Michelangelo
33
00:03:56.045 --> 00:03:57.185
deep learning team.
34
00:03:57.520 --> 00:04:05.620
And while I was there working on this horizontal problem, you know, we worked with a lot of customers that had very similar patterns that emerged
35
00:04:06.215 --> 00:04:07.515
to mine where
36
00:04:08.135 --> 00:04:12.635
we often found that there was the struggle to get something that was productionizable,
37
00:04:13.095 --> 00:04:15.780
like, at scale. Right? And there was also
38
00:04:16.240 --> 00:04:18.100
a repeating pattern of
39
00:04:18.480 --> 00:04:23.220
there just being frankly more in all problems than there was in all expertise of the company.
40
00:04:23.534 --> 00:04:27.634
And so from a kind of horizontal platform standpoint of Uber AI
41
00:04:28.175 --> 00:04:33.650
tasked with figuring out, you know, how can we help get models into production faster, we kind of
42
00:04:34.030 --> 00:04:36.930
realized that there was a need for better abstractions,
43
00:04:37.630 --> 00:04:39.330
better infrastructure more generally.
44
00:04:39.630 --> 00:04:40.370
And so
45
00:04:40.715 --> 00:04:49.830
the tool Ludwig that's my cofounder Puro put together ended up being a perfect encapsulation of that vision of being able to say, you know, let's let
46
00:04:50.370 --> 00:04:55.189
researchers build state of the art models and put them into as components into this framework.
47
00:04:55.729 --> 00:05:05.610
And then that gives the vertical teams in the company this very easy declarative interface to just kind of swap in and out different components for their data without having to
48
00:05:06.009 --> 00:05:12.509
rewrite, you know, huge swaths of Python code every time. And we realized that this was like a very successful pattern
49
00:05:12.884 --> 00:05:16.664
for Uber. And at the same time, you know, Ludwig became open source,
50
00:05:16.965 --> 00:05:26.070
saw that it resonate very strongly with the community, and realized that there was, like, a very real need for this kind of better abstraction layer, if you will, in the industry
51
00:05:26.610 --> 00:05:29.830
and decide to form prediabase with the intent kind of pushing
52
00:05:30.425 --> 00:05:42.470
the state of the art forward in terms of what kinds of tools data science and machine learning teams have available to them to make them more productive and kind of decrease the time to value of machine learning in the enterprise.
53
00:05:43.169 --> 00:05:54.175
In terms of the audience that you're focused on, particularly given that you're very early in your journey of being a new company and starting to work with some of the first sets of customers,
54
00:05:54.730 --> 00:05:56.910
I'm curious how you think about the
55
00:05:57.450 --> 00:06:00.590
priorities that you're trying to support and how that influences
56
00:06:00.890 --> 00:06:06.915
the areas of focus that you're investing in and the user experience and feature development that you're prioritizing?
57
00:06:07.935 --> 00:06:15.699
Yeah. So we like to say that we don't expect a prediabase that we're gonna be your first machine learning model that you've ever trained. Right? So oftentimes,
58
00:06:16.319 --> 00:06:18.419
we're coming into organizations that
59
00:06:18.800 --> 00:06:21.460
have a lot of machine learning problems that they wanna solve.
60
00:06:21.854 --> 00:06:34.250
Maybe they have some kind of horizontal team that's focused on trying to build out a platform for doing machine learning. And maybe they've tried using some AutoML tools in the past and have struggled with getting them into production.
61
00:06:34.629 --> 00:06:44.194
And so what we identify is, you know, we see these teams that have struggled in this way, and the value proposition that we wanna bring to them is to say, you know, if you've
62
00:06:44.735 --> 00:06:55.490
used, like, you know, some traditional ML systems in the past, like Spark MLlib or what have you, and you're struggling to kind of uplevel to deep learning and more state of the art techniques in the industry,
63
00:06:55.895 --> 00:07:15.444
you know, we provide, like, a platform that gives you those capabilities in a form factor that's, like, much more familiar to you. And if you're struggling to kinda keep up with the amount of problems that the organization has, like, maybe you have teams of engineers that have ML problems that maybe are not the top priority for the whole company, but a very important priority for that team.
64
00:07:15.824 --> 00:07:17.685
Credit based provides a platform
65
00:07:18.384 --> 00:07:24.000
that allows those teams to be unblocked and allows everyone in the organization to collaborate together
66
00:07:24.540 --> 00:07:26.160
towards building these solutions.
67
00:07:26.780 --> 00:07:28.400
And so this focus on,
68
00:07:28.755 --> 00:07:29.575
you know, collaboration
69
00:07:29.955 --> 00:07:41.310
and kind of mixed modality, like, you know, very broad set of tasks that people might wanna solve, those are very core focuses for us when we look at companies that we wanna partner with at this stage.
70
00:07:41.849 --> 00:07:51.295
1 of the interesting things that you mentioned is that you're working with companies who have a lot of machine learning problems to solve. And I'm wondering if you can talk to
71
00:07:51.915 --> 00:08:05.595
what that really means. Like, how you can identify that a problem that you have is a machine learning problem or whether machine learning is the right approach to being able to provide value and utility for a a given objective that you're trying to achieve?
72
00:08:06.055 --> 00:08:10.635
I would say that it comes in a few different flavors. Like, on the 1 hand, you have
73
00:08:11.000 --> 00:08:11.660
kind of
74
00:08:12.280 --> 00:08:19.020
a traditional data warehouse type systems that have tables, that have historical data or transactional data.
75
00:08:19.545 --> 00:08:30.990
And so very often, the story there is people wanna do some kind of predictive analytics. Right? So we know who churned last month. You know, we wanna predict who's gonna churn next month. So that kind of forward looking
76
00:08:31.530 --> 00:08:32.589
predictive capability
77
00:08:32.970 --> 00:08:36.670
is, like, 1, like, type of problem that we see a lot with companies
78
00:08:37.315 --> 00:08:47.459
that fits very nicely into machine learning. So you have a lot of data in your database. You wanna be able to, you know, predict or, like, make forward looking statements about that data.
79
00:08:47.760 --> 00:09:19.075
That's 1 area where we saw it in. But beyond that kind of structured problem, there's also this question of unstructured data as well. Right? And so you have a lot of companies that have text data or image data or audio data sitting around that they've collected, and they don't really know what to do with it. And it's maybe not so much a question of, you know, I have data that says what customers submitted and support tickets in the past. I wanna predict what support tickets are gonna say in the future. It's not anything like that, but it's more about just understanding semantically what's going on in the data and then how that can be used to
80
00:09:19.440 --> 00:09:30.215
better inform the predictive forward looking models that we want to build on that more transactional data. So this idea of unlocking the power of unstructured data is another really core 1. And so
81
00:09:30.595 --> 00:09:40.520
1 of the things I think is very unique about Predabase and Ludwig is the ability that we can kind of bridge this gap between structure and unstructured data. So the platform is is very flexible. It's
82
00:09:40.899 --> 00:09:51.695
data oriented in such a way that if you have some transactional tabular data and you also have some unstructured image or audio or text data, those can be combined together into a unified
83
00:09:52.160 --> 00:09:54.740
machine learning model in a very simple and straightforward
84
00:09:55.120 --> 00:09:57.380
way. And so we can unblock organizations
85
00:09:57.760 --> 00:10:03.675
that have all this disparate data and they wanna derive value from it, but they haven't figured out how in an effective way,
86
00:10:04.055 --> 00:10:30.209
that's where we think there's a lot of power for machine learning, particularly on credit based to kind of slot it. Yeah. 1 of the ways that I've seen those different applications categorized is the difference between predictive analytics, which is the first category that you mentioned versus prescriptive analytics of this is what you should do, and then descriptive analytics to say, I just wanna understand what this is trying to tell me. Right. Right. Absolutely. I think that descriptive component
87
00:10:30.670 --> 00:10:33.250
is 1 that not a lot of people have tapped into.
88
00:10:33.790 --> 00:10:35.329
In terms of the
89
00:10:35.895 --> 00:10:42.235
way that you have formulated the product that you're building at prediabase, you're positioning it as declarative ML.
90
00:10:42.695 --> 00:10:44.635
And you've mentioned earlier that
91
00:10:45.220 --> 00:10:50.120
may have had experience trying to use the category of tools called AutoML.
92
00:10:50.580 --> 00:10:53.524
And I'm wondering if you can just talk to the differences
93
00:10:53.825 --> 00:10:55.045
in the nomenclature
94
00:10:55.425 --> 00:10:56.404
as far as
95
00:10:56.705 --> 00:11:00.725
what that really means and how the sort of expectations
96
00:11:01.105 --> 00:11:03.110
are different between an AutoML
97
00:11:03.810 --> 00:11:07.029
category of tool and a declarative ML category of tool.
98
00:11:07.490 --> 00:11:09.910
Absolutely. Yeah. I think this is, like, a very key
99
00:11:10.355 --> 00:11:10.855
differentiator
100
00:11:11.155 --> 00:11:19.335
between how we're thinking about the problem and how a lot of other companies out there are. So the way that we think about it is that at a high level, there are very similar capabilities
101
00:11:19.730 --> 00:11:31.245
in terms of being able to put ML in the hands of non experts at kind of the starting point. But we believe that declarative ML provides a more principled and flexible path forward.
102
00:11:31.545 --> 00:11:42.370
So whereas a lot of AutoML solutions, I think in a degenerate state, AutoML often becomes this kind of kitchen sink style approach where the system will throw everything it can at the problem
103
00:11:42.670 --> 00:12:02.890
they can think of. And if something works, then great. You know, you kinda take that baton and run with it. If something doesn't work, then there's not really a lot of options that you have in terms of what's going to happen next. Like, how someone who is maybe a domain expert or a data expert can come in and kind of help out with, you know, unblocking things.
104
00:12:03.274 --> 00:12:10.415
Where we think declarative ML provides a difference here is that because it gives you this very complete
105
00:12:10.795 --> 00:12:11.295
specification,
106
00:12:12.280 --> 00:12:18.220
you can start at something very high level and say, you know, I just wanna predict this target given these input variables.
107
00:12:18.760 --> 00:12:28.165
And you can get a baseline from there, but it's not the end of the story. And so because it's very explicit about saying, here's everything that the system did. And you can modify
108
00:12:28.590 --> 00:12:40.214
and customize any aspect of this down to, you know, individual, you know, layers of a neural network. Right? It allows people to then iterate on these systems, on these implementations over time
109
00:12:40.675 --> 00:12:42.855
and kind of build towards
110
00:12:43.235 --> 00:12:51.460
a working solution in a more principled way. So for example, they can say, oh, well, I initially tried building this model with this set of parameters.
111
00:12:52.000 --> 00:13:03.085
And then for v 2, I swapped out this model architecture, and I changed the learning rate from this to this. And it gives you that audit trail of being able to say, here are all the things I tried. Here's what changed,
112
00:13:03.400 --> 00:13:11.020
and here is the effect that that had on on model performance. And so we think that this is also very powerful for enabling collaboration.
113
00:13:11.735 --> 00:13:16.475
So if, you know, someone who is an engineer maybe wants to train their first model
114
00:13:16.935 --> 00:13:32.645
in prediabase, they can do that without having to know a lot of details about the how of what's going on under the hood. But if they get a solution and then they want to maybe have a more expert data scientist take a look and say, you know, what do you think we should try in order to get better performance?
115
00:13:33.105 --> 00:13:52.024
They can take a look at the config and say, oh, well, you know, I see that you're using this parameter here. Let's maybe try swapping that out, see what happens. And so, you know, it gives you that ability to make these incremental changes in a very simple way that if you were kind of going down to just pure low level tools like PyTorch or TensorFlow,
116
00:13:52.564 --> 00:13:56.264
it'd be much more difficult to do that because you'd be having to ship over
117
00:13:56.650 --> 00:14:09.235
entire, you know, Jupyter Notebooks and Python libraries and, you know, what's the execution environment for all this. It's much more difficult for someone to just quickly take a look and kind of provide feedback and kind of next steps.
118
00:14:09.775 --> 00:14:22.675
And we also believe this ties in very nicely to our version of AutoML, which we would call iterative ML or iter ML, where we see it being much more of a conversation that you're having with the system where
119
00:14:23.055 --> 00:14:28.995
you try something out, the system can propose some new things to modify in the in the specification
120
00:14:29.375 --> 00:14:33.260
for the model. You can choose to either accept or reject any of those things,
121
00:14:33.640 --> 00:14:37.100
train for a little bit more, and then use the results of that previous
122
00:14:37.615 --> 00:14:42.915
run to then inform what you're gonna try next. So it becomes a very in the loop back and forth
123
00:14:43.295 --> 00:14:50.820
process that progresses in a much more in a way that we think is much more like how traditional software development is done. Right? Where you have a git repo,
124
00:14:51.200 --> 00:14:52.420
make some code commits,
125
00:14:52.775 --> 00:14:57.915
and over time, you can see the code evolve and change to kind of better conform to the end state
126
00:14:58.215 --> 00:15:10.920
as opposed to just trying to get all out there at once and, you know, not have any way of knowing, you know, what was the history, what was the effect of every change that we made. In this category of declarative ML,
127
00:15:11.324 --> 00:15:13.904
another company that I've seen using that terminology
128
00:15:14.445 --> 00:15:17.985
is Continual, which is based on being able to
129
00:15:18.530 --> 00:15:26.070
build machine learning pipelines on top of your data warehouse so that you can just treat your machine learning workflow as SQL effectively.
130
00:15:26.685 --> 00:15:28.464
I'm wondering if you can just characterize
131
00:15:28.925 --> 00:15:34.385
the relative strengths and use cases of what you're building at prediabase versus what they're building at continual.
132
00:15:34.940 --> 00:15:48.195
Yeah. Absolutely. So I guess the first thing I'd say is that, you know, in general, we're very glad to see other companies kind of validating the idea behind declarative ML, you know, from following the work that Tristan and the folks that continue have been doing.
133
00:15:48.575 --> 00:15:55.120
It's always been very nice to see that they've referenced our work on Ludwig and Overton. So 1 of our cofounders, Chris Ray, was
134
00:15:55.520 --> 00:16:22.235
had a company called Lattice that had a product called Overton that was acquired by Apple, which was another early declared of a mail system. And so I think in general, there's like a really good shared vision of kind of moving the conversation forward about better abstractions in a mountain. So I think there's definitely an element of, you know, rising tides, you know, raising all ships to it. Where I think there are differences would be they definitely are very leaned in to the kind of modern data stack operation side of things.
135
00:16:22.615 --> 00:16:23.515
I think that
136
00:16:23.895 --> 00:16:43.945
their value proposition resonates very nicely with people who are active DBT users, for example. That's a big part of kind of how they're approaching the problem, which I think is a totally valid way to think about it. For us, we definitely think that we can do a lot, not just on the operation side, but on the model development side as well. So with Ludwig,
137
00:16:44.600 --> 00:17:01.695
we provide a framework that is also pushing forward the state of the art of what ML models can do. Right? And so that's a big part of the story for us is trying to figure out how do we help users get good models in the first place and do it in a way that is very
138
00:17:02.070 --> 00:17:15.835
low barriers to entry, but very high performance and high ceiling. Right? We also believe the operations component is a big part of that, but it's not the only part. I think there's still a lot of work that needs to be done on just getting to a good model that you wanna put in production.
139
00:17:16.215 --> 00:17:17.835
And so that's where I think
140
00:17:18.170 --> 00:17:22.030
Ludwig is a bit different from what some of the other tools out there provide and that
141
00:17:22.410 --> 00:17:26.905
we're also tackling that aspect of the problem in the big. And as far as
142
00:17:27.365 --> 00:17:35.909
the implementation and architecture of what you've built at prediabase, can you talk to the overall system design and the ways that you've thought about the
143
00:17:36.210 --> 00:17:36.710
architecture
144
00:17:37.090 --> 00:17:41.029
of how to approach this problem of making declarative ML
145
00:17:41.570 --> 00:17:42.070
accessible
146
00:17:42.370 --> 00:17:54.330
and easy to operate so that teams who don't necessarily want to invest in building the entire MLOps stack can be able to pick it up and run with it and start to gain value very quickly.
147
00:17:54.950 --> 00:18:00.010
Credit base is built as a multi cloud native platform. We're built on top of
148
00:18:00.375 --> 00:18:00.875
Kubernetes.
149
00:18:01.255 --> 00:18:04.635
And so, you know, we have deployments that run AWS,
150
00:18:04.935 --> 00:18:08.475
Azure, GCP, and also on some on premise Kubernetes clusters.
151
00:18:08.940 --> 00:18:20.315
So we believe that that's like a very core part to make it flexible so that wherever your data happens to live, you know, we can push the compute down to be as close to that data as possible to minimize
152
00:18:21.015 --> 00:18:24.554
the latency and minimize the egress costs and all those things.
153
00:18:24.860 --> 00:18:26.960
So that's a very core part of how it's architected.
154
00:18:27.419 --> 00:18:32.159
We also have a separation between what we would call our control plane and data plane
155
00:18:32.475 --> 00:18:42.140
of the system. Our data plane is built on top of the Ludwig on Ray open source work that we've done. So we use Ray for doing the distributed aspect
156
00:18:42.520 --> 00:18:44.940
of, you know, scaling the to large datasets
157
00:18:45.240 --> 00:18:50.060
and paralyzing the work. We use Horovod for doing distributed data parallel training.
158
00:18:50.445 --> 00:18:59.505
And then we also have a serving layer as well that's built on top of that. And then we also have a separate control plane that provides a serverless
159
00:18:59.820 --> 00:19:06.560
abstraction layer on top of this data plane so that, you know, from the user perspective, they don't need to be as concerned about
160
00:19:07.144 --> 00:19:07.644
provisioning
161
00:19:07.945 --> 00:19:15.085
ray clusters that run and, like, how to right size it for the workload and whether I wanna use this GPU or that GPU.
162
00:19:15.770 --> 00:19:24.350
So that's a big aspect of what we provide on the infra side is this kind of intelligent provisioning life cycle management of the compute resources
163
00:19:25.065 --> 00:19:28.445
and making sure that these long running training and prediction workloads
164
00:19:28.905 --> 00:19:51.309
can be processed end to end in an efficient way. And then, of course, there's a whole another serving stack as well that we're building out that's built on top of NVIDIA Trident, and and we'll have a lot hopefully on the to say on in terms of our work there, kind of some blog post coming out in the future. But that's something that we're also looking to push into the open source to some extent as well as some of the serving capabilities
165
00:19:51.690 --> 00:19:54.270
for Ludwig that we're bringing to the enterprise.
166
00:19:54.970 --> 00:20:02.595
As you started to go down the path of starting to build out this platform and explore the capabilities that you wanted to offer,
167
00:20:02.895 --> 00:20:04.674
I'm wondering how the initial
168
00:20:04.975 --> 00:20:05.875
design and
169
00:20:06.190 --> 00:20:11.570
ideas and vision around where you wanted to end up have shifted and evolved and
170
00:20:11.950 --> 00:20:26.190
some of the directions that you have moved in order to be able to accommodate some of the early feedback that you've gotten as you worked with design partners and just some of the overall evolution of the platform as you started to dig deeper into this space?
171
00:20:26.570 --> 00:20:48.130
That's a great question. So definitely, we had a certain set of assumptions coming in about what the market was looking for and kind of what level the user wanted to think about the problem. Right? Whether this was a production problem first and foremost for them, a research problem somewhere in between. Right? And so we definitely had a very early focus on thinking a lot about the analyst use case
172
00:20:48.485 --> 00:20:50.825
and how, you know, there are people who have
173
00:20:51.285 --> 00:20:56.505
data but don't have a background in ML who want to be, you know, up level with that capability.
174
00:20:57.330 --> 00:21:20.860
And so we thought a lot about, you know, making it kind of operations and production oriented to begin with. What we found in kind of the early working with early customers is that there's still a lot of interest in, you know, the model development aspect. And, you know, you're never going to at least, you know, how AI machine learning works today gets that perfect model every time, like, without any kind of manual intervention or kind of domain expertise.
175
00:21:21.320 --> 00:21:26.295
And so we definitely, from a very early on working with customers, realized that having
176
00:21:26.915 --> 00:21:28.535
a teaching element of the platform
177
00:21:28.995 --> 00:21:35.620
was very important as well explaining, you know, here are the options you have available to you in this declared a specification.
178
00:21:36.240 --> 00:21:47.405
Here's how you should think about using 1 option versus the other, what the appropriate ranges are, you know, how you should go about doing model development in terms of starting with, you know, a really complex model or a baseline,
179
00:21:48.185 --> 00:21:52.549
understanding how the model performed kind of in a post hoc way and saying,
180
00:21:52.850 --> 00:22:06.144
you know, these were the features that contributed the most to the model's predictions or these were fields that maybe, you know, were imbalanced in some way, and so there are other corrections we need to make. So definitely that aspect of
181
00:22:06.620 --> 00:22:07.120
iteration
182
00:22:07.980 --> 00:22:10.320
and instilling kind of machine learning best practices
183
00:22:10.860 --> 00:22:18.055
is something that has been a learned experience. And so, know, 1 of the most recent additions we made to the platform
184
00:22:18.595 --> 00:22:20.375
just before coming out of stealth was
185
00:22:20.675 --> 00:22:29.410
investing really heavily in a Python SDK that's very similar to what the Ludwig Python SDK does, but provides but with some more enterprise features
186
00:22:30.045 --> 00:22:32.385
that really make it well integrated into
187
00:22:32.925 --> 00:22:33.905
a data science
188
00:22:34.205 --> 00:22:35.665
stack where you're able to
189
00:22:36.205 --> 00:22:38.865
experiment with data, experiment with models, iterate
190
00:22:39.299 --> 00:22:41.799
as opposed to just going straight for the production
191
00:22:42.179 --> 00:22:48.280
model from day 1. Right? So that was definitely something we learned early on in the process of working with customers.
192
00:22:48.915 --> 00:22:50.935
And it's also interesting to think about
193
00:22:51.555 --> 00:22:53.575
which areas of the overall
194
00:22:54.035 --> 00:22:58.690
machine learning problem and life cycle you're looking to be able
195
00:22:59.230 --> 00:23:00.049
to facilitate
196
00:23:00.590 --> 00:23:03.809
because there are, you know, boundless capabilities
197
00:23:04.269 --> 00:23:05.809
where, you know, there's the
198
00:23:06.175 --> 00:23:14.355
experiment tracking and model tracking. There's the model monitoring to be able to understand, you know, what the concept drift is happening once it's in production.
199
00:23:15.100 --> 00:23:17.040
You know, there's the kind of pipelining.
200
00:23:17.580 --> 00:23:23.920
So there are definitely a number of different areas that you could try to focus on, a number of different directions you could try to push into.
201
00:23:24.245 --> 00:23:24.905
I'm curious
202
00:23:25.445 --> 00:23:29.785
given the fact that you were starting with Ludwig as the kind of core
203
00:23:30.085 --> 00:23:30.985
building block,
204
00:23:31.420 --> 00:23:33.840
how that helped to shape your overall
205
00:23:34.540 --> 00:23:35.040
consideration
206
00:23:35.340 --> 00:23:42.475
of the appropriate scope for what you're trying to build and how you were thinking about what are the maybe boundaries and interfaces
207
00:23:42.935 --> 00:23:48.235
that you want to incorporate to be able to let prediabase fit into the overall
208
00:23:49.039 --> 00:23:52.340
workflow and life cycle of machine learning in an organization
209
00:23:53.039 --> 00:23:55.700
while being able to be very opinionated
210
00:23:56.000 --> 00:23:56.500
and
211
00:23:56.885 --> 00:24:02.265
drive the conversation around the areas that you wanted to own? It's a really good question, I think, because
212
00:24:02.804 --> 00:24:20.950
this, I think, is at the core of the problem of startups in the space trying to define their category. Right? Because I think when you look at the space in general, what you see is that there are a lot of really good tools that are what you might call point solutions that are solving 1 aspect of the problem, whether it's explainability,
213
00:24:21.410 --> 00:24:23.110
model training, model serving,
214
00:24:23.410 --> 00:24:31.455
what have you. Right? But really, what organizations need is they they do need something that is end to end. Right? That is actually a platform that is
215
00:24:31.755 --> 00:24:32.975
fundamentally delivering
216
00:24:33.355 --> 00:24:40.540
business value, not just, you know, model training or something like that. So the way that we think about it broadly is that
217
00:24:41.000 --> 00:24:41.500
we
218
00:24:41.960 --> 00:24:43.420
want to be able to provide
219
00:24:43.880 --> 00:24:51.135
a story that goes from data to deployment for the user. So connecting data from your data warehouse or database,
220
00:24:51.835 --> 00:24:54.815
providing best in class model training at scale
221
00:24:55.140 --> 00:24:56.679
with the serverless infrastructure,
222
00:24:57.540 --> 00:25:01.559
and then providing a really clean and simple path to deployment
223
00:25:02.075 --> 00:25:06.335
that can be either a REST API for low latency real time prediction
224
00:25:06.875 --> 00:25:12.720
or the PQL SQL Life Language that provides batch prediction capabilities to the user.
225
00:25:13.100 --> 00:25:17.040
And starting with that core vision of, like, this is the journey for the user.
226
00:25:17.419 --> 00:25:22.144
There are, as you said, a lot of other aspects to it as well, like, you know, model explainability,
227
00:25:23.085 --> 00:25:26.065
data preparation and data quality and data versioning,
228
00:25:26.605 --> 00:25:28.465
model monitoring, model drift detection.
229
00:25:28.820 --> 00:25:43.125
And the way we're thinking about these things today is pretty similar to how we think about them actually on the open source side with Ludwig, where we want to try to be as integrated with the community as possible. Right? So on Ludwig, we have integrations with CommonML,
230
00:25:43.425 --> 00:25:44.405
Weights and Biases,
231
00:25:44.785 --> 00:25:46.280
y logs, MLflow,
232
00:25:47.059 --> 00:25:56.315
AIMstack that we're working on. And so these tools, you know, provide, like, different capabilities at different parts of the process. Right? Like experiment tracking or
233
00:25:56.695 --> 00:25:57.595
model monitoring.
234
00:25:58.054 --> 00:26:08.450
But the way we wanna think about it is if you already have a tool that you like that solves these problems, like, we don't want to have to say prediabase is a rip and replace
235
00:26:08.885 --> 00:26:24.170
solution for you. Right? We wanna be well integrated. So if you wanna use weights and biases or comment, you know, you just give us an API key, and we'll log things there and then have a nice way to link back and forth between the 2. Or if you're using y logs slash y labs for doing
236
00:26:24.470 --> 00:26:40.970
model monitoring, you know, we're thinking about ways that we can integrate there to do automated model retraining based on triggers that come from from ylabs. So that's the way we're thinking about the problem today is let's integrate as much as possible in the parts of the platform where we don't feel we're providing
237
00:26:41.430 --> 00:26:46.495
strong differentiated value or that we could provide, like, a best in class value proposition on
238
00:26:46.795 --> 00:26:47.535
while still
239
00:26:47.915 --> 00:26:53.295
telling the user, like, hey. If you are starting from scratch, right, and you don't have an ML platform today,
240
00:26:53.679 --> 00:26:57.700
credit base isn't a point solution. It's something that will get you end to end from
241
00:26:58.000 --> 00:27:05.675
the data to a deployed model that can start delivering value. And then you can layer on more tooling on top of that, you know, as we see fit.
242
00:27:06.054 --> 00:27:36.510
And so in terms of that workflow, in the case where somebody is greenfielding, they say, I want to adopt ML. This is my first foray into that. I'm going to use prediabase to be able to experiment with how can I take this data that I have and turn it into something useful that I can do with it? I was wondering if you can just talk through that kind of end to end workflow of starting with the data and ending with, I have a model running in production, and I'm doing something with it. In the tool, there are different ways that users can do it. We do have a web UI that people can use
243
00:27:36.890 --> 00:27:38.110
to take all the actions.
244
00:27:38.575 --> 00:28:02.495
Everything that you can do in the platform can also be done through our sequel like language people as well as the Python SDK. So we have many different views depending on the persona that's using the platform that do the same thing. But regardless of which entry point you choose to use, the steps are largely the same as you first start with the data. So if you have your data in Snowflake or s 3 or BigQuery or whatever,
245
00:28:02.850 --> 00:28:08.870
You just give us some credentials, point us to what table or what bucket you're interested in working with,
246
00:28:09.170 --> 00:28:17.375
and then we can start with any data that is structured in some kind of table like form. Right? So that can be an actual database table.
247
00:28:17.675 --> 00:28:20.390
That can also be a parquet file, a CSV,
248
00:28:20.870 --> 00:28:21.850
anything like that.
249
00:28:22.230 --> 00:28:28.890
And then, you know, maybe a question that comes after that is, what if I wanna use unstructured data like images or audio?
250
00:28:29.215 --> 00:28:33.235
So the way we think about that today is that give us the URLs
251
00:28:33.615 --> 00:28:47.835
to those images or those audio files as columns in your tabular data, and then we can pick those up and we'll join all that together until, like, a single flat tabular view for training. Right? Once you've pointed us to the data that you wanna work with, we automatically
252
00:28:48.215 --> 00:29:06.345
do all the metadata extraction and schema extraction from the data for you. So we know, you know, what data types the data is and and that sort of thing. And then you can start creating models. So you, you know, go into the model builder UI or use the SDK to build a model in a way that's very similar to how you do it in Ludwig.
253
00:29:06.645 --> 00:29:18.880
And all you need to specify to get started is just the target or targets since we support multitask learning that you wanna predict. From there, you can customize any aspect of the training through either layering on full
254
00:29:19.259 --> 00:29:20.559
kind of, like, hyperparameter
255
00:29:20.940 --> 00:29:21.440
optimization,
256
00:29:22.220 --> 00:29:22.720
AutoML
257
00:29:23.434 --> 00:29:24.895
suggested like configurations
258
00:29:25.355 --> 00:29:26.015
for everything,
259
00:29:26.554 --> 00:29:34.620
or start with just a very simple baseline. Either way, you can go like any level of the extremes, right, or any level of customization between,
260
00:29:34.920 --> 00:29:41.980
and then start training a model. Once you start training a model, it will be sent to 1 of the, what we call, engines. So that's 1 of our serverless
261
00:29:42.665 --> 00:29:55.180
clusters that does the computation for you that lives, you know, wherever your data happens to live and, you know, the same kind of region. Right? Model will get trained. And from there, you can start using PQL or the Python SDK to
262
00:29:55.480 --> 00:30:01.645
validate. We also provide a full set of visualizations for the user to explore in terms of, you know, understanding
263
00:30:02.025 --> 00:30:05.645
the explainability of, like, feature importance. We also have calibration
264
00:30:05.945 --> 00:30:06.445
plots,
265
00:30:06.880 --> 00:30:19.695
all sorts of other things like confusion matrices, etcetera, that you can dig into. And from there, you can either iterate on the model, continue to develop it in kind of a incremental way that with a fully kind of versioned and lineage process.
266
00:30:20.315 --> 00:30:30.050
And then once you're happy with the model, there's kind of a 1 click deployment that we have where you can deploy it to a rest end point and then start curling it with, you know, JSON objects
267
00:30:30.350 --> 00:30:31.250
as you see fit.
268
00:30:31.605 --> 00:30:39.304
And then if you'd like to, you know, retire the model or replace it with a new model version, it's a similar 1 click kind of deployment process as well.
269
00:30:39.610 --> 00:30:42.590
And then there's, of course, ways that you can automate this as well-to-do
270
00:30:43.210 --> 00:30:43.710
retraining
271
00:30:44.169 --> 00:30:48.554
as well as do validation to determine when you want to trigger
272
00:30:48.855 --> 00:30:57.674
redeployment. Right? So if you say I have a held out test dataset, I only wanna redeploy when the, you know, new model does better than the old model on that dataset.
273
00:30:58.120 --> 00:31:09.054
You know, that's something that you can configure within the platform as well. And at a high level, that's the journey that we see as being the core flow that the user wants to go through. So connect data, train model, deploy.
274
00:31:09.355 --> 00:31:15.054
And then there's a lot in between, of course, that kind of fills in the gaps, but that's fundamentally what the platform provides.
275
00:31:15.659 --> 00:31:28.205
And you mentioned the PQL dialect a couple of times. And I noticed that when I was going through some of the blog posts and some of the early material that you have about what you're building at prediabase. And I'm wondering if you can
276
00:31:28.585 --> 00:31:29.325
speak to
277
00:31:29.625 --> 00:31:30.525
some of the motivation
278
00:31:30.905 --> 00:31:43.965
behind creating this new dialect and this new, I guess, language you could call it, and some of the ways that you think about the semantic and syntactic design of it. Yeah. So with people, we think it's a very natural
279
00:31:44.345 --> 00:31:48.845
way to extend the declarative idea because I think it plays very nicely into
280
00:31:49.320 --> 00:31:53.260
the way that we think about ML systems today compared to databases
281
00:31:53.640 --> 00:32:05.655
a few decades back, right, where you had in the early days, like low level languages like COBOL that people would be writing, would be interacting with databases and and then SQL comes along and provides this very nice declarative
282
00:32:06.260 --> 00:32:07.080
way of expressing
283
00:32:07.460 --> 00:32:11.000
all sorts of complex data analysis that you might wanna do.
284
00:32:11.380 --> 00:32:16.039
And we see people as being the natural extension of that idea to the ML domain.
285
00:32:16.404 --> 00:32:19.304
Whereas since you already have this, declared specification
286
00:32:19.684 --> 00:32:21.145
that provides a very tight
287
00:32:21.605 --> 00:32:22.105
semantic
288
00:32:22.485 --> 00:32:25.705
link between your data fields, the fields of your dataset,
289
00:32:26.120 --> 00:32:30.620
and the fields that are the inputs and outputs of your model, and then everything that happens in between.
290
00:32:30.920 --> 00:32:47.350
PEQOL provides a very natural way to express, you know, the model prediction request that you might wanna do. So what I think is very powerful about PQL is that you can do something as complex as a batch prediction over, you know, a 10 terabyte dataset
291
00:32:48.130 --> 00:32:51.990
using some model that you then wanna write out to a downstream table.
292
00:32:52.485 --> 00:33:00.425
Normally, you would end up writing like an ETL job and spark to do something like this. But that's just a 1 line people query, which would be predict
293
00:33:01.059 --> 00:33:02.279
target given
294
00:33:02.740 --> 00:33:13.895
select star from data or whatever. Right? And you can, of course, then do all sorts of more complex things from there in terms of joining tables across different datasets, filtering them, doing sliced analysis.
295
00:33:14.195 --> 00:33:16.135
You have ways of doing what we call hypothetical
296
00:33:16.755 --> 00:33:36.490
queries that are kind of similar to what you might do for real time prediction, where you want to take, like, an entirely new data point and then express it as a query that then can be predicted on. And so I think, you know, certainly 1 powerful use case of people is this idea of a more efficient way of doing batch prediction that fits in nicely in to other tools that
297
00:33:36.870 --> 00:33:45.455
do ELT. Like dbt is a really good example there where we already have a dbt integration that we we've written that some of our customers are using. And so if you
298
00:33:45.755 --> 00:33:48.015
want to be able to express your
299
00:33:48.395 --> 00:33:50.255
prediction pipeline as
300
00:33:50.555 --> 00:33:59.490
SQL, essentially, like people provides a very natural way to do that. But we also think that people is a very powerful enabler of putting ML
301
00:33:59.950 --> 00:34:05.195
inference and, like, letting people interact with and understand the model, stakeholders who might not
302
00:34:05.575 --> 00:34:22.625
today ever really interact with an ML model. Right? So anyone now who understands SQL can start making predictions and start to play around with understanding, like, what do I have to change in this input to make the model predict something else? Really, it's very fun kind of just interactive process that users can go through.
303
00:34:23.005 --> 00:34:36.535
And these sorts of people queries are a very nice sharing point as well. So if the data scientist has a model that they've trained and the analyst wants to play around with it or wants to see the result of some prediction on some slice of data,
304
00:34:36.915 --> 00:34:42.775
you just need to share that people query with them and then, you know, say, hey. Go run this and, you know, let me know how it goes
305
00:34:43.099 --> 00:34:55.934
instead of having to ship whole notebooks and Python files and whatever with it. So that's kind of where we see the value of people is batch prediction as well as for kind of pipelining and doing, you know, ELT type workloads
306
00:34:56.635 --> 00:35:02.260
as well as this kind of shareability and making NL inference more accessible to the broader organization.
307
00:35:03.040 --> 00:35:04.660
Noting the PQL
308
00:35:05.280 --> 00:35:05.780
acronym,
309
00:35:06.640 --> 00:35:11.885
my initial thought when I first read that in the blog post was, oh, obviously, the p stands for prediabase,
310
00:35:12.185 --> 00:35:13.645
but it stands for predictive.
311
00:35:14.025 --> 00:35:18.760
And so I'm wondering if you can talk to what your overall vision is for this syntax
312
00:35:19.700 --> 00:35:22.840
and if you intend for it to be something that is
313
00:35:23.140 --> 00:35:28.385
maybe adopted outside of prediabase as kind of a general standard for this
314
00:35:28.845 --> 00:35:33.220
means of interacting with machine learning and just some of the overall vision there?
315
00:35:33.700 --> 00:35:46.765
So the interesting thing there is that the name people actually predated the name Credit Base. So we had the name people in mind before we came up with Credit Base. But to your point, I definitely do see people being something
316
00:35:47.305 --> 00:35:50.845
larger than Prettabase in a lot of ways. Like, we want to see
317
00:35:51.350 --> 00:36:00.810
more folks in the industry adopted. And so the vision for people is that, you know, you do have a lot of the AI tools today that have very tight integration with SQL.
318
00:36:01.255 --> 00:36:12.060
And we'd like to be able to see, you know, very nice integration with people on a lot of these tools in the future as well. You know, thinking about how we can make it a standard that the larger community
319
00:36:12.520 --> 00:36:13.020
embraces.
320
00:36:13.560 --> 00:36:32.079
I think there is a lot of value to that. And so we do work very closely with some companies in the BI space through the Linux Foundation, actually, where Ludwig and Horovod, the projects that we maintain are hosted. And so we do have a collaboration there with the AI plus bi committee that is working on exactly this problem of
321
00:36:32.515 --> 00:36:35.335
integrating, you know, machine learning prediction into
322
00:36:36.115 --> 00:36:37.895
BI systems. And that's
323
00:36:38.195 --> 00:36:43.860
where I think things can go if, you know, the standard ends up becoming well adopted in the future.
324
00:36:44.320 --> 00:36:45.780
And another interesting element
325
00:36:46.080 --> 00:37:08.950
of the overall ML space is the question of collaboration. You mentioned PQL allows you to say, I've got this model. I wanna pass it off to this analyst to be able to play with it and experiment with its capabilities, maybe provide some feedback on ways that I should tweak it to, you know, make it more powerful for a certain use case. And I'm just curious how you think about that collaboration aspect of prediabase
326
00:37:09.425 --> 00:37:12.244
and how you've designed the platform to be able to
327
00:37:12.785 --> 00:37:15.125
be kind of idiomatic and recognizable
328
00:37:15.505 --> 00:37:17.445
for different roles and stakeholders
329
00:37:17.850 --> 00:37:21.230
across the organization who are interacting with the different capabilities
330
00:37:21.530 --> 00:37:23.230
of the model and the overall workflow?
331
00:37:23.930 --> 00:37:30.485
So I do think that collaboration is very core to what we're doing because we see this as being a tool not just for
332
00:37:30.785 --> 00:37:31.445
an individual
333
00:37:31.825 --> 00:37:34.965
data scientist or engineer, but a tool for an organization.
334
00:37:35.265 --> 00:37:37.690
Right? And so we do have different
335
00:37:38.150 --> 00:37:41.450
metaphors that we think about that relate to different stakeholders
336
00:37:41.829 --> 00:37:55.455
that you can see kind of visions of each of them in the platform. So for folks who are more on the analytics side, we do have a query editor built into the UI that lets you just start writing people or even ordinary SQL queries.
337
00:37:55.810 --> 00:37:58.710
The parser is expressive enough to kinda support both
338
00:37:59.010 --> 00:38:10.175
in the editor and kind of playing around with things as you would if you were using superset or some other, like, BI slash analytics tool. But we also then, for the kind of data scientists and the engineer personas,
339
00:38:10.715 --> 00:38:16.790
we have a lot of tools that kind of adhere to more of like a GitHub style workflow where, you know, folks
340
00:38:17.250 --> 00:38:34.430
will be able to kind of incrementally update models in a way that is, you know, versioned. And so you can kind of diff between different models and then have this ability to do experiments in separate branches. And then once you're happy with how the experiment is doing, saying, oh, this experiment is now doing better than what is currently in production.
341
00:38:34.890 --> 00:38:39.070
Let's, you know, merge this back into the main line similar to how you would
342
00:38:39.395 --> 00:38:50.160
in Git. And then, you know, that kind of becomes a concept very similar to a poll request where people can comment and kind of say, hey. I don't agree with this particular parameter choice. Can we maybe revisit this?
343
00:38:50.540 --> 00:38:57.200
So there are different ways that you make it more approachable to people that we've thought about doing it to make it more approachable to people by having
344
00:38:57.595 --> 00:39:10.750
those callouts that's gonna hearken back to things that they are already familiar with, but at the same time, giving them something that's net new. Right? Like, I think the problem with just, you know, Git using ML today is that Git doesn't provide a story around the
345
00:39:11.050 --> 00:39:28.680
non source code artifacts. Right? And so you need to use external tools for that, and things are not super tightly integrated. And that's where something like prediabase can slot in to fill that gap for that particular persona. Right? So it's all about large part providing the right metaphors for the right person that's using the tool.
346
00:39:29.059 --> 00:39:40.355
Digging into the Ludwig aspect of what you're building, as you mentioned, it's an open source project. It predates the business. You have used it as sort of the core building block of what you're providing.
347
00:39:40.920 --> 00:39:48.140
I'm curious if you can talk to some of the ways that you're thinking about the governance of the open source project and how you identify
348
00:39:48.995 --> 00:39:58.055
which pieces of the engineering that you're doing on and around Ludwig are part of the business and which parts belong with the open source project. And
349
00:39:58.400 --> 00:40:02.980
along with that, some of the ways that your work on prediabase has fed back into the Ludwig project.
350
00:40:03.520 --> 00:40:04.980
From the governance standpoint,
351
00:40:05.714 --> 00:40:12.855
we have been making a concerted effort to get more folks involved and, you know, we hold regular monthly meetings with the community.
352
00:40:13.480 --> 00:40:17.020
Talk about the road map, get buy in from different people about
353
00:40:17.320 --> 00:40:31.075
what features are important. So right now, 1 thing that we've been working on in the open source side, because a lot of other companies have been interested in is working on a model hub that provides, you know, some ability to share different trained Ludwig models and configurations.
354
00:40:31.890 --> 00:40:41.030
So that's something that's definitely been a community driven effort today. And then I would say that in terms of how we see the relationship between what's preda base and what's Ludwig,
355
00:40:41.454 --> 00:40:55.320
we do have a very substantial part of the engineering team that works almost exclusively on the open source. And so it's very important to us that, you know, we not just take Ludwig as something that we consume downstream, but that we also actively,
356
00:40:55.620 --> 00:41:12.950
you know, are investing back into Ludwig and making it better, that that will make credit based better by default. Right? But a really good example there is on the work we've done on scalability recently. You know, we have some customers that we've worked with that are training on larger datasets, you know, terabyte plus.
357
00:41:13.329 --> 00:41:15.750
And so we've had to think a lot about,
358
00:41:16.095 --> 00:41:27.650
you know, what are the bottlenecks in Ludwig. You know, Ludwig is a very complex system in a lot of ways. We deal with every type of modality of data at the same time potentially. Right? I need to have an efficient way to
359
00:41:28.030 --> 00:41:32.530
pipeline it all in terms of both the data processing and the model training and the prediction.
360
00:41:32.934 --> 00:41:36.315
And so we've invested quite a lot in building that out
361
00:41:36.694 --> 00:41:37.194
specifically
362
00:41:37.494 --> 00:41:44.309
to improve preda base. But the nice thing is all of those features then, you know, become ultimately part of Ludwig because that's where
363
00:41:44.609 --> 00:41:53.535
the core of those capabilities live. I'd say 2 other big features that are coming to Ludwig being driven by requirements on the credit based side
364
00:41:53.915 --> 00:41:59.560
have been 1 would be the improved AutoML capabilities that we've been investing in the tools. So this would be kind of suggesting
365
00:42:00.180 --> 00:42:00.680
configurations
366
00:42:00.980 --> 00:42:03.320
and suggesting hyperparameter search ranges
367
00:42:04.020 --> 00:42:06.680
based on the data, based on past trials,
368
00:42:07.195 --> 00:42:14.975
trainings, and things like that. And then the other is on the serving side. 1 thing we definitely found on Credit Base is that there's a very strong need
369
00:42:15.280 --> 00:42:18.660
to make sure that the serving environment is isolated
370
00:42:19.200 --> 00:42:22.020
and doesn't have tons of external dependencies
371
00:42:22.875 --> 00:42:25.855
that blow up the deployments and, you know, adds to your overhead.
372
00:42:26.395 --> 00:42:35.240
Since moving from TensorFlow to PyTorch last year, we've invested quite a lot in building out a Torch script layer for doing serving, which allows us to
373
00:42:35.620 --> 00:42:37.720
strip out all of the Python dependencies
374
00:42:38.100 --> 00:42:40.200
on Ludwig at serving time
375
00:42:40.515 --> 00:42:42.055
and provide a very low latency
376
00:42:42.755 --> 00:42:47.494
end to end servable that does not only model inference, but also does the preprocessing,
377
00:42:47.875 --> 00:42:49.095
so the data transformation
378
00:42:49.839 --> 00:42:51.300
as well as the post processing.
379
00:42:51.839 --> 00:43:00.905
And this is something that, you know, was very core to what we're doing in front of base, but we've made all of that open source as part of Ludwig as well. So now the community can take advantage of it as well.
380
00:43:01.285 --> 00:43:02.425
In terms of
381
00:43:02.965 --> 00:43:03.945
the early applications
382
00:43:04.485 --> 00:43:15.500
of the preda based platform that you're building and how you've been working with some of your early design partners, I'm wondering what are some of the most interesting or innovative or unexpected ways that you've seen the platform used.
383
00:43:15.855 --> 00:43:22.035
Yeah. There have definitely been some ones that surprised us. So we definitely expected that
384
00:43:22.415 --> 00:43:29.060
Tabular was going to be a very important use case for folks, And so we invested a lot in making sure that we had state of the art
385
00:43:29.360 --> 00:43:33.220
architectures and capabilities on tabular data. And so that turned out to be
386
00:43:33.525 --> 00:43:36.665
true, but we've also found there are quite a lot of really interesting
387
00:43:37.285 --> 00:43:40.345
unstructured datasets that people have been working with as well,
388
00:43:40.650 --> 00:43:48.670
where they're trying to predict, you know, anomalies and like image data, very large image data sets, or doing kind of really interesting
389
00:43:48.970 --> 00:43:49.789
mixed modality
390
00:43:50.315 --> 00:43:52.415
training with, like, text and tabular.
391
00:43:52.955 --> 00:44:02.230
We've also found that there are a lot of situations where users wanna do machine learning training without a lot of labeled data. And that's, I think, a particularly interesting 1 because
392
00:44:02.690 --> 00:44:13.035
it's been leading us to invest a lot more heavily and building out self supervised learning capabilities into Luke Lake. And so, you know, 1 thing that we're working on actively right now
393
00:44:13.414 --> 00:44:24.730
is building out a really sleek pre training API for Ludwig so that you can, without needing to specify, you know, a target call or anything like that, do some initial training to learn a good representation of the data
394
00:44:25.110 --> 00:44:31.994
that you can then apply downstream to a lot of different tasks. And so that's 1 that has definitely been informed by
395
00:44:32.295 --> 00:44:38.359
what we've been seeing from customers as being a a very critical need for them. That's now informing a lot of product roadmap.
396
00:44:38.819 --> 00:44:39.880
In your own experience
397
00:44:40.260 --> 00:44:41.559
of going from
398
00:44:41.940 --> 00:44:52.405
working at Uber and helping to solve the problems that they have for machine learning and producing these useful open source projects that have been available to the community
399
00:44:52.890 --> 00:45:05.994
and then turning that into building a business around those capabilities and from the lessons that you learned in Uber. I'm wondering if you can just talk to some of the most interesting or unexpected or challenging lessons that you've learned in the process of building prediabase.
400
00:45:06.775 --> 00:45:09.035
Yeah. So I would say that
401
00:45:09.460 --> 00:45:11.000
there have been some really interesting
402
00:45:11.540 --> 00:45:19.345
problems that mirror a lot of problems that we encountered at Uber. So I think that when you look back at my time at Uber,
403
00:45:19.885 --> 00:45:24.305
the story there was that I was very keen on unification
404
00:45:24.765 --> 00:45:25.505
of infrastructure.
405
00:45:26.289 --> 00:45:41.815
And so 1 of the things that I was really heavily pushing towards the end of my time there was on moving away from a kind of spark and then plus random bespoke like training architecture built on top of or about and some other things towards using Ray as a unified
406
00:45:42.115 --> 00:45:42.615
infrastructure
407
00:45:42.915 --> 00:45:45.770
layer. And so that very heavily informed
408
00:45:46.309 --> 00:45:51.050
the direction that we took with Credit Base in terms of building out our training
409
00:45:51.430 --> 00:45:51.930
system
410
00:45:52.395 --> 00:45:56.255
of being capable of being this single compute cluster
411
00:45:56.875 --> 00:45:58.975
that is capable of doing the preprocessing,
412
00:45:59.355 --> 00:46:00.095
the training,
413
00:46:00.619 --> 00:46:16.775
batch prediction, kind of the whole thing end to end. That's worked out really well. And then, you know, when we were starting to build credit base, then we had to take this kind of data plane that came from all these years of working at Uber and, like, all the lessons that we learned along the way. And then think about, okay. Now how are we gonna make this into
414
00:46:17.109 --> 00:46:18.410
a truly serverless
415
00:46:18.710 --> 00:46:30.155
enterprise experience. Right? And so we did a lot in terms of the early days of, like, building out the control plane layer. I think there were quite a lot of lessons we learned along the way about
416
00:46:30.615 --> 00:46:32.475
how you should think about coupling
417
00:46:32.775 --> 00:46:36.075
in these sorts of big complex distributed systems
418
00:46:36.830 --> 00:46:44.610
where, you know, we had early on some, like, interface boundaries between the control plane and the data plane that were not particularly well
419
00:46:45.095 --> 00:46:50.154
defined. You know, there was a lot of tight coupling. So sometimes failures would occur, and
420
00:46:50.615 --> 00:47:03.474
certain things that should not have failed would fail because there was too much coupling in there. And what we've done over time is rearchitect the platform to be much better isolated so that we use more kind of event driven architecture.
421
00:47:03.934 --> 00:47:08.194
So more message brokers and things like that that kind of make things very cleanly separated.
422
00:47:08.494 --> 00:47:09.875
And that's been a very big
423
00:47:10.280 --> 00:47:30.089
learning building an enterprise platform is, you know, how important it is to really define the service boundaries well between the different components of the system. And overall, you know, we found that reliability, robustness, stability, these have been, like, concerns that when you start building the company, you don't initially think, oh, yeah. These are gonna be the top things that I'm gonna put on the road map. Right?
424
00:47:30.390 --> 00:47:45.710
But now that's definitely, like, top of mind for us at all times is, you know, how do we build the platform in a way where we account for as many things going wrong as possible and have a story around making sure that at the end of the day, the user gets a very clean and a very responsive
425
00:47:46.170 --> 00:47:50.030
experience, right, that doesn't fail in some weird unexpected way.
426
00:47:50.365 --> 00:47:59.345
Because of the fact that you are running a large and scalable and multi cloud system with a lot of distributed systems going on, I'm curious
427
00:47:59.730 --> 00:48:02.869
how you have approached the kind of testing and validation
428
00:48:03.170 --> 00:48:08.630
so that as you iterate on the product, you're able to very quickly get feedback as to
429
00:48:08.944 --> 00:48:16.645
whether a change has caused your aggression in terms of your, you know, ability to quickly recover or being able to identify
430
00:48:17.150 --> 00:48:20.849
potential issues with fault tolerance and just how you're able to think about
431
00:48:21.230 --> 00:48:21.730
managing
432
00:48:22.430 --> 00:48:29.955
forward progress and iterative development on the platform while ensuring that you maintain those principles of stability and scalability and fault tolerance?
433
00:48:30.575 --> 00:48:32.995
Yeah. That has been, I'd say, 1 of the more
434
00:48:33.400 --> 00:48:43.435
difficult challenges to solve. I'd say that we're still figuring out the right way to think about some of these things. But we've definitely invested quite a lot in both
435
00:48:43.735 --> 00:48:45.195
ensuring, like, the benchmarking
436
00:48:45.575 --> 00:48:51.595
of the Ludwig sign. And so there's an active project from 1 of our employees working on building out an entire
437
00:48:52.040 --> 00:48:55.740
benchmarking pipeline for Ludwig so that every time a change happens,
438
00:48:56.200 --> 00:48:59.900
we can, you know, validate it against different workloads and make sure that
439
00:49:00.205 --> 00:49:08.225
model performance is good, GPU utilization is good, memory utilization is good, that sort of all the kind of metrics that we care about for the workload
440
00:49:08.620 --> 00:49:10.960
are there at that level so that we know
441
00:49:11.500 --> 00:49:32.059
that, okay, it's not a change in Ludwig that is causing memory to spike or something like that after this change is made. So that's, I think, the first aspect we have to get right is, like, making sure that the open source is very stable and, you know, meets the requirements that we have set. And then from there, we have quite a lot that we've done on the platform side in terms of building out
442
00:49:32.359 --> 00:49:33.339
continuous integration
443
00:49:33.720 --> 00:49:35.740
and different tiers of deployments
444
00:49:36.175 --> 00:49:40.595
for the whole system to make sure that it's all well tested before we do a release.
445
00:49:40.974 --> 00:49:44.915
So we do have a regular release cadence that we have set up with our customers.
446
00:49:45.570 --> 00:49:50.550
Every change we make goes into a live staging environment that we test out internally,
447
00:49:51.090 --> 00:50:06.420
goes through a full battery of integration tests that actually run on a Kubernetes cluster on, you know, live compute resources and make sure that all the different models that we, you know, regularly test out are working correctly and, not failing in any unexpected ways.
448
00:50:06.799 --> 00:50:16.935
And then we've also invested a lot on the observability side as well, in terms of making sure that we know, okay, so if this workload used to take a minute to run and now it takes 5 minutes,
449
00:50:17.235 --> 00:50:29.655
you know, what's the part of the system that's suddenly taking longer? Like, what's the part of the system that's suddenly taking more memory? Right? Be able to see what that trend line looks like and what the inflection point was. Right? And so
450
00:50:29.955 --> 00:50:33.175
that's been a big area of focus for us lately because
451
00:50:33.555 --> 00:50:45.070
it's just very important for us to ensure that as we get more and more people contributing to the code base and more and more moving parts, that we identify as quickly as possible, like, when something changes
452
00:50:45.375 --> 00:51:07.165
and then can go back and and address it. Right? So having every single commit go through the full CI process has been very critical to that. And I think pretty good policies in place where, you know, we make sure that we don't commit anything to the main line if, you know, the tests aren't in a good state and that, you know, we always make sure that we prioritize stability and bug fixes above
453
00:51:07.545 --> 00:51:14.490
new feature development. So all of those best practices, I think, are very key to getting it right. It's still something we're learning as we go.
454
00:51:14.790 --> 00:51:15.850
And so for
455
00:51:16.310 --> 00:51:17.690
individuals or organizations
456
00:51:18.230 --> 00:51:20.010
that are looking to be able to
457
00:51:20.555 --> 00:51:25.375
accelerate the rate at which they're able to experiment with and adopt machine learning to
458
00:51:25.675 --> 00:51:26.815
address some of the
459
00:51:27.115 --> 00:51:27.615
organizational
460
00:51:27.915 --> 00:51:32.830
and product problems that they're trying to solve for? What are the cases where Predabase is the wrong choice?
461
00:51:33.290 --> 00:51:36.830
I mean, that's, I think, a very valid question. And I think there are definitely
462
00:51:37.625 --> 00:51:41.005
times when it might not be the right choice for your organization.
463
00:51:41.705 --> 00:51:44.605
When we think about, like, where the market segments,
464
00:51:44.910 --> 00:51:54.290
you know, you can kind of think of it in 4 quadrants, I guess. Right? On the 1 hand or maybe 2 axes to make it sound like, on the 1 hand, you have organizations that have low data
465
00:51:54.755 --> 00:52:04.375
versus organizations that have high data. And then on the other axis, you have organizations that have high ML experience and low ML experience. Right?
466
00:52:04.740 --> 00:52:12.200
And so definitely, you know, the bread and butter customer for us would be a company that's very high in terms of, like, data volume and quantity,
467
00:52:12.645 --> 00:52:24.700
but not as high in terms of, you know, having a big sophisticated ML team. Can certainly have an ML team, but, you know, not 1 that wouldn't want to necessarily say that, like, Google Research would be a target customer. That's right.
468
00:52:25.560 --> 00:52:27.500
And then on the flip side, you have organizations
469
00:52:27.880 --> 00:52:37.775
that maybe don't have a lot of data at all. And certainly, I think there are companies out there that are trying to think about ways that they can bring in all to companies that don't have data.
470
00:52:38.220 --> 00:53:07.994
But, you know, for specialized use cases where it's like using pre trained models and things like that. But that's not what we're currently looking at. We definitely still are thinking, you know, companies that have a lot of data and don't quite know how to get enough value out of it. That's very core to to what we do well. Right? And I would also say that it's very important for a customer prediabase to have some variety of of use cases that they wanna solve. You know, it's definitely not a prerequisite, but I would say that when you look at the market, there are companies that
471
00:53:08.295 --> 00:53:15.300
only do fraud detection or only do computer vision or something like that. And, you know, I won't necessarily wanna say
472
00:53:15.600 --> 00:53:25.555
that private base is gonna beat all of them all the time on every task. Right? Like, what I would say is that we provide a very good solution for time to value relative
473
00:53:26.095 --> 00:53:32.070
to these other platforms. Right? If you have a good variety of different things you wanna do on the ML space.
474
00:53:32.450 --> 00:53:36.550
So certainly, I think if you wanna do, you know, computer vision and NLP
475
00:53:37.175 --> 00:53:46.715
from, like, a purely cost benefit standpoint, I think that we have a much stronger value proposition there than if you were to try to do point solutions for all of these different things. Right? So
476
00:53:47.110 --> 00:53:51.930
that's the other aspect that's maybe less of a hard requirement, but still, I think an important differentiator.
477
00:53:52.950 --> 00:53:53.770
As you continue
478
00:53:54.150 --> 00:54:05.515
to iterate on the product and now that you have gone out of stealth and you're starting to accept new customers onto the platform, what are some of the things you have planned for the near to medium term or any particular
479
00:54:06.130 --> 00:54:09.190
areas of focus or new features that you're excited to dig into?
480
00:54:09.650 --> 00:54:14.869
I'm certainly very excited about having a fully SaaS version of the product that people can try out.
481
00:54:15.265 --> 00:54:39.494
Right now, we're in a closed beta. So, you know, we are certainly really excited when people come to us and say they wanna try it out, and we'll set up some time to do a pilot with them. But I'm, you know, very excited about the possibility of having a website people should just log in to, start using it, you know, without any commitment. Right? And so that's something that we're definitely working on right now and and thinking about how we can put that in people's hands.
482
00:54:39.795 --> 00:54:45.890
From a product standpoint, there's also a lot that we're thinking about now. So I mentioned the self supervised learning work before.
483
00:54:46.270 --> 00:54:50.050
There's also some work that we're doing to the open source community as well around better
484
00:54:50.515 --> 00:54:52.375
support for custom components
485
00:54:52.675 --> 00:55:05.210
and kind of user defined functions, if you will. So, you know, with Ludwig and prediabase, there's quite a lot of flexibility in terms of, like, your degree to specify, like, every parameter of the model. But if you wanna add new model architectures,
486
00:55:05.829 --> 00:55:14.005
it's possible today, but we wanna make that experience even easier for folks so that it's just a very lightweight interface you implement and then you can register that component
487
00:55:14.490 --> 00:55:20.030
as just another option in your config within prediabase that other people in the organization can use.
488
00:55:20.410 --> 00:55:24.225
And then also this concept of the model hub slash model registry,
489
00:55:24.865 --> 00:55:32.245
I think is 1 that I'm very excited that will be provide benefits for both the open source users as well as the commercial users
490
00:55:32.640 --> 00:55:35.060
where you can do things like define canonical
491
00:55:35.920 --> 00:55:43.285
components that you wanna use in your organization. So if there's, like, a feature that gets used all the time in different models. If I remember at Uber,
492
00:55:43.585 --> 00:55:46.645
we had some features, like, related to, like, customers
493
00:55:47.025 --> 00:55:48.405
related to, like,
494
00:55:48.865 --> 00:56:05.065
locales that were just used in all different types of models. Right? So being able to have canonical encoders for those that are maybe pre trained even on, you know, a very large datasets, so there's very low cost to fine tuning them. I'm very excited about building out that capability as well.
495
00:56:05.365 --> 00:56:18.890
Well, for anybody who wants to get in touch with you and follow along with the work that you're doing, I'll have you add your preferred contact information to the show notes. And as a final question, I'd like to get your perspective on what you see as being the biggest barrier to adoption for machine learning today.
496
00:56:19.305 --> 00:56:24.444
So definitely, I think that there is a very big barrier to adoption that comes from just
497
00:56:24.984 --> 00:56:25.805
not having
498
00:56:26.160 --> 00:56:27.460
good enough abstractions
499
00:56:27.840 --> 00:56:36.444
to start getting value out of machine learning. So I think the analogy that I like to draw here is really about software and kind of what's enabled
500
00:56:36.904 --> 00:57:05.079
software to eat the world as the famous article in the Wall Street Journal once said. Right? And it really comes down to this idea of modularity and being able to kind of stand on the shoulders of giants. So instead of being able to reimplement every, you know, great new idea that comes along, like you just download a library and use that software. I think that ML hasn't had this abstraction before. Right? And I think that it's been a very big inhibitor to people actually being able to adopt it as you know, great new idea comes out from research, but
501
00:57:05.380 --> 00:57:18.415
companies aren't able to productionize it and actually get it to deliver value because they're too busy trying to reinvent the wheel and reinventing the infrastructure and figuring out how to get data from 1 place to another, clean up their data.
502
00:57:19.020 --> 00:57:23.119
So definitely, I think having better abstractions and better canonical
503
00:57:23.500 --> 00:57:44.000
sources of data as well are the 2 biggest barriers in my opinion. So I think once you get to a point where all the data is clean and then standard data warehouse systems and is ready for machine learning, and then you have very powerful abstractions like CrediBase that allow you to take best in class models and just run it right on this, you know, nice clean canonical data source,
504
00:57:44.300 --> 00:57:48.000
then you'll have a very, very fast pass to production. And so,
505
00:57:48.355 --> 00:58:05.970
we definitely think we can move the needle on the modeling side. And I think, you know, certainly companies like DBT, Snowflake, others are doing a great job on the data side. Then once these 2 things converge, then hopefully we'll be able to really start, you know, delivering more value. But that's definitely, I think, where companies struggle the most today.
506
00:58:06.444 --> 00:58:13.984
Alright. Well, thank you very much for taking the time today to join me and share the work that you've been doing at prediabase. It's definitely a very interesting
507
00:58:14.285 --> 00:58:24.530
platform and product that you're building there. So I'm excited to see where you go from here. So thank you again for all the time and energy that you and your team are putting into making it easier for organizations
508
00:58:25.795 --> 00:58:37.820
to get onboarded with ML and be able to experiment with it and gain some of the value from its capabilities. So thank you again for that, and I hope you enjoy the rest of your day. Awesome. Thank you, Tobias. I really appreciate it and as well.
509
00:58:40.200 --> 00:58:53.220
Thank you for listening. Don't forget to check out our other shows, the Data Engineering Podcast, which covers the latest on modern data management, and the Machine Learning Podcast, which helps you go from idea to production with machine learning. Visit the site at pythonpodcast.com
510
00:58:54.320 --> 00:59:02.085
to subscribe to the show, sign up for the mailing list, and read the show notes. And if you learned something or tried out a project from the show, then tell us about it. Email hostspythonpodcast.com
511
00:59:03.505 --> 00:59:04.325
with your story.
512
00:59:04.704 --> 00:59:05.920
And to help other people