WEBVTT
00:00:02.879 -->
00:00:14.960The AI Fundamentalists, a podcast about the fundamentals of safe and resilient modeling systems behind the AI that impacts our lives and our businesses.
00:00:15.599 -->
00:00:19.920Here are your hosts, Andrew Clark and Sid Mongelik.
00:00:20.239 -->
00:00:22.879Welcome to today's episode of the AI Fundamentalists.
00:00:23.039 -->
00:00:31.679Today we are excited to bring on Christoph Molner, a returning guest of the show and an expert in machine learning and interpretability.
00:00:31.920 -->
00:00:40.880He's now written his fifth book on machine learning, and we highly recommend his mindful modeler newsletter on Substack to listeners of this podcast.
00:00:41.119 -->
00:00:42.960Christoph, welcome back.
00:00:43.520 -->
00:00:45.439Thanks for having me again.
00:00:46.799 -->
00:00:48.399Yeah, we're so glad to have you back.
00:00:48.479 -->
00:00:55.520And you've been on a bit of a streak now on your newsletter talking about these tabular foundational models.
00:00:55.600 -->
00:00:56.960And I think they're really exciting.
00:00:57.039 -->
00:01:00.240And I think that it's definitely time for us to start talking about them.
00:01:00.560 -->
00:01:03.920So I'll give a quick little intro and then we can start digging into this.
00:01:04.079 -->
00:01:10.560So just for the audience, tabular foundational models might be the next frontier in tabular prediction.
00:01:10.719 -->
00:01:20.640This field has been traditionally dominated by stuff like tree-based algorithms, XGBoost like GBM, and in some simpler use cases, just logistic and linear regression tasks.
00:01:21.280 -->
00:01:23.599This is a fundamentally different way of doing this.
00:01:23.760 -->
00:01:28.000Do you want to start walking us through how this method works and why it might be better?
00:01:28.319 -->
00:01:34.079Yeah, for me, it has been also learning experiments over the last few weeks, and just by writing about it.
00:01:34.319 -->
00:01:39.760As I learned about Tabler Foundation models, I also wrote my in my blog about it.
00:01:40.159 -->
00:01:45.760And yeah, it was quite a different way of thinking about supervised machine learning.
00:01:46.079 -->
00:01:59.040So in traditional like regression classification, we have this train and predict paradigm where we train a model with an algorithm, get a model, use it for prediction.
00:01:59.200 -->
00:02:03.280And tabular foundation models change how we approach this.
00:02:03.359 -->
00:02:12.800So these are basically large neural networks based on transformer models, which are now pre-trained on a lot of data.
00:02:13.120 -->
00:02:21.919And instead of then skipping the actual training process and doing just prediction, which is basically in context learning.
00:02:22.000 -->
00:02:33.360So you provide at prediction time the entire training data sets plus your data test data set, and in a single forward pass, push it through these tabular foundation models to get out the predictions.
00:02:33.520 -->
00:02:36.879And during this process, there's no parameter updates or anything.
00:02:37.039 -->
00:02:48.319So all the learning happened before, and during this prediction new type of prediction step, the model basically does in context learning.
00:02:48.400 -->
00:02:53.039So it compares the test data to the training data to make to get to the predictions.
00:02:53.199 -->
00:03:01.360So coming from this like how we used to think in traditional machine learning is quite a paradigm shift.
00:03:02.159 -->
00:03:07.439Yeah, and I think for listeners at home, I think that you might be hearing some really deep parallels with LLMs.
00:03:07.520 -->
00:03:10.159And that's because this paradigm is actually extremely similar.
00:03:10.319 -->
00:03:15.439The same idea with LLMs is that we don't want to train an LM for every little task we do.
00:03:15.599 -->
00:03:37.439There should be some foundational model underneath it trained on a massive amount of highly diverse data, which we can then use to either just run inference directly in our tasks or then do a little bit of extra fine-tuning on tap on top for, let's say, insurance use cases or for casualty use cases, or really anything else that we that these tabular are used for, even soft time series.
00:03:37.520 -->
00:03:39.840And we'll get into that a little bit later.
00:03:40.080 -->
00:03:50.400For people out there who are looking to just hop off this podcast and start doing things, some examples might be like Cart or XTAB or Tab LLM or Tab PFN or Table CL.
00:03:50.560 -->
00:03:54.479There's a lot of great models out there, and I'm sure by the time we shoot this, two more will already come out.
00:03:54.560 -->
00:03:56.479It's a very happening space right now.
00:03:56.719 -->
00:04:07.919But I think one thing that's going to be really interesting that you talk about in your posts is unlike with language data, where the scope of human language is so vast, we have so much language to build a great probabilistic model.
00:04:08.319 -->
00:04:13.520How do you get enough data to train a foundational tabular model?
00:04:14.159 -->
00:04:34.319Yeah, I think that's also a very exciting part of all this development with tabular foundation models, and also change my view on synthetic data because there's different ways in which you can train these tabular foundation models, but a lot of them are trained purely on synthetic data.
00:04:34.560 -->
00:04:49.839So it can be shown that it helps to also train them on real data, but you get a lot of performance already out of training them on synthetic data, meaning you have a kind of a process from which you draw new data sets.
00:04:50.079 -->
00:05:05.839Most of these work with structural causal models where you sample nodes and edges, and the nodes represent your features and the edges, the relationship between them, and you sample from those and then create a like a huge data set.
00:05:05.920 -->
00:05:08.160So we're talking millions of data sets.
00:05:08.399 -->
00:05:12.800And these are then all used in pre-training these models.
00:05:13.120 -->
00:05:26.480And I think it's quite interesting that this actually works because you have to come up with a data generating process that helps then later to make actual predictions with real data.
00:05:26.720 -->
00:05:39.360And many of these table foundation models are only trained on synthetic data and still deliver very high performance, like good predictive performance on real data, which I find very surprising.
00:05:40.240 -->
00:05:44.879So this was certainly also like a new thing to me that this works so well.
00:05:47.040 -->
00:05:47.600Yeah.
00:05:48.079 -->
00:05:57.759One thing that I just conceptually at a very high level, thinking about it without the architecture, it does sound like they make more sense than an LML, like how it could work this way.
00:05:58.079 -->
00:06:15.360Because like it texts like the potential, if you think like a matrix of numbers and things, the having being able to have really robust, we don't know exactly what's happening around these things, but the covariance and correlation analysis between the inputs and like the target is to Sid's point, it seems a little bit more bounded than you would have in the language context.
00:06:15.519 -->
00:06:21.839So, like a at a non-mathy level, it does intuitively make sense that these this could be an interesting paradigm.
00:06:22.480 -->
00:06:26.240Yeah, and also if you look at these the architectures of what they do.
00:06:26.319 -->
00:06:30.879So we have this transformer-based architectures with the attention mechanism.
00:06:31.199 -->
00:06:36.959And while with large language models, as you said, we have these diverse like text as input.
00:06:37.279 -->
00:06:46.879With tables, we also have some different types of diversity, like the that they can come with different number of columns, for example, and different types of columns.
00:06:47.199 -->
00:07:18.959So how most of these foundation models work is that they have attention mechanisms that can work between like uh within the row or within a column, and basically to make predictions, then later they accumulate for so they represent internally each table cell basically with an embedding vector, and enrich this vector like during multiple transformer steps by paying attention to other columns and other rows.
00:07:19.600 -->
00:07:32.720Yeah, that's the basic unit on which these uh transformer-based networks for tabular data make predictions then or build up kind of the information needed.
00:07:33.120 -->
00:07:42.720And when digging into your post, one thing that I thought was really interesting and love for us to touch on as well is the fact that it uses like some structured causal modeling under the scenes as well to help do the relationships.
00:07:43.360 -->
00:07:47.519Yeah, that's for the creating the synthetic data actually.
00:07:47.759 -->
00:07:59.040So also what's interesting, these are called priors because you can motivate these tabular foundation models with the pre-training and then context learning later through Bayesian inference, interestingly.
00:07:59.120 -->
00:08:35.919So that you're basically do modeling the posterior predictive distribution of your Y test, like your target, and your the prior is basically your the data sets or the mechanism that generates your data sets, and many use these structural causal models, which kind of makes sense because then I think these could be or seemingly are a good model of how tabular data actually is created in reality, that we have some kind of analyzed underlying causal model.
00:08:36.159 -->
00:08:50.000So we have so structural causal model is in part can be represented by a direct acyclic graph, that you have nodes which represent variables which are later than features of the target and edges between them.
00:08:50.159 -->
00:09:00.639And these are directed edges, so we can say that one node influences the other, so they later in your tablet data set, it would mean that you have correlated columns.
00:09:01.120 -->
00:09:08.399And by using these structural causal models, you can create a lot of diverse data sets.
00:09:08.480 -->
00:09:16.559You can create small ones, large ones, ones with highly correlated features, and so on, and also with different data types.
00:09:16.879 -->
00:09:25.120And in this data generating process, they also usually sample the connections between these features.
00:09:25.279 -->
00:09:26.639Is it a linear relationship?
00:09:26.720 -->
00:09:31.039Is it maybe a tree-based relationship with a huge jump and so on?
00:09:31.519 -->
00:09:41.360So they also create so these labs that train or pre-train these models create a diverse set of these data sets to train on.
00:09:42.720 -->
00:09:51.840And this really rings true with an episode we just did in the podcast about metaphysics and causality, which is this idea that there's networks of causal relationships.
00:09:52.000 -->
00:10:00.000So we might look at like height, weight, smoking, and how that interacts with heart attacks, and then how heart attacks interact with mortality.
00:10:00.240 -->
00:10:02.240And it's more than just simple correlation.
00:10:02.320 -->
00:10:05.519Like there's actually probabilistic dynamics between these variables.
00:10:05.679 -->
00:10:14.000And if we can model this in synthetic data, we can create a kind of synthetic data which is actually learnable and actually delivers real insights to tabular models.
00:10:14.159 -->
00:10:19.279We've talked a lot about synthetic data on this podcast and how like synthetic language data is actually really questionable, right?
00:10:19.360 -->
00:10:24.559Because it doesn't necessarily capture a lot of this diversity or a lot of these like relationships in language.
00:10:24.879 -->
00:10:30.559But in math and in numbers, we can really directly derive those relationships out in data again.
00:10:30.879 -->
00:10:41.759Yeah, also I think it's really interesting because with if you have tables, you capture like your variables in whatever industry you're in.
00:10:42.639 -->
00:10:50.559But they are generated through some process which you you can make assumptions about and like from your domain knowledge, okay, A influences B and so on.
00:10:50.799 -->
00:11:04.320And I think these structural models, structural causal models, are a good or seemingly a good way to mimic this, that how data in the real world is generated.
00:11:04.480 -->
00:11:22.000So what they also do, which I find is interesting, is that they also sample like which of these nodes then later become a feature, or only a set like a subsample of nodes from such a graph are actually then used in a table that is then later used in pre-training the tablet foundation models.
00:11:22.159 -->
00:11:36.960So then we have these effects of hey, the target could actually be like a cause for another feature if we think in causality and not the other way around, or that we maybe don't observe all the confounders.
00:11:37.039 -->
00:11:54.080If we again think in causal language in general, that we only observe like our causal network partially, so we don't have all the knowledge, which is true for the real world, that we sometimes cannot measure all the features that would be necessary, and we don't know all the causal structures, but only observe part of it.
00:11:54.240 -->
00:12:00.000And also the during pre-training, the model doesn't know this structural graph.
00:12:00.080 -->
00:12:03.440It's just a table that goes into the pre-training.
00:12:04.799 -->
00:12:08.080And uh, you mentioned this is just to round back on scale.
00:12:08.399 -->
00:12:10.000Foundation models can be a little bit scary.
00:12:10.159 -->
00:12:11.759We talk about these language foundation models.
00:12:11.840 -->
00:12:15.200These are like, as far as we know, trillion parameter models.
00:12:15.360 -->
00:12:20.799These don't fit on a single GPU, these have to go in an entire cluster just to run inference in one of these models.
00:12:21.120 -->
00:12:24.320With these models, we're talking about millions of tabular data sets.
00:12:24.480 -->
00:12:27.120How much training data is going in and how big are these final models?
00:12:27.279 -->
00:12:29.279Could I just run this on my laptop?
00:12:29.919 -->
00:12:30.639Yeah, you can.
00:12:30.879 -->
00:12:41.840So these I don't have I don't know the parameter numbers by heart, but like the these the models, like the model weights are like a couple of hundred megabytes.
00:12:42.159 -->
00:12:46.639And I tried a few of them on my MacBook M1.
00:12:46.720 -->
00:12:56.080So without a graphic card, just on the CPU, can be a tad slow, so it's definitely slower than using boosted trees, for example.
00:12:56.320 -->
00:12:58.720But you can run this on your machine, yeah.
00:12:59.919 -->
00:13:01.279Yeah, I think that's really exciting.
00:13:01.360 -->
00:13:08.559I and I think that if we can just use ScikitLearn, download this model, maybe it's just like a gigabyte large, that makes this extremely accessible.
00:13:08.720 -->
00:13:13.840And so even if these aren't better, I think that this makes them absolutely worth being part of our toolbox.
00:13:14.159 -->
00:13:27.360But digging in a little bit on what you were saying, if this model now just exists on our computer as something that we can just immediately run predict on, we no longer need to do this fit piece.
00:13:27.679 -->
00:13:32.320Do you think that this moves us a little bit more towards a prompting era for tabular models?
00:13:32.639 -->
00:13:38.240Or like an era where like people are just doing few shot learning, like really just here's 10, 20 samples.
00:13:38.960 -->
00:13:52.879I think the prompt analogy only works in part because we don't the prompt is the training date, like the your data points, and you can definitely like play around with this.
00:13:53.039 -->
00:13:58.080So you're providing a smaller context, which basically means smaller training data part.
00:13:58.240 -->
00:14:06.480And when I say training data in this context, it means the label part of your data that goes into the in-context learning.
00:14:06.720 -->
00:14:12.080And you can think of it if you have a data set, you paste basically together your training and test data.
00:14:12.320 -->
00:14:35.679Actually, you keep it separated because they most of these Chapel Foundation models still use the scikit-learn API language, but internally it's basically paste it together then, or concatenated, and then the test data can pay attention basically to the training data to so for the model to make predictions.
00:14:35.840 -->
00:14:41.120And of course, you can now change the context for your data for making predictions.
00:14:41.279 -->
00:14:52.240That's also one of the tricks because inference is expensive with Tabula foundation models, especially compared to training, because training like the fit process doesn't exist really.
00:14:52.320 -->
00:14:56.240It's basically loading model weights and doing some pre-processing of the data.
00:14:56.399 -->
00:14:58.639Pre-training again is expensive, of course.
00:14:58.799 -->
00:15:05.200But if you already have the pre-trained model, then you no longer need the training process, but only do the prediction.
00:15:05.279 -->
00:15:09.360And this is more expensive than boosted trees, at least for now.
00:15:09.600 -->
00:15:10.960Yeah, and then you can.
00:15:11.039 -->
00:15:17.679So one of the tricks to bring this time down to make faster predictions is to provide a smaller context window.
00:15:17.759 -->
00:15:24.159So maybe you know that a subset of your training data is enough to provide as context for your predictions.
00:15:24.320 -->
00:15:26.879So there's a few of these tricks that you can use.
00:15:27.200 -->
00:15:39.600Yeah, and in terms of practical performance, then when I use this kind of model, I'm definitely finding like we're getting like 90, 80, sometimes 100% as accurate as a traditional model, and I'm not having to do any training, which is really wonderful.
00:15:39.759 -->
00:15:44.399But on some use cases, I'm definitely getting significantly degraded performance.
00:15:44.639 -->
00:15:47.600What's our recourse for trying to fine-tune one of these models?
00:15:47.679 -->
00:15:49.039Is that accessible?
00:15:50.720 -->
00:15:53.679So these are neural networks that you can fine-tune.
00:15:53.759 -->
00:15:59.200And I think that's one of the big selling points that you can fine-tune them if you have very specific data.
00:15:59.519 -->
00:16:04.080And yeah, beforehand, you also don't know will it work for my case, will it not work?
00:16:04.399 -->
00:16:17.759And in many cases, the Tableau Foundation models are at beating or on par with state of the art, like the which are mostly boosted trees, but it's of course not guaranteed.
00:16:18.159 -->
00:16:25.600In another case, you might have that it doesn't outperform them or much worse, actually.
00:16:25.759 -->
00:16:38.960And yeah, then there's a potential that you can fine-tune those models on very specific tabular data that you might have that may look similar to what your data looks like.
00:16:39.200 -->
00:16:52.320But this also means that you have to put in the assumptions like to generate new data, or if you want to use synthetic data, or maybe you have a lot of similar data sets already labeled that you can use for fine-tuning.
00:16:52.480 -->
00:17:02.320So this really depends then if you have other data that looks like your data set, or you can create synthetic data which has the similar structure.
00:17:03.600 -->
00:17:07.839Yeah, this definitely starting to smell a little bit like a business, or like what we saw with Medbert.
00:17:08.240 -->
00:17:14.960It sounds like there's room for organizations to collect a lot of the synthetic data for a specific use case and make their own tabular foundational models.
00:17:15.119 -->
00:17:16.880Have you seen any of that going around?
00:17:17.440 -->
00:17:26.799Yeah, there's actually a lot of companies now, or startups that center on training these foundational models.
00:17:26.960 -->
00:17:30.559Something I haven't seen with other types of models.
00:17:30.720 -->
00:17:33.920Like the I haven't seen any specific random forest company.
00:17:34.079 -->
00:17:37.279Probably there's one out there, and I'm like no complaining.
00:17:37.440 -->
00:17:40.720But there's now a lot of startups pre-training these models.
00:17:40.799 -->
00:17:49.519And yeah, one of the I think the main things to you, or IP or what's valuable is really the prior.
00:17:49.599 -->
00:17:55.839Like, how do you generate the data to then cover certain use cases?
00:17:56.640 -->
00:17:57.279Yeah.
00:18:00.079 -->
00:18:06.960And then I guess a question for them and for us here is it sounds like we might be taking on some trade-offs of these models.
00:18:07.359 -->
00:18:14.000Since we are going to a foundational model, we are inherently putting a black box in our pipeline, right?
00:18:14.079 -->
00:18:18.480There's some part of the model which we don't fully understand the mechanisms of.
00:18:18.559 -->
00:18:23.440This is just because transformer models are so dense and they're neural networks, and we can run inference on them.
00:18:23.599 -->
00:18:24.880But what's our what are our options?
00:18:24.960 -->
00:18:32.799How can we get back some of our explainability in a space where with trees and with boosting, we have like almost perfect explainability?
00:18:33.680 -->
00:18:54.000In a way, we are in the same situation because the reason why we have the interpretability is because of arguably they're also black boxes, and we just learn to use a lot of tools like feature importance or Shapley values or partially pendence slots to then increase the interpretability.
00:18:54.319 -->
00:18:58.160Inherently, they're still like it's just describing how they behave.
00:18:58.799 -->
00:19:07.680And since most of these tools are model agnostic for interpretability, then we can use them still for tabular foundation models.
00:19:08.000 -->
00:19:20.319One of the problems is that since predicting is more expensive now, and also for tree-based models, we have a lot of faster implementations.
00:19:20.400 -->
00:19:37.200For example, or especially for Shapley values where we have tree sharp, which is much faster than like these model agnostic implementations, we have now the situation that we can do interpretability still for these tabular foundation models, but it's more expensive and just takes much longer.
00:19:37.440 -->
00:19:44.160But I expect that there will be more research in this area going on and maybe we get some faster implementations there as well.
00:19:44.559 -->
00:19:49.440I think that's really interesting and critical point of that you still have the same toolbox.
00:19:49.599 -->
00:19:56.400So it's not, yay, it's an LLM, but you still have the same sort of explainability hypothetically that you would with XGBoost.
00:19:56.480 -->
00:19:59.920So that's I think that's a key part, is you can still have a lot of that.
00:20:00.000 -->
00:20:10.720Yes, it's slower now, but I would imagine that's definitely another area of the startup area around the space that we talked about of like the there's definitely some optimization techniques and things we could do to get that a lot faster longer term.
00:20:11.599 -->
00:20:12.079Yeah.
00:20:12.400 -->
00:20:25.680And also we have maybe a new era area open up, like with because of for large language models, we have mechanistic interpretability in these things, like visualizing our attention and stuff like this, which might be valuable.
00:20:25.839 -->
00:20:35.759I haven't seen much on this yet, or not done any deep dive, but could also be a potential area for additional insights into the model.
00:20:38.880 -->
00:20:44.480Would we have an opportunity to use stuff like we've talked about before, like the uncertainty quantification or conformal prediction?
00:20:44.559 -->
00:20:46.319Are these still options in this world?
00:20:47.440 -->
00:20:55.839Since they also are model agnostic, like conformal prediction still works.
00:20:56.079 -->
00:21:03.839And there you don't have the drawback of the predictions being slow because conformal prediction is something you do as a post-processing tool.
00:21:04.480 -->
00:21:14.960So, an example for someone who doesn't know what conformal prediction is, it's basically a toolbox for conformalizing uncertainty quantification.
00:21:15.119 -->
00:21:30.000So if you have a quantile regression, like with 10 to 8, 90% quantile, and you want to ensure that for each for the predictions it's like an 80% coverage, at least on average, then you can conformalize it with conformal prediction tools.
00:21:30.160 -->
00:21:37.039Conformalizing basically means that you get guarantees that on average it really covers 80% of the real value.
00:21:38.160 -->
00:21:44.079And you can apply these tools still to tabular foundation models as well.
00:21:44.640 -->
00:21:55.359Interestingly, also the tabular foundation models, the way they are trained, especially for regression, you also get already a model that can do like quantile predictions.
00:21:55.599 -->
00:21:59.839So at least it's true for Tap PFN and TAPICM.
00:22:00.160 -->
00:22:14.240They are both trained in a way that you don't not only, if you're using it for regression, you not only get the mean prediction, but actually the entire, or like m mostly the entire distribution.
00:22:14.480 -->
00:22:23.680So you can also ask the model to give you the prediction for the median or for the 30% quantile, and which I think is pretty neat.
00:22:24.720 -->
00:22:26.240Yeah, I think that's super exciting.
00:22:26.319 -->
00:22:34.319And I think it just shows even further that this is just expanding our toolbox machine learning tools even further past our simple good old-fashioned AI techniques.
00:22:34.720 -->
00:22:39.759In the landscape of tabular modeling, then how do you see this fitting in?
00:22:39.920 -->
00:22:42.240Do you think that this is going to be an utter replacement?
00:22:42.400 -->
00:22:45.839Or do you think that this is a new and unique way of doing things?
00:22:46.160 -->
00:22:49.680Yeah, my latest poster wrote about this.
00:22:49.839 -->
00:22:55.279Okay, how will this change data science or especially tabular machine learning?
00:22:55.519 -->
00:23:03.440And so the first scenario is it's just gonna be one of one of many models that you try out.
00:23:03.680 -->
00:23:21.519So just like the latest boosted tree model, maybe it's even state of the art in the best model, but still you're in a situation that you maybe try out 10 models or 10 algorithms and see what works best on your data set, stick with the one that works best and continue from there.
00:23:22.559 -->
00:23:32.240We're clearly there that I would say that the software is there and it's usable, at least for small and medium-sized data sets, and you can just try it out, use it.
00:23:32.400 -->
00:23:44.160And if the Tableau Foundation model outperforms all the other models, then maybe you want to go with the model, depending on your use case, of course, like how many predictions do you need to make all the time and so on.
00:23:44.720 -->
00:24:00.400Second scenario would be okay, it becomes the default model, and you because it works so well, it's just like you're all in one model because you can already do quantum regression and so on, and maybe there's an ecosystem around it, so it's just convenient to use it and becomes your go-to.
00:24:00.720 -->
00:24:08.960But on a third level would be that we have this huge paradigm shift, a little bit like a large language model said for an NLP.
00:24:09.359 -->
00:24:19.519That is such a strong new paradigm that it's kind of really becomes the go-to that you always would try this first, and this is your area where you feel comfortable.
00:24:19.680 -->
00:24:29.039And basically works in 90 plus percent of your cases, and you always use this to solve your tabular machine learning uh tasks.
00:24:29.279 -->
00:24:30.319And yeah.
00:24:31.279 -->
00:24:42.160But this really depends on how this all develops, especially in terms of compute and the ecosystem around it, like with all the interpretability, uncertainty, quantification, all these tools.
00:24:43.519 -->
00:24:56.000Yeah, because at the current state, it sounds like it it's a really good option for you to do for batch processing, but it's kind for most companies that are using any sort of online inference or things, it's too slow to really be considered at the moment.
00:24:56.160 -->
00:24:56.960Would you agree?
00:24:57.359 -->
00:24:58.160Yeah, 100%.
00:24:58.640 -->
00:25:02.240So that comes with the idea of in-context learning.
00:25:02.400 -->
00:25:14.319So if you have the scenario that you train a model and then make a prediction maybe once a day or once an hour, but only one by one, you always have to provide the entire context.
00:25:14.960 -->
00:25:19.680So you can do some optimizations there if maybe smaller context is enough.
00:25:20.079 -->
00:25:28.559But if you don't do that, you would always provide your entire train training data set to make just one prediction.
00:25:28.960 -->
00:25:36.319And this is also like for if you would be using uh random forest or cat boost or whatever, it's pretty fast.
00:25:36.400 -->
00:25:40.400You do one data point, you just make prediction because training was expensive.
00:25:40.559 -->
00:25:42.799Predicting is not so expensive, at least.
00:25:43.039 -->
00:25:49.440But yeah, with the tablet foundation models, it's the better scenario is that you do, as you said, batch prediction.
00:25:49.519 -->
00:25:57.680So maybe just run it once a night, like for all your customers or whatever, and it's fine if it takes one hour instead of 10 minutes, maybe, or once one second.
00:25:58.400 -->
00:26:03.920But if you have this ongoing online uh prediction type of situation, then it's really expensive, yeah.
00:26:04.160 -->
00:26:17.039And just to start getting our concluding thoughts out there, I think one really interesting insight that you had on the blog post is that because this is so methodologically distinct, these are potentially really good for ensembling models.
00:26:17.519 -->
00:26:23.680And I think that one thing I would also add above that, too, is that we now have a new strong baseline, right?
00:26:23.839 -->
00:26:29.119Now you have your most frequent class, you have your average, and now you have can something out of the box do it?
00:26:29.200 -->
00:26:32.079And I think that's a question that a lot of implementers are looking at now.
00:26:32.160 -->
00:26:36.960It's am I gonna build the software solution or is ChatGPT gonna do it just as well or better?
00:26:37.200 -->
00:26:41.839And if you can't clear that bar, then you know, why spend an RD to develop something?
00:26:42.000 -->
00:26:45.279I think that this is like a new level of proof we need to provide.
00:26:45.599 -->
00:26:46.079Yeah.
00:26:46.319 -->
00:26:47.279Yeah, I think so.
00:26:47.359 -->
00:26:55.279For me, when I encounter like a new machine learning task, I have the reflex to try out the random forest.
00:26:55.519 -->
00:27:00.400It's for the simple reason that it works already without hyperparameter tuning.
00:27:00.640 -->
00:27:07.359It's fast, so it's very convenient, and you always get like this first idea of okay, how well can I predict the task?
00:27:07.440 -->
00:27:10.319How well like where's the line?
00:27:10.400 -->
00:27:34.400And of course, you can improve it then afterwards, maybe to by using boosted cream models like XGBoost or Cat Boost, but it gives you this first idea, and I think that these tabular foundation models now can be used in the same way because you don't need to hyperparameter tune and kind of work for classification and regression.
00:27:34.640 -->
00:27:54.400So you can just and you have also that they work quite well, so in many cases at least, so you can just throw that your problem if you encounter it the first time and get like a first baseline and know where you stand, and afterwards, if then it's a concern that predictions expensive and so on, you might switch to a different model.
00:27:54.559 -->
00:28:00.720But for this first encounter with a new machine learning problem, it's a good way to start us.
00:28:01.039 -->
00:28:01.599Yeah.
00:28:02.160 -->
00:28:07.440I I just have one last question, which is looking back on your supervised machine learning for science book, would you have added this?
00:28:07.599 -->
00:28:09.359Would this have made the inclusion?
00:28:10.000 -->
00:28:11.359Yeah, it's an interesting case.
00:28:11.519 -->
00:28:29.519The way we wrote the supervised machine learning for science book, we said, okay, there's this way we do supervised machine learning, but it's not enough for science, so we need all these tools from interpretability to robustness to uncertainty quantification, and wrote it in such a way that these are model agnostic.
00:28:30.480 -->
00:28:44.240And even though the paradigm now changed with the tabular foundation models, these tools remained the same, just as we talked about, you can still use interpretability tools, you can still use uncertainty quantification in the same way.
00:28:44.559 -->
00:28:51.759So many of these tools are actually the same, or it's well, you can still use them.
00:28:52.880 -->
00:29:13.039What's a bit different now is the idea of this pre-training, which I think is adds a new layer to all of this, because now you could say, hey, maybe for my very specific scientific application, maybe it has a very specific structure, and I want to put more domain knowledge into my modeling approach.
00:29:13.200 -->
00:29:16.640Now with these tableau foundation models, you can do pre-training.
00:29:16.799 -->
00:29:24.400So you could fine-tune them on synthetic data or similar data and give this prior knowledge into the modeling process in this way.
00:29:24.640 -->
00:29:30.960I think for the domain, like instilling domain knowledge into your model, I think this adds a new layer.
00:29:31.119 -->
00:29:37.200So I think in this place, I would see this new, I would maybe make an update in the book, yeah.
00:29:37.519 -->
00:29:37.759Great.
00:29:37.920 -->
00:29:40.880Any other clearing thoughts from Angela or Susan on this?
00:29:42.319 -->
00:29:44.640Christoph, it's always a pleasure to have you on.
00:29:44.880 -->
00:29:46.240This is a great topic.
00:29:46.559 -->
00:29:53.759We thoroughly enjoy following your work and excited for whatever next topic or book, and love to have you back if you're willing.
00:29:54.000 -->
00:29:55.599Thank you for the show.
00:29:56.000 -->
00:29:57.759Yeah, I echoed it.
00:29:58.000 -->
00:29:59.440Yeah, I echo that, Christoph.
00:29:59.759 -->
00:30:09.759Earlier in the discussion, when you were talking about the compute power and everything like that, I was thinking about what's going on right now with Apple chips and people now actually going out and buying.
00:30:10.079 -->
00:30:17.680Apple can't keep up with their supply of Mac minis because like people have now found that they can do this very locally on like very little computes.
00:30:17.920 -->
00:30:19.039I was excited to put that together.
00:30:19.119 -->
00:30:20.160So thank you.
00:30:20.480 -->
00:30:21.599Yeah, thanks for having me.
00:30:21.759 -->
00:30:23.279Always a pleasure talking to you.
00:30:23.440 -->
00:30:28.480And also, yeah, especially with the tablet foundation models, also something I'm just excited about at the moment.
00:30:28.559 -->
00:30:31.119So it was really cool to chat with you about this.
00:30:33.200 -->
00:30:34.720Yeah, absolutely.
00:30:35.039 -->
00:30:35.680All right.
00:30:35.920 -->
00:30:41.680And we, with that, for our listeners, we have one short announcement for you.
00:30:41.839 -->
00:30:45.279This will be my last podcast here with the team.
00:30:45.440 -->
00:30:50.319I'm moving on and we'll actually get to admire our work from afar.
00:30:50.880 -->
00:30:58.000And so now, from as you keep listening to our episodes and keep contributing your thoughts, you're probably going to hear from a new face.
00:30:58.079 -->
00:31:04.720His name is Mike Moore, and he will be helping Sid and Andrew get their thoughts and guests together for the show.
00:31:04.960 -->
00:31:09.680I really appreciate all of the feedback that listeners have given us over the past three years.
00:31:09.920 -->
00:31:14.960Again, for all our guests and Sid Andrew, thank you for everybody listening.
00:31:15.039 -->
00:31:21.200If you have questions about the show, same email address, ai fundamentalist at monetar.ai.
00:31:21.680 -->
00:31:23.519Then we'll see you next time.
