#156 Bayesian Experimental Design & Active Learning, with Adam Foster의 전체 녹취록

was just who I was around right when I joined my PhD EYT and Tom Rainforth, both have done lot of serious Bayesian stuff. But I think also it was the topic of experimental design and iterated experimental design where I felt like just, yeah, Bayesianism was just sort of a language that allows you to talk about that quite concretely and quite naturally. And that was sort of maybe missing from other frameworks. So I think they actually kind of came onto the scene kind of together a little bit, ⁓ where, know, me getting into Bayesianism was also driven by me wanting to think more about, ⁓ experimental design. And then the experimental design specifically. So I think it was Tom Rainforth, my, my, ⁓ co-supervisor at Oxford, who first actually brought this topic to my attention. ⁓ and then I went to do an internship with Uber AI Labs, ⁓ and now sort of defunct part of Uber. And there I worked with Noah Goodman and he was also really interested. So I kind of had these two like people who were pushing me in that direction. ⁓ And it did seem like a really interesting underexplored area. know, ⁓ everyone's doing, ⁓ you have your data set, download MNIST or download, you know, some tabular data set and like try and get a better number on that, get a better sort of marginal likelihood. And this was a completely different setup and a different problem that felt underexpored. speaker-0: Hmm, okay. Yeah, so it was really, it was really motivated by something that you saw as a gap in the, in the literature at that time, if I understood correctly. speaker-1: Yeah, definitely. And also, know, Noah had a lot of practical stuff with, sort of cogsight experiments and psychology experiments. So it's sort of like, okay, there's there's a reason to do this. There's a gap in the literature and it's a very fun problem that I think gets to the heart of why we do Bayesian. I really think it does get to the heart of why we do Bayesian stuff in the first place. speaker-0: Okay, yeah, so actually let's dig a bit into that. know when you say and when I say Bayesian experimental design What does that mean and what problem are you actually solving? speaker-1: Yeah, interesting. why don't, yeah, let's start from the sort of problem and the practical side. By the way, we have written a couple of papers trying to answer this question. So it's not, it's not that clear where the boundaries lie. And I think some people might, might be doing things that we are actually classifying as based on experimental design. But anyway, that's a bit by the by. So experimental design, right? You, you don't have your data set yet. Or you maybe only have a partial data set and you're going to collect more data and you have control over some of the parameters of that collection. Right. So for example, you're going to do a, you want to go and collect data from humans. Well, how are you going to interact with them? If it's a questionnaire, what's what are the questions? Or you're going to go into the chemistry lab. Well, what do you do? What are the reagents and what are the conditions? Right. I guess it's sort of upstream from data collection. Whereas typically, like I was saying, you have a fixed data set and you just start doing analysis. So it's a slight change in mindset there. And then once you have this, then obviously anyone would ask, well, how do I choose the settings? Right. Like I can talk to a hundred people. Well, what should I do? Or, you know, I have this budget or whatever. I have this much time. Well, what settings should I choose? So that's just experimental design in general. And there's this like many, many ways you could try and tackle that. In Bayesian experimental design, I think we take the view that your model, right, your Bayesian model already has uncertainty within it. That's one of the reasons why people really like Bayesian, right? It comes with this uncertainty measure, which you wouldn't necessarily get from other paradigms. And then you could ask the question, well, what new data would reduce uncertainty? And that I think is really the Zen of Bayesian experimental design. You're asking, well, What new data could I add that would then reduce uncertainty? And then you could flesh that out and say, uncertainty in exactly what? ⁓ and that, you know, there are different answers depending on what your aim is. speaker-0: Yeah, yeah, yeah. why should a practitioner even care about patient experimental design? Just to collect more data. Why would that even be interesting? speaker-1: Well, how are you going to collect the data? Right. That's the whole question. Like, yes, you could, you could go out. mean, I think it just depends on what your topic is. Right. If you're doing LLM, you can just go and crawl the internet and that is data. And maybe there are some parameters about how you would do the crawl, but the data is just kind of exists. I mean, if you're writing a questionnaire, well, what are the questions? It's sort of like, you can't avoid the question of. Sorry, you can't avoid the topic of experimental design. If you're in that setting, right? You have to fill in something. Well, then you can ask the question, why not just put random stuff or just what feels right? ⁓ I mean, often you can, and if you're really going in blind, you have no model, you have no expectations, then maybe that's legitimate. ⁓ If you have some prior or if you have a model or if you have some concept of how you're going to train the model afterwards. Right. I think this is important. If you already have an idea of what, what you're going to do after you've collected the data, you can sort of forecast that ahead, right? You can say, okay, well, when I've got the data, I would do this. So let's analyze that actually before doing the kind of committal step. Right. The committal step is to, you know, do the survey, pay people or, or spend volunteers time or, you know, ⁓ build the object and, I don't know, blow it up, whatever your

experiment is. ⁓ That's the committal step. So why not do some analysis of then, well, now let's imagine this was the data set. Well, what would it look like at the end? ⁓ So I think that's the general pitch. And then whether you would choose a specific algorithm would come a little bit more down to your exact model and your exact assumptions. speaker-0: Hmm, okay. Can you maybe clarify Epidant for listeners and maybe give them a lay of the land on when you think these kinds of methods and models are most useful? speaker-1: ⁓ I think, I think they're most useful. First of all, are you in a position to actively collect data? Right. So, so if you, for example, if you're running your website and you're just tracking what people do, but you don't, you don't actively engage with them in a way to change the data, then it probably doesn't make sense. Right. It's like you are just observing the world as it goes past and you're not going to do anything to it. So that's probably a case where you wouldn't really think much about experimental design, although you could think about it. So we'd think more about the case where you are, you have control parameters, right? So like I said, you're writing a questionnaire, you are choosing reagents to put into your reaction. You are designing an object that you're going to explode in some exciting way. So are you in that case? Right. And then if you want to now use sort of specific methods that I've worked on, you would think about, do I have a model of this process? And can I capture some of my uncertainty in a Bayesian model in the sense that if you were to look at the prior predictive distribution of your Bayesian model, would give you a good idea. Okay. You'd say, okay, yeah, this is what typical data sets might look like. If you're in that setting, then You you've, you've hit the money. Like that is the time to kind of switch on your Bayesian experimental design. ⁓ Another case could be that you've already done some data collection and you want to, you want to scale up. Right. Cause then you can, you're not just going in with pure assumptions, right? You can validate a lot of assumptions on your small data set. You can use that to build yourself a Bayesian model that, you know, again, you think sort of captures a lot of your uncertainties about the data generating process. and then you want to do that scale up so you want to spend more on data collection, well that would be the time to do it. speaker-0: Okay, okay. Yeah. Yeah, thanks. I think this is very practical and things that the audience appreciate. ⁓ actually, you know, keeping staying on the practical side, do you see this kind of methods? So a Bayesian approach to experiment design? Do you see actually changing how scientists work in practice or is it still mostly a research endeavor? Let's say, because I think you're in a very interesting position because you've been in there for years. You're at the forefront of the research and I'm guessing that you

have an interesting way of things, how things are going. speaker-1: Yeah, I think it's really dependent on the area and what exact experiments people do and how they've typically analyzed their data. So, you know, I've been fortunate to just sort of get the odd email from a practitioner and they've been from really diverse areas, you know, like manufacturing to some sort of like neutron physics and a whole range of really interesting applications. I think in most of these cases, it was about ⁓ research and is this a new way that we could do things? I know that there are some areas that are a bit more advanced and I think if they're quite advanced, they've kind of redeveloped Bayesian experimental design a little bit because it was such a natural fit to what they had to do. I think the domain specialists have kind of rediscovered it in some way. You often find this. think Bayesian experimental design is one of the most rediscovered ideas because it sort of keeps coming back. with a slightly different name and things like that. I know, for example, incensor placement is quite advanced, for example, because I think that's just a place where the practicalities really can use spatial experimental design and it's quite high value. And yeah, know, you know, with people like Noah and doing the COGS-I research, think it's relatively studied there, for example. So I think it very much depends on the domain. and how the data is analyzed and what tools people are familiar with and whether they think they could get a big gain from it. It would be very valuable. speaker-0: Yeah. I, do you think there is one domain, like, is there one domain in your, in your mind that is still not applying your research and that you think would, get a big gain in that, know, like it would be a low hanging fruit. And I'm, and when I'm seeing your research, it's not like, only your personally, but like your fields research. speaker-1: Yeah, yeah, the subfield. Nothing really springs to mind because I think you do need to be a little bit expert. I think that would be slightly arrogant of me to say, oh, look at these silly people. Why are they not using this? sure. Often there are reasons why people do things the way they do them. I know, for example, you know, like with A B testing, right? It's so studied. And you could say, well, speaker-0: Now for speaker-1: Do they use it based on experimental design? if not, why not? And I think one of the reasons they, they, I think they do use it. Some people would use it, but one of the reasons people don't is that, um, they, they have, they just have other criteria by which they judge things and they have kind of these unknown unknowns that they are a bit worried about. And so they're looking to, they just have slightly different criteria, um, with what they do and. They're not, I think they're not comfortable basically relying on prior knowledge from previous experiments. Right. That would be the way that you would apply in A-B testing. You'd say, well, actually a lot of the variations we're experimenting with, we already have data on. So our prior is strong. So we should, we should invest less time in those. I think some people adopt that mentality, but I think other people are like, Ooh, but there could have been an unknown change that we don't have eyes on. So we actually just need to just run the baseline again. with exactly the same sample size. So yeah, I would, I would very much hesitate to come out and say, this is the specific application that should use it. I think it's an area that a lot of, a lot of fields could, could investigate and, and I think really interesting research could be done. I think it also revolves around models and how advanced are the models, how confident are people with the level of uncertainty? Do they think that's accurate or is it just, you know, something that's unreliable itself. speaker-0: Yeah. Yeah. Yeah. and something I'm, I'm always curious about when we talk about methods on the show is when are they useful, which I think you, you covered pretty well already, but also when they break down, know, so when, when would you say Bayesian experimental design breaks down or underperforms, ⁓ simpler heuristics? when would you tell someone, Hey, you know, Like, don't bother with that. Just keep doing the classic thing. speaker-1: Yeah, I think there's many cases where I would say that. So one would be if just picking up random data is just really easy for you, just do that. Right. Like I think there's no point doing a lot of computational work to design your experiment. If the data is so cheap, you can just get 10 times more of it. So I think if you're, if you're, and that you think it's valuable, right. Like I think there are settings where you can get 10 times more data. but it's just all telling you nothing. And then you obviously would want to invest in some experimental design. but there are cases where you can get quite reasonable data at a very low cost and you can sort of apply simple heuristics about, you know, what your train, what your test set looks like. And you can sort of correlate your data with that. And then you're like, okay, well, this data should help answer my question. ⁓ so I think it would be a case where maybe your experiments are quite expensive. ⁓ typically to sort of justify wanting to design it properly. and, and I think the other thing is about models, right? ⁓ you need to have some kind of model that you're somewhat committed to. It doesn't mean it has to be a hundred percent totally right, but it has to represent pretty good guess at how things work, ⁓ in the system. I mean, I think this is, this is a topic of, of much debate in the, in the field. Can you take a black box model, ⁓ know, random forest, Bayesian neural network? ⁓ And can you run experimental design with that? And how much, how, how much initial training data would you need? Right? Obviously I think your Bayesian neural network untrained prior is not

going to tell you a great deal. But you might be able to bootstrap it with a small amount of data or something. I think that can work. Definitely. I mean, we've done research on that topic and we've, we've shown that it, that it definitely can help. ⁓ but, you know, if you, yeah, I think if you are just really not wanting to commit to any model, ⁓ then that would also be a sign maybe that things aren't working. Or the other thing would be that your model is not Bayesian or it doesn't express any uncertainties that you would want, right? That's quite common. You know, people have a model. They know it's wrong, but it just makes these point estimates and that they have some kind of bias that's not quantifiable. And there's no parameter of the model that you could vary that would sort of change those predictions. So think that would be another setting where it maybe wouldn't make sense. speaker-0: Okay, okay, very valuable. ⁓ Thanks for entertaining me here. And actually, I remembered I had already an episode about Bayesian experimental design with Desi Ivanova. That was episode 117. folks, if you want more background about this, I remember we dug quite a lot into the... into the intuitions and the concept about Bayesian experimental designs. Do look into that because, well, what we did with Adam was more of a refresher here and we're also going to talk about other topics. So definitely check out the show notes for that. Adam, actually something you also like, yeah, something I've seen a lot in your work is something that's called the expected information gain. So Can you tell us what it is measuring intuitively and why maximizing it is the right objective? speaker-1: First of all, everyone should go and listen to that episode with Desi. I've worked with her pretty closely back at the beginning of her PhD. And yeah, I'm sure she's got some probably quite complimentary things to say, given her slightly different background. So yeah, I would really recommend people to check that out. Okay. So I to expect an information gain or EIG, you know, as we acronymatize it. So as I was telling you, ⁓ Alex, what, what we do is we, have parameters of our experiment and we want to sort of decide them, The questions of your questionnaire or something. ⁓ and there's a sense that you could choose them optimally. So what would that mean? Well, optimization in this case means that you can associate a score to every one of your designs. And then you choose the one with the highest design, ⁓ you choose the design with the highest score. And that would be your optimal design. Okay. And then the score is this thing, expected information gain. So it's the amount of information you expect to gain about your model parameters by performing an experiment with a given design. Now, so why don't we unpack that a little bit? Cause I think that's, you know, that's sort of maths language just put out into an English sentence. I'm just trying to think of an example that might, that might kind of, bring this home, you, well, if you remember, we said you have uncertainty, right? Your Bayesian model is sort of reflects the uncertainty that you have. If you choose a design of your experiment, you can also forecast what the datasets would look like through your prior predictive distribution. Right? So now you can take a dataset, a prior predictive sample, which is a simulated dataset. Then you can do your Bayesian. difference, and then you can look at how the uncertainty changed. And then you can repeat that for various different synthetic data sets. And then you take the average and that's your expected information gain. That's one way to arrive at it. I think what's really cool about it is that, you know, if you actually work through the maths, you can arrive at it in quite different from quite different starting points. Here's a second starting point, which sounds completely different to the first one, but actually arrives at the same quantity. So I don't know if you're familiar with this. It's in Brooklyn nine nine. It's this question about you have, I think it's like 12 prisoners on an island. One of them is slightly heavier or lighter. You know, how would you find, how would you find out which one it is? ⁓ Okay. I don't know if you know how to solve that problem, but basically the, kind of the way to think about solving that is. And that's an experimental design problem, right? Because you have settings, which of these people you want to weigh against, which other ones, and you have outcomes, which is, well, did the scales balance or not balance, right? I don't know if, like, have I told you enough about the setting? Or do want me to just clarify that little problem? speaker-0: Yeah, maybe clarify it, think, for... for listeners. speaker-1: Yeah. Okay. I think this is quite a fun, a fun puzzle. and people might enjoy it. So I think it's 12. yeah, I'm pretty sure it's you have, you have 12 people on an island and you were told that one of them is slightly heavier or lighter than all the rest. Okay. And you have access to like a ginormous balance scale, right? One of those old fashioned scales where you weigh two things against the other and it's either level or it's tipping one way or tipping the other way, right? Like my mum has one, still is like kind of like a nice old one. And you have a stack of weights that you compare to your flower or something. Okay. So now imagine you have a giant one that can weigh people against each other. ⁓ And then the question is, well, how would you most efficiently find this odd, odd person, right? Of the 12, you've got an odd one out. would you find them most efficiently with the fewest weighings? That's the puzzle. And the kind of, I won't tell you the answer. I think it might be fun for people to think about it a little bit. It's in the show and I think they can't solve it in the show, at least if I remember rightly. But the best way to think about it is there are three outcomes, right?

The scale can either tip left, tip right, or it can balance. And ideally what you'd want is for all three of those to be equally represented. Now, why would you want that? It's because you can, you're kind of dividing your search space down as quickly as possible. Right. If you, if you, if all three of them are equally likely with your different hypotheses that you might have. ⁓ then if you see it tipped left, so for example, suppose that it was a third, a third, a third based on the hypotheses that you have, then When, you see at tip left, the search space has divided by three. So you've made really good progress on identifying, you know, the true hypothesis. If the probabilities were not a third, a third, a third, you might have only reduced the search space by quite a small amount. So you can, and you can, you can mathematize this and you can, you can talk about the, right? So the entropy of the outcome is the, right? If you made all three likely. And it turns out that for this experiment, the ray approaches to max out outcome. So I think hopefully with that hint people could now figure out the EIG. This has become a very... ⁓ There's one... speaker-0: I you tell us the solution but at the end of the show speaker-1: Okay, well, I actually don't know it. I only know how to derive it. I will have to email you afterwards with an actual solution. all I know is what I've told you, which is that you need to divide the search space in three every time. But then you need to come up with a decision tree that actually does that, which I can't do off the top of my head. ⁓ Yeah. speaker-0: that makes it yeah so go ahead sorry I was okay taken in the math enigma speaker-1: So I think to kind of get people back in the flow of where we were, right? We're talking about EIG. And I've given you kind of two arguments about how to design an experiment. One was about reducing uncertainty in the model parameters. And then this one is about maximizing the uncertainty of the outcome. And it turns out there's kind of one missing piece, which is, you know, in this sort of balance scale problem. There's no notion of noise, right? So you, you, want a really uncertain outcome because that would divide your search space. But you can imagine that actually some experiments have noise and you want to avoid experiments that are very noisy, right? Because they could be very uncertain because they're just noise, right? If you, if on the island, you also had a six sided dice, you'd be like, ⁓ wow, like that has log six of information because it has six outcomes. Whereas my mass balance only has three. Yeah, but it's not correlated with your problem. It's not correlated with the problem of the sailors who you're weighing. So you need to correct for this noise problem. And once you correct for this, have two formulas. One is about the reduction in uncertainty about your model parameters. And one is about how uncertain is your outcome accounting for noise. Right. And it turns out that these are equal. If you actually just work through, they are the same thing and that is EIG. And that's why I think EIG is really central because it kind of, you can, you can re-derive it in many ways. It keeps coming back. As I said before, it's one of those things that's been re-derived at least 10 times ⁓ by different people. So I think the fact that people keep coming back to it as the kind of objective criterion for choosing experiments tells us that it's quite important. And then just to kind of link that back then to the, to the big topic. So I said, you know, it's all about choosing inputs for experiments. So the idea now is you have a scalar value associated with every candidate design, which is this EIG, and then you just choose the candidate design with the highest EIG score. It's very simple. speaker-0: Yeah, I mean, it's very simple once you explain it. So, and then how do you compute the IG score? Because I've understood from what I've read to prepare for the show that it can be hard. So what are ways you can do that? And when it's too hard analytically, are there ways to get around it? speaker-1: Yeah, so this is actually now touching on my own sort of real research, actual papers written by me. But let me just back up to say it may not be hard, right? So with this prisoner island question, there's a finite number of outcomes. There's a finite number of hypotheses. It's just a very small, like you could just do it with a small NumPy array, right? So it's not hard. It's actually very easy to do on your laptop to run all the calculations. ⁓ So it really depends on your model, the size of your search space, the size of your hypothesis space, the size of the outcome space and the size of your design space, right? And the type of outcome, right? Is it continuous? Is it a classification task? For example, like we had a three-way classification, right? Left, right or balance. ⁓ Or in your questionnaire, it could be a yes, no tick box, or the person could be writing free text. Obviously they require completely different models to analyze and to sort of predict. mean, if your, if your model... is meant to be of a generative model of text, that's quite a bit more complicated than a checkbox that could be on or off. Now, one case that I looked at was about continuous outcomes, right? So suppose it's kind of a regression model, multi-dimensional regression or something like that. And in this case, it does get a little bit kind of hairy due to this, what we called, mean, Tom generated this term, double intractability, which means ⁓ if you think that Bayesian inference is already kind of intractable, right? Cause it involves this, ⁓ this, this denominator that you have to integrate over all of your prior space, right? ⁓ the much, the marginal likelihood. If you think that's already intractable, then in experimental design, if you recall, I was talking about synthesizing lots of datasets and repeating experimental design, or sorry,

inference, right? To compute the score. That's kind of the heart of your double intractability. So you would synthesize data sets, do Bayesian inference, and then do that in a loop. So you're kind of doing this very hard intractable thing many times over. Now, like I said, with the prisoners on the island or maybe with a classification model, there are ways around that. And it might be that it's just not really a big deal. But in the general case with these continuous outcomes, you have to do something about it. ⁓ And that's what we looked at in this paper, Variational Bayesian Experimental Design. ⁓ I think it's Variational Bayesian Optimal Experimental Design. ⁓ And we looked at, like how could you use variational inference? I mean, variational inferences or amortized variational inference is a way to obviously sort of get to the posterior just as a neural network function of the data. Right. That's the purpose of amortized inference is that you, you feed in your data and it will immediately tell you how the posterior should behave approximately. And that is obviously going to be really useful for this concept of synthesizing many data sets and doing inference, right? You could, well, ⁓ I'm to amortize that. So I won't have to keep solving ginormous integrals. I'll just take forward passes through my network. That's one of I mean, that is quite a dense paper. I think if someone enjoys a of maths, that would be a good one to read. That's just one aspect of what we studied. We looked at other kind of estimators because the EIG can be rewritten, right? As I said, you know, had these two different viewpoints on it ⁓ because it can be rewritten so many ways you can estimate it in different ways. And it sort of depends on what the dimensions of different things are in your model. ⁓ But yeah, that was kind of the thrust of this paper on variational. ⁓ Bayesian optimal experimental design. speaker-0: Yeah, yeah, and I think these were these were really interesting and and brilliant papers. So please do add that in the show notes because here we can only do them partial justice. First math in audio form is always harder and second these are long. So so definitely for people who want to dig deeper to check these out and we'll put the links in the show notes. You'll have that on the website. And yeah. Do check these out because I think if you're interested in these topics and they are something you're working on, you'll definitely want to check that out. ⁓ And actually, ⁓ you've also written about the connection between so Bayesian active learning by disagreement, which is a new concept. haven't talked about that yet on the show and Bayesian experimental design. So there is a lot to unpack here. So first, can you define what active learning Ease for us. speaker-1: Now that is, that's really hard question. I don't think there's a universally accepted definition of active learning. I think it's more a vibe, right? The vibe is, it's quite similar to what I've been talking about, but it's usually people imagine the model now to be a neural network and people imagine that it's about getting improved test set performance or validation set performance, right? So you, again, you can actively choose what kind of data points you want. And it might be that you have a pool, right? You have a pool of candidates, but they don't have labels and you want to add labels. That's, that's kind of a typical setup. Now I would sort of controversially state that this is a special case of experimental design, right? Like you have, you have certain inputs, which is your set of candidates to be labeled. And you say you have experimental outcomes, which are the labels that you're going to reveal. ⁓ And then you, might have a Bayesian model. ⁓ so for example, with, with, with Freddie Bickford Smith, we worked on a lot of stuff about active learning and there we kind of boiled it down to, yeah, it's about, ⁓ doing, doing experimental design with a view to improving your performance on a certain test set or, you know, target set, for example, you sort of know what predictions you want the model to be good at and you try and then find data that would help the model to achieve that goal. people use active learning in many different ways. think some people use it to mean experimental design in general. ⁓ So what I've given you is very much my biased definition of active learning. And I'm sure if there are any professors of active learning listening, they're going to be shaking their fist at the speaker. But that's my... speaker-0: We started a feud, ⁓ speaker-1: It's not- this is not the beginning of the feud. This has been going on for a while. speaker-0: ⁓ yeah, we just reignited, put fire on the... put oil on the fire. speaker-1: ⁓ And so yeah, that's my definition of active learning. ⁓ And then bold, right, we were talking about basic active learning by disagreement. Well, what is that? ⁓ So this comes from a 2011 paper by Neil Hulsby. think that's, I believe, the first ⁓ use of the term. It's basically EIG again, right? So I said it keeps being reinvented. It's just the EIG. It's just a different way of writing it as an objective function. I think this paper, right, was one of the first to talk about this rearrangement that I was talking about where I said, well, have uncertainty, reducing uncertainty on your model parameters, and then sort of increasing uncertainty on the outcomes whilst accounting for noise. And bold is one of the first papers where they really focused on that rewriting and the fact that those two are equivalent. And then they. They showed that kind of, I think the intuitive one is the first one, right? I have model uncertainty on one or reduce it. Makes sense, right? That's probably the most intuitive. But then maybe the more computationally useful one is the second one, which says, well, have, I have, I want to have an uncertain outcome subject to not being too noisy, but you can prove that they're

the same. And then you can calculate obviously the second one. If you, if you think it's easier to calculate. And then you can score designs using that. it's really, I would say it's, it is the EIG. It's a different way to write the EIG. And it can be really good. It can be much more efficient to do it that way for a classifier model, for example, right? Neural network classifier. So yeah, that's, that's what I would say bold is. And, and, you know, just to say, yeah, you know, I wasn't writing papers in 2011. So, so this whole concept. ⁓ it's not due to me, it's due to either Neil Hulsby or somebody before him, basically. speaker-0: Yes, and bold. to make sure people understood correctly, that's Beijing active learning by disagreement. So that's the active learning part. Now some people are fighting right now as they listen to you, Adam. I'm sure you already have a lot of angry mail, but let's let's continue. Then what is the disagreement part about, you know, and then let's let's talk about how that connect back. to Beijing experimental design. speaker-1: Yeah. I think that's a great question. I really like the term actually. think it's because I think it's kind of a nice way to think about what the EIG is. So the disagreement is between candidate hypotheses of your model. Right. So it could be as samples of your prior or if you have some old data, it's samples from the posterior conditional on your old data. So actually let's think about that island prisoner or sailor example again. There's 24 hypotheses, right? There's 12 sailors and either one is heavier or one is lighter. So there's 24 candidate hypotheses in this model space. And then disagreement would mean the predictions of the experimental outcome made by different hypotheses are different. Right? So for example, right, we talked about, well, why don't you just roll a dice on the beach? Well, none of the hypotheses will make a different prediction about the outcome. But now if I weigh two sailors between them, right? If, if, you know, hypothesis, I don't know, 15 might say that this one is the heavy one, then that would make a different prediction to a hypothesis that says neither of them is, in which case it would balance. Right. So that's where the disagreement comes from. And I think it sort of connects to thinking of your model as a sort of mixture of experts. I think that's a nice way to think about it. Right. So. If you had a finite number of hypotheses, right? So your posterior is just a weight over a finite number of hypotheses, then active learning by disagreement would basically boil down to, well, what are experimental designs where the predictions of different experts disagree? And again, you can sort of write out the maths and then it all comes back to this sort of magical EIG quantity that's I think really at the center of a lot of this. speaker-0: Okay, okay. Yeah. Yes. All right. This is actually a good name. It's usually because we have a lot of bad names in statistics, but that one I would have to say is actually probably helping to understand what this is all about. Right. So can you also come back very, like give us the elevator pitch again to make sure it was clear to listeners because this was a long and technical answer, but to make sure that listeners understand how bold connects to BD. speaker-1: Yeah, it's the same thing. And is my, is my statement. It's just so B E D I think this is why people find it a bit hard to get into. It's, extremely abstract. Yeah. Right. So it can kind of like it, it's sort of like a chameleon in that it can appear like a completely different algorithm in a different, in a different model class in a different setting. Bold, I think as it's used now, which is by the way, is not, I would say the full extent of what was introduced. in the original paper, but a sub case of bold would be you have a classifier model. You sample the weights of your neural net with dropout. That gives you a variety of predictions and you find out where those predictions maximally disagree. Right. ⁓ I would say that's an approach to doing BED with a specific model, with a specific way to generate hypotheses, right? It's sort of Bayesian posterior and Yeah, with a certain approximation to the EIG, Because to get the true EIG, you would need to sample like the entire hypothesis space. So that would probably be infinite. So yeah, I think that's the link. It's sort of, you know, BED is the sort of chameleon, the sort of very abstract concepts and bold is a very concrete, you could use it, right? But you could use it because it's a specific choice. There's so many free choices. in the BED framework and it's more like here's a lift of specific choices that could work in certain types of problem such as classification. speaker-0: Yeah, yeah, I completely agree with that. would say, yeah, probably one of the problems of PDs that it's very large, it's very broad, ⁓ whereas bold is going to be much more like it's a bit clearer what it is. it's also probably less intimidating to people new to it. ⁓ speaker-1: Yeah, I think just because it's so much more specific, Like it's a specific type of model and a specific type of problem. And BED sort of isn't, but because it's so general, it's hard to work out what, well, what is BED for my case? I don't know, right? You have to do a lot of thinking to answer that question. speaker-0: Yeah, no exactly. So again though as I asked you for BD, when do you think bold is appropriate and when is it not? And when it's not appropriate, what does it get wrong and what's the key insight behind the fix? speaker-1: Yeah, I'm well, there's kind of two answers to this, right? ⁓ So one is about supposing that you want to reduce uncertainty in your model parameters. Is it the right thing to do? To which I would say probably, right, like I said, it's more specific. Do you check all the boxes that bold requires you to check? your model classification model? Right? ⁓ Is it? And

do you have a way to sort of generate hypotheses? So some kind of approximate posterior. So, you know, are you happy with that? Do you think that the prior predictive of that is going to be reasonable, right? Like throwing dropout on your neural network is, you know, it's a little risky, let's say. You have to decide whether you are okay with that. So I think if you tick the boxes of using bold, yeah, go for it. Right. Use bold. And I think most of what I've written. Like in my own research is not like an alternative. It's more like understanding why is that? What is that algorithm really doing on a sort of more math-sy level? Right. So I wouldn't say like you had to choose between BED and BOLD. That's actually completely wrong. Now there's another paper written with, with, ⁓ Freddie, right? Freddie Bickford-Smith who I worked with quite a bit. Where we said, actually is reducing uncertainty in the model parameters what you want to do? Probably not. Well, it may be not. And it could be that what you're interested in is specific target predictions. Right. So actually it's not the parameters, the model, right. That are important. It's the predictions that you want to make. And you might have a good idea of the kind of inputs where you want to make good quality predictions. And in this case, there's something that we came up with called EPIG. You should be able to find that. which is just slightly changing the objective in such a way that you should now be focused on making predictions on the specific inputs that are of interest to you. speaker-0: Okay. Yeah. There are so many things. ⁓ Make sure to make sure to add these links, please, to the show notes because I think this is going to be extremely helpful. I already had added a few of them, but you are going to be much, better than me, of course, to do that. And so to add a bit more to the complexity, of course, I've also read on your your blog ⁓ about deep adaptive design. So. What is that? What is the difference with what we just talked about? So bold. And again, when would that be useful? speaker-1: Yeah, again, I think these are all just different forms of the same thing. It's like making something more specific. So this is still BED, it's still based in experimental design. It's just a different sort of way to do it and with different kinds of needs and assumptions. So again, it really comes back to like, what is important in any specific setting? And we kind of make the case in this Deep Adaptive Design paper. that sometimes you're in a case where, for example, in a human computer interaction setting, where you cannot go and do a very large calculation to determine what's the optimal design to ask somebody. so imagine it's like, it's kind imagine it's like an interview, but imagine like one of us is an AI basically. And you want to kind of extract information from, okay, let's, I'm sorry. put this on you, but imagine you're the AI, right? You're the interviewer. I'm the interviewee. You want

to extract information from me and you want to do that as efficiently as possible. Well, use BED, right? We've just been talking about that. How would you extract information from a human? You use BED. The problem is, right? I give you some information. You need to sort of update your internal model where you need to do Bayesian inference. And then you need to sort of go and do this EIG calculation, find the best, you know, candidate model, blah, blah. So, you know, it's not going to be a very natural conversation because, know, your, your kind of AI machine would have to go into this huge calculation. Now, obviously you could just use a fullback, right? You could use a much, a much faster, cheaper heuristic algorithm. And that's what most people do. Right. So for example, in the age of LLM, obviously you could just get the LLM do in context, ⁓ inference, right? Or in sort of more old school stuff, people defined sort of heuristics for certain types of experiment, right? We looked at a, ⁓ it's basically behavioral economics, right? So you, basically, it's kind of like, it's like barter, right? Or, haggling. You're offering people, ⁓ would you take this financial trade off? Would you take this financial trade off? ⁓ You know, would you, I could give you a hundred pounds in a year, or I could give you a 75 today, which would you prefer? Right. And you're, you're trying to build a model of their decision-making process. In that case, you want your AI or your experimental design to respond very fast. And in that case, what you want to do is you want to kind of absorb the new data and generate the optimal design immediately afterwards. So you need to kind of cut out this intermediate computation. So then the idea of deproductive design is that you could pre-train all of that. So you could have a neural network that is sort of pre-trained to take in data and It's kind of like amortized inference. like the step above amortized inference, right? ⁓ Because you know, yeah, but it's like the inputs and outputs are different, right? So with amortized inference, you just get the posterior back. But with this, you just jump straight to the best design, right? So you actually skip the posterior. And so the posterior is just kind of learned implicitly by the model you would think, or the right aspects of it are. Now, obviously there's a bunch of approximations that go into that. speaker-0: it like that? speaker-1: And it's only as good as your neural network design space. But that's basically the idea. So again, it's just BED again, but it's BED in a case where you want this fast call in response and you want to have a pre-trained network that can be adaptive. Right. Because you could also imagine a case where you cannot or don't need to be adaptive. Then you wouldn't use it. Right. Like for example, you know, you're doing high throughput screening in the chemistry lab. Well, it's high, they go in parallel. Right. So it's, it's high throughput because it's parallel. Well, then it's not adaptive because you're not deciding one design on the basis of the first experiment. You have to do them all simultaneously. So it just really comes back to what your, what your need is and how BED actually translates into an algorithm for this case. ⁓ So yeah, deep adaptive design is just about this problem of how do you, how do you adapt very quickly to new data and get very good designs that will keep reducing uncertainty. ⁓ And how do you build one kind end-to-end neural network that can do that? speaker-0: Okay, yes. This is, yeah, that's really fascinating. I didn't know that was possible actually. So I put a link to your blog post about that in the show notes for people who want to give it a look. I will also put episode 107 that we had with Marvin Schmidt from the Baseflow team that explains what Armortized Patient Inference is, for you to understand what that would be when you would use that. And actually next episode, just after you, Adam, will be with Stefan Radeff, who is the creator of Baseflow, the Python package for Amortize patient inference. And yeah, we'll dig a bit more into API, what it is, how to do it right now, these days, because episode 107 was a few months ago and things have changed quite a lot. So how do you... use ABI now in your workflows with ⁓ the new version of Baseflow, the 2.01. But also, ⁓ Stefan and I worked on an agent skill to help you do that when you're working with AI agents. So to train the agent to actually do real good amortized patient inference, like having Stefan ⁓ teaching your agent. how to do that. if you're interested in that, make sure to tune in for next episode. ⁓ So yeah, all of that will be in the show notes. In the meantime, Adam, you've also worked and you work often on protein dynamics, as you were saying at the beginning of the show and quantum chemistry. This is super interesting to me. What is that about? And how does Bayesian experimental design show up there or does it even show up? don't know. speaker-1: Yeah, that's a great question. So, ⁓ yeah, these are some things that I've worked on more recently in my job in Microsoft. ⁓ I think the connection to Bayesian experimental design is maybe a little tangential. think it's something I'd like to bring into those topics more. I think, as I mentioned, you need your model to be Bayesian if you want to use Bayesian experimental design. And I think example, with protein dynamics models, they currently are just sort of like feed forward neural network basically. ⁓ There is an uncertainty module in some of them, but it's not quite the same as a, like a multi, like a hypothesis based model. ⁓ So yeah, I think TBD on the Bayesian experimental design, I mean, I think what people, everyone is agrees on in, for example, in the protein structure sampling community is that more experimental data is good. Experimental data is very important and helpful. And then there's a question about, how do we collect it how do we

choose it? I think, yeah, having worked in the field, I now realize that it's quite a complicated question. I think this is another reason why BED maybe is only kind of like nascent in a lot of fields is that The design space is incredibly complicated. Like this is, this is state of the art stuff. If you're talking about cutting edge, ⁓ you know, ⁓ like biological or biochemical experiments. ⁓ and they come with certain sort of cost implications and, and, ⁓ there are kind of like tricks you might be able to play that, that maybe you would want to play that are not kind of represented necessarily in your kind of Bayesian model. So. I think I would say for protein structure sampling, I'd say really interesting experimental data, incredibly, incredibly important. of TBD on our BED, sorry, it's two acronyms next to each other. It's to be decided how Bayesian experimental design will, will fit into that. think that's, that's pretty cutting edge. and involves speaker-0: Very good, well done, well done. speaker-1: really ⁓ quite a lot of complexity, more than maybe me in my kind of PhD days just doing abstract mathematics might ever appreciate it. Now you also asked about quantum chemistry. So I think that's interesting because the type of quantum chemistry I worked on, it's not supervised learning actually. It's really interesting. It's unsupervised. So there actually isn't any data that you have to get. It's just about doing doing big computation and trying to amortize these big computations. So it's kind of like the deep productive design effort in the deep learning aspects in a way, right? You're trying to amortize a very, expensive computation. So if people are interested in this, it's, ⁓ it's a, I have a blog post called all about all performer. It's all performer in the title. That's the name for the model. ⁓ so yeah, that's about trying to essentially amortize solutions of the Schrodinger equation. ⁓ and in that, in that case, ⁓ yeah, it's not, it's not based on labels. Essentially it's based on, it's based on just pure computation and trying to amortize that. Now there are other parts of the quantum chemistry stack where you, what you, what you might want to do, where you might want to get experimental data, or you might want to use a very expensive quantum chemistry method to label a less expensive one. And I think that's, so that's another experimental design problem, right? Well, which, which ones do I label and which method do I use? What fidelity do I use? I think in certain aspects, this is very studied, not by me, but other people. For example, if you're, if you're training a machine learning force field. So, ⁓ I'm thinking of papers like, NECWIP, which is a machine learning force fields paper. I'm 99 % sure that what they did in that paper was. They sort of ran a kind of molecular dynamic simulation with their model until, they, and they had a Bayesian model that was based on, again, this is, this is my memory of the paper, apologies if

it's wrong. They had a GP based model. So it had its Bayesian uncertainty built in and they basically run their molecular dynamic simulation until that uncertainty was too high. And then they said, okay, now we're going to label more data. ⁓ Again, apologies if my memory of that paper is faulty. ⁓ So that is, that's BED. That's that's a form of BED. It's not, it's not globally optimal across the whole design space, right? You're, using your MD simulation to explore a small part of the design space and then you're choosing based on, on high uncertainty. ⁓ But you know, I think the principle applies. It's the same principle. And the principle does just sort of keep coming back. And I think we'll see. Because that's all in Silico, which is a scientist way of saying it's done on a computer. ⁓ It's kind of easier to create the back and forth ⁓ iterative setup that you would need, right? Where you create, you add more data, you fit your model and you could just leave that running autonomously. Doing that with wet lab is really hard because, you know, these things take months sometimes. speaker-0: Yeah, I can guess. But that was the, I think it was the first time I heard about quantum chemistry. So I was, I was really curious about what that was. Um, which sounds like we could even make a whole episode about that actually. So maybe you should, maybe you should come back on the show later. Um, so I'm going to start closing this out here because it's gonna, uh, it's going to be late for you at some point, but, um, I'm curious, you know, to wrap things up for someone who's listened to us in the past hour and wants to start applying BD and bold practically, not at the frontier like you're doing, but just usefully in the work. Where do you point them? Are there any packages that you recommend, things like that, that people can practically pick up and use to get started? speaker-1: That's a great question. ⁓ I think the first, you know, as we've said, the struggle is that it's such an abstract and multifaceted sort of way of viewing the world that it's very hard to kind of be very specific. So I think the first question is, do you use a Bayesian model? Right. Are you doing Bayesian analysis on your data? If the answer is yes. And do you like it, right? Like do you feel that it's Good is giving you the right uncertainties. You're looking at your posterior predictors and you're happy with it. If that's all check, check, check, then you're a good candidate. Okay. And in that case, the assumption would be that you want to, you want to collect more data that's either very similar or it's literally just more data of the same type that you already have. ⁓ now if, if the same type, that's exactly the setting that's, you know, studied mathematically. So you fit your posterior and you, generate candidate hypotheses and you scan over the candidate designs and you choose the best one. So I think that is, so that would basically be bold, right? I think bold is a good starting point. If you can make bold fit in your setting, just that's probably a great starting point, right? And then there's more stuff you can explore after that by kind of reading around. And then if whatever reason it doesn't work. or it doesn't give you the outcomes you're looking for, then you kind of need to think about, why doesn't it? Right. So I mean, this, you know, I've written a number of papers, right. And they all kind of dig into a different aspect. For example, you might have continuous outcomes and you might feel that just picking a hundred hypotheses and integrating over those to estimate your marginal likelihood is rubbish. Then you might need something like what we studied in the variational paper. You might have a very huge design space of continuous design parameters where you'd be like, you know what, I want to take gradients in the design space. That's something that we studied in another paper. ⁓ You might be in the case, right, as we discussed with this deep adaptive design, where the problem is that you're interacting with humans and you need a very fast kind of adaptive response is you're going to, you're going to kind of consume what they've told you and then immediately give the next question. Then you would want to look at something like that. Or you might be in the case where, like I mentioned, you're interested in predictions on a target set. So your bold is like quite good, but it's not, it's kind of exploring stuff that you don't need to know about because it's not relevant to what you want to test on. And then you might look into Epic and things like that. So I think, yeah. So I recently interacted with, I won't talk about the domain or her words, but with some scientists who wanted to start on this. And I kind of said, well, the very first thing you should do is do the really simple version, right? So you have a small set of candidate designs. You do kind of bold. You generate, you generate a finite set of candidate hypotheses. You look at the predictions for each one, and then you apply the bold formula, which, you know, I don't want to speak it because that would be incomprehensible, but it's kind of a simple formula that's a difference of two entropies basically. And you see if that works. And if that works, you're doing great. And then if you find, ⁓ it doesn't work because I'm estimating entropy of continuous variables, you know, that's one direction to go in. If it doesn't work because all my design space is too huge, you know, that's another direction. So I think I think bolded can be a nice starting point. And I think it's, that is the oldest, the older work within the kind of machine learning that I'm most familiar with. ⁓ where it's specifically thinking about computation. Obviously a lot of people have studied the theory before. so I think that can be a nice starting point and then you can kind of think, well, what's the problem? And then you, and you need to look for the solution. speaker-0:

Yeah, yeah, that makes sense to me and something we often recommend here on the show, like always start with the simplest one and then build your way up to that because otherwise you can get lost pretty easy. ⁓ Maybe one last question before the last questions, of course. ⁓ Just, know, more broadly, as with your vision as a researcher, what do you think is the most underrated or unexplored or underexplored direction in patient experimental design as a whole right now. speaker-1: ⁓ wow. That's really interesting question. The most underexplored area within Bayesian experimental design. think application. think it's been picked up by a sort of statistical machine learning community. And I think there's a lot of really interesting, cool stuff being done there. And I really do think the next step is sort of finding collaborators, finding, and finding quite a few of them. and getting more experience about, okay, well, where does this really change the way people do experiments, right? Is there something that's kind of crying out for this, but hasn't tried or hasn't succeeded due to like computational or theoretical limits, right? Which is the thing that statistical machine learning departments could help you with. They're not going to help you with your specific setup of your wet lab experiments. So I think... Yeah, for a while I've been, the only thing that I've been doing maybe in last year or so is yeah, like answering emails from people who've contacted me, who've been working on very, very specific problems. And I've been sort of trying to help them to set up BED in their specific case. And it's been really, really interesting. I've learned a lot and I think, but also every setting is different, right? And every setting has different requirements and hits different kind of walls and stuff. So. I think that's what's needed for the field is to sort of go deep on certain applications and it's happening, right? mean, Tom Rainforth, think, ⁓ yeah, by the way, if you're like looking to do a PhD or postdoc, you should go and work with Tom Rainforth. If this is the topic, I mean, he's, he's really one of the leading people in this field. And, ⁓ he is pushing forward on, on these questions, right? Like, what, kinds of applications are interesting and where can BED make an impact? But I still think that's probably the thing if I was. back doing that research full time that I would also be interested in, as sort of building collaborations with non-mathematicians who are doing experiments. speaker-0: Yeah, yeah, yeah. And thanks for the call for PhD applications for sure. Like if people are interested in that, should definitely see if you could go to Tom's department. speaker-1: I hope he is actually taking people on. speaker-0: Yeah, now that I said that. ⁓ Well, Adam, that was great. Thank you so much for taking the time. think ⁓ we can call it a show. We've covered so much ground. I'm very happy about that. I think it's going to be a very useful episode and practical one, hopefully for listeners. Before letting you go, of course, I'm going to ask you the last two questions I ask every guest at the end of the show. So first one. If you had unlimited time and resources, which problem would you try to solve? speaker-1: Yeah. I mean, you said unlimited. So I'm going to, I'm going to sort of just take a massive, massive challenge, which is basically AI for science, right? Which is, what I'm working on right now. How do you build an AI model that, understands the real world, like the scientific and physical world? I think it's one of the most intellectually interesting and most impactful for human society challenges that are out there. I mean, it's just, that's a ginormous challenge, but you know, you've given me unlimited resources. So I could burn up a couple of earth copies to solve that. And then, you know, I'll come back to you with my AI model that can understand all of science. speaker-0: Man, yeah, love that. Let me know how I can help. I would definitely be down for that kind of project. yeah, for sure. Definitely agree with the objective and the idea. ⁓ And second question, if you could have dinner with any great scientific mind, dead, alive or fictional, who would be? speaker-1: Yeah, I think, I think just for BED, I would choose Dennis Lindley, very underrated as a statistician from the 1950s. He kind of first wrote down this EIG equation. And I think, you know, it'd be interesting for him to know where it's gone and that people are still working on it. I don't know if he'd be a barrel of laughs. No idea, you know, what his character was, was thought to be like, or what he was like on a personal level. But I, you know, I think, you know, That's the BED choice, let's say. ⁓ If it was more just like a random, exciting person. ⁓ I don't know. No, I stick with Dennis Lindley. speaker-0: Okay, let's do that. ⁓ Awesome. Well, Adam, that's it for today. Thank you so much for taking the time. It was really, really detailed and I really loved it. That was really awesome. Again, folks, show notes are going to be quite detailed for that one. Again, remember that we have ⁓ a new website and a new format for the show notes that I hope is easier for you to pass through. with some key takeaways and the related episodes in their own section and the show notes and even a small blog post I write each time after when the episode is ready to publish. So yeah, Mike, for sure, folks, feel free to look at that if you want to dig deeper. And Adam, thank you again for taking the time and being on this show. speaker-1: Thank you for having me. speaker-0: This has been another episode of Learning Bayesian Statistics. Be sure to rate, review and follow the show on your favorite podcatcher and visit learnbaystats.com for more resources about today's topics as well as access to more episodes to help you reach true Bayesian state of mind. That's learnbaystats.com. Our theme music is Good Bayesian by Baba Brinkman. Fit MC Lass and Meghiraam. Check out his awesome work at bababrinkman.com. I'm your host. Alex and Dora. can follow me on Twitter at Alex underscore and Dora like the country. You can support the show and unlock exclusive benefits by visiting Patreon.com slash LearnBasedDance. Thank you so much for listening and for your support. You're truly a good Bayesian. Change your predictions after taking information in and if you're thinking I'll be less than amazing. Let's adjust those expectations. Let me show you how to be. Good day change calculations after taking fresh data those predictions that your brain is making Let's get them on a solid foundation