TRANSCRIPT 🔗
speaker-0: Yeah
speaker-1: anything you wanna add on that, you know, like ideas that ⁓ that you're paying attention attention to in the adaptation and preconditioning world?
speaker-2: ⁓ yeah, I mean I I not so much adaptation and and preconditioning. I know that ⁓ diffusion models are like all the all the rave right now. and ⁓ I think there's a lot of people working on how to use those to pre condition densities. I I personally don't know much about about those, nor I I'm also not sure if there are actual like software projects, you know, open source projects for wide use, you know, ⁓ with those implemented. ⁓ yeah, I think I guess broadly what I'm most interested in moving forward is different ways of parallelizing MCMC, I think, and making it ⁓ you know scalable to ultra high dimensions, you know, because MCMC is inherently like a very sequential algorithm ⁓ While meanwhile, a lot of the you know, computational advances that we're getting in hardware are for parallel ⁓ computation. So I think that's kind of an interesting there's there's some interesting ideas on how to bridge that gap, but I I would agree that for for preconditioning, yeah, it seem seems like NutPy's state of the art.
speaker-1: Yeah. Mm-hmm. Yeah, I was gonna I was gonna ask you actually, you know what you're gonna what are you gonna focus on in the in the coming month, ⁓ things you wanna learn in particular or or explore?
speaker-2: Yeah, I ⁓ I'm particularly interested in a ⁓ a new way of parallelizing MCMZ over trajectory length. So there's ⁓ so there's a method of ⁓ you basically ch can solve ⁓ an optimization problem for to solve a full MCMT trajectory instead of computing ⁓ in sequence, you can make an initial guess for a whole trajectory and then iterate on it and it provably will converge to the true trajectory, which ⁓ I think could be really cool for problems where the chain actually is ⁓ somewhat predictable to some extent, but the log p evaluations are ⁓ really expensive, for instance. I think it doesn't really work it it's I mean part of what I'm interested in is how That can be done for adaptive ⁓ samplers where it becomes a little more complicated. So that's one thing I'll be focusing on a little bit.
speaker-1: Yeah. ⁓ yeah, sounds like fun. And and Adrian, what about you? Because you always do you always do some fun stuff, investigating some new ways of doing things. ⁓ so yeah, like I'm I'm curious what's on your mind for ⁓ the coming month.
speaker-0: Yeah, as always, I've got more projects running than I can actually finish or I don't know. So that's always a little bit of a problem kind of focusing on particular things that I can actually get done. But ⁓ as I mentioned, MCLMC and kind of the non-adjusting methods, that's something I've been looking into. some reason, I haven't really gotten it to work as well as I wanted to. So I'm not sure how much... ⁓ Maybe it's entirely possible I'm just doing something wrong, of course, so I don't want to bash on the methods at all in any way. But ⁓ so far I didn't have too much luck, but that could be me. So it's a question of how much I want to try that more. ⁓ Definitely I'm looking into kind of what I mentioned a little bit in the beginning is of kind of using higher order autodiff ⁓ as well. So not just the Fisher divergence. But what happens if we also use second order derivatives? I think there's really cool theory that could tell us how to do that. ⁓ So that could be really, really useful. ⁓ interesting. No clue if it's actually will turn out to be useful, but fun for sure.