Bootstrap Your Flow
Laurence Illing Midgley, Vincent Stimper, Gregor N. C. Simm, Jos\'e, Miguel Hern\'andez-Lobato

TL;DR
This paper introduces FAB, a novel training method combining normalizing flows with annealed importance sampling using alpha-divergence, enabling accurate approximation of complex distributions where previous methods struggle.
Contribution
The paper proposes FAB, a new bootstrapping approach that integrates flows with AIS and alpha-divergence, improving the approximation of challenging target distributions.
Findings
FAB produces accurate approximations to complex distributions.
It outperforms previous flow-based methods on challenging targets.
FAB effectively combines flow models with AIS for better sampling.
Abstract
Normalizing flows are flexible, parameterized distributions that can be used to approximate expectations from intractable distributions via importance sampling. However, current flow-based approaches are limited on challenging targets where they either suffer from mode seeking behaviour or high variance in the training loss, or rely on samples from the target distribution, which may not be available. To address these challenges, we combine flows with annealed importance sampling (AIS), while using the -divergence as our objective, in a novel training procedure, FAB (Flow AIS Bootstrap). Thereby, the flow and AIS improve each other in a bootstrapping manner. We demonstrate that FAB can be used to produce accurate approximations to complex target distributions, including Boltzmann distributions, in problems where previous flow-based methods fail.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGaussian Processes and Bayesian Inference · Adversarial Robustness in Machine Learning · Model Reduction and Neural Networks
