Variational Transdimensional Inference

TL;DR

This paper introduces CoSMIC, a novel normalizing flow architecture, and VTI, a stochastic variational inference method, enabling efficient Bayesian inference over complex, transdimensional model spaces.

Contribution

The paper presents CoSMIC, a new flow-based model for transdimensional inference, and VTI, a training approach combining Bayesian optimization and Monte Carlo gradients.

Findings

VTI performs well on high-cardinality model spaces.

CoSMIC enables inference over multi-model, transdimensional distributions.

The approach scales to complex, real-world problems.

Abstract

The expressiveness of flow-based models combined with stochastic variational inference (SVI) has expanded the application of optimization-based Bayesian inference to highly complex problems. However, despite the importance of multi-model Bayesian inference for problems defined on a transdimensional joint model and parameter space, such as Bayesian structure learning and model selection, flow-based SVI has been limited to problems defined on a fixed-dimensional parameter space. We introduce CoSMIC, normalizing flows (COntextually-Specified Masking for Identity-mapped Components), an extension to neural autoregressive conditional normalizing flow architectures that enables use of a single flow-based variational density for inference over a transdimensional (multi-model) conditional target distribution. We propose a combined stochastic variational transdimensional inference (VTI) approach to training CoSMIC, flows using ideas from Bayesian optimization and Monte Carlo gradient estimation. Numerical experiments show the performance of VTI on challenging problems that scale to high-cardinality model spaces.