Bernstein Flows for Flexible Posteriors in Variational Bayes

TL;DR

This paper introduces Bernstein flow variational inference (BF-VI), a novel method combining normalizing flows and Bernstein polynomials to accurately approximate complex posteriors in Bayesian inference, outperforming existing VI techniques especially in higher dimensions.

Contribution

The paper proposes BF-VI, a new flexible variational inference method that effectively models complex multivariate posteriors using Bernstein polynomial transformations.

Findings

BF-VI accurately approximates true posteriors in low-dimensional models.

BF-VI outperforms other VI methods in higher-dimensional models.

Demonstrated application of VI in semi-structured models for melanoma data.

Abstract

Variational inference (VI) is a technique to approximate difficult to compute posteriors by optimization. In contrast to MCMC, VI scales to many observations. In the case of complex posteriors, however, state-of-the-art VI approaches often yield unsatisfactory posterior approximations. This paper presents Bernstein flow variational inference (BF-VI), a robust and easy-to-use method, flexible enough to approximate complex multivariate posteriors. BF-VI combines ideas from normalizing flows and Bernstein polynomial-based transformation models. In benchmark experiments, we compare BF-VI solutions with exact posteriors, MCMC solutions, and state-of-the-art VI methods including normalizing flow based VI. We show for low-dimensional models that BF-VI accurately approximates the true posterior; in higher-dimensional models, BF-VI outperforms other VI methods. Further, we develop with BF-VI a Bayesian model for the semi-structured Melanoma challenge data, combining a CNN model part for image data with an interpretable model part for tabular data, and demonstrate for the first time how the use of VI in semi-structured models.