Multiscale Invertible Generative Networks for High-Dimensional Bayesian Inference

TL;DR

This paper introduces MsIGN, a multiscale invertible generative network designed for high-dimensional Bayesian inference, which effectively captures complex posteriors and improves sample quality by leveraging multiscale structures and a multi-stage training process.

Contribution

The paper presents a novel multiscale invertible generative network that addresses high-dimensional Bayesian inference challenges using a multi-stage training method minimizing Jeffreys divergence.

Findings

MsIGN outperforms previous methods in posterior approximation and mode capturing.

MsIGN achieves better bits-per-dimension in natural image synthesis.

MsIGN provides interpretability of neurons in intermediate layers.

Abstract

We propose a Multiscale Invertible Generative Network (MsIGN) and associated training algorithm that leverages multiscale structure to solve high-dimensional Bayesian inference. To address the curse of dimensionality, MsIGN exploits the low-dimensional nature of the posterior, and generates samples from coarse to fine scale (low to high dimension) by iteratively upsampling and refining samples. MsIGN is trained in a multi-stage manner to minimize the Jeffreys divergence, which avoids mode dropping in high-dimensional cases. On two high-dimensional Bayesian inverse problems, we show superior performance of MsIGN over previous approaches in posterior approximation and multiple mode capture. On the natural image synthesis task, MsIGN achieves superior performance in bits-per-dimension over baseline models and yields great interpret-ability of its neurons in intermediate layers.