Variational Inference with Continuously-Indexed Normalizing Flows

TL;DR

This paper introduces a novel way to incorporate Continuously-Indexed Flows into variational inference, enabling more expressive posterior approximations for complex distributions, and demonstrates improved performance over baseline flows.

Contribution

The authors develop a method to use CIFs within an auxiliary VI framework, leveraging their structure to produce low-variance estimators and better approximate complex posteriors.

Findings

CIFs can be integrated into VI as auxiliary models.

Empirical results show CIFs outperform baseline flows in complex posteriors.

Improved evidence estimation in Bayesian inference and maximum likelihood.

Abstract

Continuously-indexed flows (CIFs) have recently achieved improvements over baseline normalizing flows on a variety of density estimation tasks. CIFs do not possess a closed-form marginal density, and so, unlike standard flows, cannot be plugged in directly to a variational inference (VI) scheme in order to produce a more expressive family of approximate posteriors. However, we show here how CIFs can be used as part of an auxiliary VI scheme to formulate and train expressive posterior approximations in a natural way. We exploit the conditional independence structure of multi-layer CIFs to build the required auxiliary inference models, which we show empirically yield low-variance estimators of the model evidence. We then demonstrate the advantages of CIFs over baseline flows in VI problems when the posterior distribution of interest possesses a complicated topology, obtaining improved results in both the Bayesian inference and surrogate maximum likelihood settings.