Model-Informed Flows for Bayesian Inference

TL;DR

This paper introduces Model-Informed Flows (MIF), a novel variational inference architecture that combines flow-based models with prior information to better approximate complex hierarchical Bayesian posteriors.

Contribution

It provides a theoretical link between VIP and autoregressive flows, leading to the development of MIF, which improves posterior approximation quality.

Findings

MIF achieves tighter posterior approximations.

MIF matches or exceeds state-of-the-art performance.

MIF effectively handles hierarchical Bayesian models.

Abstract

Variational inference often struggles with the posterior geometry exhibited by complex hierarchical Bayesian models. Recent advances in flow-based variational families and Variationally Inferred Parameters (VIP) each address aspects of this challenge, but their formal relationship is unexplored. Here, we prove that the combination of VIP and a full-rank Gaussian can be represented exactly as a forward autoregressive flow augmented with a translation term and input from the model's prior. Guided by this theoretical insight, we introduce the Model-Informed Flow (MIF) architecture, which adds the necessary translation mechanism, prior information, and hierarchical ordering. Empirically, MIF delivers tighter posterior approximations and matches or exceeds state-of-the-art performance across a suite of hierarchical and non-hierarchical benchmarks.