VINA: Variational Invertible Neural Architectures

TL;DR

This paper introduces a unified variational framework for invertible neural networks and normalizing flows, providing theoretical guarantees on their approximation quality and demonstrating practical effectiveness in complex inverse problems.

Contribution

It offers a novel variational approach that unifies INNs and NFs, with theoretical performance guarantees under realistic assumptions, and applies it to real-world inverse problems.

Findings

Theoretical guarantees for posterior and distributional accuracy.

Design principles and practical guidelines for INNs and NFs.

Successful application to ocean-acoustic inversion.

Abstract

The distinctive architectural features of normalizing flows (NFs), notably bijectivity and tractable Jacobians, make them well-suited for generative modeling. Invertible neural networks (INNs) build on these principles to address supervised inverse problems, enabling direct modeling of both forward and inverse mappings. In this paper, we revisit these architectures from both theoretical and practical perspectives and address a key gap in the literature: the lack of theoretical guarantees on approximation quality under realistic assumptions, whether for posterior inference in INNs or for generative modeling with NFs. We introduce a unified framework for INNs and NFs based on variational unsupervised loss functions, inspired by analogous formulations in related areas such as generative adversarial networks (GANs) and the Precision-Recall divergence for training normalizing flows. Within this framework, we derive theoretical performance guarantees, quantifying posterior accuracy for INNs and distributional accuracy for NFs, under assumptions that are weaker and more practically realistic than those used in prior work. Building on these theoretical results, we conduct extensive case studies to distill general design principles and practical guidelines. We conclude by demonstrating the effectiveness of our approach on a realistic ocean-acoustic inversion problem.