How Regularization Terms Make Invertible Neural Networks Bayesian Point Estimators

TL;DR

This paper explores how specific regularization terms in invertible neural networks can produce Bayesian point estimators like the posterior mean and MAP, enhancing stability and interpretability in inverse problems.

Contribution

It introduces two novel regularization terms that enable invertible neural networks to approximate classical Bayesian estimators, with theoretical analysis and numerical validation.

Findings

Regularization terms can induce Bayesian estimator properties in invertible neural networks.

The first regularizer approximates the posterior mean, the second resembles the MAP estimator.

Numerical experiments confirm the stability and interpretability of the proposed methods.

Abstract

Can regularization terms in the training of invertible neural networks lead to known Bayesian point estimators in reconstruction? Invertible networks are attractive for inverse problems due to their inherent stability and interpretability. Recently, optimization strategies for invertible neural networks that approximate either a reconstruction map or the forward operator have been studied from a Bayesian perspective, but each has limitations. To address this, we introduce and analyze two regularization terms for the network training that, upon inversion of the network, recover properties of classical Bayesian point estimators: while the first can be connected to the posterior mean, the second resembles the MAP estimator. Our theoretical analysis characterizes how each loss shapes both the learned forward operator and its inverse reconstruction map. Numerical experiments support our findings and demonstrate how these loss-term regularizers introduce data-dependence in a stable and interpretable way.