Bayesian view on the training of invertible residual networks for solving linear inverse problems

TL;DR

This paper explores how Bayesian analysis can clarify the influence of data structure on training invertible residual networks for linear inverse problems, revealing limitations and advantages of different training strategies.

Contribution

It provides a Bayesian perspective on training invertible residual networks, analyzing data dependency and introducing a reconstruction-based training approach with theoretical and empirical insights.

Findings

Reconstruction-based training introduces unique data dependency.

Approximation training cannot achieve the same level of data dependency.

Numerical experiments support the theoretical analysis.

Abstract

Learning-based methods for inverse problems, adapting to the data's inherent structure, have become ubiquitous in the last decade. Besides empirical investigations of their often remarkable performance, an increasing number of works addresses the issue of theoretical guarantees. Recently, [3] exploited invertible residual networks (iResNets) to learn provably convergent regularizations given reasonable assumptions. They enforced these guarantees by approximating the linear forward operator with an iResNet. Supervised training on relevant samples introduces data dependency into the approach. An open question in this context is to which extent the data's inherent structure influences the training outcome, i.e., the learned reconstruction scheme. Here we address this delicate interplay of training design and data dependency from a Bayesian perspective and shed light on opportunities and limitations. We resolve these limitations by analyzing reconstruction-based training of the inverses of iResNets, where we show that this optimization strategy introduces a level of data-dependency that cannot be achieved by approximation training. We further provide and discuss a series of numerical experiments underpinning and extending the theoretical findings.