Instance Optimal Decoding and the Restricted Isometry Property

TL;DR

This paper establishes conditions for robust, noise-tolerant decoding in non-linear inverse problems using a non-linear Restricted Isometry Property, with applications to neural network invertibility.

Contribution

It introduces a non-linear LRIP framework for analyzing instance optimal decoders and provides new conditions for non-uniform recovery in inverse problems.

Findings

Necessary and sufficient conditions for instance optimal decoding.

New formulation of the non-linear LRIP.

Application to invertibility of neural networks with random weights.

Abstract

In this paper, we address the question of information preservation in ill-posed, non-linear inverse problems, assuming that the measured data is close to a low-dimensional model set. We provide necessary and sufficient conditions for the existence of a so-called instance optimal decoder, i.e., that is robust to noise and modelling error. Inspired by existing results in compressive sensing, our analysis is based on a (Lower) Restricted Isometry Property (LRIP), formulated in a non-linear fashion. We also provide sufficient conditions for non-uniform recovery with random measurement operators, with a new formulation of the LRIP. We finish by describing typical strategies to prove the LRIP in both linear and non-linear cases, and illustrate our results by studying the invertibility of a one-layer neural net with random weights.