Implicit Regularization of the Deep Inverse Prior Trained with Inertia

TL;DR

This paper establishes convergence and recovery guarantees for self-supervised neural networks used in inverse problems, demonstrating accelerated training dynamics with inertia and Hessian-driven damping.

Contribution

It provides the first theoretical analysis of inertia-based training algorithms for inverse problem neural networks, including convergence rates and recovery guarantees.

Findings

Continuous-time analysis shows optimal accelerated exponential convergence.

Inertial algorithms achieve similar recovery guarantees with linear convergence.

The study bridges dynamical systems and neural network training for inverse problems.

Abstract

Solving inverse problems with neural networks benefits from very few theoretical guarantees when it comes to the recovery guarantees. We provide in this work convergence and recovery guarantees for self-supervised neural networks applied to inverse problems, such as Deep Image/Inverse Prior, and trained with inertia featuring both viscous and geometric Hessian-driven dampings. We study both the continuous-time case, i.e., the trajectory of a dynamical system, and the discrete case leading to an inertial algorithm with an adaptive step-size. We show in the continuous-time case that the network can be trained with an optimal accelerated exponential convergence rate compared to the rate obtained with gradient flow. We also show that training a network with our inertial algorithm enjoys similar recovery guarantees though with a less sharp linear convergence rate.