Convergence rates for critical point regularization

TL;DR

This paper studies the convergence rates of critical point regularization in inverse problems, especially when using non-convex neural network-based regularizers, and shows finite iteration sufficiency for near-minimization.

Contribution

It extends regularization theory to critical points in non-convex settings and demonstrates finite iteration sufficiency for convergence rates.

Findings

Convergence rates are established for critical point regularization using Bregman distances.

Finite iterations are sufficient for near-minimization without losing convergence properties.

Theoretical analysis applies to non-convex regularizers like neural networks.

Abstract

Tikhonov regularization involves minimizing the combination of a data discrepancy term and a regularizing term, and is the standard approach for solving inverse problems. The use of non-convex regularizers, such as those defined by trained neural networks, has been shown to be effective in many cases. However, finding global minimizers in non-convex situations can be challenging, making existing theory inapplicable. A recent development in regularization theory relaxes this requirement by providing convergence based on critical points instead of strict minimizers. This paper investigates convergence rates for the regularization with critical points using Bregman distances. Furthermore, we show that when implementing near-minimization through an iterative algorithm, a finite number of iterations is sufficient without affecting convergence rates.