Characterizations of inexact proximal operators

TL;DR

This paper characterizes inexact proximal operators, exploring their properties and impact on the convergence of various proximal algorithms, especially when approximations are non-ideal or errors persist.

Contribution

It extends the characterization of proximal operators to inexact versions, analyzing their regularity, approximation quality, and effects on algorithm convergence.

Findings

Inexact proximal operators can be effectively characterized for non-convex penalties.

Convergence of proximal algorithms is maintained under certain inexactness conditions.

Non-summable and non-vanishing errors can still allow convergence with proper inexactness control.

Abstract

Proximal operators are now ubiquitous in non-smooth optimization. Since their introduction in the seminal work of Moreau, many papers have shown their effectiveness on a wide variety of problems, culminating in their use to construct convergent deep learning methods. The characterization of these operators for non-convex penalties was completed recently in [Gribonval et al, A characterization of proximity operators, 2020]. In this paper, we propose to follow this line of work by characterizing inexact proximal operators, thus providing an answer to what constitutes a good approximation of these operators. We propose several definitions of approximations and discuss their regularity, approximation power, and their fixed points. Equipped with these characterizations, we investigate the convergence of proximal algorithms in the presence of errors that may be non-summable and/or non-vanishing. In particular, we look at the proximal point algorithm, and at the forward-backward, Peaceman-Rachford and Douglas-Rachford algorithms when we minimize the sum of a weakly convex function (whose proximal operator is approximated) and a strongly convex function.