# Bregman Proximal Mappings and Bregman-Moreau Envelopes under Relative Prox-Regularity

## TL;DR

This paper investigates the properties of Bregman proximal mappings and Bregman--Moreau envelopes for nonconvex functions under relative prox-regularity, extending classical concepts to broader, possibly nonconvex domains with applications to optimization algorithms.

## Contribution

It introduces the concept of relative prox-regularity for Bregman proximal mappings, extending the class of functions for which local properties are established, and applies these results to Bregman minimization algorithms.

## Key findings

- Established local single-valuedness of Bregman proximal mappings under relative prox-regularity.
- Demonstrated local smoothness of Bregman--Moreau envelopes for a broad class of nonconvex functions.
- Applied the theory to interpret Bregman minimization algorithms in a new framework.

## Abstract

We systematically study the local single-valuedness of the Bregman proximal mapping and local smoothness of the Bregman--Moreau envelope of a nonconvex function under relative prox-regularity - an extension of prox-regularity - which was originally introduced by Poliquin and Rockafellar. As Bregman distances are asymmetric in general, in accordance with Bauschke et al., it is natural to consider two variants of the Bregman proximal mapping, which, depending on the order of the arguments, are called left and right Bregman proximal mapping. We consider the left Bregman proximal mapping first. Then, via translation result, we obtain analogue (and partially sharp) results for the right Bregman proximal mapping. The class of relatively prox-regular functions significantly extends the recently considered class of relatively hypoconvex functions. In particular, relative prox-regularity allows for functions with a possibly nonconvex domain. Moreover, as a main source of examples and analogously to the classical setting, we introduce relatively amenable functions, i.e. convexly composite functions, for which the inner nonlinear mapping is component-wise smooth adaptable, a recently introduced extension of Lipschitz differentiability. By way of example, we apply our theory to locally interpret joint alternating Bregman minimization with proximal regularization as a Bregman proximal gradient algorithm, applied to a smooth adaptable function.