The Generalized Bregman Distance

TL;DR

This paper introduces the generalized Bregman distance, extending classical Bregman distances to a broader setting involving maximally monotone operators, with applications to optimization and information theory.

Contribution

It defines and analyzes the generalized Bregman distance, providing conditions for convexity and coercivity, and explores its connections to the Fitzpatrick function and Kullback-Leibler divergence.

Findings

Established convexity and coercivity conditions for the generalized Bregman distance.

Connected the generalized Bregman distance to the Fitzpatrick function and conjugates.

Demonstrated applications involving the Kullback-Leibler divergence and Lambert W function.

Abstract

Recently, a new distance has been introduced for the graphs of two point-to-set operators, one of which is maximally monotone. When both operators are the subdifferential of a proper lower semicontinuous convex function, this distance specializes under modest assumptions to the classical Bregman distance. We name this new distance the generalized Bregman distance, and we shed light on it with examples that utilize the other two most natural representative functions: the Fitzpatrick function and its conjugate. We provide sufficient conditions for convexity, coercivity, and supercoercivity: properties that are essential for implementation in proximal point type algorithms. We establish these results for both the left and right variants of this new distance. We construct examples closely related to the Kullback--Leibler divergence, which was previously considered in the context of Bregman distances, and whose importance in information theory is well known. In so doing, we demonstrate how to compute a difficult Fitzpatrick conjugate function, and we discover natural occurrences of the Lambert $\W$ function, whose importance in optimization is of growing interest.