Equivalence of Informations Characterizes Bregman Divergences

TL;DR

This paper characterizes Bregman divergences as the unique class of divergences that ensure two formulations of information content in weighted vector collections always agree, highlighting their fundamental role in information theory.

Contribution

It proves that the property of agreement in information content formulations uniquely characterizes Bregman divergences among all divergence functions.

Findings

Bregman divergences cause two formulations of information content to agree.

This agreement property uniquely characterizes Bregman divergences.

The result links divergence properties to fundamental information measures.

Abstract

Bregman divergences are a class of distance-like comparison functions which play fundamental roles in optimization, statistics, and information theory. One important property of Bregman divergences is that they cause two useful formulations of information content (in the sense of variability or non-uniformity) in a weighted collection of vectors to agree. In this note, we show that this agreement in fact characterizes the class of Bregman divergences; they are the only divergences which generate this agreement for arbitrary collections of weighted vectors.