# Equivalence of Informations Characterizes Bregman Divergences

## TL;DR

This paper shows that Bregman divergences are uniquely defined by their ability to equate two measures of information in data sets.

## Contribution

The paper proves that Bregman divergences are the only class of divergences that equate Jensen gap and divergence information for arbitrary weighted data sets.

## Key findings

- Bregman divergences uniquely equate Jensen gap and divergence information measures.
- This equivalence characterizes the entire class of Bregman divergences.
- The result holds for arbitrary weighted sets of data vectors.

## Abstract

Bregman divergences form a class of distance-like comparison functions which plays fundamental roles in optimization, statistics, and information theory. One important property of Bregman divergences is that they generate agreement between two useful formulations of information content (in the sense of variability or non-uniformity) in weighted collections of vectors. The first of these is the Jensen gap information, which measures the difference between the mean value of a strictly convex function evaluated on a weighted set of vectors and the value of that function evaluated at the centroid of that collection. The second of these is the divergence information, which measures the mean divergence of the vectors in the collection from their centroid. In this brief note, we prove that the agreement between Jensen gap and divergence informations in fact characterizes the class of Bregman divergences; they are the only divergences that generate this agreement for arbitrary weighted sets of data vectors.