On conformal divergences and their population minimizers

TL;DR

This paper characterizes the population minimizers of conformal divergences, a broad class including total Bregman divergences, using a geometric framework, and explores their robustness and clustering properties.

Contribution

It introduces conformal divergences as a geometric extension of Bregman divergences and characterizes their population minimizers analytically and geometrically.

Findings

Conformal divergences are essentially exhaustive for their population minimizers.

Total Bregman divergences are characterized as the minimizers of conformal divergences.

The robustness of population minimizers to outliers is extended and analyzed.

Abstract

Total Bregman divergences are a recent tweak of ordinary Bregman divergences originally motivated by applications that required invariance by rotations. They have displayed superior results compared to ordinary Bregman divergences on several clustering, computer vision, medical imaging and machine learning tasks. These preliminary results raise two important problems : First, report a complete characterization of the left and right population minimizers for this class of total Bregman divergences. Second, characterize a principled superset of total and ordinary Bregman divergences with good clustering properties, from which one could tailor the choice of a divergence to a particular application. In this paper, we provide and study one such superset with interesting geometric features, that we call conformal divergences, and focus on their left and right population minimizers. Our results are obtained in a recently coined $(u, v)$-geometric structure that is a generalization of the dually flat affine connections in information geometry. We characterize both analytically and geometrically the population minimizers. We prove that conformal divergences (resp. total Bregman divergences) are essentially exhaustive for their left (resp. right) population minimizers. We further report new results and extend previous results on the robustness to outliers of the left and right population minimizers, and discuss the role of the $(u, v)$-geometric structure in clustering. Additional results are also given.