Symmetrizing Bregman Divergence on the Cone of Positive Definite Matrices: Which Mean to Use and Why

TL;DR

This paper investigates the principles behind symmetrizing Bregman divergences on positive definite matrices, identifying canonical means for different symmetrization types and practical mirror maps.

Contribution

It establishes the canonical means for forward and reverse symmetrizations, linking them to arithmetic, log-Euclidean, and harmonic means for common mirror maps.

Findings

The arithmetic mean is canonical for forward symmetrization.

The arithmetic, log-Euclidean, and harmonic means are canonical for reverse symmetrization.

Results clarify and guide the choice of means in symmetrizing Bregman divergences.

Abstract

This work uncovers variational principles behind symmetrizing the Bregman divergences induced by generic mirror maps over the cone of positive definite matrices. We show that computing the canonical means for this symmetrization can be posed as minimizing the desired symmetrized divergences over a set of mean functionals defined axiomatically to satisfy certain properties. For the forward symmetrization, we prove that the arithmetic mean over the primal space is canonical for any mirror map over the positive definite cone. For the reverse symmetrization, we show that the canonical mean is the arithmetic mean over the dual space, pulled back to the primal space. Applying this result to three common mirror maps used in practice, we show that the canonical means for reverse symmetrization, in those cases, turn out to be the arithmetic, log-Euclidean and harmonic means. Our results improve understanding of existing symmetrization practices in the literature, and can be seen as a navigational chart to help decide which mean to use when.