On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means

TL;DR

This paper introduces a generalized Jensen-Shannon divergence using abstract means, providing closed-form formulas for specific distribution families like exponential and Cauchy, enhancing divergence computation and clustering methods.

Contribution

It proposes a novel generalization of Jensen-Shannon divergence via abstract means, enabling closed-form expressions for complex distributions and matrix divergences.

Findings

Closed-form formulas for geometric Jensen-Shannon divergence in exponential families.

Closed-form formulas for harmonic Jensen-Shannon divergence in scale Cauchy distributions.

Application of generalized Jensen-Shannon divergences to matrix and quantum divergences.

Abstract

The Jensen-Shannon divergence is a renown bounded symmetrization of the unbounded Kullback-Leibler divergence which measures the total Kullback-Leibler divergence to the average mixture distribution. However the Jensen-Shannon divergence between Gaussian distributions is not available in closed-form. To bypass this problem, we present a generalization of the Jensen-Shannon (JS) divergence using abstract means which yields closed-form expressions when the mean is chosen according to the parametric family of distributions. More generally, we define the JS-symmetrizations of any distance using generalized statistical mixtures derived from abstract means. In particular, we first show that the geometric mean is well-suited for exponential families, and report two closed-form formula for (i) the geometric Jensen-Shannon divergence between probability densities of the same exponential family, and (ii) the geometric JS-symmetrization of the reverse Kullback-Leibler divergence. As a second illustrating example, we show that the harmonic mean is well-suited for the scale Cauchy distributions, and report a closed-form formula for the harmonic Jensen-Shannon divergence between scale Cauchy distributions. We also define generalized Jensen-Shannon divergences between matrices (e.g., quantum Jensen-Shannon divergences) and consider clustering with respect to these novel Jensen-Shannon divergences.