On a generalization of the Jensen-Shannon divergence

TL;DR

This paper introduces a new vector-skew generalization of Jensen-Shannon divergence, explores its properties, and provides an algorithm for computing Jensen-Shannon-type centroids for various probability distributions.

Contribution

It presents a novel vector-skew generalization of Jensen-Shannon divergence and an iterative algorithm for centroid computation in mixture families.

Findings

New vector-skew Jensen-Shannon divergence introduced

Properties of the divergence studied and characterized

Algorithm for centroid computation demonstrated on categorical distributions

Abstract

The Jensen-Shannon divergence is a renown bounded symmetrization of the Kullback-Leibler divergence which does not require probability densities to have matching supports. In this paper, we introduce a vector-skew generalization of the scalar $\alpha$-Jensen-Bregman divergences and derive thereof the vector-skew $\alpha$-Jensen-Shannon divergences. We study the properties of these novel divergences and show how to build parametric families of symmetric Jensen-Shannon-type divergences. Finally, we report an iterative algorithm to numerically compute the Jensen-Shannon-type centroids for a set of probability densities belonging to a mixture family: This includes the case of the Jensen-Shannon centroid of a set of categorical distributions or normalized histograms.