A family of statistical symmetric divergences based on Jensen's inequality

TL;DR

This paper introduces a new parametric family of symmetric divergences based on Jensen's inequality, unifying well-known divergences like Jeffreys and Jensen-Shannon, and provides algorithms for computing centroids with experimental validation.

Contribution

It proposes a novel family of symmetric divergences based on Jensen's inequality and develops algorithms for centroid computation, bridging Jeffreys and Jensen-Shannon divergences.

Findings

Unified divergence family connecting Jeffreys and Jensen-Shannon

Algorithm for computing divergence-based centroids

Experimental validation demonstrating practical effectiveness

Abstract

We introduce a novel parametric family of symmetric information-theoretic distances based on Jensen's inequality for a convex functional generator. In particular, this family unifies the celebrated Jeffreys divergence with the Jensen-Shannon divergence when the Shannon entropy generator is chosen. We then design a generic algorithm to compute the unique centroid defined as the minimum average divergence. This yields a smooth family of centroids linking the Jeffreys to the Jensen-Shannon centroid. Finally, we report on our experimental results.