A note on the quasiconvex Jensen divergences and the quasiconvex Bregman divergences derived thereof

TL;DR

This paper introduces and studies quasiconvex Jensen and Bregman divergences, proposing methods to address their pseudo-divergence issues and demonstrating their applications in probability density comparisons.

Contribution

It defines quasiconvex Jensen and Bregman divergences, proposes $ ext{delta}$-averaged versions to ensure proper divergence properties, and links these divergences to probability density comparisons.

Findings

Quasiconvex Jensen divergences are asymmetric distances.

$ ext{delta}$-averaged quasiconvex Bregman divergences are proper divergences.

These divergences relate to Kullback-Leibler divergences for nested distributions.

Abstract

We first introduce the class of strictly quasiconvex and strictly quasiconcave Jensen divergences which are oriented (asymmetric) distances, and study some of their properties. We then define the strictly quasiconvex Bregman divergences as the limit case of scaled and skewed quasiconvex Jensen divergences, and report a simple closed-form formula which shows that these divergences are only pseudo-divergences at countably many inflection points of the generators. To remedy this problem, we propose the $\delta$-averaged quasiconvex Bregman divergences which integrate the pseudo-divergences over a small neighborhood in order obtain a proper divergence. The formula of $\delta$-averaged quasiconvex Bregman divergences extend even to non-differentiable strictly quasiconvex generators. These quasiconvex Bregman divergences between distinct elements have the property to always have one orientation finite while the other orientation is infinite. We show that these quasiconvex Bregman divergences can also be interpreted as limit cases of generalized skewed Jensen divergences with respect to comparative convexity by using power means. Finally, we illustrate how these quasiconvex Bregman divergences naturally appear as equivalent divergences for the Kullback-Leibler divergences between probability densities belonging to a same parametric family of distributions with nested supports.