Nonextensive Generalizations of the Jensen-Shannon Divergence

TL;DR

This paper introduces a nonextensive generalization of the Jensen-Shannon divergence using Tsallis entropy, extending convexity concepts through a new q-convexity framework.

Contribution

It proposes a novel q-convexity concept and constructs the Jensen-Tsallis q-difference, broadening divergence measures in information theory.

Findings

Defines q-convexity satisfying Jensen's q-inequality

Constructs Jensen-Tsallis q-difference as a nonextensive divergence

Characterizes convexity and extrema of the new divergence

Abstract

Convexity is a key concept in information theory, namely via the many implications of Jensen's inequality, such as the non-negativity of the Kullback-Leibler divergence (KLD). Jensen's inequality also underlies the concept of Jensen-Shannon divergence (JSD), which is a symmetrized and smoothed version of the KLD. This paper introduces new JSD-type divergences, by extending its two building blocks: convexity and Shannon's entropy. In particular, a new concept of q-convexity is introduced and shown to satisfy a Jensen's q-inequality. Based on this Jensen's q-inequality, the Jensen-Tsallis q-difference is built, which is a nonextensive generalization of the JSD, based on Tsallis entropies. Finally, the Jensen-Tsallis q-difference is charaterized in terms of convexity and extrema.