On Shore and Johnson properties for a Special Case of Csisz\'ar f-divergences

TL;DR

This paper investigates the properties of minimizing Tsallis divergence, a special case of Csiszár f-divergences, highlighting its relation to power-law distributions and examining key properties like the Pythagorean property.

Contribution

It analyzes the Shore and Johnson properties for Tsallis divergence, extending the understanding of divergence minimization in the context of power-law distributions.

Findings

Tsallis divergence exhibits Shore and Johnson properties.

Minimization of Tsallis divergence relates to power-law distributions.

The Pythagorean property holds for Tsallis divergence.

Abstract

The importance of power-law distributions is attributed to the fact that most of the naturally occurring phenomenon exhibit this distribution. While exponential distributions can be derived by minimizing KL-divergence w.r.t some moment constraints, some power law distributions can be derived by minimizing some generalizations of KL-divergence (more specifically some special cases of Csisz\'ar f-divergences). Divergence minimization is very well studied in information theoretical approaches to statistics. In this work we study properties of minimization of Tsallis divergence, which is a special case of Csisz\'ar f-divergence. In line with the work by Shore and Johnson (IEEE Trans. IT, 1981), we examine the properties exhibited by these minimization methods including the Pythagorean property.