On some entropy functionals derived from R\'enyi information divergence

TL;DR

This paper explores maximum entropy problems using Rnyi entropy under different expectation constraints, deriving optimal distributions with power-law behaviors and analyzing properties of associated entropy functionals.

Contribution

It introduces new entropy functionals derived from Rnyi divergence, characterizes their properties, and connects them to known entropies in specific cases.

Findings

Derived maximum entropy distributions with power-law behavior.

Defined and analyzed properties of new entropy functionals.

Connected these functionals to classical entropies in special cases.

Abstract

We consider the maximum entropy problems associated with R\'enyi $Q$-entropy, subject to two kinds of constraints on expected values. The constraints considered are a constraint on the standard expectation, and a constraint on the generalized expectation as encountered in nonextensive statistics. The optimum maximum entropy probability distributions, which can exhibit a power-law behaviour, are derived and characterized. The R\'enyi entropy of the optimum distributions can be viewed as a function of the constraint. This defines two families of entropy functionals in the space of possible expected values. General properties of these functionals, including nonnegativity, minimum, convexity, are documented. Their relationships as well as numerical aspects are also discussed. Finally, we work out some specific cases for the reference measure $Q(x)$ and recover in a limit case some well-known entropies.