Entropies, cross-entropies and R\'enyi divergence: sharp three-term inequalities for probability density functions

TL;DR

This paper establishes a new sharp inequality linking differential Re9nyi entropy, divergence, and cross-entropy, with equality conditions and applications to various informational functionals, advancing theoretical understanding in information theory.

Contribution

The paper introduces a novel sharp inequality involving Re9nyi entropy, divergence, and cross-entropy, with equality conditions and broad applications to informational measures.

Findings

Sharp inequality involving Re9nyi entropy, divergence, and cross-entropy.

Equality condition when one density is an escort of the other.

Bounded Re9nyi divergence by quotients of informational functionals.

Abstract

A new sharp inequality featuring the differential R\'enyi entropy, the R\'enyi divergence and the R\'enyi cross-entropy of a pair of probability density functions is established. The equality is reached when one of the probability density function is an escort density of the other. This inequality is applied, together with a general framework of a pair of transformations reciprocal to each other, to derive a number of further inequalities involving both classical and new informational functionals. A remarkable fact is that, in all these inequalities, the R\'enyi divergence of two probability density functions is sharply bounded by quotients of informational functionals of cross-type and single type. More precisely, we derive sharp inequalities composed by relative and cross versions of the absolute moments, or of the Fisher information measures (among others), and involving two and three probability density functions.