Weighted Approach to R\'enyi Entropy

TL;DR

This paper introduces a weighted R'enyi entropy framework that enables estimation of the entropy of source mixtures and their entropy dimensions, providing new theoretical tools for entropy analysis.

Contribution

It presents a novel weighted R'enyi entropy definition and establishes its equivalence to classical R'enyi entropy, facilitating entropy estimation for mixtures.

Findings

Derived estimations for R'enyi entropy of mixtures

Established relation between weighted and classical R'enyi entropy

Enabled computation of R'enyi entropy dimension for mixtures

Abstract

R\'enyi entropy of order \alpha is a general measure of entropy. In this paper we derive estimations for the R\'enyi entropy of the mixture of sources in terms of the entropy of the single sources. These relations allow to compute the R\'enyi entropy dimension of arbitrary order of a mixture of measures. The key for obtaining these results is our new definition of the weighted R\'enyi entropy. It is shown that weighted entropy is equal to the classical R\'enyi entropy.