The use of the logarithm of the variate in the calculation of differential entropy among certain related statistical distributions

TL;DR

This paper introduces a method to calculate differential entropy and its variance using only the logarithmic transform of the variate, simplifying estimation for certain related distributions.

Contribution

It presents a novel approach to estimate entropy and its variance solely from the logarithmic variate's statistics, bypassing traditional parameter dependence.

Findings

Entropy and variance can be estimated from the logarithmic variate.

The method applies to distributions that are multiples and powers of a common variate.

Estimation relies only on the mean and variance of the log-transformed variate.

Abstract

This paper demonstrates that basic statistics (mean, variance) of the logarithm of the variate itself can be used in the calculation of differential entropy among random variables known to be multiples and powers of a common underlying variate. For the same set of distributions, the variance of the differential self-information is shown also to be a function of statistics of the logarithmic variate. Then entropy and its "variance" can be estimated using only statistics of the logarithmic variate plus constants, without reference to the traditional parameters of the variate.