Entropy Moments Characterization of Statistical Distributions

TL;DR

This paper introduces two new moment-based extensions of entropy to better distinguish between statistical distributions with similar traditional entropy values, enhancing the analysis of distribution characteristics.

Contribution

It proposes two novel entropy moment extensions and demonstrates their effectiveness in differentiating distributions with similar traditional entropies.

Findings

Entropy moments can distinguish distributions with identical traditional entropy.

The proposed extensions are based on moments of the entropy function.

Alternative entropy moments provide additional discriminative power.

Abstract

This letter reports two moment extensions of the entropy of a distribution. By understanding the traditional entropy as the average of the original distribution up to a random variable transformation, the traditional moments equation become immediately applicable to entropy. We also suggest an alternative family of entropy moments. The discriminative potential of such entropy moment extensions is illustrated with respect to different types of distributions with otherwise undistinguishable traditional entropies.