Bounds of Shannon entropy and Extropy and their application in exploring the extreme value behavior of a large set of data

TL;DR

This paper establishes bounds for Shannon entropy and extropy in the context of extreme value theory, characterizes the exponential distribution as maximizing these measures, and proves their convergence to Gumbel distribution values for large samples.

Contribution

It derives bounds for Shannon entropy and extropy related to extreme values, characterizes the exponential distribution as a maximizer, and offers an alternative proof of convergence to Gumbel distribution.

Findings

Bounds for entropy and extropy of maximum values are derived.

Exponential distribution maximizes entropy and extropy in large samples.

Shannon entropy and extropy converge to Gumbel distribution values for large samples.

Abstract

This paper derives bounds for two omnipresent information theoretic measures, the Shannon entropy and its complementary dual, the extropy. Based on a large size data set from a logconcave model, the said bounds are obtained for the entropy and the extropy of the distribution of the largest order statistic and the respective normalized sequence, in the extreme value theory setting. A characterization of the exponential distribution is provided as the model that maximizes the Shannon entropy and the extropy which are associated with the distribution of the maximum value, in a large sample size regime. This characterization is exploited to provide an alternative, immediate proof of the convergence of Shannon entropy and extropy of the normalized maxima of a large size sample to the respective measures for the Gumbel distribution, studied recently for Shannon entropy in Johnson (2024) and references therein.