R\'enyi entropy, abundance distribution and the equivalence of ensembles

TL;DR

This paper establishes a mathematical link between abundance distributions and Re9nyi entropy using thermodynamic analogies, enabling geometric construction of diversity measures and highlighting limitations when the thermodynamic limit is not well-defined.

Contribution

It introduces a novel thermodynamic framework relating abundance distributions to Re9nyi entropy, providing geometric tools and insights into the behavior of diversity measures.

Findings

Re9nyi entropy relates to abundance distributions via Legendre transform.

Non-concave rank-frequency regions cause kinks in Re9nyi entropy.

Limitations of ensemble equivalence are highlighted outside the thermodynamic limit.

Abstract

Distributions of abundances or frequencies play an important role in many fields of science, from biology to sociology, as does the R\'enyi entropy, which measures the diversity of a statistical ensemble. We derive a mathematical relation between the abundance distribution and the R\'enyi entropy, by analogy with the equivalence of ensembles in thermodynamics. The abundance distribution is mapped onto the density of states, and the R\'enyi entropy to the free energy. The two quantities are related in the thermodynamic limit by a Legendre transform, by virtue of the equivalence between the micro-canonical and canonical ensembles. In this limit, we show how the R\'enyi entropy can be constructed geometrically from rank-frequency plots. This mapping predicts that non-concave regions of the rank-frequency curve should result in kinks in the R\'enyi entropy as a function of its order. We illustrate our results on simple examples, and emphasize the limitations of the equivalence of ensembles when a thermodynamic limit is not well defined. Our results help choose reliable diversity measures based on the experimental accuracy of the abundance distributions in particular frequency ranges.