Inferring Diversity: Life After Shannon

TL;DR

This paper explores the use of the Maximum Entropy method as a versatile tool for inferring community diversity, extending beyond Shannon's index by incorporating various information types, and relates it to thermodynamic entropy.

Contribution

It demonstrates how the Maximum Entropy framework can generalize diversity measures, surpassing Shannon's index, and connects ecological diversity inference with thermodynamic entropy.

Findings

Maximum Entropy reproduces Shannon's diversity index

ME allows inclusion of diverse information types

Entropy derived aligns with thermodynamic entropy

Abstract

The diversity of a community that cannot be fully counted must be inferred. The two preeminent inference methods are the MaxEnt method, which uses information in the form of constraints and Bayes' rule which uses information in the form of data. It has been shown that these two methods are special cases of the method of Maximum (relative) Entropy (ME). We demonstrate how this method can be used as a measure of diversity that not only reproduces the features of Shannon's index but exceeds them by allowing more types of information to be included in the inference. A specific example is solved in detail. Additionally, the entropy that is found is the same form as the thermodynamic entropy.