The maximum entropy of a metric space

TL;DR

This paper introduces a family of entropies on metric spaces, proves the existence of a universal maximum entropy measure, and links the asymptotics of maximum entropy to geometric properties like volume and dimension.

Contribution

It generalizes Shannon and Rényi entropies to metric spaces and establishes the existence and properties of a universal maximum entropy measure.

Findings

Maximum entropy measure exists on any compact metric space.

Maximum entropy grows with space scaling, revealing geometric info.

The asymptotics of maximum entropy relate to volume and dimension.

Abstract

We define a one-parameter family of entropies, each assigning a real number to any probability measure on a compact metric space (or, more generally, a compact Hausdorff space with a notion of similarity between points). These entropies generalise the Shannon and R\'enyi entropies of information theory. We prove that on any space X, there is a single probability measure maximising all these entropies simultaneously. Moreover, all the entropies have the same maximum value: the maximum entropy of X. As X is scaled up, the maximum entropy grows; its asymptotics determine geometric information about X, including the volume and dimension. We also study the large-scale limit of the maximising measure itself, arguing that it should be regarded as the canonical or uniform measure on X. Primarily we work not with entropy itself but its exponential, called diversity and (in its finite form) used as a measure of biodiversity. Our main theorem was first proved in the finite case by Leinster and Meckes.