Typicality, entropy and the generalization of statistical mechanics

TL;DR

This paper explores how the concept of typicality and entropy, fundamental in statistical mechanics, can be extended to complex systems beyond traditional assumptions, using toy models and entropy generalizations.

Contribution

It introduces a framework connecting typicality, entropy measures, and thermodynamic quantities, proposing ways to generalize statistical mechanics to complex systems.

Findings

Typicality relates to Shannon, Renyi, and Tsallis entropies.

Connections between typical sets, free energy, and partition functions.

Potential generalizations of typicality to non-traditional systems.

Abstract

When at equilibrium, large-scale systems obey conventional thermodynamics because they belong to microscopic configurations (or states) that are typical. Crucially, the typical states usually represent only a small fraction of the total number of possible states, and yet the characterization of the set of typical states -- the typical set -- alone is sufficient to describe the macroscopic behavior of a given system. Consequently, the concept of typicality, and the associated Asymptotic Equipartition Property allow for a drastic reduction of the degrees of freedom needed for system's statistical description. The mathematical rationale for such a simplification in the description is due to the phenomenon of concentration of measure. The later emerges for equilibrium configurations thanks to very strict constraints on the underlying dynamics, such as weekly interacting and (almost) independent system constituents. The question naturally arises as to whether the concentration of measure and related typicality considerations can be extended and applied to more general complex systems, and if so, what mathematical structure can be expected in the ensuing generalized thermodynamics. In this paper we illustrate the relevance of the concept of typicality in the toy model context of the "thermalized" coin and show how this leads naturally to Shannon entropy. We also show an intriguing connection: The characterization of typical sets in terms of Renyi and Tsallis entropies naturally leads to the free energy and partition function, respectively, and makes their relationship explicit. Finally, we propose potential ways to generalize the concept of typicality to systems where the standard microscopic assumptions do not hold.