Combinatorial Entropy for Distinguishable Entities in Indistinguishable States

TL;DR

This paper explores how different entropy measures arise from combinatorial arrangements of distinguishable entities in indistinguishable states, introducing a new entropy measure for degenerate states.

Contribution

It derives a novel entropy measure for distinguishable entities in degenerate states, expanding the understanding beyond Shannon entropy.

Findings

Degenerate states lead to a new entropy measure involving Stirling numbers.

Non-degenerate cases recover Shannon entropy asymptotically.

Different combinatorial arrangements yield distinct entropy functions.

Abstract

The combinatorial basis of entropy by Boltzmann can be written $H= {N}^{-1} \ln \mathbb{W}$, where $H$ is the dimensionless entropy of a system, per unit entity, $N$ is the number of entities and $\mathbb{W}$ is the number of ways in which a given realization of the system can occur, known as its statistical weight. Maximizing the entropy (``MaxEnt'') of a system, subject to its constraints, is then equivalent to choosing its most probable (``MaxProb'') realization. For a system of distinguishable entities and states, $\mathbb{W}$ is given by the multinomial weight, and $H$ asymptotically approaches the Shannon entropy. In general, however, $\mathbb{W}$ need not be multinomial, leading to different entropy measures. This work examines the allocation of distinguishable entities to non-degenerate or equally degenerate, indistinguishable states. The non-degenerate form converges to the Shannon entropy in some circumstances, whilst the degenerate case gives a new entropy measure, a function of a multinomial coefficient, coding parameters, and Stirling numbers of the second kind.