Classical dynamical coarse-grained entropy and comparison with the quantum version

TL;DR

This paper develops a rigorous classical observational entropy framework for non-equilibrium thermodynamics, compares it with quantum entropy, and explores its properties and correspondence with quantum concepts.

Contribution

It introduces a detailed classical observational entropy framework, linking it to quantum entropy and analyzing its properties and classical-quantum correspondence.

Findings

Classical observational entropy is well-defined out of equilibrium.

It is additive and approaches thermodynamic entropy at equilibrium.

The classical-quantum entropy correspondence is direct but affected by non-commutativity.

Abstract

We develop the framework of classical Observational entropy, which is a mathematically rigorous and precise framework for non-equilibrium thermodynamics, explicitly defined in terms of a set of observables. Observational entropy can be seen as a generalization of Boltzmann entropy to systems with indeterminate initial conditions, and describes the knowledge achievable about the system by a macroscopic observer with limited measurement capabilities; it becomes Gibbs entropy in the limit of perfectly fine-grained measurements. This quantity, while previously mentioned in the literature, has been investigated in detail only in the quantum case. We describe this framework reasonably pedagogically, then show that in this framework, certain choices of coarse-graining lead to an entropy that is well-defined out of equilibrium, additive on independent systems, and that grows towards thermodynamic entropy as the system reaches equilibrium, even for systems that are genuinely isolated. Choosing certain macroscopic regions, this dynamical thermodynamic entropy measures how close these regions are to thermal equilibrium. We also show that in the given formalism, the correspondence between classical entropy (defined on classical phase space) and quantum entropy (defined on Hilbert space) becomes surprisingly direct and transparent, while manifesting differences stemming from non-commutativity of coarse-grainings and from non-existence of a direct classical analogue of quantum energy eigenstates.

Figures

Tables

Table 1. Table 1: Classical and quantum descriptions of an isolated physical system.

Classical	Quantum
phase-space $\Gamma$	Hilbert space $\mathscr{H}$
defines a classical system,	defines a quantum system,
is a space of all possible classical states,	is a space of all possible quantum states,
all states are orthogonal,	employs Hilbert space inner product,
employs $L^2$-inner product for observables	employs Hilbert-Schmidt inner product for observables.
Point in phase-space $(𝒙,𝒑)$	Vector in Hilbert space $|\psi ⟩$ – wave-function
describes a state of a classical system,	describes a state of a quantum system,
is evolved through Hamilton’s equations of motion,	is evolved through Schrödinger equation,
is also called a microstate.	is also called a microstate.
phase-space density $\rho$	Density matrix $\hat{\rho }$
describes a classical system of indeterminate state,	describes a quantum system of indeterminate state,
is evolved through Liouville’s equation.	is evolved through von Neumann equation.
Point in phase-space is described by a $\delta$-function,	Vector in Hilbert space is described by a rank-1
$\rho (\widetilde{𝒙},\widetilde{𝒑})=\delta (\widetilde{𝒙}-𝒙,\widetilde{𝒑}-𝒑)$.	density matrix, ${\hat{\rho }}^2=\hat{\rho }$.
Coarse-graining $𝒞={\left\{P_i\right\}}_i$	Coarse-graining $𝒞={\left\{{\mathscr{H}}_i\right\}}_i$
is a partition of phase-space, $\Gamma ={\bigcup }_i{P}_i$.	is a partition of Hilbert space, $\mathscr{H}={⨁}_i{\mathscr{H}}_i$.
Equivalently, $𝒞={\left\{{\hat{P}}_i\right\}}_i$	Equivalently, $𝒞={\left\{{\hat{P}}_i\right\}}_i$
is a complete set of orthogonal window functions, ${\sum }_i{\hat{P}}_i=1$.	is a complete set of orthogonal projectors, ${\sum }_i{\hat{P}}_i=\hat{I}$.
$P_i$ or equivalently ${\hat{P}}_i$ is called a macrostate.	${\mathscr{H}}_i$ or equivalently ${\hat{P}}_i$ is called a macrostate.
A joint coarse-graining of multiple coarse-grainings always	A joint coarse-graining exists only if the coarse-grainings
exists.	commute.
Observable $A$	Observable $\hat{A}$
is a real-valued function $A:(𝒙,𝒑)\to a$ acting on	is a Hermitian operator acting on a Hilbert space,
a phase-space,
admitting spectral decomposition $A={\sum }_{a}a{\hat{P}}_a$,	admitting spectral decomposition $\hat{A}={\sum }_{a}a{\hat{P}}_a$,
with expectation value $⟨A⟩=(A,\rho )$,	with expectation value $⟨\hat{A}⟩=\mathrm{tr}\left[\hat{A}\hat{\rho }\right]$,
where the probability of observing $a$ is $p_a=({\hat{P}}_a,\rho )$,	where the probability of observing $a$ is $p_a=\mathrm{tr}\left[{\hat{P}}_a\hat{\rho }\right]$,
and coarse-graining given by an observable is ${𝒞}_A={\left\{{\hat{P}}_a\right\}}_a$.	and coarse-graining given by an observable is ${𝒞}_{\hat{A}}={\left\{{\hat{P}}_a\right\}}_a$.
Observational entropy $S_{O({𝒞}_1,\dots ,{𝒞}_n)}\left(\rho \right)=-{\sum }_{𝒊}{p}_{𝒊}\ln \frac{p_{𝒊}}{V_{𝒊}}$,	Observational entropy $S_{O({𝒞}_1,\dots ,{𝒞}_n)}\left(\hat{\rho }\right)=-{\sum }_{𝒊}{p}_{𝒊}\ln \frac{p_{𝒊}}{V_{𝒊}},$
where $p_{𝒊}={\int }_{\Gamma }\rho {\hat{P}}_{i_1}\cdots {\hat{P}}_{i_n}𝑑𝒙𝑑𝒑$ is a probability of belonging	where $p_{𝒊}=\mathrm{tr}\left[{\hat{P}}_{i_n}\cdots {\hat{P}}_{i_1}\hat{\rho }{\hat{P}}_{i_1}\cdots {\hat{P}}_{i_n}\right]$ is a probability
to multi-macrostate $𝒊=(i_1,\dots ,i_n)$,	of belonging to multi-macrostate $𝒊=(i_1,\dots ,i_n)$,
and $V_{𝒊}=\mu {\int }_{\Gamma }{\hat{P}}_{i_1}\cdots {\hat{P}}_{i_n}𝑑𝒙𝑑𝒑$ is volume of multi-macrostate $𝒊$.	and $V_{𝒊}=\mathrm{tr}\left[{\hat{P}}_{i_n}\cdots {\hat{P}}_{i_1}\cdots {\hat{P}}_{i_n}\right]$ is volume of multi-macrostate $𝒊$.
A point in phase-space belongs only into a single	A wave-function can span over several macrostates.
macrostate.
$S_{O({𝒞}_1,{𝒞}_2)}=S_{O({𝒞}_2,{𝒞}_1)}$ always holds.	$S_{O({𝒞}_1,{𝒞}_2)}=S_{O({𝒞}_2,{𝒞}_1)}$ holds for commuting coarse-grainings,
	but does not hold for non-commuting coarse-grainings.
$S_G\left(\rho \right)\le S_{O({𝒞}_1,\dots ,{𝒞}_n)}\left(\rho \right)\le \ln V$	$S_{VN}\left(\hat{\rho }\right)\le S_{O({𝒞}_1,\dots ,{𝒞}_n)}\left(\hat{\rho }\right)\le \ln dim\mathscr{H}$
$S_{O({𝒞}_1,\dots ,{𝒞}_n)}\left(\rho \right)\le S_{O({𝒞}_1,\dots ,{𝒞}_{n-1})}\left(\rho \right)$	$S_{O({𝒞}_1,\dots ,{𝒞}_n)}\left(\hat{\rho }\right)\le S_{O({𝒞}_1,\dots ,{𝒞}_{n-1})}\left(\hat{\rho }\right)$
Dynamical thermodynamic entropy	Dynamical thermodynamic entropy
${𝑺}_{𝒙𝑬}\equiv S_{O({𝒞}_X,{𝒞}_H)}=-{\sum }_{𝒏,E}{p}_{𝒏E}\ln \frac{p_{𝒏E}}{V_{𝒏E}}$,	${𝑺}_{𝒙𝑬}\equiv S_{O({𝒞}_{\hat{X}},{𝒞}_{\hat{H}})}=-{\sum }_{𝒏,E}{p}_{𝒏E}\ln \frac{p_{𝒏E}}{V_{𝒏E}}$,
where ${𝒞}_X={\left\{P_{𝒏}\right\}}_{𝒏}$ is a set of a local particle number	where ${𝒞}_{\hat{X}}={\left\{{\hat{P}}_{𝒏}\right\}}_{𝒏}$ is a set of a local particle number
macrostates, and ${𝒞}_H={\left\{P_E\right\}}_E$ is a set of energy macrostates,	macrostates, and ${𝒞}_{\hat{H}}={\left\{{\hat{P}}_E\right\}}_E$ is a set of energy macrostates,
which are defined as energy shells of the Hamiltonian,	which are defined as projectors onto energy eigenstates,
($dE$ independent of $E$).	${\hat{P}}_E=|E⟩⟨E|$, $\hat{H}|E⟩=E|E⟩$.
Converges to thermodynamic entropy. Is not additive.	Converges to thermodynamic entropy. Is additive.
	Is not additive if a finite resolution in energy $\Delta E$ is used.
	${𝑺}_{𝑬𝒙}\equiv S_{O({𝒞}_{{\hat{H}}^{\left(\Delta E\right)}},{𝒞}_{\hat{X}})}=-{\sum }_{𝒏,E}{p}_{E𝒏}\ln \frac{p_{E𝒏}}{V_{E𝒏}}$,
	where ${𝒞}_{{\hat{H}}^{\left(\Delta E\right)}}={\left\{{\hat{P}}_{E,\Delta E}\right\}}_E$ is a set of energy macrostates,
	which are defined as projectors onto coarse-grained energy
	macrostates, .
Dynamical thermodynamic entropy	Dynamical thermodynamic entropy
${𝑺}_F\equiv S_{O\left({𝒞}_{X𝑬}\right)}=-{\sum }_{𝒏,E}{p}_{𝒏𝑬}\ln \frac{p_{𝒏𝑬}}{V_{𝒏𝑬}}$,	${𝑺}_F\equiv S_{O\left({𝒞}_{{\hat{H}}_1}\otimes \cdots \otimes {𝒞}_{{\hat{H}}_m}\right)}=-{\sum }_{𝑬}{p}_{𝑬}\ln \frac{p_{𝑬}}{V_{𝑬}}$,
where ${𝒞}_{X𝑬}={\left\{P_{𝒏𝑬}\right\}}_{𝒏𝑬}$ is a set of a local particle number	where ${𝒞}_{{\hat{H}}_1}\otimes \cdots \otimes {𝒞}_{{\hat{H}}_m}$ is a set of local energy
and local energy macrostates.	macrostates.
Converges to thermodynamic entropy. Is additive.	Converges to thermodynamic entropy. Is additive.
	Is modified to include coarse-graining in local particle
	numbers if a finite resolution in energy $\Delta E$ is used.${}^{\text{<a><span>17</span></a>}}$

Equations

$$\rho (\mathbf{x},\mathbf{p};t)$$

$${\int }_{\Gamma }\rho (\mathbf{x},\mathbf{p};t)d\mathbf{x}d\mathbf{p}=1.$$

$$V_i={\int }_{P_{\mathbf{i}}}\mu d\mathbf{x}d\mathbf{p}.$$

$$\mu =\frac{1}{h^{Nd}},$$

$$p_i(t)={\int }_{P_i}\rho (\mathbf{x},\mathbf{p};t)d\mathbf{x}d\mathbf{p}.$$

$${\hat{P}}_i(\mathbf{x},\mathbf{p})=\{\begin{array}{ll}1,& (\mathbf{x},\mathbf{p})\in P_i\\ 0,& (\mathbf{x},\mathbf{p})\notin P_i\end{array}$$

$$V_i$$

$$p_i(t)$$

$$(f,g)\equiv {\int }_{\Gamma }f(\mathbf{x},\mathbf{p})g(\mathbf{x},\mathbf{p})d\mathbf{x}d\mathbf{p},$$

$$V_i$$

$$p_i(t)$$

$$S_{O(\mathcal{C})}(t)=-\sum _i{p}_i(t)\ln \frac{p_i(t)}{V_i}.$$

$$\mathcal{C}=\{P_i{\}}_i\equiv \{{\hat{P}}_i{\}}_i.$$

$${\mathcal{C}}_{1,\dots ,n}=\{P_{i_1,\dots ,i_n}{\}}_{i_1,\dots ,i_n}$$

$$P_{\mathbf{i}}$$

$${\hat{P}}_{\mathbf{i}}$$

$$S_{O({\mathcal{C}}_1,\dots ,{\mathcal{C}}_n)}(t)\equiv -\sum _{\mathbf{i}}{p}_{\mathbf{i}}(t)\ln \frac{p_{\mathbf{i}}(t)}{V_{\mathbf{i}}},$$

$$V_{\mathbf{i}}$$

$$p_{\mathbf{i}}(t)$$

$$S_{O(\mathcal{C})}(\rho )=S_B(\rho )=\ln V_i.$$

$$S_{O({\mathcal{C}}_1)}(\rho )\ge S_{O({\mathcal{C}}_2)}(\rho ).$$

$$S_G(\rho )\le S_{O({\mathcal{C}}_1,\dots ,{\mathcal{C}}_n)}(\rho )\le \ln V$$

$$S_{O({\mathcal{C}}_1,\dots ,{\mathcal{C}}_n)}(\rho )\le S_{O({\mathcal{C}}_1,\dots ,{\mathcal{C}}_{n-1})}(\rho )$$

$$S_{O({\mathcal{C}}_1,\dots ,{\mathcal{C}}_n)}(t)\equiv -\sum _{\mathbf{i}}{p}_{\mathbf{i}}(t)\ln p_{\mathbf{i}}(t)+\sum _{\mathbf{i}}{p}_{\mathbf{i}}(t)\ln V_{\mathbf{i}}$$

$$\begin{array}{rl}& P_{\mathbf{n}}\equiv \{(\mathbf{x},\mathbf{p})|n_1\text{ particles with position }x\in [L_1,L_1+\Delta x),\\ & \dots ,n_m\text{ particles with position }x\in [L_1+(m-1)\Delta x,L_2)\}.\end{array}$$

$$P_{E,\Delta E}\equiv \{(\mathbf{x},\mathbf{p})|E\le H(\mathbf{x},\mathbf{p})<E+\Delta E\},$$

$$S_{xE}(t)\equiv S_{O({\mathcal{C}}_X,{\mathcal{C}}_H)}(t)=-\sum _{\mathbf{n},E}{p}_{\mathbf{n}E}(t)\ln \frac{p_{\mathbf{n}E}(t)}{V_{\mathbf{n}E}}.$$

$$H(\mathbf{x},\mathbf{p},t)=\{\begin{array}{ll}{H}_1(\mathbf{x},\mathbf{p})=H_0(\mathbf{x},\mathbf{p})+U_1(\mathbf{x}),& t<0\\ H_2(\mathbf{x},\mathbf{p})=H_0(\mathbf{x},\mathbf{p})+U_2(\mathbf{x}),& t\ge 0\end{array}$$

$$U_1(\mathbf{x})$$

$$U_2(\mathbf{x})$$

$$S_{\mathrm{micro}}^{(H)}(E)=\ln V_{E,\Delta E}^{(H)}=\ln (\rho (E)\Delta E)=S_{\mathrm{micro}}(E,\mathcal{V},N).$$

$$S_{\mathrm{th}}^{(H)}\equiv S_{O({\mathcal{C}}_H)},$$

$$S_{xE}=S_{O({\mathcal{C}}_X,{\mathcal{C}}_{H_2})}=S_{O({\mathcal{C}}_{H_2},{\mathcal{C}}_X)}\le S_{O({\mathcal{C}}_{H_2})}=S_{\mathrm{th}}^{(H_2)}.$$

Full text

I Motivation

From the introductory pedagogical level to contemporary fundamental research, and in both classical and quantum contexts, the concept of entropy has caused a great deal of confusion. Not only are there many definitions of different type of entropies appropriate to different contexts Jaynes (1965); Wehrl (1978); Swendsen (2011); Balibrea (2016); Goldstein et al. (2017, 2019), but, we would contend, there are two distinct fundamental notions of to what entropy is meant to refer.

On one hand, entropy per the definitions of Shannon, Gibbs, or von Neumann, is an information-theoretic quantity associated with the probabilities attributed to states of a system. This entropy is preserved in a closed system undergoing evolution via the classical Liouville equation or a unitary quantum operator, reflecting the preservation of information in such systems, but the entropy may change (and will generally rise) if interactions with an external system are allowed.

On the other hand, entropy can measure how “generic” some state-of-affairs is, as described at a coarse-grained or macroscopic level. Thermodynamic entropy, and that defined by Boltzmann as the number of microstates associated with a given macrostate, share this character. This sort of entropy can (and tends to) increase in a closed system, satisfying some version of the Second Law of thermodynamics.

These notions are often conflated because they tend to coincide in equilibrium systems, all converging to the logarithm of the number of states compatible with some set of fixed constraints – whether those states are cells in classical state space, energy levels in quantum theory, or defined otherwise. Yet if we wish to describe systems out of equilibrium, in which entropy can evolve, or to evaluate entropy for small numbers of particles, more conceptual and mathematical precision is necessary.

In this paper we argue that there is a natural and rigorous definition of entropy at the classical level that:

Is well-defined in any classical system with a fixed phase-space and probability measure over that phase-space; in particular it is well-defined out of equilibrium and for small numbers of particles. 2. 2.

Constitutes a generalization of, and an interpolation between, classical Gibbs and Boltzmann entropies. 3. 3.

Is defined in terms of a coarse-graining that corresponds to a partitioning of phase-space. 4. 4.

Evolves continuously in time, and generically toward larger values, corresponding to a Second Law. 5. 5.

Can be cleanly transcribed into the quantum context – and in fact is the classical version of the quantum “Observational entropy” introduced by Šafránek et al. (2019a, b).

and in addition,

Can be used to describe the dynamics of classical systems. Specifically, it can be used to define “dynamical thermodynamic entropy” that changes with the evolution of a system.

The idea of coarse-graining has a very long history, going back to Boltzmann Boltzmann (2015), and coarse-grained entropies or coarse-grained free energies have emerged in numerous applications, such as fluid dynamics and Navier-Stokes equations Español et al. (1997); Gao et al. (2017) (clearly present in the continuum hypothesis Batchelor and Batchelor (1967)), statistical mechanics of fields and renormalization group Fisher (1998); Kardar (2007a, b); Ma (2018), in chemical engineering in computing the entropy of mixing Smith (1950); Guggenheim (1956); Callen (1998), and in field theory in the guise of renormalization (leading to the 1982 Nobel Prize in Physics for work on critical phenomena using the renormalization group Wilson (1971)). And there are definitions of entropy using some type of rigorously-defined coarse-graining, such as entropy of partition Jost (2006), Kolmogorov-Sinai entropy Farmer (1982); Latora and Baranger (1999); Frigg (2004); Jost (2006), or topological entropy Farmer (1982); Jost (2006). (These mostly apply to dynamical systems.)

Nonetheless, in many works coarse-graining is treated in a rather ad-hoc and non-rigorous manner, with many subtleties swept under the rug. The coarse-grained entropy that we are going to argue for here has also appeared previously in literature. The quantum version of this entropy predates the classical; it was first introduced by von Neumann von Neumann (2010, 1955) as a resolution to the fact that the [von Neumann] entropy does not increase in isolated systems, then briefly mentioned by Wehrl Wehrl (1978) as “coarse-grained entropy.” It was further developed and generalized to include multiple coarse-grainings by the present authors in Šafránek et al. (2019a, b), and termed “Observational entropy,” because it can be interpreted as the amount of knowledge an observer has about the system, when they have access only to a set of coarse-grained measurements (and each measurement corresponds to an observable). Quantum Observational entropy has been found to dynamically describe thermalization of isolated quantum systems Šafránek et al. (2019b); Lent (2019), has been discussed in relationship with other types of entropies Goldstein et al. (2019), has been found to increase under Markovian stochastic maps Gemmer and Steinigeweg (2014), has been argued for as a natural candidate for entropy production Strasberg (2019); Strasberg and Winter (2020) because its definition does not need an explicit temperature dependence, found to follow an integral fluctuation theorem and providing an alternative perspective on the conventional definition of mechanical work in quantum systems Strasberg and Winter (2020), and has been shown to illustratively model isolated systems out of equilibrium by studying its extreme fluctuations Faiez et al. (2019).

Classical Observational entropy has appeared previously Wehrl (1978); Latora and Baranger (1999); Nauenberg (2004); Kozlov and Treshchev (2007); Piftankin and Treschev (2008); Županović and Kuić (2018), but generally just as a mention in passing,111Eq. (1.26) in Wehrl (1978), Eq. (1) in Latora and Baranger (1999) (up to a minus sign and an additive constant), Eq. (A.1) in Nauenberg (2004), Eq. (1.1) in Kozlov and Treshchev (2007), Def. 4.1 in Piftankin and Treschev (2008) and Eq. (11) in Županović and Kuić (2018). but never with a clear or comprehensive discussion of it, nor with any compelling treatment of applications to thermodynamics.

Thus the aim of this paper is to motivate and define a rigorous mathematical framework of classical Observational entropy, which we believe gives an elegant unification of many coarse-graining techniques, then derive its various properties, interpret it both from an information-theoretic perspective and from a physical perspective (by connecting it to thermodynamic entropy), and finally spell out its relations to the quantum version. In other words, we will both argue that our definition of coarse-grained entropy satisfies the desiderata in points $1$–$6$ above, and also provide the reader intuition about behavior of this entropy, by making the connection between the classical and quantum versions clear.

II Boltzmann entropy

Although there are variations, the type of “state counting” entropy defined by Boltzmann Boltzmann (2015, 2003); Brush (2004) generally attributes a number $V$ of fundamental microstates to a given macrostate – defined in some terms – attributing an entropy proportional to $\ln V$ to the macrostate. To be more precise, we make use of the following definitions.

The state space of a system can be partitioned into non-overlapping subspaces that sum up to the full state space. This partition is called a “coarse-graining” and denoted by ${\mathcal{C}}$. An element of this partition is a called a macrostate.

For a microstate $m$ in a macrostate, we attach Boltzmann entropy of $S_{B}(m)=\ln(V)$ to this microstate (as well as to the macrostate of which it is an element) where $V$ is the number of microstates contained in the macrostate.

This entropy typically rises, at least on average, in any classical dynamical system out of equilibrium. Consider for example the prototypical system of a small box of gas that is opened within a larger box, as depicted in Fig. 1, panel (a). In the Boltzmann view, we consider the phase-space of gas particles in the full box, so that immediately post-opening, the gas is in a low-entropy macrostate that might be described as “all particles in the small box.” Under natural evolution the microstate tends to wander out of this macrostate and into the much larger macrostate “particles spread throughout large box.” This is depicted in Fig. 2 (top panel.)

This scheme has the advantage of being defined throughout the process, not just in the equilibrium states. But it is problematic in that it changes discontinuously as the microstate transitions from one macrostate to another, and in that it requires perfect knowledge of the microstate, which is never in practice available. What if one would like to describe a more realistic situation, in which the observer has only a partial knowledge about the state of the system? We will show that in such situations, classical Observational entropy extends the definition of Boltzmann, and leads to the continuously evolving quantity.

Another goal of this paper, as previously mentioned, is to use this generalized and smoothly-varying definition of Boltzmann entropy to describe thermodynamic entropy as a dynamical quantity, as opposed to the standard definition, which yields a fixed value that is completely determined by the external parameters of a system. To elaborate, in equilibrium systems, thermodynamic entropy (or more precisely, microcanonical entropy) is defined in terms of the density of states near a given overall conserved energy. Illustrated on our example, thermodynamic entropy defined this way changes only during the sudden quench of the Hamiltonian, which changes discontinuously due to the removal of the barrier during stage $1\rightarrow 2$ in Fig. 1 (a). In other words, this prescription ascribes exactly the same entropy to states 2 and 4, as illustrated in Fig. 1 (b) (top). This conflicts with the fact that the state 2 is a highly non-equilibrium state from which it should be in principle possible to extract a larger amount of work than from the equilibrium state 4. We will show that with a suitable choice of coarse-graining it is indeed possible to define a thermodynamic entropy that does not depend on the barrier removal, but rather on the underlying state of a system, as shown of Fig. 1 (b) (bottom). Moreover, this quantity will correspond to the standard thermodynamic entropy in its respective equilibrium states, increasing continuously from one equilibrium configuration (closed box) to the next (open box after a long time), making it a reasonable definition of a dynamical thermodynamic entropy.

III Classical Observational entropy

We desire a generalization of Boltzmann entropy that can be applied to probability distributions given by phase-space density

[TABLE]

over microstates. This dynamical entropy should act as smoothing-out of the Boltzmann entropy, reducing to the actual Boltzmann entropy for a fully determined system. We also choose the phase-space density to be normalized as

[TABLE]

By phase-space density $\rho(t):({\boldsymbol{x}},{\boldsymbol{p}})\rightarrow\rho({\boldsymbol{x}},{\boldsymbol{p}};t)$ we mean a function parametrized by time $t$ which attaches a non-negative-valued probability density to each point in phase-space. Thus the probability that the state of the system is in the infinitesimal phase-space volume $d{\boldsymbol{x}}d{\boldsymbol{p}}$ at time $t$ is given by $\rho({\boldsymbol{x}},{\boldsymbol{p}};t)d{\boldsymbol{x}}d{\boldsymbol{p}}$.

In the case of indistinguishable particles, the phase-space density is additionally required to be fully symmetric under the interchange of any two particles.

We assume that an observer – guided by some physical motivation – chooses a certain partition of this space, $\Gamma=\bigcup_{i}P_{i}$. We collect these disjoint subsets – regions of phase-space – into a what we call a coarse-graining, and denote ${\mathcal{C}}=\{P_{i}\}_{i}$.

Each region $P_{i}$, called a macrostate, is commonly defined by some inequality conditions on points of phase-space $({\boldsymbol{x}},{\boldsymbol{p}})$. It usually represents a collection of points of phase-space consistent with some macroscopically observed value or property.

The number of states in each macrostate, which we simply call its volume, is linearly proportional to its phase-space volume. The volume of each macrostate is computed as an integral over the macrostate with a constant222We demand the constant measure for the following reason: Due to Liouville’s evolution, the phase-space volume of any state described by a phase-space density $\rho$ is constant. If measure $\mu$ varied depending on $({\boldsymbol{x}},{\boldsymbol{p}})$, then the volume $V_{\rho}=\int_{\{({\boldsymbol{x}},{\boldsymbol{p}})|\rho({\boldsymbol{x}},{\boldsymbol{p}};t)\neq 0\}}\mu d{\boldsymbol{x}}d{\boldsymbol{p}}$ associated with phase-space density would change in time. Since this volume is supposed to represent the number of microstates within it, and since it is reasonable to demand that this number stay the same during the evolution, we must demand a constant measure $\mu$. More generally, Liouville’s equation and the property of time independent phase-space volume is one of the primary reasons for using phase-space to describe thermodynamics, as opposed to for example configuration space, where the volume of states does not stay constant during time evolution. measure $\mu$,

[TABLE]

This constant measure determines the inverse of the size of each microstate, and we will typically choose a conventional

[TABLE]

where $d$ is number of degrees of freedom of each particle ($d=1$ for a particle moving on a line, and $d=3$ for particle in a 3-dimensional space), $N$ is number of particles, and $h$ is Planck’s constant. The advantage of this specific choice is that its dimensions render $V_{i}$ unitless (as required to take a logarithm), and because $V_{i}$ then measures the actual number of quantum microstates within each macrostate, since each quantum microstate is considered to take a phase-space volume of $h^{Nd}$, see e.g. Balian (2007a); Goodstein (2014).

In the case of indistinguishable particles, the microstate where the $k^{\mathrm{th}}$ particle is at position and momenta $(x,p)$ and the microstate where $l^{\mathrm{th}}$ particle is at the same point in phase space, are considered to be the same microstate. As a result, Eq. (3) is overcounting the number of microstates, and has to be modified by a standard factor Balian (2007b) to $V_{i}=\frac{1}{N!}\int_{P_{\boldsymbol{i}}}\mu\ d{\boldsymbol{x}}d{\boldsymbol{p}}.$

To calculate the probability of the state being in macrostate $P_{i}$, we integrate the phase-space density over this region,

[TABLE]

This gives a positive value, and all probabilities sum up to one, since by definition, integral over the entire phase-space is equal to one by Eq. (2).

Defining projectors as window functions

[TABLE]

we can write

[TABLE]

where the integral now goes over the entire phase-space. This can be further compacted using the $L^{2}$-inner product on real-valued functions,

[TABLE]

so that

[TABLE]

($V_{i}=\frac{1}{N!}(\hat{P}_{i},\mu)$ for indistinguishable particles.)

Since $V_{i}$ denotes the number of microstates in macrostate $i$, and $p_{i}$ denotes the probability of being in that macrostate, then assuming an observer cannot distinguish between any two microstates within the same macrostate, they associate equal probability $p_{i}^{(k)}:=p_{i}/V_{i}$ to each microstate $k$ within that macrostate. The Shannon entropy that this observer assigns to a system, given their inability to distinguish microstates within the same macrostate, is therefore $-\sum_{i,k}p_{i}^{(k)}\ln p_{i}^{(k)}=-\sum_{i}V_{i}\frac{p_{i}}{V_{i}}\ln\frac{p_{i}}{V_{i}}$. This motivates the definintion of classical Observational entropy with coarse-graining ${\mathcal{C}}$ as

[TABLE]

Since the coarse-graining ${\mathcal{C}}$ can be uniquely defined either by a set of macrostates $P_{i}$, or by a set of corresponding projectors $\hat{P}_{i}$, we can identify these two otherwise mathematically distinct objects, and write

[TABLE]

So far, our definition of coarse-grained entropy is mathematically identical to those used by Refs. Wehrl (1978); Nauenberg (2004); Kozlov and Treshchev (2007); Piftankin and Treschev (2008); Županović and Kuić (2018); the next section introduces an important generalization to multiple coarse-grainings.

IV Multiple coarse-grainings

The above definition applies for any coarse-graining, but some coarse-grainings are more relevant than others. To define a thermodynamically relevant specification of Observational entropy, and for other purposes, it is necessary to generalize this entropy to include multiple coarse-grainings. This is done as follows.

Suppose we have several coarse-grainings of the phase-space, $({\mathcal{C}}_{1},\dots,{\mathcal{C}}_{n})$. We define a joint coarse-graining

[TABLE]

where the corresponding multi-macrostates are given by overlap of the previous macrostates, and the projectors as a multiple of projectors,

[TABLE]

In the above, we have also employed multi-index ${{\boldsymbol{i}}=(i_{1},\dots,i_{n})}$.

Inserting the joint coarse-graining into the definition, Eq. (10), motivates the definition of the Observational entropy with multiple coarse-grainings as

[TABLE]

where

[TABLE]

($V_{\boldsymbol{i}}=\frac{1}{N!}(\hat{P}_{\boldsymbol{i}},\mu)$ for indistinguishable particles.) Indeed, from the definition it follows $S_{O({\mathcal{C}}_{1},\dots,{\mathcal{C}}_{n})}=S_{O({\mathcal{C}}_{1,\dots,n})}$. (Note the subtle notational difference.) That is, a set of coarse-grainings can also be considered as a single composite coarse-graining.333As discussed below, this statement does not transfer to quantum mechanics unless the coarse-grainings commute.

V Properties

We briefly mention some properties of classical Observational entropy. These are classical equivalents of theorems proved for the quantum Observational entropy Šafránek et al. (2019b). The proofs are mostly analogous to those quantum mechanics (with some important subtleties), although we were able to slightly simplify some of them due to a simpler structure of the phase-space as compared to the Hilbert space (see Appendix A).

Theorem 1**.**

(Observational entropy is a generalization of the Boltzmann entropy) For a single point in phase-space $({\boldsymbol{x}},{\boldsymbol{p}})\in P_{i}$, equivalent to a delta function $\rho(\tilde{\boldsymbol{x}},\tilde{\boldsymbol{p}})=\delta(\tilde{\boldsymbol{x}}-{\boldsymbol{x}},\tilde{\boldsymbol{p}}-{\boldsymbol{p}})$, we have

[TABLE]

Definition 1**.**

(Finer coarse-graining) We say that coarse-graining ${\mathcal{C}}_{2}$ is finer than coarse-graining ${\mathcal{C}}_{1}$ (and denote ${\mathcal{C}}_{1}\hookrightarrow{\mathcal{C}}_{2}$) when for every $P_{i_{1}}\in{\mathcal{C}}_{1}$ there exists an index set $I^{(i_{1})}$ such that $P_{i_{1}}=\bigcup_{i_{2}\in I^{(i_{1})}}P_{i_{2}}$, ${P}_{i_{2}}\in{\mathcal{C}}_{2}$. (That is, each element of ${\mathcal{C}}_{1}$ can be partitioned using elements of ${\mathcal{C}}_{2}$.)

When ${\mathcal{C}}_{1}\hookrightarrow{\mathcal{C}}_{2}$, we can also write $\hat{P}_{i_{1}}=\sum_{i_{2}\in I^{(i_{1})}}\hat{P}_{i_{2}}$.

Theorem 2**.**

(Observational entropy is a monotonic function of the coarse-graining.) If ${\mathcal{C}}_{1}\hookrightarrow{\mathcal{C}}_{2}$ then

[TABLE]

Definition 2**.**

(Coarse-graining given by an observable) Let $A:({\boldsymbol{x}},{\boldsymbol{p}})\rightarrow a$ be a classical observable, that assigns value $a$ (property) to each point in phase-space. We define macrostates associated with value $a$ as ${P_{a}=\{({\boldsymbol{x}},{\boldsymbol{p}})|A({\boldsymbol{x}},{\boldsymbol{p}})=a\}}$ in case of observable with discrete values, or as ${P_{a}=\{({\boldsymbol{x}},{\boldsymbol{p}})|a\leq A({\boldsymbol{x}},{\boldsymbol{p}})<a+da\}}$ in case of a continuous observable.444The infinitesimal increment $da$ plays the role of the resolution in measuring the observable $A$. We define coarse-graining given by the observable555Note 1: Although $\rho$ is not usually considered an observable, it fulfills our definition and we will treat it as such in the theorem that follows. Note 2: we can also define the spectral decomposition of an observable as $A=\sum_{a}a\hat{P}_{a}$, or $A=\int_{-\infty}^{\infty}a\hat{P}_{a}da$, where eigenvalues $a$ are considered to be different from each other. This spectral decomposition is unique, and it plays an identical role in phase-space with inner product (8) as the spectral decomposition of a quantum observable plays in a Hilbert space. $A$ as ${\mathcal{C}}_{A}=\{P_{a}\}_{a}$.

Theorem 3**.**

(Observational entropy with multiple coarse-grainings is bounded)

[TABLE]

where $S_{G}(\rho)\equiv-\int_{\Gamma}\frac{\rho({\boldsymbol{x}},{\boldsymbol{p}})}{\mu}\ln\frac{\rho({\boldsymbol{x}},{\boldsymbol{p}})}{\mu}\ \mu d{\boldsymbol{x}}d{\boldsymbol{p}}$, and $V\equiv\int_{\Gamma}\mu d{\boldsymbol{x}}d{\boldsymbol{p}}$ is the total volume of the phase-space. $S_{G}(\rho)=S_{O({\mathcal{C}}_{1},\dots,{\mathcal{C}}_{n})}(\rho)$ if and only if ${\mathcal{C}}_{\rho}\hookrightarrow{\mathcal{C}}_{1,\dots,n}$, i.e., if the joint coarse-graining is fine enough to distinguish between points in phase-space that have different assigned probabilities.

$S_{G}(\rho)$ represents Gibbs entropy.666Definition of Gibbs entropy is not consistent in literature. Our definition seems to be the most common, and is used for example in Jaynes (1965). Sometimes, Gibbs entropy is defined as Shannon entropy of probabilities of energy distribution (which has a quantum equivalent called diagonal entropy Polkovnikov (2011)), and what is called Gibbs entropy in Županović and Kuić (2018) is actually our Observational entropy. Gibbs entropy is invariant under Liouville’s evolution. It is zero for a single point in phase-space, and it is a property of a state, and not of a coarse-graining. This quantity also appears as the functional $H$ in the classical H-theorem, as interpreted by Tolman Tolman (1979). Since $\mu$ is the inverse of the phase-space volume of a single microstate, $p_{i}=\frac{\rho({\boldsymbol{x}},{\boldsymbol{p}})}{\mu}$ is the probability of being in a microstate, so we can as well write $S_{G}(\rho)\equiv-\sum_{i}p_{i}\ln p_{i}$, i.e. the Shannon entropy of these probabilities. The quantum mechanical equivalent of Gibbs entropy is the von Neumann entropy, which is invariant under unitary evolution, is zero for pure states, and which is also property of a state and not of a coarse-graining.

Theorem 4**.**

(Observational entropy is non-increasing with each added coarse-graining.)

[TABLE]

for any set of coarse-grainings $({\mathcal{C}}_{1},\dots,{\mathcal{C}}_{n})$ and any phase-space density $\rho$.

These theorems show that Observational entropy can be elegantly interpreted as the amount of knowledge an observer would obtain if he or she were to measure the macroscopic observables that define the coarse-grainings. While Theorem 2 says that an observer with better resolution will get to know more about the system, Theorem 3 says that no matter what coarse-grained measurements they choose to perform, their knowledge will be still limited by an inherent uncertainty in the system given by the Gibbs entropy. On the other hand, no matter the coarse-graining, their knowledge cannot be worse than that measured by the maximal entropy, which signifies complete uncertainty about the system’s state. Theorem 4 then shows an intuitive statement that each additional macroscopic measurement will provide better knowledge of the system, at least on average.

In addition, rewriting Eq. (15) as

[TABLE]

provides an intuitive information-theoretic interpretation. The first term denotes an uncertainty as measured by the Shannon entropy regarding to which macrostate the microstate of the system belongs. In other words, if one were to make a coarse-grained measurement at time $t$ given by the coarse-graining (for example a measurement determining the system’s energy) the first term measures the uncertainty in the measurement outcomes in such a coarse-grained measurement. The second term measures the average remaining uncertainty about the microstate after this coarse-grained measurement was done. Put together, Observational entropy measures the average amount of uncertainty about a microstate of a system, from a point of view of an observer that can track only certain macroscopic properties by his/her ability to perform coarse-grained measurement.

VI Thermodynamically relevant non-equilibrium entropies: introduction

The treatment thus far has pertained to any possible coarse-graining. However, even if entropy increase is generic, there is no reason to expect that an arbitrary coarse-graining will be closely connected with thermodynamics, which in particular relates temperature, energy and entropy.

In the following sections we introduce two types of (composite) coarse-grainings that each lead to an entropy that is relevant for describing dynamics of isolated quantum systems: $S_{xE}$ (Observational entropy with local particle coarse-graining and global energy coarse-grainings) and $S_{F}$ (Observational entropy with local particle and local energy coarse-grainings).777In the quantum context Šafránek et al. (2019b), $S_{F}$ is also named “Factorized Observational entropy” or FOE for short, due its tensor-product form of its local energy coarse-graining. Both will be perfectly defined for systems outside of equilibrium, both will grow to the equilibrium thermodynamic value when the system thermalizes, and both will describe physical regions coming into equilibrium with each other.

However, there are some important distinctions in their properties closely related to distinctions in their meanings: as explained in the following sections, $S_{xE}$ can be interpreted as the value of entropy the system would attain in the long-time limit if the regions were allowed to exchange energy, but not particles. $S_{F}$, on the other hand, represents the value of entropy associated with the system if regions equilibrate but cannot exchange either energy or particles.

This interpretation of $S_{xE}$ shows that it is not in general additive on subsystems. For example, given two equally-sized regions consisting of the same number of particles but with very different temperature, $S_{xE}$ assigns an entropy value appropriate to the two systems’ temperatures having already equalized through interaction between the regions. $S_{F}$, on the other hand, is additive, and equal to the sum of the thermodynamic entropies of the two regions. Additionally, since measuring local energies also determines the global value of energy, in classical physics $S_{F}\leq S_{xE}$ always.

Since in real systems both particles and energy are exchanged between the regions, both $S_{xE}$ and $S_{F}$ are inherently time-dependent quantities outside of equilibrium.

In the following Sections, we focus on $S_{xE}$ (Sections VII and VIII), not because it is of more interest but because its definition requires only the global Hamiltonian, making it more transparent. We will derive its dynamical properties and show a simulation illustrating its behavior. Defining $S_{F}$ requires a notion of local Hamiltonians. Despite this, its properties are derived in a close analogy to those of $S_{xE}$ (Section IX).888 While both definitions can be written in the standard Hamiltonian formalism which assumes there is a fixed number of particle in the entire system, the description is much more elegant when one develops a formalism of the classical analogue of Fock space – the Fock phase-space – which allows for describing classical systems with a variable number of particles. After all, each subsystem (both for $S_{xE}$ and $S_{F}$) has a variable number of particles in it, and therefore it should be possible to describe each subsystem independently. Observables of local particle number and local energies will be then seen as observables on the Fock phase-space of a subsystem. The coarse-grainings defining $S_{F}$ is then created as a Cartesian product of local coarse-grainings. This will be done elsewhere Šafránek et al. (2020).

Both of these entropies, $S_{xE}$ and $S_{F}$, originated as fully quantum mechanical versions published in Šafránek et al. (2019a, b). However, there are some important differences. For example, unlike classical $S_{xE}$, the quantum $S_{xE}$ is (quite surprisingly!) an additive quantity. These will be discussed in detail in Sec. X.

VII $S_{xE}$: entropy of measuring local particle numbers and global energy

In this section we introduce a (composite) coarse-graining that defines one of the thermodynamically relevant entropies, $S_{xE}$.

Let us consider a system of $N$ particles contained in a 1-dimensional box from position $L_{1}$ to $L_{2}$, so of size $L=L_{2}-L_{1}$. We coarse-grain this box into $m$ physical regions (bins) of size $\Delta x=\frac{L}{m}$. Considering the vector of positions as ${\boldsymbol{x}}=(x_{1},\dots,x_{N})$, and vector of number of particles in each part of the box as ${\boldsymbol{n}}=(n_{1},\dots,n_{m})$ (where $n_{1}+\cdots+n_{m}=N$) we define local particle number (configuration) macrostates as

[TABLE]

Clearly, we can easily generalize this to any number $d$ of spatial dimensions. This coarse-graining corresponds to measuring number of particles in each one of the $m$ bins, i.e., to a coarse-grained measurement of local particle numbers, and we denote it ${\mathcal{C}}_{X}=\{P_{\boldsymbol{n}}\}_{\boldsymbol{n}}$.999We denote coarse-graining ${\mathcal{C}}_{X}$ with lower index $X$ signifying position mostly for historical reasons, because in the case of indistinguishable particles focused on in the quantum version of observational entropy Šafránek et al. (2019a, b), measuring coarse-grained positions of particles is the same as measuring the local number of particles. Thus also an equivalent name for ${\mathcal{C}}_{X}$ is positional coarse-graining. For example, $P_{(3,0,1)}$ corresponds to a macrostate where three particles are in the first bin, zero in the second, and one in the third.

We also define energy macrostates with width $\Delta E$, as

[TABLE]

where $H$ denotes the Hamiltonian. When $\Delta E\equiv dE$ is an infinitesimal increment, we simply denote $P_{E}\equiv P_{E,dE}$, its corresponding projector as $\hat{P}_{E}\equiv\hat{P}_{E,dE}$, and volume as $V_{E}\equiv V_{E,dE}$. We call the coarse-graining with such an infinitesimal energy increment a “fine-graining” in energy, and denote it ${\mathcal{C}}_{E}=\{P_{E}\}_{E}$ (considering definition (2), ${\mathcal{C}}_{E}\equiv{\mathcal{C}}_{H}$). It is important to emphasize here that $dE$ is fixed to be the same for all energies $E$.101010For example, one could also think of an energy coarse-graining $C_{E}^{(\mathrm{eigen})}=\{P_{E}^{(\mathrm{eigen})}\}_{E}$, consisting of projectors onto “eigenstates” of the classical Hamiltonian, where we fit the width of each energy shell $dE$ so that $V_{E}^{(\mathrm{eigen})}=1$, meaning that each macrostate corresponds to a single microstate. In that case $dE=dE(E)$ will generally depend on the energy. Projectors (window functions) $\hat{P}_{E}^{(\mathrm{eigen})}$ are then a classical equivalent of projectors onto quantum energy eigenstates $|E\rangle\langle E|$. However, this would not give a desirable long-time behavior for $S_{xE}$; in particular, the entropy assigned to a microstate in a given energy macrostate $P_{E}$ would not be the correct microcanonical entropy, which is defined as a logarithm of number of microstates in energy shell $[E,E+dE)$. Projectors $\hat{P}_{E}\in{\mathcal{C}}_{E}$ with fixed $dE$ correspond to the sum $\sum_{E\leq\tilde{E}<E+dE}|\tilde{E}\rangle\langle\tilde{E}|$ in quantum mechanics. We also note that correspondence between projectors onto eigenstates of the Hamiltonian $|E\rangle\langle E|$ (called stationary Liouville eigenstates) and $\hat{P}_{E}^{(\mathrm{eigen})}$ have been explored in Wilkie and Brumer (1997) and references therein. It was shown that the Wigner function of $|E\rangle\langle E|$ converges to a distribution that can be viewed as a highly peaked $\hat{P}_{E}^{(\mathrm{eigen})}$ in the $h\rightarrow 0$ limit (see Eq. (24) in Wilkie and Brumer (1997)).

The local particle number-energy macrostates are then defined as ${P_{{\boldsymbol{n}}E}\equiv P_{{\boldsymbol{n}}}\cap P_{E}}$, and the respective projectors are ${\hat{P}_{{\boldsymbol{n}}E}=\hat{P}_{{\boldsymbol{n}}}\hat{P}_{E}}$. With macrostate volumes $V_{{\boldsymbol{n}}E}=(\hat{P}_{{\boldsymbol{n}}E},\mu)$, and probabilities $p_{{\boldsymbol{n}}E}=(\hat{P}_{{\boldsymbol{n}}E},\rho_{t})$, we define observational entropy corresponding to measuring local particle numbers and the global energy as

[TABLE]

Figure 3 gives an example of these quantities for a single particle in a Harmonic potential.

VIII Thermodynamic behavior of $S_{xE}$

We now take a closer look at the thermodynamic behavior of $S_{xE}$, using the prototypical example of a gas at first contained in the half of a box, then expanding into its full volume, as shown in Fig. 1. We will show that for the cases where all particles are only in one of the regions, $S_{xE}$ is equal to the thermodynamic entropy of that region, and that as the system thermalizes by letting the particles spread, $S_{xE}$ grows to the thermodynamic entropy of the full system. This entropy is therefore enough to effectively model, for example, the expansion of an ideal gas.

We model our system by a Hamiltonian undergoing quench (a sudden discontinuous change) at time $t=0$,

[TABLE]

where $H_{0}$ denotes Hamiltonian of the particles themselves (which may or may not contain interaction between the particles), and

[TABLE]

denote two infinite potential wells.

For Hamiltonian $H$, the microcanonical entropy is defined as the logarithm of number of microstates in each energy shell Sethna (2006), which we can write in several ways as

[TABLE]

$V_{E,\Delta E}^{(H)}$ is the volume of an energy macrostate, and $\rho(E)$ denotes the density of energy states. $\Delta E$ is the width of the energy macrostate which can be fixed to some small but non-zero value111111Since $\rho(E)$ rises exponentially with $E$ and $\ln(\rho(E)\Delta E)=\ln\rho(E)+\ln\Delta E$, the choice of $\Delta E$ ultimately does not matter, because $\ln\Delta E$ acts only as an additive constant which is small compared to the first term. Moreover, since we are usually interested in the changes in entropy, rather than in absolute values, in such situations value of this additive term becomes completely irrelevant. that is considered to be independent of $E$. ${\cal V}$ is the physical volume occupied by $N$ particles of total energy $E$. More generally, we can define thermodynamic entropy as

[TABLE]

which coincides with the definition of microcanonical entropy for a single point in phase-space, and with canonical entropy for a canonical distribution $\rho({\boldsymbol{x}},{\boldsymbol{p}})=\frac{1}{Z}e^{-\beta H({\boldsymbol{x}},{\boldsymbol{p}})}$, where $\beta$ denotes inverse temperature, and $Z=\int_{\Gamma}e^{-\beta H({\boldsymbol{x}},{\boldsymbol{p}})}d{\boldsymbol{x}}d{\boldsymbol{p}}$ is the partition function.

In the following, we consider $S_{xE}\equiv S_{O({\mathcal{C}}_{X},{\mathcal{C}}_{H_{2}})}$ with a local particle number coarse-graining ${\mathcal{C}}_{X}$ that halves the box (${\boldsymbol{n}}=(n_{1},n_{2})$, $L_{1}=0$, $L_{2}=L$, $\Delta x=\frac{L}{2}$), and with energy coarse-graining ${\mathcal{C}}_{H_{2}}$ given by the Hamiltonian after the quench.

From Theorem 4, we can immediately see that thermodynamic entropy of the entire box bounds the $S_{xE}$,121212Symmetry $S_{O({\mathcal{C}}_{X},{\mathcal{C}}_{H_{2}})}=S_{O({\mathcal{C}}_{H_{2}},{\mathcal{C}}_{X})}$ holds for classical Observational entropy, but not for quantum Observational entropy Šafránek et al. (2019b).

[TABLE]

Now we move onto studying the actual dynamics of $S_{xE}$.

VIII.1 Dynamics of $S_{xE}$ for determinate initial conditions

First we assume a fully-determined state of $N$ particles contained in the left part of the box, described by a state $({\boldsymbol{x}}_{0},{\boldsymbol{p}}_{0})\in P_{(N,0)E}$. We have

[TABLE]

The first equality comes from Theorem 1, and the second comes from the fact that macrostate of $N$ particles in the left side of the box with energy $E$ is identical to the energy macrostate of the initial Hamiltonian, $P_{(N,0)E}=P_{E}^{(H_{1})}$. Therefore, for initial particles contained in the left side of the box, $S_{xE}$ gives the microcanonical entropy of this side of the box.

As the particles evolve, some of them will go to the right side of the box and some remain. After some time, the state of the system becomes what we can describes as “about a half of the particles are on the left, and about a half of the particles are on the right side of the box”.

We can say the same by saying that the point in phase-space will wander around, and will most likely end up in one of the largest macrostates. In the current situation, the largest macrostate corresponds to a statement “half of the particles are on the left side of the box, half are on the right, and the total energy is $E$”. This is schematically depicted as stages $1\rightarrow 2$ in Fig. 5.

Due to the slow growth of the logarithm, the entropy associated with either of those large macrostates will not differ much: there will be some corrections, but they will become irrelevant in the thermodynamic limit. So in order to find the long-time behavior of $S_{xE}$, it is therefore enough to calculate the entropy for the largest of the macrostates, which is what we do in Appendix C. Assuming indistinguishable particles, we find that for $m$ bins,131313The proof of an equivalent long-time limit behavior for quantum $S_{xE}$ is very different than the present classical case; instead of computing entropy of the largest macrostate, it uses ansatz of random Hamiltonian, and assumption that in the long-time limit the phases in energy basis can be considered to be drawn randomly from a uniform ensemble (which is equivalent of saying that ratios of any two energy eigenvalues are irrational) Šafránek et al. (2019b).

[TABLE]

where $S_{\mathrm{micro}}(E,L,N)=S_{\mathrm{micro}}^{(H_{2})}(E)$ denotes the total microcanonical entropy of the full system. This means that in the case of perfect knowledge of the system, $S_{xE}$ matches microcanonical entropy of the full system, up to some small corrections. The second term becomes negligible compared to the first in the thermodynamic limit of particle number $N\rightarrow\infty.$ The last term represents a finite-size effect, which for small number of bins $m$ (or equivalently, large bin sizes) is also small in comparison with the first term. We will see in the next section that these corrections are an artifact of taking the determinate initial conditions: they disappear in case of indeterminate initial conditions.

VIII.2 Dynamics of $S_{xE}$ for indeterminate initial conditions

Let us define the phase-space density of a microcanonical ensemble as

[TABLE]

Equivalently, we can write

[TABLE]

It is easy to see that for this microcanonical ensemble, as long as $dE\ll\Delta E$ (i.e., the energy coarse-graining is fine enough to determine the width of the ensemble), $S_{xE}$ gives the microcanonical entropy:

[TABLE]

where $\rho(E)$ denotes the energy density of states.

More generally, in Appendix A we prove a theorem that gives meaning to all stationary states: For phase-space densities that are mixtures of energy macrostates (such as the microcanonical and canonical ensembles), $S_{xE}$ gives the thermodynamic entropy:

Theorem 5**.**

For phase-space density of form $\rho=\mu\sum_{E}f(E)\hat{P}_{E}$, where $f(E)$ is any function of energy $E$ normalized as $\sum_{E}f(E)V_{E}=1$,

[TABLE]

Now for the dynamics. We consider an initial state at some time $t<0$ that is a microcanonical state of $N$ particles contained in the left part of the box, and none in the other half, which we can denote as a microcanonical ensemble of the initial Hamiltonian,

[TABLE]

Since this is a stationary state of $H_{1}$, $\rho(t)=\rho_{0}$ for $t<0$.

Similar to Eq. (32), we find that for $t<0$,

[TABLE]

(See Appendix B for details.) In other words, $S_{xE}$ of the initial state is equal to the microcanonical entropy of the first half of the box.

At time $t=0$, the available phase-space suddenly expands, and phase-space density $\rho(t)$ starts to explore the full extent of it. That first leads to a quick increase of entropy for the same reasons as for the case of determinate conditions, i.e., because the phase-space denstity will wander into the largest macrostate. However, due to mixing Arnold and Avez (1968), with time the positions of the particles become increasingly uncertain, and the phase-space density becomes uniformly smeared over all points in each energy shell. This is depicted as stages $2\rightarrow 3$ in Fig. 5.

Mathematically, this means that the phase-space density converges to microcanonical state of the second Hamiltonian,

[TABLE]

According to Theorem 5, $S_{xE}$ of this state must be equal to the microcanonical entropy of the entire box,141414In order for this convergence to hold, we must take the width of energy coarse graining to be small, but non-zero, $0<dE\ll\Delta E$. $0<dE$ comes from the fact that Eq. (40) does not hold in a strict mathematical sense (there is however a way to write an exact mathematical statement, on which we will not elaborate here Arnold and Avez (1968); Volovich (2009)). This is because the phase-space density behaves like an incompressible fluid and as such it never uniformly fills up the energy shells of phase-space. However, it becomes dense in each shell, meaning that after waiting a long time, from the coarse-grained description given by non-zero $dE$ there will be no observable difference between the real microcanonical state and a state that is dense in an energy shell. And this is enough for the entropy $S_{xE}$ to not to register a difference between such states. We are taking $dE\ll\Delta E$ because we want Eq. (36) to hold.

[TABLE]

Clearly, one can generalize this to any initial ensembles, since due to mixing, any initial phase-space density will become a stationary state of Hamiltonian $H_{2}$, in a sense of Theorem 5.

Having indeterminate initial conditions therefore results in $S_{xE}$ converging to thermodynamic entropy exactly, without the corrections of Eq. (33).

VIII.3 Simulations

To support our analytical arguments, we have performed a simulation of a thermodynamic system of gas in $d=2$ spatial dimension, and for 4 and 16 partitions. The case of 4 partitions is depicted in Fig. 5.

We take $N=64$ particles of identical mass $m=1$, initialize them in the lower left corner of size $8\times 8$, within the full box of size $16\times 16$ with periodic boundary conditions. The velocity of each particle has been randomly drawn from the Normal distribution. Particles interact via a Lennard-Jones potential

[TABLE]

where we took $\varepsilon=1/120$ and $r_{m}=1$ as parameters of the model, and $r$ denotes distance between each two particles. Particles are then evolved via a velocity Verlet algorithm with time step $10^{-4}$.

As particles and heat spread from one region to the others, entropy $S_{xE}$ grows from thermodynamic entropy of the first bin to the thermodynamic entropy of the full system, up to some finite-size corrections as expected from Eq. (33), effectively modelling thermalization of an expanding gas.

VIII.4 Interpretation

We have shown that $S_{xE}$, which is well-defined out of equilibrium, corresponds to the thermodynamical entropy of the initial region if all the particles are contained within this region, and it grows to thermodynamic entropy of the full system, for both cases of determinate and indeterminate initial conditions.

Physically, we can interpret $S_{xE}$ as a measure of equilibrium between the different regions defined by the coarse-graining in local particle numbers, but not necessarily as a measure of thermal equilibrium. If particles are uniformly distributed between the regions (the number of particles in each region being proportional the size of the region), $S_{xE}$ is equal to thermodynamic entropy. When $S_{xE}$ is low, it means that many particles are contained in one or a few small regions, and the system is therefore in a highly non-equilibrium state. $S_{xE}$ is therefore a measure of uniformity of particle density.

To illustrate this mathematically, for a single point in phase-space $({\boldsymbol{x}},{\boldsymbol{p}})(t)\in P_{(n_{1},\cdots,n_{m})E}$, $S_{xE}$ gives value

[TABLE]

which is exactly the thermodynamic entropy that one would assign to a system of $N$ particles distributed into $m$ bins, in the situation where the energy is allowed to be exchanged between the bins, but particle number in each bin is fixed.

To imagine this quantity operationally, imagine a situation where an observer/experimenter inserts elastic membranes in between the bins at time $t$ (infinitely quickly), which allow for energy transfer but not for particle transfer, and then waits until the system relaxes. The value of thermodynamic entropy he or she would assign to that system after it relaxes to thermal equilibrium is $S_{xE}(t)$.

This brings out problematic feature of this entropy, which is that the coarse-graining in global energy does not distinguish any microstates that differ in energy locally. This means that the same entropy will be associated to any microstate that has $E_{1}+E_{2}=E$, with $E_{1}$ and $E_{2}$ being the energy of the left region and of the right region respectively. As an example, imagine a situation with $10$ particles in the left region, and $10$ particles in the right region. Situations when the left part is really hot and the right part really cold, and when both parts have equal temperatures, will both have the same associated $S_{xE}=\ln V_{(10,10)E}$ as long as the total energy $E$ is the same. This also means that $S_{xE}$ is not in general additive. 151515The studied example of an expanding gas was a special case, because there were no particles in the right part of the box, therefore this part did not contribute to the total entropy (explained in our operational view: no energy could be transferred if there were no particles on the right when the elastic membrane was inserted).

As we’ll see, factorized Observational entropy $S_{F}$ solves this problem, by building on the definition of $S_{xE}$ but using the local energy coarse-grainings, instead of the global energy coarse-graining.

IX $S_{F}$: entropy of measuring local particle numbers and local energies

In the same 1-D system introduced in the last section, we define local particle number-local energy macrostates as

[TABLE]

where ${\boldsymbol{n}}=(n_{1},\dots,n_{m})$ and ${\boldsymbol{E}}=(E_{1},\dots,E_{m})$ are vectors representing the local particle numbers and local energies. $N=n_{1}+\cdots+n_{m}$, $E=E_{1}+\cdots+E_{m}$, and $L=L_{2}-L_{1}$ are the total particle number, the total energy, and the total spatial volume (length of the box). This can be generalized to any spatial dimension $d$ and spatial volume ${\cal V}$, with number of bins defined as $m=\frac{{\cal V}}{\Delta x^{d}}$.

Observational entropy with coarse-graining ${\mathcal{C}}_{X{\boldsymbol{E}}}=\{P_{{\boldsymbol{n}}{\boldsymbol{E}}}\}_{{\boldsymbol{n}}{\boldsymbol{E}}}$ corresponding to measuring local particle numbers and local energies

[TABLE]

satisfies all properties required of dynamical thermodynamic entropy.

Assuming indistinguishable particles of a sufficiently dilute and weakly interacting gas, volume of macrostate $P_{{\boldsymbol{n}}{\boldsymbol{E}}}$ is equal to the product of local volumes, $V_{{\boldsymbol{n}}{\boldsymbol{E}}}=V_{n_{1}E_{1}}\cdots V_{n_{m}E_{m}}$ (see Appendix D). As a result, for a single point in phase-space $({\boldsymbol{x}},{\boldsymbol{p}})(t)\in P_{{\boldsymbol{n}}{\boldsymbol{E}}}$,

[TABLE]

where $S_{\mathrm{micro}}(E_{k},\Delta x^{d},n_{k})=\ln V_{n_{k}E_{k}}$ is the microcanonical entropy of the $k^{\mathrm{th}}$ bin. In other words, unlike $S_{xE}$, for a single point in phase-space $S_{F}$ is equal to the sum of local microcanonical entropies, and is therefore also additive on independent systems.

In the long time limit, this entropy converges to the total microcanonical entropy,

[TABLE]

up to some corrections which are about a factor of two larger than those for $S_{xE}$. (For $d=1$, ${\cal V}=L$.) These corrections are due to determinate initial conditions (starting as a single point in phase-space), and they disappear in the long time limit when starting with indeterminate conditions due to mixing. The proof is very similar to the proof of the same property for $S_{xE}$, Eqs. (33) and (41), and can be also found in Appendix D.

Similar to $S_{xE}$, $S_{F}$ is always defined even for systems out of equilibrium, and for systems with indeterminate initial conditions, given by a phase-space density $\rho$.

Since the coarse-graining used by $S_{F}$ is finer than that of $S_{xE}$, from Theorem 2 follows that

[TABLE]

This holds up to some finite size corrections, which comes from the fact that $S_{F}$ uses local Hamiltonians161616For the definition of local Hamiltonians, see Appendix D. that ignore interaction between the bins.

$S_{F}$ is a measure of thermal equilibrium between the regions. If the value of $S_{F}$ is high, it means that both particles and energy are uniformly distributed across the entire system. If the value of $S_{F}$ is low, it means that there are regions with a much higher density of particles and energy compared to other regions, which corresponds to a highly non-equilibrium state. $S_{F}$ is therefore a measure of uniformity of both particle and energy density.

Operationally, $S_{F}(t)$ corresponds to a situation when an observer/experimenter inserts walls in between the bins at time $t$ (infinitely quickly), which do not allow for either energy or particle transfer between the bins. Waiting until the system relaxes means that each region is in a thermal equilibrium with itself, but not with other regions. The thermodynamic entropy the experimenter would assign to that system after this type of relaxation is $S_{F}(t)$.

X Comparison of classical and quantum Observational entropy

Having investigated in detail particular coarse-grainings that connect classical Observational entropy to thermodynamics, we compare Observational entropy, and these particular coarse-grainings, across the classical and quantum case.

In the same way that a point in phase-space describes a classical system, a vector in a Hilbert space describes a quantum system. This state-vector (or wave-function) encodes every property of a quantum system and is evolved using the Schrödinger equation. More generally, a quantum system with indeterminate initial condition is described by a density matrix ${\hat{\rho}}$, which is a positive semi-definite operator acting on the Hilbert space. The density matrix is a quantum equivalent of phase-space density, and is evolved through the von Neumann equation, as compared to Liouville’s equation in classical system.

In classical systems, the coarse-graining is defined as a partitioning of phase-space. These partitions/regions — subsets of phase-space — are called the macrostates. To define coarse-graining in quantum physics, we have to partition a Hilbert space. However, partitions of Hilbert space are not composed of subsets but of subspaces combined in a direct sum: $\mathcal{H}=\mathcal{H}_{1}\oplus\mathcal{H}_{2}\oplus\cdots$. We can form these subspaces for example by choosing a basis of the Hilbert space, and putting every basis vector into a group. A subspace is then created by taking all the linear combinations of vectors in the group. For example, real three-dimensional space could be partitioned as $\mathbb{R}^{3}=\mathbb{R}^{2}\oplus\mathbb{R}$, where subspace $\mathbb{R}^{2}=\mathrm{span}\{(1,0,0),(0,1,0)\}$ describes the x-y plane, and $\mathbb{R}=\mathrm{span}\{(0,0,1)\}$ the z-axis. We call these subspaces macrostates, and their collection a coarse-graining, ${\mathcal{C}}=\{\mathcal{H}_{i}\}_{i}$.

As we did with the classical system in Eq. (6), also here we can switch from describing the coarse-graining as a collection of subspaces, and describe it as a collection of projectors that project onto these subspaces instead. In other words, for each subspace $\mathcal{H}_{i}$ there exists a unique projector $\hat{P}_{i}$ that projects onto this subspace, and the coarse-graining ${\mathcal{C}}=\{\hat{P}_{i}\}_{i}$ is then defined as a complete set of orthogonal projectors ($\hat{P}_{i}^{2}=\hat{P}_{i}$, $\hat{P}^{\dagger}=\hat{P}$, $\hat{P}_{i}\hat{P}_{j}=\delta_{ij}\hat{P}_{i}$, $\sum_{i}\hat{P}_{i}=\hat{I}$).

With an ordered set of coarse-grainings $({\mathcal{C}}_{1},\dots,{\mathcal{C}}_{n})$, we define volume of a multi-macrostate ${\boldsymbol{i}}=(i_{1},\dots,i_{n})$ as

[TABLE]

and the probability of being in a macrostate as

[TABLE]

This probability can be also interpreted as a probability of obtaining the sequence of outcomes ${\boldsymbol{i}}$ when performing a sequence of measurements in measurement bases $({\mathcal{C}}_{1},\dots,{\mathcal{C}}_{n})$ on the system.

The (quantum) Observational entropy is defined identically to the classical case, Eq. (15), but with the use the above definitions for $V_{\boldsymbol{i}}$ and $p_{\boldsymbol{i}}({\hat{\rho}})$:

[TABLE]

We compare descriptions of classical and quantum systems in Table 1. Notably, coarse-grainings in the classical scenario always commute, while in quantum scenario they need not. As a consequence, a joint coarse-graining may not exist in the quantum case, and switching order of non-commuting coarse-grainings leads to different Observational entropies. Surprisingly, this does not affect many other properties, as theorems equivalent to 1-5 still hold.

More stark differences appear when defining thermodynamically relevant coarse-grainings. Due to the non-commutativity of coarse-grainings, there are two quantum versions of $S_{xE}$, one that corresponds to first measuring the the local particle numbers, and then energy (denoted $S_{xE}$), and one that corresponds to first measuring energy and then local particle numbers (denoted $S_{Ex}$). The behavior of quantum $S_{xE}$ or $S_{Ex}$ further depends heavily on whether the energy resolution is finite or infinite, i.e whether it is assumed that the experimenter cannot distinguish between two energy eigenstates with energies $|E_{1}-E_{2}|<\Delta E$. Since classical energy eigenstates do not exist, the classical $S_{xE}$ inherently entails such finite resolution in its definition.

This leads to a complex and rather subtle set of correspondences. When $\Delta E$ is non-negligible (relative say to the temperature of the system) and the temperature is high, these differences are negligible, and classical and quantum $S_{xE}$ and $S_{Ex}$ all behave in the same way. (However it should be noted that in this case – as touched upon in Sec. VIII.4 and detailed in Appendix E – $S_{xE}$ is not always additive.) For $\Delta E>0$ at low temperatures and/or small bin sizes $\Delta x$, $S_{xE}$ and $S_{Ex}$ can differ appreciably due to large effects of non-commutativity. Finally, for $\Delta E=0$ (requiring the quantum case), $S_{xE}$ and $S_{Ex}$ are quite distinct: $S_{xE}$ behaves much like quantum $S_{F}$ (and is additive; see Appendix E) while $S_{Ex}$ is constant in time, equal to thermodynamic entropy $S_{E}$, and zero on any global energy eigenstate.

Turning to $S_{F}$, there is no issue of commutation, but the quantum case again gives an option of $\Delta E=0$ or finite $\Delta E.$ The $\Delta E=0$ case defined $S_{F}$ in Šafránek et al. (2019a, b), and was shown to be very similar to quantum $S_{xE}$. One could also define a quantum $S_{F}$ with $\Delta E>0$ to closely correspond to the classical case.171717For $\Delta E>0$ it is necessary to include also coarse-graining in local particle numbers, in order for this entropy to give the correct thermodynamic values. The definition then reads $\boldsymbol{S}_{F}\equiv S_{O({\mathcal{C}}_{\hat{N}_{1}}\otimes\cdots\otimes{\mathcal{C}}_{\hat{N}_{m}},{\mathcal{C}}_{\hat{H}_{1}^{(\Delta E)}}\otimes\cdots\otimes{\mathcal{C}}_{\hat{H}_{m}^{(\Delta E)}})}=-\sum_{{\boldsymbol{n}},{\boldsymbol{E}}}p_{{\boldsymbol{n}}{\boldsymbol{E}}}\ln\frac{p_{{\boldsymbol{n}}{\boldsymbol{E}}}}{V_{{\boldsymbol{n}}{\boldsymbol{E}}}}$. For $\Delta E=0$ this was not necessary, because measuring local energy eigenstates uniquely determined the local particle numbers, for particle conserving Hamiltonians. In all cases $S_{F}$ is additive.

Generally, one could expect differences between classical and quantum definitions of entropies for systems that where value of Planck’s constant is comparable to quantities with the same dimensions which appear in the problem Balian (2007c).

XI Conclusion and outlook

This paper has discussed in detail the framework of classical “Observational entropy,” an idea previously defined in the quantum context Šafránek et al. (2019a, b). This quantity is defined precisely in or out of equilibrium, is generically non-decreasing, and corresponds to thermodynamic entropy in equilibrium. Our treatment has aimed to define this quantity clearly and rigorously, while also exhibiting in detail three core sets of relations.

First, the treatment shows how Observation entropy generalizes and interpolates between classical “Gibbs” and “Boltzmann” entropies. The latter is often thought of as Gibbs entropy where equal probability is attributed to all microstates compatible with a given set of macroscopic constraints. Here, we see Boltzmann entropy as a limit of Observational entropy in which all probability is attributed to a single microstate, while Gibbs entropy appears in the limit in which the coarse-graining is as fine as possible so that each microstate constitutes a macrostate.

Second, Observational entropy has both an information-theoretic and a thermodynamic interpretation. Every additional measurement on a system – corresponding to an additional coarse-graining – that better pins down a particular subset of phase-space yields more information in a clear and quantifiable way. At the same time, here (Eq. (33)) and in previous work Šafránek et al. (2019a, b) we have shown that with an appropriate choice of coarse-graining, Observational entropy describes dynamically thermalization of a gas, converging to thermodynamic entropy when the system equilibrates. These coarse-graining also have an elegant interpretation out of equilibrium, corresponding to the thermodynamic entropy the system would attain in equilibrium if particular constraints were imposed on the sharing of particles and energy across spatial regions.

Third, this work demonstrates a clear and well-defined correspondence between the quantum and classical cases. By defining projection operators that on Hilbert space and classical phase-space, respectively, carefully defining volume units for classical phase-space, and suitably defining quantum and classical density operators, we show that formulas and most theorems concerning Observational entropy carry over directly between the classical and quantum case. The only key fundamental differences arise in the non-commutation of coarse-grainings in the quantum case, and from the non-existence of a direct classical analogue of quantum energy eigenstates. In contrast, quantum entanglement has no direct classical analog, so entanglement entropy lacks – to our knowledge – a clear correspondence with a classical quantity that behaves similarly.

Thus we view Observational entropy as a framework that relates and unifies a number of disparate views and definitions of entropy, as well as provides useful new perspectives on the connection between quantum and classical treatments of phenomena. We hope that this treatment proves both illuminating and useful in understanding how entropy manifests and operates in a wide range of physical systems.

Acknowledgements.

We are grateful to Dana Faiez and Joseph Schindler for helpful discussions. This research was supported by the Foundational Questions Institute (FQXi.org), of which AA is Associate Director, and by the Faggin Presidential Chair Fund.

Appendix A Proofs

We are going to use Jensen’s inequality: Let $f$ be a strictly concave function, $0\leq a_{i}\leq 1$, $\sum_{i}a_{i}=1$. Then for any $b_{i}\in\mathbb{R}$,

[TABLE]

$f(\sum_{i}a_{i}b_{i})=\sum_{i}a_{i}f(b_{i})$ if and only if $(\forall i,j|a_{i}\neq 0,a_{j}\neq 0)(b_{i}=b_{j}).$

Now we proceed with proofs of the paper’s theorems.

Proof.

(Theorem 1) (Observational entropy is a generalization of the Boltzmann entropy) Clearly, for a single point in phase-space $({\boldsymbol{x}},{\boldsymbol{p}})\in P_{i}$, we have $p_{i}=1$ and $p_{j}=0$ for $j\neq i$. Therefore

[TABLE]

∎

Proof.

(Theorem 2) (Observational entropy is a monotonic function of the coarse-graining.) ${\mathcal{C}}_{1}\hookrightarrow{\mathcal{C}}_{2}$ means that $P_{i_{1}}=\bigcup_{i_{2}\in I^{(i_{1})}}P_{i_{2}}$ for each $i_{1}$, therefore $p_{i_{1}}=\sum_{i_{2}\in I^{(i_{1})}}p_{i_{2}}$ and $V_{i_{1}}=\sum_{i_{2}\in I^{(i_{1})}}V_{i_{2}}$. Then from Jensen’s inequality applied on concave function $f(x)=-x\ln x$, taking $a_{i_{2}}=\frac{V_{i_{2}}}{V_{i_{1}}}$ and $b_{i_{2}}=\frac{p_{i_{2}}}{V_{i_{2}}}$, we have

[TABLE]

The equality conditions from the Jensen’s inequality show that $S_{O({\mathcal{C}}_{1})}({\hat{\rho}})=S_{O({\mathcal{C}}_{2})}({\hat{\rho}})$ if and only if

[TABLE]

To determine the constant $c^{(i_{1})}$ we multiply the equation by $V_{{i}_{2}}$ and sum over all $\forall i_{2}\in I^{(i_{1})}$, which gives

[TABLE]

And considering that for all $V_{i_{2}}=0$ also $p_{i_{2}}=0$, we can simplify Eq. (55), and obtain that $S_{O({\mathcal{C}}_{1})}({\hat{\rho}})=S_{O({\mathcal{C}}_{2})}({\hat{\rho}})$ if and only if

[TABLE]

∎

Proof.

(Theorem 3) (Observational entropy with multiple coarse-grainings is bounded) Since Observational entropy with multiple coarse-grainings can be rewritten as Observational entropy with a joint coarse-graining,

[TABLE]

it is enough to prove the inequalities just for a single coarse-graining ${\mathcal{C}}$,

[TABLE]

For the second inequality we define coarse-graining with a single element – the entire phase-space – ${\mathcal{C}}_{\Gamma}=\{\Gamma\}$. Clearly, this coarse-graining is coarser than any other coarse-graining, therefore from Theorem 2 we have

[TABLE]

The first inequality comes from the fact that one can choose a coarse-graining where macrostates are single points in phase-space. This coarse-graining is finer than any other coarse-graining, and one can easily derive that the Observational entropy is then equal to the Gibbs entropy. By the same argument we can therefore obtain the first inequality. However, this argument does not give us the equality conditions, for which we will have to make a more elaborate derivation as follows.

To prove the first inequality and the equality conditions, we define the spectral decomposition of the phase-space density in its eigenvector projectors $\rho=\sum_{({\boldsymbol{x}},{\boldsymbol{p}})}\rho({\boldsymbol{x}},{\boldsymbol{p}})\hat{P}_{({\boldsymbol{x}},{\boldsymbol{p}})}$ (meaning $\rho(\tilde{{\boldsymbol{x}}},\tilde{{\boldsymbol{p}}})=\sum_{({\boldsymbol{x}},{\boldsymbol{p}})}\rho({\boldsymbol{x}},{\boldsymbol{p}})\hat{P}_{({\boldsymbol{x}},{\boldsymbol{p}})}(\tilde{{\boldsymbol{x}}},\tilde{{\boldsymbol{p}}})$, where eigenvalues $\rho({\boldsymbol{x}},{\boldsymbol{p}})$ do not have to be different for different $({\boldsymbol{x}},{\boldsymbol{p}})$. The eigenvector projectors project onto infinitesimal regions surrounding $({\boldsymbol{x}},{\boldsymbol{p}})$ are defined as

[TABLE]

We also define spectral decomposition of the density matrix in a form where the eigenvector projectors associated with the same eigenvalue are now grouped together,

[TABLE]

where eigenvalues $\lambda$ are now different from each other. This decomposition is unique. It follows that for each $({\boldsymbol{x}},{\boldsymbol{p}})$ there exists $\lambda$ such that $\rho({\boldsymbol{x}},{\boldsymbol{p}})=\lambda$.

Now recall Eq. (9): the probability of a state being in a macrostate $i$ of volume $V_{i}=(\hat{P}_{i},\mu)$ is $p_{i}=(\hat{P}_{i},\rho)$.

Defining

[TABLE]

for $V_{i}\neq 0$ and $a_{({\boldsymbol{x}},{\boldsymbol{p}})}^{(i)}\equiv 0$ for $V_{\boldsymbol{i}}=0$, and then using the spectral decomposition of $\rho$ we have

[TABLE]

Since $\sum_{({\boldsymbol{x}},{\boldsymbol{p}})}\hat{P}_{({\boldsymbol{x}},{\boldsymbol{p}})}=1$,

[TABLE]

and since $\sum_{i}\hat{P}_{i}=1$, also

[TABLE]

A series of equalities and inequalities follow:

[TABLE]

We have used Eqs. (64) and (66) for the equalities, and applied Jensen’s Theorem on function $f(x)=-x\ln x$ to derive the inequality. We have chosen $a_{({\boldsymbol{x}},{\boldsymbol{p}})}\equiv a_{({\boldsymbol{x}},{\boldsymbol{p}})}^{(i)}$ and $b_{({\boldsymbol{x}},{\boldsymbol{p}})}\equiv\frac{\rho({\boldsymbol{x}},{\boldsymbol{p}})}{\mu}$ for the Theorem.

According to Jensen’s Theorem, the inequality becomes equality if and only if

[TABLE]

That is, the inequality becomes equality when for a given index $i$, all eigenvector projectors $\hat{P}_{(\tilde{{\boldsymbol{x}}},\tilde{{\boldsymbol{p}}})}$ of the phase-space density such that $(\hat{P}_{i},\hat{P}_{({\boldsymbol{x}},{\boldsymbol{p}})})\neq 0$ have the same associated eigenvalue $\rho({\boldsymbol{x}},{\boldsymbol{p}})$ with them. In other words, we can associate this unique eigenvalue to the index $i$ itself, $\rho_{i}\equiv\rho({\boldsymbol{x}},{\boldsymbol{p}})$, where $\rho({\boldsymbol{x}},{\boldsymbol{p}})$ is given by any representative $({\boldsymbol{x}},{\boldsymbol{p}})$ such that $(\hat{P}_{i},\hat{P}_{({\boldsymbol{x}},{\boldsymbol{p}})})\neq 0$. Realizing that $(\hat{P}_{i},\hat{P}_{({\boldsymbol{x}},{\boldsymbol{p}})})\neq 0$ is the same as saying $P_{i}\cap P_{({\boldsymbol{x}},{\boldsymbol{p}})}\neq\O$, or $\hat{P}_{({\boldsymbol{x}},{\boldsymbol{p}})}\hat{P}_{i}\neq 0$ we can rewrite Eq. (68) as

[TABLE]

Assuming that this holds, we can write

[TABLE]

where we have used $\sum_{({\boldsymbol{x}},{\boldsymbol{p}})}\hat{P}_{({\boldsymbol{x}},{\boldsymbol{p}})}=1$. Summing the above equation over $i$, and using $\sum_{i}\hat{P}_{i}=1$, we obtain

[TABLE]

i.e., $\rho$ can be decomposed using coarse-graining ${\mathcal{C}}=\{\hat{P}_{i}\}$. Defining sets $I^{(\lambda)}=\{i|\rho_{i}=\lambda\}$, we can rewrite this equation as

[TABLE]

and since decomposition Eq. (62) is unique, it must be that

[TABLE]

which by definition means ${\mathcal{C}}_{\rho}\hookrightarrow{\mathcal{C}}$. For multiple corse-graining this then means that ${\mathcal{C}}_{\rho}\hookrightarrow{\mathcal{C}}_{1,\dots,n}$.

Conversely, we assume that Eq. (73) holds. Points $({\boldsymbol{x}},{\boldsymbol{p}}),(\tilde{{\boldsymbol{x}}},\tilde{{\boldsymbol{p}}})$ such that $\hat{P}_{({\boldsymbol{x}},{\boldsymbol{p}})}\hat{P}_{i}\neq 0,\hat{P}_{(\tilde{{\boldsymbol{x}}},\tilde{{\boldsymbol{p}}})}\hat{P}_{i}\neq 0$ belong into the same macrostate $\hat{P}_{i}$, and therefore by Eq. (73) they must have the same associated eigenvalue,

[TABLE]

which means that Eq. (69) holds, thus inequality in (67) becomes equality. ∎

Proof.

(Theorem 4) (Observational entropy is non-increasing with each added coarse-graining.) Since joint coarse-graining ${\mathcal{C}}_{1,\dots,n}$ is finer than joint coarse-graining ${\mathcal{C}}_{1,\dots,n-1}$, from Theorem 2 we have

[TABLE]

Statement of the theorem then follows from Eq. (58). ∎

Proof.

(Theorem 5) Using $\hat{P}_{E}\hat{P}_{{\boldsymbol{n}}\tilde{E}}=\delta_{E,\tilde{E}}\hat{P}_{{\boldsymbol{n}}\tilde{E}}$ we have

[TABLE]

from which

[TABLE]

∎

Appendix B Details for dynamics of $S_{xE}$ for a microcanonical ensemble

Here we provide details for the calculation done in subsection VIII.2, showing that $S_{xE}=S_{O({\mathcal{C}}_{x},{\mathcal{C}}_{H_{2}})}$ for a microcanonical state of particles confined in the left part of the box is equal to the microcanonical entropy of this left part.

The initial state is a microcanonical state of $N$ particles contained in the left part of the box. It can be written in two ways as

[TABLE]

$\hat{P}_{E,\Delta E}^{(H_{1})}$ and $\hat{P}_{E,\Delta E}^{(H_{2})}$ denote projectors onto energy macrostates given by the first and the second Hamiltonian, and $\hat{P}_{(N,0)}$ is the projector onto a local particle number macrostate, corresponding to a statement that $N$ particles are in the left side of the box, and zero on the right.

We take $S_{xE}=S_{O({\mathcal{C}}_{x},{\mathcal{C}}_{H_{2}})}$ with local particle number coarse-graining that halves the box (${\boldsymbol{n}}=(n_{1},n_{2})$, $L_{1}=0$, $\Delta x=\frac{L}{2}$), and that has energy macrostates given by the second Hamiltonian, $H_{2}$. Using

[TABLE]

where $\delta_{n_{1},N}$ denotes Kronecker delta, we have

[TABLE]

from which we obtain

[TABLE]

In other words, $S_{xE}$ of the initial state is equal to the microcanonical entropy of the first half of the box.

Appendix C Dynamics of $S_{xE}$ for a single point in phase-space

Here we provide calculation on the long-time limit of $S_{xE}$ in the case of an initial single point in phase-space. We note that the derivation of $S_{xE}$ of an initial state follows exactly the same pattern as in Appendix B, with the only difference that we have $dE$ instead of $\Delta E$.

We now focus on calculating microcanonical entropy of the largest macrostate of phase-space jointly coarse grained by ${\mathcal{C}}_{X}$ and ${\mathcal{C}}_{H_{2}}$ (we will further denote $H\equiv H_{2}$).

Consider $N$ identical classical particles in $d$ dimensions evolving through a Hamiltonian $H$. Their positions and momenta are described by a point in phase-space $\Gamma\equiv({\boldsymbol{x}}_{1},\dots,{\boldsymbol{x}}_{N},{\boldsymbol{p}}_{1},\dots,{\boldsymbol{p}}_{N})$. They are confined to a box of spatial volume ${\cal V}=L^{d}$, where $L$ is a linear dimension. We subdivide the box into smaller ones of linear dimensions $\Delta x$, so that there are m\equiv\big{(}\frac{L}{\Delta x}\big{)}^{d} boxes. An arbitrary $\Gamma$ will be in some coarse-grained region $P_{{\boldsymbol{n}}E}$, and this will have $n_{1}$ particles in the first box, $n_{2}$ in the second, and in general $n_{i}$ in the $i$’th box. We would like to know how many distinct ways, $\mathcal{N}$, there are of arranging the $N$ particles having precisely these ${\boldsymbol{n}}=(n_{1},\dots,n_{m})$. The answer is the multinomial distribution

[TABLE]

where $\sum_{i=1}^{m}n_{i}=N$.

$\mathcal{N}$ is maximimized when the $n_{i}$’s are uniformly distributed, that is, for large $\frac{N}{m}$, $n_{i}=\frac{N}{m}$.

The spatial volume of a coarse grained region is $\Delta x^{Nd}\mathcal{N}(\{n_{1},\dots,n_{m}\})$. If we fine grain in energy, we can write the (phase-space) volume of the $i$th region that $\Gamma$ may be in:

[TABLE]

where \hat{P}_{E}(\{{\boldsymbol{x}}_{i}\}_{i},\{{\boldsymbol{p}}_{i}\}_{i})=\delta\big{(}E-H(\{{\boldsymbol{x}}_{i}\}_{i},\{{\boldsymbol{p}}_{i}\}_{i}) is the projector onto energy shell. $P_{{\boldsymbol{n}}E}$ contains all permutations of particles. This means that if we consider a single $\Gamma$ in this region, $P_{{\boldsymbol{n}}E}$ will also contain other $\Gamma$’s with all of particle permutations consistent the same particle numbers, ${\boldsymbol{n}}=(n_{1},\dots,n_{m})$, and the same energy $E$.

C.1 Case of dilute gas

Consider the case where the interaction between the particles is small. We will also take the particles to be indistinguishable, meaning all phase-space volumes are divided by $N!$. In the limit of small interactions, the Hamiltonian only depends on the ${\boldsymbol{p}}$s and we can integrate over $x$, and write

[TABLE]

Defining the number of microstates of $N$ indistinguishable particles in energy shell $E$ occupying spatial volume ${\cal V}$ as

[TABLE]

we can write

[TABLE]

$\Omega(E,{\cal V},N)$ defines the microcanonical entropy,181818We remind that the microcanonical entropy is defined as by the right hand side of Eq. (36), in which notation $V_{E}\equiv\Omega(E,{\cal V},N)\equiv\rho(E)\Delta E$. Here we use $\Omega(E,{\cal V},N)$ instead of the other two to spell out its dependence on the three variables. $S_{\mathrm{micro}}(E,{\cal V},N)=\ln[\Omega(E,{\cal V},N)]$ (taking $k_{B}=1$). For small interactions considered here, we can approximate (Eq. (2.40) in Schroeder (1999))

[TABLE]

where $M$ is the mass of a gas particle.191919This expression leads to the famous Sackur-Tetrode equation Ma (1985), S_{\mathrm{micro}}(E,{\cal V},N)=N\bigg{(}\ln\bigg{(}\frac{{\cal V}}{N}\bigg{(}\frac{4\pi ME}{Nd}\bigg{)}^{\frac{d}{2}}\bigg{)}+1+\frac{d}{2}\bigg{)}, but we will not use it here directly.

$\mathcal{N}$ is maximized with $n_{i}=\frac{N}{m}$. In this case, using Stirling’s approximation,

[TABLE]

Combining Eqs. (87), (88), and (89), and realizing that $\Delta x^{d}=\frac{{\cal V}}{m}$ (where ${\cal V}$ denotes the full spatial volume), we obtain $S_{xE}$ for the point in phase-space $\Gamma$ which wandered into the largest macrostate,

[TABLE]

If the point in phase-space did not wander into the largest macrostate, but, let’s say, the second largest (given by $n_{1}=\frac{N+1}{m}-1,n_{i}=\frac{N+1}{m}$ for $i>1$), there would be minor modifications to the above formula, which would be negligible compared to the first term - the microcanonical entropy. In the end, it does not matter into which of the large macrostate the particle wanders – the leading term will be always the microcanonical entropy.

To understand to correction term in more detail, we can look at the entropy per particle. Then the relevant quantity to consider here is $\ln(\mathcal{N}_{max})/N$.

[TABLE]

We take a limiting process where we fix the particle density, and send $N$ to infinity, while keeping the particle density constant. For large $N$ we can ignore the second term on the right hand side. $N_{m}\equiv\frac{N}{m}$ is the average number of particles per box. In the limit where we fix the density average number of particles and increase the box length so that $N_{m}$ becomes large, the last term on the right hand side $-(1/2N_{m})\ln(2\pi N_{m})$ becomes small, and vanishes as $N_{m}\rightarrow\infty$. This correction term therefore represents a finite size effect.

C.2 General classical systems

The case of a dilute gas can be extended to a general homogeneous classical system that is extensive. The system with maximum entropy will be the one where the number in each box is the same. This follows from the fact that all boxes must have the same chemical potential in equilibrium. The system is most likely to find itself in this coarse-grained region. The logarithm of Eq. (84) represents the entropy of a system that has been partitioned. As compared to the entropy of the entire system, each particle is confined to a box of width $\Delta x$. It is as if barriers had been added to prevent the exchange of particles between regions. However energy can still be exchanged, therefore the temperature of each sub-system is the same. If the system has a density $\rho$ and temperature $T$, corresponding to its total energy, then the entropy per particles $s(E,T)$ will not depend on system size in the thermodynamic limit. There are correction to this due to finite size effects and are particularly pronounced at critical points Privman and Fisher (1984). They have the general behavior as above, becoming negligible in the limit of large box size. The actual error will depend on the universality class of the system being studied.

Appendix D Properties of $S_{F}$

First we rephrase the coarse-graining ${\mathcal{C}}_{X{\boldsymbol{E}}}=\{P_{{\boldsymbol{n}}{\boldsymbol{E}}}\}_{{\boldsymbol{n}},{\boldsymbol{E}}}$, Eq. (44), in mathematically precise terms. For simplicity we set $L_{1}=0$ and $L_{2}=L$.

The local particle number-local energy macrostates are then

[TABLE]

In order to define this in mathematical terms, we need to define the notion of local Hamiltonian. While the general theory that applies also to a phase-space density will be done elsewhere Šafránek et al. (2020), here we can still treat the case of a single point of phase-space. At every time $t$, there there is some number of particles in each of the $m$ bins. Let’s say we have $n_{1}$ particle between $[0,\Delta x)$. For this particular time, the local Hamiltonian $H_{1}$ will be a function of phase-space points of $n_{1}$ particles,202020For a different time, it can be a function of a different number of particles – this hints on the construction for the general case which requires the construction of a Fock phase-space Šafránek et al. (2020). which maps a point in phase-space to the energy

[TABLE]

where $x_{i}\in[0,\Delta x)$, $i=1,\dots,n_{1}$ (note that the lower index here is just a label denoting variables – we are not saying that exactly the first $n_{1}$ particles are in the first bin; in case of indistinguishable particles, this label does not even matter). For instance, in the case of non-interacting particles we have $H_{1}=\sum_{i=1}^{n_{1}}\frac{p_{i}^{2}}{2M}$. If there is some interaction, the interaction between particles within the bin is accounted for in the local Hamiltonian, while the interaction between different bins is ignored.

We denote $S_{N}$ the set of permutations on the set $\{1,2,\dots,N\}$. With these tools, we can write

[TABLE]

Clearly, independently of the permutation, the phase-space volume will be always the same for a specific permutation $\pi$. The question, then, is how many permutations lead to a specific distribution of particles ${\boldsymbol{n}}$ between the bins. The answer is again

[TABLE]

which is the total number of permutations, divided by the possible number of permutations within the first bin, within the second bin, and so on. We can therefore write the volume of macrostate $P_{{\boldsymbol{n}}{\boldsymbol{E}}}$ as

[TABLE]

Assuming generalization to spatial dimension $d$, we similarly derive $V_{{\boldsymbol{n}}{\boldsymbol{E}}}=\Pi_{k=1}^{m}\Omega(E_{k},\Delta x^{d},n_{k})$. This means that for a single point in phase-space $\gamma\in P_{{\boldsymbol{n}}{\boldsymbol{E}}}$,

[TABLE]

and therefore $S_{F}$ is additive.

Now, let us study the long-time limit. After a long time, assuming identical particles of a dilute and weakly interacting gas, the largest macrostate is given by $E_{k}=\frac{E}{m}$, $n_{k}=\frac{N}{m}$, (where $m=\frac{\cal V}{\Delta x^{d}}$), giving

[TABLE]

Here we have used Eq. (88) and Stirling’s approximation for the factorials. The correction term $\frac{m-1}{2}\ln\frac{d}{2}$ represents a finite-size effect, which result in an order-1 correction in the thermodynamic limit.

Appendix E Additivity of quantum $S_{xE}$ and non-additivity of classical

As explained in Section (VIII.4), the classical $S_{xE}$ is not additive. This is because the classical global energy macrostate is defined as

[TABLE]

This means that any microstate $({\boldsymbol{x}},{\boldsymbol{p}})$ that has energy $E_{1}$ in the left region and energy $E_{2}$ in the right region, $E_{1}+E_{2}=E$, will fall into the same macrostate $P_{E}$ with any other states given by $\tilde{E}_{1}+\tilde{E}_{2}=E$.

Quantum mechanically, for infinite resolution in energy ($\Delta E=0$), this is not true. This is because in a generic system, even states with very similar sum of energies $E\approx E_{1}+E_{2}\approx\tilde{E}_{1}+\tilde{E}_{2}$ are not exactly the same, so energy eigenstates of the global Hamiltonian $|E_{1}+E_{2}\rangle$ and $|\tilde{E}_{1}+\tilde{E}_{2}\rangle$ constitute their own macrostates, $\hat{P}_{E_{1}+E_{2}}=|E_{1}+E_{2}\rangle\langle E_{1}+E_{2}|$ and $\hat{P}_{\tilde{E}_{1}+\tilde{E}_{2}}=|\tilde{E}_{1}+\tilde{E}_{2}\rangle\langle\tilde{E}_{1}+\tilde{E}_{2}|$. (We ignored corrections coming from ignoring the interaction terms between the two regions that also contribute to the total Hamiltonian and therefore also to local energy eigenstates.)

If a finite resolution in energy is used ($\Delta E>0$), then the situation for quantum $S_{xE}$ is very similar to the classical case, because then the quantum macrostates are constituted by

[TABLE]

Put simply, classical energy macrostates correspond to quantum macrostate with $\Delta E>0$, but for $\Delta E=0$, the quantum macrostates are much smaller – actually consisting of a single microstate. This means that for $\Delta E=0$ the global energy macrostates are actually small enough to distinguish between different local energies, which is why the quantum $S_{xE}$ is additive.

We have confirmed this numerically in some particular cases, as exhibited in Fig. 6.

Making the analytical argument for additivity precise would require similar arguments to those employed in appendices of Ref. Šafránek et al. (2019b). That is, identifying the interaction part of the Hamiltonian with a random matrix, and then studying the overlaps between energy eigenstates the full Hamiltonian of the system and those of the Hamiltonian system without the interaction, which gives the probability $p_{E}=|\langle E_{1},E_{2}|E\rangle|^{2}$ that goes into $S_{xE}$. We leave this for future work.