Gibbs and Boltzmann Entropy in Classical and Quantum Mechanics

TL;DR

This paper compares Gibbs and Boltzmann entropies in classical and quantum systems, highlighting their differences in equilibrium and non-equilibrium states, and advocates for Boltzmann entropy as the thermodynamic entropy, especially in quantum thermalization.

Contribution

It provides a detailed comparison of Gibbs and Boltzmann entropies in classical and quantum contexts, emphasizing the relevance of Boltzmann entropy for non-equilibrium and quantum thermalization.

Findings

Gibbs and Boltzmann entropies agree at equilibrium for macroscopic systems.

Boltzmann entropy better describes thermodynamic entropy in non-equilibrium.

Quantum Boltzmann entropy supports the individualist view of thermalization.

Abstract

The Gibbs entropy of a macroscopic classical system is a function of a probability distribution over phase space, i.e., of an ensemble. In contrast, the Boltzmann entropy is a function on phase space, and is thus defined for an individual system. Our aim is to discuss and compare these two notions of entropy, along with the associated ensemblist and individualist views of thermal equilibrium. Using the Gibbsian ensembles for the computation of the Gibbs entropy, the two notions yield the same (leading order) values for the entropy of a macroscopic system in thermal equilibrium. The two approaches do not, however, necessarily agree for non-equilibrium systems. For those, we argue that the Boltzmann entropy is the one that corresponds to thermodynamic entropy, in particular in connection with the second law of thermodynamics. Moreover, we describe the quantum analog of the Boltzmann entropy, and we argue that the individualist (Boltzmannian) concept of equilibrium is supported by the recent works on thermalization of closed quantum systems.

Equations

$$S_{\mathrm{G}}(\rho )=-k{\int }_{\mathcal{X}}dx\rho (x)\log \rho (x),$$

$$S_{\mathrm{B}}(X)=k\log \mathrm{vol}\Gamma (X),$$

$$\mathcal{X}=\bigcup _{\nu }{\Gamma }_{\nu },$$

$$S_{\mathrm{B}}(X)=S_{\mathrm{B}}(\nu ):=k\log \mathrm{vol}{\Gamma }_{\nu }.$$

$$\frac{d{S}_{\mathrm{G}}({\rho }_t)}{dt}=0,$$

$$\frac{\partial\rho}{\partial t}=-\sum_{i=1}^{6N}\frac{\partial}{\partial x_{i}}\bigl{(}\rho(x)\,v(x)\bigr{)}\,,$$

$$\frac{d{X}_i}{dt}=v_i(X(t))$$

$$S_{\mathrm{G}}(\rho \circ {\Phi }^{-1})=S_{\mathrm{G}}(\rho ).$$

$$S=-k{\int }_{{\mathcal{X}}_1}{d}^3\mathbf{q}{d}^3\mathbf{v}\tilde{f}(\mathbf{q},\mathbf{v})\log \tilde{f}(\mathbf{q},\mathbf{v})$$

$$S_{\mathrm{vN}}(\hat{\rho })=-k\mathrm{tr}(\hat{\rho }\log \hat{\rho }).$$

$$\mathcal{H}=\underset{\nu }{⨁}{\mathcal{H}}_{\nu },$$

$$S_{\mathrm{qB}}(\nu )=k\log \dim {\mathcal{H}}_{\nu }.$$

$$S_{\mathrm{qB}}(\nu )=S_{\mathrm{qB}}({\nu }_1)+S_{\mathrm{qB}}({\nu }_2).$$

$$S_{\mathrm{qB}}(\mathrm{eq})\approx k\log \dim {\mathcal{H}}_{(E-\Delta E,E]},$$

$$\hat{S}=\sum _{\nu }{S}_{\mathrm{qB}}(\nu ){\hat{P}}_{\nu },$$

$${\hat{P}}_{\nu }\psi$$

$$S_{\mathrm{qB}}(\mathcal{S}),$$

$${\hat{\rho }}_1:={\mathrm{tr}}_2|\Psi ⟩⟨\Psi |$$

$$\hat{\rho}_{\mathrm{cond}}:=\operatorname{tr}_{s_{2}}\langle x_{2}|\Psi\rangle\langle\Psi|x_{2}\rangle\Big{|}_{x_{2}=X_{2}}\,,$$

$$\nu$$

$$S_{\mathrm{B}}({\mathcal{S}}_1):=S_{\mathrm{B}}(\nu (X_1)).$$

$${\hat{\rho }}_1:={\mathrm{tr}}_2|{\psi }_t⟩⟨{\psi }_t|,$$

$${\hat{\rho }}_1\approx \frac{1}{Z}{e}^{-\beta {\hat{H}}_1},$$

$${\rho }_X(x)=\frac{1}{\mathrm{vol}\Gamma (X)}{1}_{\Gamma (X)}(x)=\{\begin{array}{ll}(\mathrm{vol}\Gamma (X){)}^{-1}& \text{if }x\in \Gamma (X)\\ 0& \text{if }x\notin \Gamma (X).\end{array}$$

$$S_{\mathrm{G}}({\rho }_X)=S_{\mathrm{B}}(X).$$

$$S_{\mathrm{G}}(\rho )=-k{\int }_{A}dx\frac{(-1)}{\mathrm{vol}A}\log \mathrm{vol}A=k\log \mathrm{vol}A.$$

$$\Gamma_{\nu}=\Bigl{\{}x\in\mathscr{X}:M_{j}(x)=\nu_{j}\>\forall j\Bigr{\}}\,,$$

$$M_1(x)=[H(x)/\Delta E]\Delta E$$

$$\mathscr{X}_{\mathrm{mc}}:=\mathscr{X}_{(E-\Delta E,E]}:=\Bigl{\{}x\in\mathscr{X}:E-\Delta E<H(x)\leq E\Bigr{\}}\,.$$

$$\frac{\mathrm{vol}{\Gamma }_{\tilde{\nu }}}{\mathrm{vol}{\mathcal{X}}_{\mathrm{mc}}}\approx 1-10^{-cN}$$

Full text

Key words: statistical mechanics; second law of thermodynamics; thermal equilibrium.

1. 1 Introduction

2. 1.1 Two Definitions of Entropy in Classical Statistical Mechanics

3. 1.2 $X$ vs. $\rho$

4. 2 Status of the Second Law

5. 2.1 Gibbs Entropy

6. 2.2 Boltzmann’s $H$ and $k\log W$

7. 3 The Quantum Case

8. 3.1 Entropy in Quantum Mechanics

9. 3.2 Open Systems

10. 3.3 Quantum Thermalization

11. 4 Subjective Entropy Is Not Enough

12. 4.1 Cases of Wrong Values

13. 4.2 Explanatory and Predictive Power

14. 4.3 Phase Points Play No Role

15. 4.4 What is Attractive About Subjective Entropy

16. 4.5 Remarks

17. 5 Boltzmann’s Vision Works

18. 5.1 Macro States

19. 5.2 Entropy Increase

20. 5.3 Non-Equilibrium

21. 5.4 Concrete Example: The Boltzmann Equation

22. 5.5 Rigorous Result

23. 5.6 Empirical vs. Marginal Distribution

24. 5.7 The Past Hypothesis

25. 5.8 Boltzmann Entropy Is Not Always $H$

26. 6 Gibbs Entropy as Fuzzy Boltzmann Entropy

27. 7 The Status of Ensembles

28. 7.1 Typicality

29. 7.2 Equivalence of Ensembles and Typicality

30. 7.3 Ensemblist vs. Individualist Notions of Thermal Equilibrium

31. 7.4 Ergodicity and Mixing

32. 7.5 Remarks

33. 8 Second Law for the Gibbs Entropy?

34. 9 Further Aspects of the Quantum Case

35. 9.1 Macro Spaces

36. 9.2 Some Historical Approaches

37. 9.3 Entanglement Entropy

38. 9.4 Increase of Quantum Boltzmann Entropy

39. 10 Conclusions

1 Introduction

Disagreement among scientists is often downplayed, and science often presented as an accumulation of discoveries, of universally accepted contributions to our common body of knowledge. But in fact, there is substantial disagreement among physicists, not only concerning questions that we have too little information about to settle them, such as the nature of dark matter, but also concerning conceptual questions about which all facts have long been in the literature, such as the interpretation of quantum mechanics. Another question of the latter type concerns the definition of entropy and some related concepts. In particular, two different formulations are often given in the literature for how to define the thermodynamic entropy (in equilibrium and non-equilibrium states) of a macroscopic physical system in terms of a microscopic, mechanical description (classical or quantum).

1.1 Two Definitions of Entropy in Classical Statistical Mechanics

In classical mechanics, the Gibbs entropy of a physical system with phase space $\mathscr{X}$, for example $\mathscr{X}=\mathbb{R}^{6N}=\{(\boldsymbol{q}_{1},\boldsymbol{v}_{1},\ldots,\boldsymbol{q}_{N},\boldsymbol{v}_{N})\}$ for $N$ point particles in $\mathbb{R}^{3}$ with positions $\boldsymbol{q}_{j}$ and velocities $\boldsymbol{v}_{j}$, is defined as

[TABLE]

where $k$ is the Boltzmann constant, $dx=N!^{-1}d^{3}\boldsymbol{q}_{1}\,d^{3}\boldsymbol{v}_{1}\cdots d^{3}\boldsymbol{q}_{N}\,d^{3}\boldsymbol{v}_{N}$ the (symmetrized) phase space volume measure, $\log$ the natural logarithm,555One actually takes the expression $u\log u$ to mean the continuous extension of the function $u\mapsto u\log u$ from $(0,\infty)$ to the domain $[0,\infty)$; put differently, we follow the convention to set $0\log 0=0$. and $\rho$ a probability density on $\mathscr{X}$.666Changing the unit of phase space volume will change $\rho(x)$ by a constant factor and thus $S_{\mathrm{G}}(\rho)$ by addition of a constant, an issue that does not matter for the applications and disappears anyway in the quantum case.

The Boltzmann entropy of a macroscopic system is defined as

[TABLE]

where $X\in\mathscr{X}$ is the actual phase point of the system, $\operatorname{\mathrm{vol}}$ means the volume in $\mathscr{X}$, and $\Gamma(X)$ is the set of all phase points that “look macroscopically the same” as $X$. Obviously, there is no unique precise definition for “looking macroscopically the same,” so we have a certain freedom to make a reasonable choice. It can be argued that for large numbers $N$ of particles (as appropriate for macroscopic physical systems), the arbitrariness in the choice of $\Gamma(X)$ shrinks and becomes less relevant.777For example, for a dilute gas and large $N$, $-S_{\mathrm{B}}(X)/k+N$ equals approximately the $H$ functional, i.e., the integral in (9) below, which does not refer any more to a specific choice of boundaries of $\Gamma(X)$. A convenient procedure is to partition the phase space into regions $\Gamma_{\nu}$ we call macro sets (see Figure 1),

[TABLE]

and to take as $\Gamma(X)$ the $\Gamma_{\nu}$ containing $X$. That is,

[TABLE]

We will give more detail in Section 5. Boltzmann’s definition (2) is often abbreviated as “$S_{\mathrm{B}}=k\log W$” with $W=\operatorname{\mathrm{vol}}\Gamma(X)$. In every energy shell there is usually one macro set $\Gamma_{\nu}=\Gamma_{\mathrm{eq}}$ that corresponds to thermal equilibrium and takes up by far most (say, more than $99.99\%$) of the volume (see Section 5.1).

1.2 $X$ vs. $\rho$

An immediate problem with the Gibbs entropy is that while every classical system has a definite phase point $X$ (even if we observers do not know it), a system does not “have a $\rho$”; that is, it is not clear which distribution $\rho$ to use. For a system in thermal equilibrium, $\rho$ presumably means a Gibbsian equilibrium ensemble (micro-canonical, canonical, or grand-canonical). It follows that, for thermal equilibrium states, $S_{\mathrm{B}}$ and $S_{\mathrm{G}}$ agree to leading order, see (31) below. In general, several possibilities for $\rho$ come to mind:

• (a)

ignorance: $\rho(x)$ expresses the strength of an observer’s belief that $X=x$.

• (b)

preparation procedure: A given procedure does not always reproduce the same phase point, but produces a random phase point with distribution $\rho$.

• (c)

coarse graining: Associate with every $X\in\mathscr{X}$ a distribution $\rho_{X}(x)$ on $\mathscr{X}$ that captures how macro-similar $X$ and $x$ are (or perhaps, how strongly an ideal observer seeing a system with phase point $X$ would believe that $X=x$).

Correspondingly, there are several different notions of Gibbs entropy, which we will discuss in Sections 4 and 6. Here, maybe (c) could be regarded as a special case of (a), and thermal equilibrium ensembles as a special case of (b). In fact, it seems that Gibbs himself had in mind that any system in thermal equilibrium has a random phase point whose distribution $\rho$ should be used, which is consistent with option (b); in his words (Gibbs, 1902, p. 152):

[…] we shall find that [the] distinction [between interaction of a system $S_{1}$ with a system $S_{2}$ with determined phase point $X_{2}$ and one with distribution $\rho_{2}$] corresponds to the distinction in thermodynamics between mechanical and thermal action.

In our discussion we will also address the status of the Gibbsian ensembles (see also Goldstein, 2019). We will argue that $S_{\mathrm{B}}$ qualifies as a definition of thermodynamic entropy whereas version (a) of $S_{\mathrm{G}}$ does not; (b) is not correct in general; and (c) is acceptable to the extent that it is not regarded as a special case of (a).

Different views about the meaning of entropy and the second law have consequences about the explanatory and predictive power of the second law that we will consider in Section 4. They also have practical consequences in the formulation of hydrodynamic equations, e.g., Navier-Stokes equations, for macroscopic variables (Goldstein and Lebowitz, 2004): such macroscopic equations can be compatible with a microscopic Hamiltonian evolution only if they make sure that the Boltzmann entropy increases.

The remainder of this article is organized as follows. In Section 2, we raise the question of the status of the second law for $S_{\mathrm{G}}$ and $S_{\mathrm{B}}$. In Section 3, we consider the analogs of Gibbs and Boltzmann entropy in quantum mechanics. The following Sections 4–8 focus again on the classical case for simplicity. In Section 4, we discuss and criticize option (a), the idea that entropy is about subjective knowledge. In Section 5, we explain why Boltzmann entropy indeed tends to increase with time and discuss doubts and objections to this statement. In Section 6, we discuss an individualist understanding of Gibbs entropy as a generalization of Boltzmann entropy. In Section 7, we discuss the status of Gibbs’s ensembles. In Section 8, we comment on a few proposals for how entropy increase should work for Gibbs entropy. In Section 9, we add some deeper considerations about entropy in quantum mechanics. In Section 10, we conclude.

2 Status of the Second Law

2.1 Gibbs Entropy

Another immediate problem with the Gibbs entropy is that it does not change with time,

[TABLE]

if $\rho$ is taken to evolve in accord with the microscopic Hamiltonian dynamics, that is, according to the Liouville equation

[TABLE]

where $v(x)$ is the vector field on $\mathscr{X}$ that appears in the equation of motion

[TABLE]

for the phase point $X(t)\in\mathscr{X}$, such as $v=\omega\nabla H$ with $6N\times 6N$ matrix \omega=\bigl{(}\begin{smallmatrix}0&I\\ -I&0\end{smallmatrix}\bigr{)} in position and momentum coordinates and $H$ the Hamiltonian function. Generally, the Gibbs entropy does not change when $\rho$ gets transported by any volume-preserving bijection $\Phi:\mathscr{X}\to\mathscr{X}$,

[TABLE]

In particular, by Liouville’s theorem of the conservation of phase space volume, this applies when $\Phi=\Phi_{t}$ is the Hamiltonian time evolution, with $X(t)=\Phi_{t}(X(0))$. The time independence of $S_{\mathrm{G}}(\rho)$ conflicts with the formulation of the second law given by Clausius, the person who coined the “laws of thermodynamics” as follows (Clausius, 1865, p. 365):

1. The energy of the universe is constant.

2. The entropy of the universe tends to a maximum.

Among the authors who took entropy to be the Gibbs entropy, some (e.g., Khinchin, 1941) have argued that Clausius’s wording of the second law is inappropriate or exaggerated, others (e.g., Mackey, 1989) that the Liouville evolution (6) is not the relevant evolution here. We will come back to this point in Section 8. As we will explain in Section 5, Clausius’s statement is actually correct for the Boltzmann entropy.

2.2 Boltzmann’s $H$ and $k\log W$

We should address right away a certain confusion about Gibbs and Boltzmann entropy that has come from the fact that Boltzmann used, in connection with the Boltzmann equation, the definition

[TABLE]

for entropy. Boltzmann used the notation $H$ (not to be confused with the Hamiltonian function) for the integral in (9); this functional gave name to the $H$-theorem, which asserts that $H$ always decreases—except in thermal equilibrium, when $H$ is constant in time.

Here, $\mathscr{X}_{1}$ is the 1-particle phase space (for definiteness, $\mathbb{R}^{6}$), $\tilde{f}=Nf$, and $f$ is a normalized distribution density in $\mathscr{X}_{1}$. The formula (9) obviously looks much like the Gibbs entropy (and has presumably inspired Gibbs’s definition, which was published after (9)). In fact, (9) is also the Gibbs entropy of $\rho(x_{1},\ldots,x_{N}):=f(x_{1})\cdots f(x_{N})$ (up to addition of the constant $kN\log N$). But here it is relevant that $f$ means the empirical distribution of $N$ points in $\mathscr{X}_{1}$ (after smoothing, or in the limit $N\to\infty$), and is thus computed from $X\in\mathscr{X}$ (see Section 5.4 below for more detail). So, the $H$ functional or the $S$ in (9) is indeed a function on $\mathscr{X}$, and in fact a special case of (2) (up to addition of the constant $kN$) corresponding to a particular choice of dividing $\mathscr{X}$ into macro sets $\Gamma_{\nu}$ (see Section 5.1).

3 The Quantum Case

Consider a macroscopic quantum system (e.g., a gas of $N>10^{23}$ atoms in a box) with Hilbert space $\mathscr{H}$. We write $\mathbb{S}(\mathscr{H})$ for the unit sphere in the Hilbert space $\mathscr{H}$.

3.1 Entropy in Quantum Mechanics

The natural analog of the Gibbs entropy is the quantum Gibbs entropy or von Neumann entropy of a given density matrix $\hat{\rho}$ on $\mathscr{H}$ (von Neumann, 1927),

[TABLE]

Von Neumann (1929) himself thought that this formula was not the most fundamental one but only applicable in certain circumstances; we will discuss his proposal for the fundamental definition in Section 9 below. In a sense, the density matrix $\hat{\rho}$ plays the role analogous to the classical distribution density $\rho$, and again, the question arises as to what exactly $\hat{\rho}$ refers to: an observer’s ignorance or what? Our discussion of options (a)–(c) above for the Gibbs entropy will apply equally to the von Neumann entropy. In addition, there is also a further possibility:

• (d)

reduced density matrix: The system $\mathscr{S}_{1}$ is entangled with another system $\mathscr{S}_{2}$, and $\hat{\rho}$ is obtained from the (perhaps even pure) state $\Psi\in\mathbb{S}(\mathscr{H}_{1}\otimes\mathscr{H}_{2})$ of $\mathscr{S}_{1}\cup\mathscr{S}_{2}$ through a partial trace, $\hat{\rho}=\operatorname{tr}_{2}|\Psi\rangle\langle\Psi|$.

As we will discuss in Section 9.3, $S_{\mathrm{vN}}$ with option (d) does not yield a good notion of thermodynamic entropy, either.

In practice, systems are never isolated. But even if a macroscopic system were perfectly isolated, heat would flow in it from the hotter to the cooler parts, and, as we will explain in this section, there is a natural sense in which entropy can be defined and increases. The idealization of an isolated system helps us focus on this sense. In Section 3.2, we will point out why the results also cover non-isolated systems.

The closest quantum analog of the Boltzmann entropy is the following. A macro state $\nu$ should correspond to, instead of a subset $\Gamma_{\nu}$ of phase space, a subspace $\mathscr{H}_{\nu}$ of Hilbert space $\mathscr{H}$, called a macro space in the following; for different macro states $\nu^{\prime}\neq\nu$, the macro spaces should be mutually orthogonal, thus yielding a decomposition of Hilbert space into an orthogonal sum (von Neumann, 1929; Goldstein et al., 2010a, b),

[TABLE]

instead of the partition (3). Now the dimension of a subspace of $\mathscr{H}$ plays a role analogous to the volume of a subset of $\mathscr{X}$, and correspondingly we define the quantum Boltzmann entropy of a macro state $\nu$ by (Griffiths, 1994; Lebowitz, 2008; Goldstein et al., 2010b)

[TABLE]

In fact, already Einstein (1914, Eq. (4a)) argued that the entropy of a macro state should be proportional to the log of the “number of elementary quantum states” compatible with that macro state; it seems that (12) fits very well with this description. To a system with wave function $\psi\in\mathscr{H}_{\nu}$ (or, for that matter, a density matrix concentrated in $\mathscr{H}_{\nu}$) we attribute the entropy value $S_{\mathrm{qB}}(\psi):=S_{\mathrm{qB}}(\nu)$.

It seems convincing that $S_{\mathrm{qB}}(\nu)$ yields the correct value of thermodynamic entropy. For one thing, it is an extensive, or additive quantity: If we consider two systems $\mathscr{S}_{1},\mathscr{S}_{2}$ with negligible interaction, then the Hilbert space of both together is the tensor product, $\mathscr{H}=\mathscr{H}_{\mathscr{S}_{1}\cup\mathscr{S}_{2}}=\mathscr{H}_{\mathscr{S}_{1}}\otimes\mathscr{H}_{\mathscr{S}_{2}}$, and it seems plausible that the macro states of $\mathscr{S}_{1}\cup\mathscr{S}_{2}$ correspond to specifying the macro state of $\mathscr{S}_{1}$ and $\mathscr{S}_{2}$, resp., i.e., $\nu=(\nu_{1},\nu_{2})$ with $\mathscr{H}_{\nu}=\mathscr{H}_{\nu_{1}}\otimes\mathscr{H}_{\nu_{2}}$. As a consequence, the dimensions multiply, so

[TABLE]

For another thing, it is plausible that, analogously to the classical case, in every energy shell (i.e., the subspace $\mathscr{H}_{(E-\Delta E,E]}$ corresponding to an energy interval $(E-\Delta E,E]$ with $\Delta E$ the resolution of macroscopic energy measurements) there is a dominant macro space $\mathscr{H}_{\tilde{\nu}}$ whose dimension is near (say, greater than $99.99\%$ of) the dimension of the energy shell, and this macro state $\tilde{\nu}$ corresponds to thermal equilibrium (e.g., Tasaki, 2016), $\mathscr{H}_{\tilde{\nu}}=\mathscr{H}_{\mathrm{eq}}$. As a consequence,

[TABLE]

and the right-hand side is well known to yield appropriate values of thermodynamic entropy in thermal equilibrium. In fact, the right-hand side agrees with $S_{\mathrm{vN}}(\hat{\rho}_{\mathrm{mc}})$, the von Neumann entropy associated with the micro-canonical density matrix $\hat{\rho}_{\mathrm{mc}}$ (i.e., the normalized projection to $\mathscr{H}_{(E-\Delta E,E]}$) and with thermal equilibrium at energy $E$.

Of course, a general pure quantum state $\psi\in\mathbb{S}(\mathscr{H})$ will be a non-trivial superposition of contributions from different $\mathscr{H}_{\nu}$’s, $\psi=\sum_{\nu}\hat{P}_{\nu}\psi$, where $\hat{P}_{\nu}$ is the projection to $\mathscr{H}_{\nu}$. One can say that $\psi$ is a superposition of different entropy values, and in analogy to other observables one can define the self-adjoint operator

[TABLE]

whose eigenvalues are the $S_{\mathrm{qB}}(\nu)$ and eigenspaces the $\mathscr{H}_{\nu}$.

Here, the question arises as to which entropy value we should attribute to a system in state $\psi$. At this point, the foundations of statistical mechanics touch the foundations of quantum mechanics, as the problem of Schrödinger’s cat (or the measurement problem of quantum mechanics) concerns precisely the status of wave functions $\psi$ that are superpositions of macroscopically different contributions, given that our intuition leads us to expect a definite macroscopic situation. The standard “Copenhagen” formulation of quantum mechanics does not have much useful to say here, but several proposed “quantum theories without observers” have long solved the issue in clean and clear ways (Goldstein, 1998):

• •

Bohmian mechanics (in our view the most convincing proposal) admits further variables besides $\psi$ by assuming that quantum particles have definite positions, too. Since Schrödinger’s cat then has an actual configuration, there is a fact about whether it is dead or alive, even though the wave function is still a superposition. In the same way, the configuration usually selects one of the macroscopically different contributions $\hat{P}_{\nu}\psi$; in fact, this happens when

[TABLE]

Artificial examples can be constructed for which this condition is violated, but it seems that (16) is fulfilled in practice. In that case, it seems natural to regard the entropy of that one contribution selected by the Bohmian configuration as the actual entropy value.

• •

Collapse theories modify the Schrödinger equation in a non-linear and stochastic way so that, for macroscopic systems, the evolution of $\psi$ avoids macroscopic superpositions and drives $\psi$ towards (a random) one of the $\mathscr{H}_{\nu}$. Then the question about the entropy of a macroscopic superposition does not arise.

• •

Maybe a many-worlds view (Everett, 1957; Wallace, 2012), in which each of the contributions corresponds to a part of reality, can be viable (see Allori et al. (2011) for critical discussion). Then the system’s entropy has different values in different worlds.

We have thus defined the quantum Boltzmann entropy of a macroscopic system,

[TABLE]

in each of these theories.

Some authors (e.g., von Neumann (1929); Safranek et al. (2017, 2018)) have proposed averaging $S_{\mathrm{qB}}(\nu)$ with weights $\|\hat{P}_{\nu}\psi\|^{2}$, but that would yield an average entropy, not the entropy (see also Section 4.1 and our discussion of (66) in Section 9.2).

Let us turn to the question of what the second law asserts in quantum mechanics. Since $S_{\mathrm{vN}}$ remains constant under the unitary time evolution in $\mathscr{H}$, the discussion given in Section 8 for $S_{\mathrm{G}}$ applies equally to $S_{\mathrm{vN}}$. About $S_{\mathrm{qB}}$, we expect the following.

Conjecture 1**.**

In Bohmian mechanics and collapse theories, for macroscopic systems $\mathscr{S}$ with reasonable Hamiltonian $\hat{H}$ and decomposition $\mathscr{H}=\oplus_{\nu}\mathscr{H}_{\nu}$, every $\nu^{*}$ and most $\psi\in\mathbb{S}(\mathscr{H}_{\nu^{*}})$ are such that with probability close to 1, $S_{\mathrm{qB}}(\mathscr{S})$ is non-decreasing with time, except for infrequent, shallow, and short-lived valleys, until it reaches the thermal equilibrium value. Moreover, after reaching that value, $S_{\mathrm{qB}}(\mathscr{S})$ stays at that value for a very long time.

Careful studies of this conjecture would be of interest but are presently missing. We provide a bit more discussion in Section 9.

3.2 Open Systems

For open (non-isolated) systems, the quantum Boltzmann entropy can still be considered and should still tend to increase—not for the reasons that make the von Neumann entropy of the reduced density matrix increase (evolution from pure to mixed states), but for the reasons that make the quantum Boltzmann entropy of an isolated system in a pure state increase (evolution from small to large macro spaces). Let us elaborate.

In fact, the first question about an open system $\mathscr{S}_{1}$ (interacting with its environment $\mathscr{S}_{2}$) would be how to define its state. A common answer is the reduced density matrix

[TABLE]

(if $\mathscr{S}_{1}\cup\mathscr{S}_{2}$ is in the pure state $\Psi$); another possible answer is the conditional density matrix (Dürr et al., 2005)

[TABLE]

where $\operatorname{tr}_{s_{2}}$ denotes the partial trace over the non-positional degrees of freedom of $\mathscr{S}_{2}$ (such as spin), the scalar products are partial scalar products involving only the position degrees of freedom of $\mathscr{S}_{2}$, and $X_{2}$ is the Bohmian configuration of $\mathscr{S}_{2}$. (To illustrate the difference between $\hat{\rho}_{1}$ and $\hat{\rho}_{\mathrm{cond}}$, if $\mathscr{S}_{1}$ is Schrödinger’s cat and $\mathscr{S}_{2}$ its environment, then $\hat{\rho}_{1}$ is a mixture of a live and a dead state, whereas $\hat{\rho}_{\mathrm{cond}}$ is either a live or a dead state, each with the appropriate probabilities.)

Second, it seems that the following analog of (16) is usually fulfilled in practice for both the reduced density matrix $\hat{\rho}=\hat{\rho}_{1}$ and the conditional density matrix $\hat{\rho}=\hat{\rho}_{\mathrm{cond}}$:

[TABLE]

As with (16), if (20) holds, then the Bohmian configuration $X_{1}$ of $\mathscr{S}_{1}$ selects the actual macro state $\nu(X_{1})$ of $\mathscr{S}_{1}$. Moreover, $\nu(X_{1})$ should usually be the same for $\hat{\rho}=\hat{\rho}_{1}$ as for $\hat{\rho}=\hat{\rho}_{\mathrm{cond}}$. Then this macro state determines the entropy,

[TABLE]

In short, the concept of quantum Boltzmann entropy carries over from isolated systems in pure states to open systems.

3.3 Quantum Thermalization

In the 21st century, there has been a wave of works on the thermalization of closed quantum systems, often connected with the key words “eigenstate thermalization hypothesis” (ETH) and “canonical typicality”; see, e.g., (Gemmer et al., 2004; Goldstein et al., 2006; Popescu et al., 2006; Goldstein et al., 2010a; Gogolin and Eisert, 2016; Goldstein et al., 2015; Kaufman et al., 2016) and the references therein. The common theme of these works is that an individual, closed, macroscopic quantum system in a pure state $\psi_{t}$ that evolves unitarily will, under conditions usually satisfied and after a sufficient waiting time, behave very much as one would expect a system in thermal equilibrium to behave; more precisely, on such a system $\mathscr{S}$ with $\psi_{0}$ in an energy shell, relevant observables yield their thermal equilibrium values up to small deviations with probabilities close to 1. For example, this happens simultaneously for all observables referring to a small subsystem $\mathscr{S}_{1}$ of $\mathscr{S}$ that interacts weakly with the remainder $\mathscr{S}_{2}:=\mathscr{S}\setminus\mathscr{S}_{1}$, with the further consequence (“canonical typicality”) that the reduced density matrix of $\mathscr{S}_{1}$,

[TABLE]

is close to a canonical one (Goldstein et al., 2006),

[TABLE]

for suitable $\beta$ and normalizing constant $Z$; here, $\hat{H}_{1}$ is the Hamiltonian of $\mathscr{S}_{1}$. For another example, every initial wave function of $\mathscr{S}$ in an energy shell will, after a sufficient waiting time and for most of the time in the long run, be close to the thermal equilibrium macro space $\mathscr{H}_{\mathrm{eq}}$ (Goldstein et al., 2010a), provided that the Hamiltonian $\hat{H}$ is non-degenerate and (ETH) all eigenvectors of $\hat{H}$ are very close to $\mathscr{H}_{\mathrm{eq}}$.

These works support the idea that the approach to thermal equilibrium need not have anything to do with an observer’s ignorance. In fact, the system $\mathscr{S}$ always remains in a pure state, and thus has von Neumann entropy $S_{\mathrm{vN}}=0$ at all times. This fact illustrates that the kind of thermalization relevant here involve neither an increase in von Neumann entropy nor a stationary density matrix of $\mathscr{S}$. Rather, $\psi_{t}$ reaches, after a sufficient waiting time, the $\varepsilon$-neighborhood of the macro space $\mathscr{H}_{\tilde{\nu}}=\mathscr{H}_{\mathrm{eq}}$ corresponding to thermal equilibrium in the energy shell (Goldstein et al., 2010a). That is the “individualist” or “Boltzmannian” version of approach to thermal equilibrium in quantum mechanics.

In fact, there are two individualist notions of thermal equilibrium in quantum mechanics, which have been called “macroscopic” and “microscopic thermal equilibrium” (Goldstein et al., 2017b). Boltzmann’s approach requires only that macro observables assume their equilibrium values (Goldstein et al., 2010a), whereas a stronger statement is actually true after a long waiting time: that all micro observables assume their equilibrium values (Goldstein et al., 2015). This is true not only for macroscopic systems, but also for small systems (Goldstein et al., 2017a), and has in fact been verified experimentally for a system with as few as 6 degrees of freedom (Kaufman et al., 2016).

We have emphasized earlier in this subsection that thermalization does not require mixed quantum states. We should add that this does not mean that pure quantum states are fully analogous to phase points in classical mechanics. In Bohmian mechanics, for example, the analog of a phase point would be the pair $(\psi,Q)$ comprising the system’s wave function $\psi$ and its configuration $Q$.

4 Subjective Entropy Is Not Enough

By “subjective entropy” we mean Gibbs entropy under option (a): the view that the Gibbs entropy is the thermodynamic entropy, and that the distribution $\rho$ in the Gibbs entropy represents an observer’s subjective perspective and limited knowledge (e.g., Jaynes, 1965; Krylov, 1979; Mackey, 1989; Garibyan and Tegmark, 2014). We would like to point to three major problems with this view.

4.1 Cases of Wrong Values

The first problem is that in some situations, the subjective entropy does not appropriately reproduce the thermodynamic entropy. For example, suppose an isolated room contains a battery-powered heater, and we do not know whether it is on or off. If it is on, then after ten minutes the air will be hot, the battery empty, and the entropy of the room has a high value $S_{3}$. Not so if the heater is off; then the entropy has the low initial value $S_{1}<S_{3}$. In view of our ignorance, we may attribute a subjective probability of 50 percent to each of “on” and “off.” After ten minutes, our subjective distribution $\rho$ over phase space will be spread over two regions with macroscopically different phase points, and its Gibbs entropy $S_{\mathrm{G}}(\rho)$ will have a value $S_{2}$ between $S_{1}$ and $S_{3}$ (in fact, slightly above the average of $S_{1}$ and $S_{3}$).888Generally, if several distribution functions $\rho_{i}$ have mutually disjoint supports, and we choose one of them randomly with probability $p_{i}$, then the resulting distribution $\rho=\sum_{i}p_{i}\rho_{i}$ has Gibbs entropy $S_{\mathrm{G}}(\sum_{i}p_{i}\rho_{i})=-k\int dx(\sum_{i}p_{i}\,\rho_{i}(x))\log(\sum_{i}p_{i}\,\rho_{i}(x))=-k\sum_{i}\int dx\,p_{i}\,\rho_{i}\log(p_{i}\,\rho_{i}(x))=\sum_{i}p_{i}\,S_{\mathrm{G}}(\rho_{i})-k\sum_{i}p_{i}\,\log p_{i}>\sum_{i}p_{i}S_{\mathrm{G}}(\rho_{i})$. In our example, $S_{2}=(S_{1}+S_{3})/2+k\log 2$. But the correct thermodynamic value is not $S_{2}$, it is either $S_{1}$ (if the heater was off) or $S_{3}$ (if the heater was on). So subjective entropy yields the wrong value.

The same problem arises with option (b), which concerns a system prepared by some procedure that, when repeated over and over, will lead to a distribution $\rho$ of the system’s phase point $X$. Suppose that the isolated room also contains a mechanism that tosses a coin or generates a random bit $Y\in\{0,1\}$ in some other mechanical way; after that, the mechanism turns on the heater or does not, depending on $Y$. We would normally say that the entropy $S$ after ten minutes is random, that $S=S_{1}$ with probability 1/2 and $S=S_{3}$ with probability 1/2. But the distribution $\rho$ created by the procedure (and the canonical distribution for each given value of $Y$) is the same as in the previous paragraph, and has Gibbs entropy $S_{\mathrm{G}}(\rho)=S_{2}$, the wrong value.

4.2 Explanatory and Predictive Power

The second major problem with subjective entropy concerns the inadequacy of its explanatory and predictive power. Consider for example the phenomenon that by thermal contact, heat always flows from the hotter to the cooler body, not the other way around. The usual explanation of this phenomenon is that entropy decreases when heat flows to the hotter body, and the second law excludes that. Now that explanation would not get off the ground if entropy meant subjective entropy: In the absence of observers, does heat flow from the cooler to the hotter? In distant stars, does heat flow from the cooler to the hotter? In the days before humans existed, did heat flow from the cooler to the hotter? After the human race becomes extinct, will heat flow from the cooler to the hotter? If not, why would observers be relevant at all to the explanation of the phenomenon?

And as with explanation, so with prediction: Can we predict that heat will flow from the hotter to the cooler also in the absence of observers? If so, why would observers be relevant to the prediction?

So, subjective entropy does not seem relevant to either explaining or predicting heat flow. That leaves us with the question, what is subjective entropy good for? The study of subjective entropy is a subfield of psychology, not of physics. It is all about beliefs.

Some ensemblists may be inclined to say that the explanation of heat flow is that it occurs the same way as if an observer observed it. But the fact remains that observers actually have nothing to do with it.

Once the problem of explanatory power is appreciated, it seems obvious that subjective entropy is inappropriate: How could an objective physical phenomenon such as heat flow from the hotter to the cooler depend on subjective belief? In fact, since different observers may have different, incompatible subjective beliefs, how could coherent consequences such as physical laws be drawn from them? And what if the subjects made mistakes, what if they computed the time-evolved distribution $\rho\circ\Phi_{t}$ incorrectly, what if their beliefs were irrational—would that end the validity of subjective entropy? Somebody may be inclined to say that subjective entropy is valid only if it is rational (e.g., Bricmont, 2019), but that means basically to back off from the thought that entropy is subjective. It means that it does not play much of a role whether anybody’s actual beliefs follow that particular $\rho$, but rather that there is a correct $\rho$ that should be used; we will come back to this view at the end of the next subsection.

Another drawback of the subjective entropy, not unrelated to the problem of explanatory power, is that it draws the attention away from the fact that the universe must have very special initial conditions in order to yield a history with a thermodynamic arrow of time. While the Boltzmann entropy draws attention to the special properties of the initial state of the universe, the subjective entropy hides any such objective properties under talk about knowledge.

4.3 Phase Points Play No Role

The third problem with subjective entropy is that $S_{\mathrm{G}}(\rho)$ has nothing to do with the properties of the phase points $x$ at which $\rho$ is significantly non-zero. $S_{\mathrm{G}}(\rho)$ measures essentially the width of the distribution $\rho$, much like the standard deviation of a probability distribution, except that the standard deviation yields the radius of the set over which $\rho$ is effectively distributed, whereas the Gibbs entropy yields the log of its volume (see (26) below). The problem is reflected in the fact, mentioned around (8), that any volume-preserving transformation $\Phi:\mathscr{X}\to\mathscr{X}$ will leave the Gibbs entropy unchanged. It does not matter to the Gibbs entropy how the $x$’s on which $\rho$ is concentrated behave physically, although this behavior is crucial to thermodynamic entropy and the second law.

Some ensemblists may be inclined to say that the kind of $\rho$ that occurs in practice is not any old density function, but is approximately concentrated on phase points that look macroscopically similar. This idea is essentially option (c) of Section 1.2, which was to take $\rho$ as a kind of coarse graining of the actual phase point $X$. Specifically, if $\Gamma(X)$ denotes again the set of phase points that look macroscopically similar to $X$, then we may want to take $\rho=\rho_{X}$ to be the flat distribution over $\Gamma(X)$,

[TABLE]

With this choice we obtain exact agreement between the Gibbs and Boltzmann entropies,

[TABLE]

Indeed, whenever $\rho$ is the flat distribution over any subset $A$ of phase space $\mathscr{X}$, $\rho(x)=(\operatorname{\mathrm{vol}}A)^{-1}\,1_{A}(x)$, then

[TABLE]

(This fact also illustrates the mathematical meaning of the Gibbs entropy of any distribution $\rho$ as the log of the volume over which $\rho$ is effectively distributed.)

Of course, if we associate an entropy value $S_{\mathrm{G}}(\rho_{X})$ with every $X\in\mathscr{X}$ in this way, then the use of Gibbs’s definition (1) seems like an unnecessary detour. In fact, we have associated with every $X\in\mathscr{X}$ an entropy value $S(X)$, and talk about the knowledge of observers is not crucial to the definition of the function $S$, as is obvious from the fact that the function $S$ is nothing but the Boltzmann entropy, which was introduced without mentioning observers.

This brings us once more to the idea that the $\rho$ in $S_{\mathrm{G}}(\rho)$ is the subjective belief of a rational observer as advocated by Bricmont (2019). One could always use the Boltzmann entropy and add a narrative about observers and their beliefs, such as: Whenever $X\in\Gamma_{\nu}$, a rational observer should use the flat distribution over $\Gamma_{\nu}$, and the Gibbs entropy of that observer’s belief is what entropy really means. One could say such words. But they are also irrelevant, as observers’ knowledge is irrelevant to which way heat flows, and the resulting entropy value agrees with $S_{\mathrm{B}}(X)$ anyway.

4.4 What is Attractive About Subjective Entropy

Let us turn to factors that may seem attractive about the subjective entropy view: First, it seems like the obvious interpretation of the density $\rho$ that comes up in all ensembles. But the Boltzmannian individualist view offers an alternative interpretation, as we explain in Section 5.

Second, it is simple and elegant. That may be true but does not do away with the problems mentioned.

Third, the subjective view mixes well with certain interpretations of quantum mechanics such as Copenhagen and quantum Bayesianism, which claim that quantum mechanics is all about information or that one should not talk about reality. These interpretations are problematical as well, mainly because all information must ultimately be information about things that exist, and it does not help to leave vague and unspecified which things actually exist (Goldstein, 1998).

Fourth, the subjective view may seem to mix well with the work of Shannon (1948), as the Shannon entropy is a discrete version of Gibbs entropy and often regarded as quantifying the information content of a probability distribution. But actually, there is not a strong link, as Shannon regarded the probabilities in his study of optimal coding of data for transmission across a noisy channel as objective and did not claim any connection with thermodynamics. (By the way, it is dangerous to loosely speak of the “amount of information” in the same way as one speaks of, e.g., the amount of sand; after all, the sand grains are equal to each other, and one does not care about whether one gets this or that grain, whereas different pieces of information are not equivalent to each other.)

Fifth and finally, a strong pull towards subjective entropy comes from the belief that “objective entropy” either does not work or is ugly—a wrong belief, as we will explain in Section 5.

4.5 Remarks

Further critiques of subjective entropy can be found in (Callender, 1999; Lebowitz and Maes, 2003; Goldstein et al., 2017a; Goldstein, 2019).

We would like to comment on another quote. Jaynes (1965), a defender of subjective entropy, reported a dictum of Wigner’s:

Entropy is an anthropomorphic concept.

Of course, this phrase can be interpreted in very different ways. Jaynes took it to express that entropy refers to the knowledge of human observers—the subjective view that we have criticized. But we do admit that there is a trait in entropy that depends partly on human nature, and that is linked to a certain (though limited and usually unproblematical) degree of arbitrariness in the definition of “looking macroscopically the same.” This point will come up again in the next section.

5 Boltzmann’s Vision Works

Many authors expressed disbelief that Boltzmann’s understanding of entropy and the second law could possibly work. Von Neumann (1929, Sec. 0.6) wrote:

As in classical mechanics, also here [in the quantum case] there is no way that entropy could always increase, or even have a predominantly positive sign of its [time] derivative (or difference quotient): the time reversal objection as well as the recurrence objection are valid in quantum mechanics as well as in classical mechanics.

Khinchin (1941, §33, p. 139):

[One often] states that because of thermal interaction of material bodies the entropy of the universe is constantly increasing. It is also stated that the entropy of a system “which is left to itself” must always increase; taking into account the probabilistic foundation of thermodynamics, one often ascribes to this statement a statistical rather than absolute character. This formulation is wrong if only becuase the entropy of an isolated system is a thermodynamic function—not a phase-function—which means that it cannot be considered as a random quantity; if $E$ and all [external parameters] $\lambda_{s}$ remain constant the entropy cannot change its value whereas by changing these parameteres in an appropriate way we can make the entropy increase or decrease at will. Some authors (footnote: Comp. Borel, Mécanique statistique classique, Paris 1925.) try to generalize the notion of entropy by considering it as being a phase function which, depending on the phase, can assume different values for the same set of thermodynamical parameters, and try to prove that entropy so defined must increase, with overwhelming probability. However, such a proof has not yet been given, and it is not at all clear how such an artificial generalization of the notion of entropy could be useful to the science of thermodynamics.

Jaynes (1965):

[T]he Boltzmann $H$ theorem does not constitute a demonstration of the second law for dilute gases[.]

Even Boltzmann himself was at times unassured. In a letter to Felix Klein in 1889, he wrote:

Just when I received your dear letter I had another neurasthenic attack, as I often do in Vienna, although I was spared them altogether in Munich. With it came the fear that the whole $H$-curve was nonsense.

But actually, the $H$-curve (i.e., the time evolution of entropy) makes complete sense, Boltzmann’s vision does work, and von Neumann, Khinchin, and Jaynes were all mistaken, so it is perhaps worth elucidating this point. Many other, deeper discussions can be found in the literature, e.g., qualitative, popular accounts in (Penrose, 1989; Lebowitz and Maes, 2003; Carroll, 2010), overviews in (Goldstein, 2001; Lebowitz, 2008; Goldstein, 2019), more technical and detailed discussions in (Boltzmann, 1898; Ehrenfest and Ehrenfest, 1911; Lanford, 1976; Garrido et al., 2004; Falcioni et al., 2007; Cerino et al., 2016; Goldstein et al., 2017a; Lazarovici, 2018). So we now give a summary of Boltzmann’s explanation of the second law.

5.1 Macro States

We start with a partition $\mathscr{X}=\cup_{\nu}\Gamma_{\nu}$ of phase space into macro sets as in Figure 1. A natural way of obtaining such a partition would be to consider several functions $M_{j}:\mathscr{X}\to\mathbb{R}$ ($j=1,\ldots,K$) that we would regard as “macro variables.” Since macro measurements have limited resolution (say, $\Delta M_{j}>0$), we want to think of the $M_{j}$ as suitably coarse-grained with a discrete set of values, say, $\{n\Delta M_{j}:n\in\mathbb{Z}\}$. Then two phase points $x_{1},x_{2}\in\mathscr{X}$ will look macroscopically the same if and only if $M_{j}(x_{1})=M_{j}(x_{2})$ for all $j=1,\ldots,K$, corresponding to

[TABLE]

one for every macro state $\nu=(\nu_{1},\ldots,\nu_{K})$ described by the list of values of all $M_{j}$. We will discuss a concrete example due to Boltzmann in Section 5.4. Since coarse-grained energy should be one of the macro variables, say

[TABLE]

with $H$ the Hamiltonian function and $[x]$ the nearest integer to $x\in\mathbb{R}$, every $\Gamma_{\nu}$ is contained in one micro-canonical energy shell

[TABLE]

Of course, this description still leaves quite some freedom of choice and thus arbitrariness in the partition, as different physicists may make different choices of macro variables and of the way and scale to coarse-grain them; this realization makes an “anthropomorphic” element in $S_{\mathrm{B}}$ explicit. Wallace (2019) complained that this element makes the Boltzmann entropy “subjective” as well, but that complaint does not seem valid: rather, $S_{\mathrm{B}}$ and its increase provide an objective answer to a question that is of interest from the human perspective. Moreover, as mentioned already, this anthorpomorphic element becomes less relevant for larger $N$. It is usually not problematical and not subject to the same problems as the subjective entropy.

Usually in macroscopic systems, there is, for every energy shell $\mathscr{X}_{\mathrm{mc}}$ (or, if there are further macroscopic conserved quantities besides energy, in the set where their values have been fixed as well), one macro set $\Gamma_{\tilde{\nu}}$ that contains most (say, more than $99.99\%$) of the phase space volume of $\mathscr{X}_{\mathrm{mc}}$ (see, e.g., Boltzmann, 1898; Lanford, 1973; Lazarovici, 2018);999There are exceptions, in which none of the macro sets dominates; for example, in the ferromagnetic Ising model with vanishing external magnetic field and not-too-high temperature, there are two macro states (the first having a majority of spins up, the second having a majority of spins down) that together dominate but have equal volume; see also (Lazarovici, 2018). But that does not change much about the discussion. in fact (Goldstein et al., 2017b, Eq. (6)),

[TABLE]

with positive constant $c$. The existence of this dominant macro state means that all macro observables are nearly-constant functions on $\mathscr{X}_{\mathrm{mc}}$, in the sense that the set where they deviate (beyond tolerances) from their dominant values has tiny volume. This macro state $\tilde{\nu}$ is the thermal equilibrium state, $\Gamma_{\tilde{\nu}}=\Gamma_{\mathrm{eq}}$, see Figure 1, and the dominant values of the macro observables are their thermal equilibrium values. That fits with thermal equilibrium having maximal entropy, and it has the consequence that

[TABLE]

where $\rho_{\mathrm{mc}}(x)=(\operatorname{\mathrm{vol}}\mathscr{X}_{\mathrm{mc}})^{-1}\,1_{\mathscr{X}_{\mathrm{mc}}}(x)$ is the micro-canonical distribution, and the (relative or absolute) error in the approximation tends to zero as $N\to\infty$.

Moreover, different macro sets $\Gamma_{\nu}$ have vastly different volumes. In fact, usually the small macro sets of an energy shell taken together are still much smaller than the next bigger one,

[TABLE]

with

[TABLE]

(There are exceptions to this rule of thumb; in particular, symmetries sometimes imply that two or a few macro sets must have approximately the same volume.)

5.2 Entropy Increase

Now increase of Boltzmann entropy means that the phase point $X(t)$ moves to bigger and bigger macro sets $\Gamma_{\nu}$. In this framework, the second law can be stated as follows.

[TABLE]

The described behavior is depicted in Figure 2. Entropy valleys (i.e., periods of entropy decrease and return to the previous level) are also called fluctuations. The physical second law then asserts that the actual phase point of a real-world closed system behaves the way described in (34) for most phase points.

As an illustration of (34) and as a step towards making it plausible, let us consider two times, 0 and $t>0$. Let $A_{t}:=\Phi_{t}(\Gamma_{\nu})$. By Liouville’s theorem, $\operatorname{\mathrm{vol}}(A_{t})=\operatorname{\mathrm{vol}}(\Gamma_{\nu})$, and thus, by (32),

[TABLE]

That is, only a small minority of points in $A_{t}$ will have entropy smaller than $S_{\mathrm{B}}(\nu)$. That is, for most points $X(0)\in\Gamma_{\nu}$,

[TABLE]

Another simple special case is the one in which the macro evolution is deterministic (Garrido et al., 2004; Goldstein and Lebowitz, 2004; De Roeck et al., 2006). For the sake of concreteness, assume that in a time step of a certain size $\tau$, $\Gamma_{\nu_{1}}$ gets mapped into $\Gamma_{\nu_{2}}$, which in turn gets mapped into $\Gamma_{\nu_{3}}$, and so on up to $\nu_{m}$:

[TABLE]

Then, by Liouville’s theorem, $\operatorname{\mathrm{vol}}\Gamma_{\nu_{i}}\leq\operatorname{\mathrm{vol}}\Gamma_{\nu_{i+1}}$, so

[TABLE]

for all $i=1,\ldots,m-1$, so entropy does not decrease. Of course, in realistic cases, the macro evolution becomes deterministic only in the limit $N\to\infty$, and as long as $N$ is finite, there are a minority of points in $\Gamma_{\nu_{i}}$ that do not evolve to $\Gamma_{\nu_{i+1}}$.

Generally, if the Hamiltonian motion is not specially desgined for the given partition of $\mathscr{X}$, then it is quite intuitive that the motion of the phase point should tend to lead to larger macro sets, and not to smaller ones. Numerical simulations exhibiting this behavior are presented in (Falcioni et al., 2007). It is also quite intuitive that the phase point would stay in $\Gamma_{\mathrm{eq}}$ for a very, very long time: If the non-equilibrium set $\Gamma_{\mathrm{noneq}}:=\Gamma^{(E-\Delta E,E]}_{<\mathrm{eq}}=\mathscr{X}_{\mathrm{mc}}\setminus\Gamma_{\mathrm{eq}}$ has only the fraction $10^{-cN}$ of the volume of the energy shell, cf. (30), then only a tiny fraction of $\Gamma_{\mathrm{eq}}$ should be able to evolve into the non-equilibrium set $\Gamma_{\mathrm{noneq}}$ in a short time; and if most points in $\Gamma_{\mathrm{noneq}}$ spend a substantial amount of time there, then it will take very, very long until a substantial fraction of $\Gamma_{\mathrm{eq}}$ has visited $\Gamma_{\mathrm{noneq}}$. The statement that points in $\Gamma_{\mathrm{eq}}$ stay there for a long time fits well with the observed stationarity of thermal equilibrium—which is why it is called “equilibrium.”

Let us briefly address two classic objections to the idea that entropy increases:

• •

Time reversal (Loschmidt’s objection) shows that entropy increase cannot hold for all phase points in $\Gamma_{\nu}$. Concretely, for relevant Hamiltonians the time reversal mapping $R:\mathscr{X}\to\mathscr{X}$, defined by

[TABLE]

with $\boldsymbol{q}_{i}$ the position and $\boldsymbol{v}_{i}$ the velocity of particle $i$, has the property

[TABLE]

Usually, $R$ maps $\Gamma_{\nu}$ onto some $\Gamma_{\nu^{\prime}}$ (where $\nu^{\prime}$ may or may not equal $\nu$), so $S_{\mathrm{B}}(R(x))=S_{\mathrm{B}}(x)$. So if some $X(0)\in\Gamma_{\nu_{1}}$ evolves to $X(t)\in\Gamma_{\nu^{\prime}}$ with $S_{\mathrm{B}}(\nu^{\prime})>S_{\mathrm{B}}(\nu_{1})$, then $R(X(t))\in\Gamma_{\nu}$ evolves to $R(X(0))$, and its entropy decreases.

• •

Recurrence (Zermelo’s objection) shows that $S_{\mathrm{B}}(X(t))$ cannot forever be non-decreasing; thus, $X(t)$ cannot stay forever in $\Gamma_{\mathrm{eq}}$ once it reaches $\Gamma_{\mathrm{eq}}$. (The Poincaré recurrence theorem states that under conditions usually satisfied in $\mathscr{X}_{\mathrm{mc}}$, every trajectory $X(t)$, except for a set of measure zero of $X(0)$s, returns arbitrarily close to $X(0)$ at some arbitrarily late time.)

Contrary to von Neumann’s statement quoted in the beginning of Section 5, the second law as formulated in (34) is not refuted by either objection: after all, the second law applies to most, not all, phase points $X(0)$, and it does not claim that $X(t)$ will stay in thermal equilibrium forever, but only for a very, very long time.

5.3 Non-Equilibrium

The term “non-equilibrium” is sometimes understood (Gallavotti, 2003) as referring to so called non-equilibrium steady states (NESS), which concerns, for example, a system $\mathscr{S}$ coupled to two infinite reservoirs of different temperature; so $\mathscr{S}$ is an open system heated on one side and cooled on another, and it will tend to assume a macroscopically stationary (“steady”) state with a temperature gradient, a nonzero heat current, and a positive rate of entropy production (Onsager, 1931; Bergmann and Lebowitz, 1955; Derrida, 2007; Goldstein et al., 2017a). In contrast, in this Section 5 we are considering a closed system (i.e., not interacting with the outside), and “non-equilibrium” refers to any phase point in $\mathscr{X}_{\mathrm{mc}}\setminus\Gamma_{\mathrm{eq}}$. Examples of non-equilibrium macro states include, but are not limited to, states in local thermal equilibrium but not in (global) thermal equilibrium (such as systems hotter in one place than in another). Other examples arise from removing a constraint or wall; such macro states may have been in thermal equilibrium before the constraint was removed but are not longer so afterwards; for example, think of a macro state in which all particles are in the left half of a box; for another example, suppose we could turn on or off the interaction between two kinds of particles (say, “red ones” and “blue ones”), and think of a macro state that is a thermal equilibrium state when the interaction is off (so that the red energy and the blue energy are separately conserved) but not when it is on, such as when both gases are in the same volume but at different temperatures.

5.4 Concrete Example: The Boltzmann Equation

Here is a concrete example of a partition of phase space due to Boltzmann. Divide the 1-particle phase space $\mathscr{X}_{1}$ into cells $C_{i}$ (say, small cubes of equal volume) and count (with a given tolerance) the particles in each cell. The macro state is described by the list $\nu=(\nu_{1},\ldots,\nu_{L})$ of all these occupation numbers; for convenience, we will normalize them:

[TABLE]

with $N\Delta f$ the tolerance in counting and $[\cdot]$ again the nearest integer. This example of a partition $\Gamma_{\nu}=\{x\in\mathscr{X}:\nu(x)=\nu\}$ is good for dilute, weakly interacting gases but not in general (Garrido et al., 2004; Goldstein and Lebowitz, 2004) (see also Section 5.8).

Boltzmann considered $N$ billiard balls of radius $a$ in a container $\Lambda\subset\mathbb{R}^{3}$, so $\mathscr{X}_{1}=\Lambda\times\mathbb{R}^{3}$. In a suitable limit in which $N\to\infty$, $a\to 0$, and the cells $C_{i}$ become small, the normalized occupation numbers $f_{i}$ become a continuous density $f(\boldsymbol{q},\boldsymbol{v})$. He argued (convincingly) that for most $x\in\Gamma_{\nu}$ this density, essentially the empirical distribution of the $N$ particles in $\mathscr{X}_{1}$, will change in time according to the Boltzmann equation, an integro-differential equation (Boltzmann, 1872, 1898; Ehrenfest and Ehrenfest, 1911; Lanford, 1976). It reads, in the version appropriate for the hard sphere gas without external forces,

[TABLE]

with the “collision term”

[TABLE]

involving a constant $\lambda>0$ and the abbreviations

[TABLE]

for the outgoing velocities of a collision between two balls with incoming velocities $\boldsymbol{v}$ and $\boldsymbol{v}_{*}$ and direction $\boldsymbol{\omega}$ between the centers of the two balls. The Boltzmann equation is considered for $\boldsymbol{v}\in\mathbb{R}^{3}$ and $\boldsymbol{q}\in\Lambda$ along with a boundary condition representing that balls hitting the boundary of $\Lambda$ get reflected there. A function $f(\boldsymbol{q},\boldsymbol{v})$ is a stationary solution if and only if it is independent of $\boldsymbol{q}$ and a Maxwellian (i.e., Gaussian) in $\boldsymbol{v}$—that is, if and only if it represents thermal equilibrium. Correspondingly, non-equilibrium macro states correspond to any density function $f$ that is not a global (i.e., $\boldsymbol{q}$-independent) Maxwellian.

The entropy turns out to be (up to addition of the constant $kN$ and terms of lower order)

[TABLE]

In the limit of small $C_{i}$, this becomes (9), i.e., $-kH$ in terms of the $H$ functional. Boltzmann further proved the $H$-theorem, which asserts that for any solution $f_{t}$ of the Boltzmann equation,

[TABLE]

with equality only if $f$ is a local Maxwellian. The $H$-theorem amounts to a derivation of the second law relative to the partition $\{\Gamma_{\nu}\}$ under consideration.

5.5 Rigorous Result

Some authors suspected that Boltzmann’s vision, and the Boltzmann equation in particular, was not valid. For example, Khinchin (1941, §33, p. 142) complained about individualist accounts of entropy:

All existing attempts to give a general proof of this postulate [i.e., $S=k\log W$,] must be considered as an aggregate of logical and mathematical errors superimposed on a general confusion in the definition of the basic quantities.

But actually, the Boltzmann equation (and with it the increase of entropy) is rigorously valid for most phase points in $\Gamma_{\nu}$, at least for a short time, as proved by Lanford (1975; 1976). Here is a summary statement of Lanford’s theorem, leaving out some technical details:

Theorem 1**.**

Let $\overline{t}>0$ and $\lambda>0$ (the constant in the Boltzmann equation) be constants. For a very large number $N$ of billiard balls of (very small) radius $a$ with $4Na^{2}=\lambda$, for every $0\leq t<\tfrac{1}{5}\overline{t}$, for any nice density $f_{0}$ in $\mathscr{X}_{1}=\Lambda\times\mathbb{R}^{3}$ with mean free time $\geq\overline{t}$, and for a coarse graining of $\mathscr{X}_{1}$ into cells $C_{i}$ that are small but not too small, most phase points $X$ with empirical distribution $f_{0}$ (relative to $\{C_{i}\}$ and within small tolerances) evolve in such a way that the empirical distribution of $X(t)$ (relative to $\{C_{i}\}$) is close to $f_{t}$, where $t^{\prime}\mapsto f_{t^{\prime}}$ is the solution of the Boltzmann equation with initial datum $f_{0}$.

It is believed but not proven that the Boltzmann equation is valid for a much longer duration, maybe of the order of recurrence times. The method of proof fails after $\tfrac{1}{5}\overline{t}$, but it does not give reason to think that the actual behavior changes at $\tfrac{1}{5}\overline{t}$.

Where is the famous Stosszahlansatz, or hypothesis of molecular chaos, in this discussion? This hypothesis was stated by Boltzmann as specifying the approximate number of collisions with parameter $\boldsymbol{\omega}$ between particles from cells $C_{i}$ and $C_{j}$ within a small time interval. In our discussion it is hidden in the assumption that the initial phase point $X(0)$ be typical in $\Gamma_{\nu}$: Both Theorem 1 and the wording (34) of the second law talk merely about most phase points in $\Gamma_{\nu}$, and for most phase points in $\Gamma_{\nu}$ ($\nu=f$) it is presumably true that the number of upcoming collisions is, within reasonable tolerances, given by the hypothesis of molecular chaos, not just at the initial time but at all relevant times. We will discuss molecular chaos further in Section 5.7.

5.6 Empirical vs. Marginal Distribution

Many mathematicians (e.g., Kac (1956); Cercignani et al. (1994) but also Tolman (1938)) considered the Boltzmann equation in a somewhat different context with a different meaning, viz., with $f$ not the empirical distribution but the marginal distribution. This means the following.

• •

The empirical distribution, for a given phase point $X=(\boldsymbol{q}_{1},\boldsymbol{v}_{1},\ldots,\boldsymbol{q}_{N},\boldsymbol{v}_{N})\in\mathscr{X}$, is the distribution on $\mathscr{X}_{1}$ with density

[TABLE]

As such, it is not a continuous distribution but becomes roughly continuous-looking only after coarse graining with cells $C_{i}$ in $\mathscr{X}_{1}$ that are not too small (so that the occupation numbers are large enough), and it becomes a really continous distribution only after taking a limit in which the cells shrink to size 0 while $N\to\infty$ fast enough for the occupation numbers to become very large.

• •

The marginal distribution starts from a distribution $\rho$ on phase space $\mathscr{X}$ and is obtained by integrating over the positions and velocities of $N-1$ particles (and perhaps averaging over the number $i$ of the particle not integrated out, if $\rho$ was not permutation invariant to begin with). The marginal distribution can also be thought of as the average of the (exact) empirical distribution: the empirical distribution $f_{\mathrm{emp}}^{X}$ associated with $X\in\mathscr{X}$ becomes a continuous function when $X$ is averaged over using a continuous $\rho$.

For example, Kac (1956, first page) wrote:

$f(r,v)dr\,dv$ is the average number of molecules in $dr\,dv$,

whereas Boltzmann (1898, §3 p. 36) wrote:

let $f(\xi,\eta,\zeta,t)d\xi\,d\eta\,d\zeta$ […] be the number of $m$-molecules whose velocity components in the three coordinate directions lie between the limits $\xi$ and $\xi+d\xi$, $\eta$ and $\eta+d\eta$, $\zeta$ and $\zeta+d\zeta$[.]

Note that Kac wrote “average number” and Boltzmann wrote “number”: For Kac, $f$ was the marginal and for Boltzmann the empirical distribution.101010That is also why Boltzmann normalized $f$ so that $\int_{\mathscr{X}_{1}}f=N$, not $\int_{\mathscr{X}_{1}}f=1$: a marginal of a probability distribution would automatically be normalized to 1, not $N$, but if $f$ means the empirical density then it is natural to take it to mean the density of particles in $\mathscr{X}_{1}$, which is normalized to $N$, not 1.

Of course, the (coarse-grained) empirical distribution is a function of the phase point $X$, and so is any functional of it, such as $H$; thus, the empirical distribution can serve the role of the macro state $\nu$, and $H$ that of Boltzmann entropy. This is not possible for the marginal distribution.

So why would anybody want the marginal distribution? Kac aimed at a rigorous derivation of the Boltzmann equation in whatever context long before Lanford’s theorem, and saw better chances for a rigorous proof if he assumed collisions to occur at random times at exactly the rate given by Boltzmann’s hypothesis of molecular chaos. This setup replaces the time evolution in phase space (or rather, since Kac dropped the positions, in $3N$-dimensional velocity space) by a stochastic process, in fact Markov jump process. (By the way, as a consequence, any density $\rho$ on $3N$-space tends to get wider over time, and its Gibbs entropy increases, contrary to the Hamiltonian evolution.) So the mathematician’s aim of finding statements that are easier to prove leads in a different direction than the aim of discussing the mechanism of entropy increase in nature.

Another thought that may lead authors to the marginal distribution is that $f(\boldsymbol{q},\boldsymbol{v})$ $d^{3}\boldsymbol{q}$ $d^{3}\boldsymbol{v}$ certainly cannot be an integer but must be an infinitesimal, so it cannot be the number of particles in $d^{3}\boldsymbol{q}\,d^{3}\boldsymbol{v}$ but must be the average number of particles. Of course, this thought neglects the idea that as long as $N$ is finite, also the cells $C_{i}$ should be kept of finite size and not too small, and the correct statement is that $f_{i}\,\operatorname{\mathrm{vol}}C_{i}$ is the number of particles in $C_{i}$ (or, depending on the normalization of $f$, $N^{-1}$ times the number of particles); when followers of Boltzmann express the volume of $C_{i}$ as $d^{3}\boldsymbol{q}\,d^{3}\boldsymbol{v}$, they merely express themselves loosely.

5.7 The Past Hypothesis

Lanford’s theorem has implications also for negative $t$: For most phase points in $\Gamma_{f}$, the Boltzmann equation also applies in the other time direction, so that entropy increases in both time directions! (See Figure 3.) That is, before time 0 the Boltzmann equation applies with the sign of $t$ reversed.

It is generally the case, not just for a dilute gas of hard spheres, that entropy increases in both time directions for most phase points in $\Gamma_{\nu}$. This fact leads to the following worry: If Lanford’s theorem, or the statement (34), persuaded us to expect that entropy increases after the time we chose to call $t=0$, should it not persuade us just as much to expect that entropy decreases before $t=0$? But this decrease does not happen. Does that mean we were unjustified in expecting from Lanford’s theorem or (34) that entropy increases after $t=0$?

Here is a variant of this worry in terms of explanation. Statement (34) may suggest that the explanation for the increase of entropy after $t=0$ is that this happens for typical phase points $X(0)$. But if we know that entropy was lower before $t=0$ than at $t=0$, then $X(0)$ was not typical. This undercuts the explanation considered for the behavior after $t=0$.

Here is the resolution of this worry. The assumption really made by Boltzmann’s followers is not that the phase point $X(0)$ at the beginning of an experiment is typical in $\Gamma_{\nu}$ but the following:

[TABLE]

Here, “typical” means that in relevant ways it behaves like most points in $\Gamma_{\nu_{0}}$. The ways relevant here are features of the macro history of the universe shared by most points in $\Gamma_{\nu_{0}}$.

Given that entropy keeps increasing, the initial macro state must be one of extremely low entropy; one estimate (Penrose, 1989) yields $10^{123}$ Joule/Kelvin less than the thermal equilibrium entropy at the same energy; thus, $\Gamma_{\nu_{0}}$ must have extremely small volume compared to the relevant energy shell. All the same, we do not know very clearly what $\nu_{0}$ actually is; one proposal is known as the Weyl curvature hypothesis (Penrose, 1979).

A related worry at this point may arise from the observation (for any macroscopic system or specifically a hard sphere gas as considered for the Boltzmann equation) that if the time evolution $\Phi_{t}$ of $\Gamma_{\nu}$ lies (say) in a macro set $\Gamma_{\nu^{\prime}}$ of much greater volume, then phase points $X(t)$ coming from $X(0)\in\Gamma_{\nu}$ would be atypical in $\Gamma_{\nu^{\prime}}$. So if the prediction of entropy increase after $t$ was based on the assumption that $X(t)$ be typical in $\Gamma_{\nu^{\prime}}$, then it could not be applied to $X(0)\in\Gamma_{\nu}$. So why should entropy still increase at $t>0$? Because Lanford’s theorem says so—at least until $\overline{t}/5$. But even after that time, it is plausible that the Boltzmann equation continues to be valid (and therefore entropy continues to increase) because it is plausible that the number of upcoming collisions of each type agrees with the value specified by the hypothesis of molecular chaos (see, e.g., Lebowitz, 2008). That is because it is plausible that, for typical $X(0)\in\Gamma_{\nu}$, $X(t)$ contains very special correlations in the exact positions and velocities of all particles concerning the collisions before $t$, but not concerning those after $t$. Likewise, we would expect for a general macroscopic system, unless the dynamics (as given by the Hamiltonian) is specially contrived for the partition $\{\Gamma_{\nu}\}$, that $X(t)\in\Gamma_{\nu^{\prime}}$ coming from a typical $X(0)\in\Gamma_{\nu}$ behaves towards the future, but of course not towards the past, like a typical point in $\Gamma_{\nu^{\prime}}$, as stated by the second law (34). The whole reasoning does not change significantly if $\Phi_{t}(\Gamma_{\nu})$ is distributed over several $\Gamma_{\nu_{1}^{\prime}},\ldots,\Gamma_{\nu_{\ell}^{\prime}}$ of much greater volume, instead of being contained in one $\Gamma_{\nu^{\prime}}$.

Putting this consideration together with the past hypothesis, we are led to expect that the Boltzmann entropy of the universe keeps increasing (except for occasional insignificant entropy valleys) to this day, and further until the universe reaches thermal equilibrium. As a consequence for our present-day experiments:

[TABLE]

This statement follows from Lanford’s theorem for times up to $\overline{t}/5$, but otherwise has not been proven mathematically; it summarizes the presumed implications of the past hypothesis (i.e., of the low entropy initial state of the universe) to applications of statistical mechanics. For a dilute gas, it predicts that its macro state will evolve according to the Boltzmann equation in the future, but not the past, of $t_{0}$, as long as it is isolated. It also predicts that heat will not flow from the cooler body to the hotter, and that a given macroscopic object will not spontaneously fly into the air although the laws of mechanics would allow that all the momenta of the thermal motion at some time all point upwards.

By the way, the development conjecture allows us to make sense of Tolman’s (1938, §23, p. 60) “hypothesis of equal a priori probabilities,” which asserts

that the phase point for a given system is just as likely to be in one region of the phase space as in any other region of the same extent which corresponds equally well with what knowledge we do have as to the condition of the system.

That sounds like $X$ is always uniformly distributed over the $\Gamma_{\nu}$ containing $X$, but that statement is blatantly inconsistent, as it cannot be true at two times, given that $\Phi_{t}(\Gamma_{\nu})\neq\Gamma_{\nu^{\prime}}$ for any $\nu^{\prime}$. But the subtly different statement (50) is consistent, and (50) is what Tolman should have written.

The past hypothesis brings out clearly that the origin of the thermodynamic arrow of time lies in a special property (of lying in $\Gamma_{\nu_{0}}$) of the physical state of the matter at the time $T_{0}$ of the big bang, and not in the observers’ knowledge or their way of considering the world. The past hypothesis is the one crucial assumption we make in addition to the dynamical laws of classical mechanics. The past hypothesis may well have the status of a law of physics—not a dynamical law but a law selecting a set of admissible histories among the solutions of the dynamical laws. As Feynman (1965, p. 116) wrote:

Therefore I think it is necessary to add to the physical laws the hypothesis that in the past the universe was more ordered, in the technical sense, than it is today—I think this is the additional statement that is needed to make sense, and to make an understanding of the irreversibility.

Making the past hypothesis explicit in the form (49) or a similar one also enables us to understand the question whether the past hypothesis could be explained. Boltzmann (1898, §90) suggested tentatively that the explanation might be a giant fluctuation out of thermal equilibrium, an explanation later refuted by Eddington (1931) and Feynman (1965; 1995). (Another criticism of this explanation put forward by Popper (1976, §35) is without merit.) Some explanations of the past hypothesis have actually been proposed in highly idealized models (Carroll, 2010; Barbour et al., 2013, 2014, 2015; Goldstein et al., 2016); it remains to be seen whether they extend to more adequate cosmological models.

5.8 Boltzmann Entropy Is Not Always $H$

Jaynes (1965) wrote that

the Boltzmann $H$ yields an “entropy” that is in error by a nonnegligible amount whenever interparticle forces affect thermodynamic properties.

It is correct that the $H$ functional represents (up to a factor $-k$) the Boltzmann entropy (2) only for a gas of non-interacting or weakly interacting molecules. Let us briefly explain why; a more extensive discussion is given in (Goldstein and Lebowitz, 2004).

As pointed out in Section 5.1, the coarse grained energy (28) should be one of the macro variables, so that the partition into macro sets $\Gamma_{\nu}$ provides a partition of the energy shell. If interaction cannot be ignored, then the $H$ functional does not correspond to the Boltzmann entropy, since restriction to the energy shell is not taken into account by $H$. When interaction can be ignored there is only kinetic energy, so the Boltzmann macro states based on the empirical distribution alone determine the energy and hence the $H$ functional corresponds to the Boltzmann entropy.

6 Gibbs Entropy as Fuzzy Boltzmann Entropy

This section is about another connection between Gibbs and Boltzmann entropy that is not usually made explicit in the literature; it involves interpreting the Gibbs entropy in an individualist spirit as a variant of the Boltzmann entropy for “fuzzy” macro sets. That is, we now describe a kind of Gibbs entropy that is not an ensemblist entropy (as the Gibbs entropy usually is) but an individualist entropy, which is better. This possibility is not captured by any of the options (a)–(c) of Section 1.2.

By a fuzzy macro set we mean using functions $\gamma_{\nu}(x)\geq 0$ instead of sets $\Gamma_{\nu}$ as expressions of a macro state $\nu$: some phase points $x$ look a lot like $\nu$, others less so, and $\gamma_{\nu}(x)$ quantifies how much. The point here is to get rid of the sharp boundaries between the sets $\Gamma_{\nu}$ shown in Figure 1, as the boundaries are artificial and somewhat arbitrary anyway. A partition into sets $\Gamma_{\nu}$ is still contained in the new framework as a special case by taking $\gamma_{\nu}$ to be the indicator function of $\Gamma_{\nu}$, $\gamma_{\nu}=1_{\Gamma_{\nu}}$, but we now also allow continuous functions $\gamma_{\nu}$. One advantage may be to obtain simpler expressions for the characterization of $\nu$ since we avoid drawing boundaries. The condition that the $\Gamma_{\nu}$ form a partition of $\mathscr{X}$ can be replaced by the condition that

[TABLE]

Another advantage of this framework is that we can allow without further ado that $\nu$ is a continuous variable, by replacing (51) with its continuum version

[TABLE]

It will sometimes be desirable to normalize the function $\gamma_{\nu}$ so its integral becomes 1; we write $\gamma^{1}_{\nu}$ for the normalized function,

[TABLE]

So what would be the appropriate generalization of the Boltzmann entropy to a fuzzy macro state? It should be $k$ times the log of the volume over which $\gamma_{\nu}^{1}$ is effectively distributed—in other words, the Gibbs entropy,

[TABLE]

Now fix a phase point $x$. Since $x$ is now not uniquely associated with a macro state, it is not clear what $S_{\mathrm{B}}(x)$ should be. In view of (52), one might define $S_{\mathrm{B}}(x)$ to be the average $\int d\nu\,S_{\mathrm{B}}(\nu)\,\gamma_{\nu}(x)$. Be that as it may, if the choice of macro states $\gamma_{\nu}$ is reasonable, one would expect that different $\nu$s for which $\gamma_{\nu}(x)$ is significantly non-zero have similar values of $S_{\mathrm{B}}(\nu)$, except perhaps for a small set of exceptional $x$s.

Another advantage of fuzzy macro states is that they sometimes factorize in a more convenient way. Here is an example. To begin with, sometimes, when we are only interested in thermal equilibrium macro states, we may want to drop non-equilibrium macro states and replace the equilibrium set $\Gamma_{\mathrm{eq}}$ in an energy shell $\mathscr{X}_{(E-\Delta E,E]}$ by the full energy shell, thereby accepting that we attribute wildly inappropriate $\nu$s (and $S_{\mathrm{B}}$s) to a few $x$s. As a second step, the canonical distribution

[TABLE]

is strongly concentrated in a very narrow range of energies, a fact well known as “equivalence of ensembles” (between the micro-canonical and the canonical ensemble). Let us take $\nu=\beta$ and $\gamma^{1}_{\nu}(x)=\rho_{\mathrm{can}}(\beta,x)$ as a continuous family of fuzzy macro states. As a third step, consider a system consisting of two non-interacting subsystems, $\mathscr{S}=\mathscr{S}_{1}\cup\mathscr{S}_{2}$, so $\mathscr{X}=\mathscr{X}_{1}\times\mathscr{X}_{2}$, $x=(x_{1},x_{2})$ and $H(x)=H_{1}(x_{1})+H_{2}(x_{2})$. Then $\gamma^{1}_{\beta}$ factorizes,

[TABLE]

whereas the energy shells do not,

[TABLE]

because a prescribed total energy $E$ can be obtained as a sum $E_{1}+E_{2}$ for very different values of $E_{1}$ through suitable choice of $E_{2}$, corresponding to different macro states for $\mathscr{S}_{1}$ and $\mathscr{S}_{2}$. One particular splitting $E=E_{1}+E_{2}$ will have the overwhelming majority of phase space volume in $\mathscr{X}_{(E-\Delta E,E]}$; it is the splitting that maximizes $S_{1}(E_{1})+S_{2}(E_{2})$ under the constraint $E_{1}+E_{2}=E$, and at the same time the one corresponding to the same $\beta$ value, i.e., with $E_{i}$ the expectation of $H_{i}(x_{i})$ under the distribution $\gamma^{1}_{i,\beta}$ ($i=1,2$). So equality in (57) fails only by a small amount, in the sense that the symmetric difference set between the left and right-hand sides has small volume compared to the sets themselves. In short, in this situation, the use of sets $\Gamma_{\nu}$ forces us to consider certain non-equilibrium macro states, and if we prefer to introduce only equilibrium mecro states, then the use of fuzzy macro states $\gamma_{\nu}$ is convenient.

The quantum analog of fuzzy macro states consists of, instead of the orthogonal decomposition (11) of Hilbert space, a POVM (positive-operator-valued measure) $\hat{G}$ on the set $M$ of macro states. That is, we replace the projection $\hat{P}_{\nu}$ to the macro space $\mathscr{H}_{\nu}$ by a positive operator $\hat{G}_{\nu}$ with spectrum in $[0,1]$ such that $\sum_{\nu}\hat{G}_{\nu}=\hat{I}$, where $\hat{I}$ is the identity operator on $\mathscr{H}$. The eigenvalues of $\hat{G}_{\nu}$ would then express how much the corresponding eigendirection looks like $\nu$. Set $\hat{G}^{1}_{\nu}:=(\operatorname{tr}\hat{G}_{\nu})^{-1}\hat{G}_{\nu}$; then $\hat{G}^{1}_{\nu}$ is a density matrix, and the corresponding entropy value would be

[TABLE]

As before when using the $\mathscr{H}_{\nu}$, a quantum state $\psi$ may be associated with several very different $\nu$s. As discussed around (16), this problem presumably disappears in Bohmian mechanics and collapse theories.

7 The Status of Ensembles

If we use the Boltzmann entropy, then the question of what the $\rho$ in the Gibbs entropy means does not come up. But a closely related question remains: What is the meaning of the Gibbs ensembles (the micro-canonical, the canonical, the grand-canonical) in Boltzmann’s “individualist” approach? This is the topic of the present section (see also Goldstein, 2019).

By way of introduction to this section, we can mention that Wallace, an ensemblist, feels the force of arguments against subjective entropy but thinks that there is no alternative. He wrote (Wallace, 2019, Sec. 10):

It will be objected by (e.g.) Albert and Callender that when we say “my coffee will almost certainly cool to room temperature if I leave it” we are saying something objective about the world, not something about my beliefs. I agree, as it happens; that just tells us that the probabilities of statistical mechanics cannot be interpreted epistemically. And then, of course, it is a mystery how they can be interpreted, given that the underlying dynamics is deterministic […]. But (on pain of rejecting a huge amount of solid empirical science […]) some such interpretation must be available.

The considerations of this section may be helpful here.

It is natural to call any measure (or density function $\rho$) that is normalized to 1 a “probability distribution.” But now we need to distinguish more carefully between different roles that such a measure may play:

• (i)

frequency in repeated preparation: The outcome of an experiment is unpredictable, but will follow the distribution $\rho$ if we repeat the experiment over and over. (This kind of probability could also be called genuine probability or probability in the narrow sense.)

• (ii)

degree of belonging: As in Section 6, $\rho$ could represent a fuzzy set; then $\rho(x)$ indicates how strongly $x$ belongs to this fuzzy set.

• (iiii)

typicality: This is how the flat distribution over a certain $\Gamma_{\nu}$ enters the past hypothesis (49) and the development conjecture (50).

7.1 Typicality

A bit more elucidation of the concept of typicality may be useful here, as the difference between typicality and genuine probability is subtle. A feature or behavior is said to be typical in a set $S$ if it occurs for most (i.e., for the overwhelming majority of) elements of $S$. Let us elaborate on this by means of an example. The digits of $\pi$, 314159265358979…, look very much like a random sequence although there is nothing random about the number $\pi$, as it is uniquely defined and thus fully determined. Also, there is no way of “repeating the experiment” that would yield a different sequence of digits; we can only do statistics about the digits in this one sequence. We can set up reasonable criteria for whether a given sequence (finite or infinite) “looks random,” such as whether the relative frequency of each digits is, within suitable tolerances, 1/10, and that of each $k$-digit subsequence $1/10^{k}$ (for all $k$ much smaller than the length of the sequence). Then the digits of $\pi$ will presumably pass the criteria, thereby illustrating the fine but relevant distinction between “looking random” and “being random.”

Looking random is an instance of typical behavior, as most sequences of digits of given length look random and most numbers between 3 and 4 have random-looking decimal expansions—“most” relative to the uniform distribution, also known as Lebesgue measure. So with respect to a certain behavior (say, the frequencies of $k$-digit subsequences), $\pi$ presumably is like most numbers.

Similarly, with respect to another sort of behavior (say, the future macroscopic history of the universe), the initial phase point of the universe is presumably like most points in $\Gamma_{\nu_{0}}$, as demanded by the past hypothesis (49); and according to the development conjecture (50), the phase point of a closed system now is like most points in its macro set.

In contrast, when considering a random experiment in probability theory, we usually imagine that we can repeat the experiment, with relative frequencies in agreement with the distribution $\rho$; that is clearly impossible for the universe as a whole (or, if it were possible, irrelevant, because we are not concerned with what happens in other universes). But even if an experiment, such as the preparation of a macroscopic body (say, a gas), can be repeated a hundred or a thousand times, the frequency distribution of the sample points (i.e., the empirical distribution (48)) consists of a mere $10^{3}$ delta peaks and thus is far from resembling a continuous distribution in a phase space of $10^{23}$ dimensions, so we do not have much basis for claiming that it is essentially the same as the micro-canonical or the canonical or any other common ensemble $\rho$. On the other hand, even a single sample point could meaningfully be said to be typical with respect to a certain high-dimensional distribution $\rho$ (and a certain concept of which kinds of “behavior” are being considered). What we are getting at is that Gibbs’s ensembles are best understood as measures of typicality, not of genuine probability.

For example, there is a subtle difference between the Maxwell-Boltzmann distribution and the canonical distribution. The former is the empirical distribution of (say) $10^{23}$ points in 6-dimensional 1-particle phase space $\mathscr{X}_{1}$, while the latter is a distribution of a single or a few points in high-dimensional phase space. The former is genuine probability, the latter typicality. This contrast is also related to the contrast between empirical and marginal distribution: On the one hand, the canonical distribution of a system $\mathscr{S}_{1}$ is often derived as a marginal distribution of the micro-canonical distribution of an even bigger system $\mathscr{S}_{1}\cup\mathscr{S}_{2}$, and a marginal of a typicality distribution is still a typicality distribution. On the other hand, the Maxwell-Boltzmann distribution is the empirical distribution that arises from most phase points relative to the micro-canonical distribution. If $\rho$-most phase points lead to the empirical distribution $f$, then that explains why we can regard $f$ as a genuine probability distribution, in the sense of being objective.

7.2 Equivalence of Ensembles and Typicality

It is not surprising that typicality and genuine probability can easily be conflated, and that various conundrums can arise from that. For example, the “mentaculus” of Albert and Loewer (Albert, 2015; Loewer, 2019) is a view based on understanding the past hypothesis (49) with genuine probability instead of typicality. That is, in this view one considers the uniform distribution over the initial macro set $\Gamma_{\nu_{0}}$ and regards it as defining “the” probability distribution of the micro history $t\mapsto X(t)$ of the universe, in particular as assigning a precisely defined value in $[0,1]$ as the probability to every conceivable event, for example the event that intelligent life forms evolve on Earth and travel to the moon.

In contrast, the typicality approach used in (49) does not require or claim that the initial phase point $X(T_{0})$ of the universe is random, merely that it looks random (just as $\pi$ merely looks random). Furthermore, the typicality view does not require a uniquely selected measure of typicality, just as $\pi$ is presumably typical relative to several measures: the uniform distribution on the interval $[3,4]$ or on $[0,10]$, or some Gaussian distribution. We elaborate on this aspect in the subsequent paragraphs, again through contrast with Albert and Loewer.

Loewer once expressed (personal communication 2018) that different measures $\rho_{1}\neq\rho_{2}$ for the initial phase point of the universe would necessarily lead to different observable consequences. That is actually not right; let us explain. To begin with, one may think that $\rho_{1}$ and $\rho_{2}$ must lead to different observable consequences if one defines (following the mentaculus) the probability of any event $E$ at time $t$ as $P(E)=\int_{\Phi_{t}^{-1}(E)}\rho_{i}$. However, a definition alone does not ensure that the quantity $P(E)$ has empirical significance; the question arises whether observers inside the universe can determine $P(E)$ from their observations. More basically, we need to ask whether observers can determine from their observations which of $\rho_{1}$ and $\rho_{2}$ was used as the initial distribution. To this end, let us imagine that the initial phase point $X(T_{0})$ of the universe gets chosen truly randomly with distribution either $\rho_{1}$ or $\rho_{2}$. Now the same point $X(T_{0})$ may be compatible with both distributions $\rho_{1},\rho_{2}$ if they overlap appropriately. Since all observations must be made in the one history arising from $X(T_{0})$, and since the universe cannot be re-run with an independent initial phase point, it is impossible to decide empirically whether $\rho_{1}$ or $\rho_{2}$ was the distribution according to which $X(T_{0})$ was chosen. Further thought in this direction shows that observers can only reliably distinguish between $\rho_{1}$ and $\rho_{2}$ if they can observe an event that has probability near 1 for $\rho_{1}$ and near 0 for $\rho_{2}$ or vice versa; that is, they can only determine the typicality class of the initial distribution. (As a consequence for the mentaculus, its main work, which is to efficiently convey hard core facts about the world corresponding to actual patterns of events over space and time, is done via typicality, i.e., concerns only events with measure near 0 or near 1.)

In the typicality view there does not have to be a fact about which distribution $\rho$ is “the right” distribution of the initial phase point of the universe; that is because points typical relative to $\rho_{1}$ are also typical relative to $\rho_{2}$ and vice versa, if $\rho_{1}$ and $\rho_{2}$ are not too different. Put differently, in the typicality view there is a type of equivalence of ensembles, parallel to the well known fact that one can often replace a micro-canonical ensemble by a canonical one without observable consequences. There is room for different choices of $\rho$, and this fits well with the fact that $\Gamma_{\nu_{0}}$ has boundaries with some degree of arbitrariness.

7.3 Ensemblist vs. Individualist Notions of Thermal Equilibrium

The two notions of Gibbs and Boltzmann entropy are parallel to two notions of thermal equilibrium. In the view that we call the ensemblist view, a system is in thermal equilibrium if and only if its phase point $X$ is random with the appropriate distribution (such as the micro-canonical or the canonical distribution). In the individualist view, in contrast, a system is in thermal equilibrium (at a given energy) if and only if its phase point $X$ lies in a certain subset $\Gamma_{\mathrm{eq}}$ of phase space.

The definition of this set may be quite complex; for example, according to the partition due to Boltzmann described in (41), the set $\Gamma_{\mathrm{eq}}$ contains those phase points for which the relative number of particles in each cell $C_{i}$ of $\mathscr{X}_{1}$ agrees, up to tolerance $\Delta f$, with the content of $C_{i}$ under the Maxwell-Boltzmann distribution. In general, the task of selecting the set $\Gamma_{\mathrm{eq}}$ has similarities with that of selecting the set $S$ of those sequences of (large) length $L$ of decimal digits that “look random.” While a reasonable test of “looks random” can be designed (such as outlined in the first paragraph of Section 7.1 above), different scientists may well come up with criteria that do not exactly agree, and again there is some arbitrariness in where exactly to “draw the boundaries.” Moreover, such a test or definition $S$ is often unnecessary for practical purposes, even for designing and testing pseudo-random number generators. For practical purposes, it usually suffices to specify the distribution (here, the uniform distribution over all sequences of length $L$) with the understanding that $S$ should contain, in a reasonable sense, the typical points relative to that distribution. Likewise, we usually never go to the trouble of actually selecting $\Gamma_{\mathrm{eq}}$ and the other $\Gamma_{\nu}$: it often suffices to imagine that they could be selected. Specifically, for thermal equilibrium, it often suffices to specify the distribution (here, the uniform distribution over the energy shell, that is, the micro-canonical distribution) with the understanding that $\Gamma_{\mathrm{eq}}$ should contain, in a reasonable sense, the typical points relative to that distribution (i.e., points that look macroscopically the way most points in $\mathscr{X}_{\mathrm{mc}}$ do).

That is why, also for the individualist, specifying an ensemble $\rho$ can be a convenient way of describing a thermal equilibrium state: It would basically specify the set $\Gamma_{\mathrm{eq}}$ as the set of those points $x$ that look macroscopically like $\rho$-most points. For the same reasons as in Section 7.2, this does not depend sensitively on $\rho$: points typical relative to the micro-canonical ensemble near energy $E$ are also typical relative to the canonical ensemble with mean energy $E$.

Let us turn to the ensemblist view. The basic problem with the ensemblist definition of thermal equilibrium is the same as with the Gibbs entropy: a system has an $X$ but not a $\rho$. To say the least, it remains open what should be meant by $\rho$. Is it subjective? But whether or not a system is in thermal equilibrium is not subjective. Does that mean that in distant places with no observers around, no system can be in thermal equilibrium? That does not sound right.

What if $\rho$ corresponded to a repeated preparation procedure? That is problematical as well. Suppose such a procedure produced a random phase point $X(0)$ that is uniformly distributed in some set $M$; now let time evolve up to $t$; the resulting phase point $X(t)$ is uniformly distributed in $\Phi_{t}(M)$, the associated Gibbs entropy is constant, and so the distribution of $X(t)$ is far from thermal equilibrium. In other words, the system never approaches thermal equilibrium. That does not sound right. We will come back to this point in Section 8.

Moreover, thermal equilibrium should have something to do with physical behavior, which depends on $X$. Thermal equilibrium should mean that the temperature, let us say the energy per degree of freedom, is constant (within tolerances) throughout the volume of the system, and that is a property of $X$. Thermal equilibrium should mean that the values of macro variables, which are functions on phase space, are constant over time, and that is true of some $X$ and not of others.

7.4 Ergodicity and Mixing

Ergodic and mixing are two kinds of chaoticity properties of a dynamical system $\mathscr{Y}$ such as an energy surface in phase space. A measure-preserving dynamical system in $\mathscr{Y}$ is called ergodic if almost every trajectory spends time, in the long run, in all regions of $\mathscr{Y}$ according to their volumes. Equivalently, a system is ergodic if time averages almost surely equal phase averages. A measure-preserving dynamical system in $\mathscr{Y}$ is called mixing if for all subsets $A,B$ of $\mathscr{Y}$, $\operatorname{\mathrm{vol}}(A\cap\Phi_{t}(B))$ tends to $\operatorname{\mathrm{vol}}(A)\,\operatorname{\mathrm{vol}}(B)/\operatorname{\mathrm{vol}}(\mathscr{Y})$ as $t\to\infty$. In other words, mixing means that for large $t$, $\Phi_{t}(B)$ will be deformed into all regions of phase space according to their volumes.

It is sometimes claimed (e.g., Mackey (1989)) that ergodicity or mixing are crucial for statistical mechanics. We would like to explain now why that is not so—with a few caveats; see also Cerino et al. (2016).

• •

Mixing would seem relevant to the ensemblist approach to thermal equilibrium because mixing implies that the uniform distribution over $\Phi_{t}(B)$ converges setwise to the uniform distribution (Liouville measure) over the energy surface $\mathscr{Y}$ (and every density converges weakly to a constant) (Krylov, 1979; Ruelle, 2007). However, for every finite $t$, the Gibbs entropy has not increased, so some ensemblists are pushed to say that the entropy does not increase within finite time, but jumps at $t=\infty$ to the thermal equilibrium value. Thus, some ensemblists are pushed to regard thermal equilibrium as an idealization that never occurs in the real world. That does not seem like an attractive option.

• •

The fact that the observed thermal equilibrium value $a_{\mathrm{eq}}$ of a macro variable $A$ coincides with its micro-canonical average $\langle A\rangle$ has sometimes been explained through the following reasoning (e.g., Khinchin, 1941, § 9): Any macro measurement takes a time that is long compared to the time that collisions take or the time of free flight between collisions. Thus, it can be taken to be infinite on the micro time scale. Thus, the measured value is actually not the value of $A(X(t))$ at a particular $t$ but rather its time average. By ergodicity, the time average is equal to the phase average, QED.

This reasoning is incorrect. In fact, ergodicity is neither necessary nor sufficient for $a_{\mathrm{eq}}=\langle A\rangle$. Not necessary because $A$ is nearly constant over the energy surface, so most phase points will yield a value close to $\langle A\rangle$ even if the motion is not ergodic. And not sufficient because the time needed for the phase point of an ergodic system to explore an energy surface is of the order $10^{N}$ years and thus much longer than the duration of the measurement. This point is also illustrated by the fact that in a macro system with non-uniform temperature, you can clearly measure unequal temperatures in different places with a thermometer faster than the temperature equilibrates.

• •

Still, ergodicity and similar properties are not completely unrelated to thermodynamic behavior. Small interactions between different parts of the system are needed to drive the system towards a dominant macro set (i.e., towards thermal equilibrium), and the same small interactions often also have the consequence of making the dynamics ergodic. That is, ergodicity is neither cause nor consequence of thermodynamic behavior, but the two have a common cause.

• •

In addition, and aside from thermodynamic behavior or entropy increase, mixing and similar properties play a role for the emergence of macroscopic randomness. For example, suppose a coin is tossed or a die is rolled; chaotic properties of the dynamics ensure that the outcome as a function of the initial condition varies over a very small scale on phase space, with the consequence that a probability density over the initial conditions that varies over a larger scale will lead to a uniform probability distribution of the outcome. (For more extensive discussion see (Dürr and Teufel, 2009, Sec. 4.1).)

• •

Boltzmann himself considered ergodicity, but not for the purposes above. He used it as a convenient assumption for estimating the mean free path of a gas molecule (Boltzmann, 1898, §10).

7.5 Remarks

Of course, Boltzmannian individualists use probability, too. There is no problem with considering procedures that prepare random phase points, or with using Gibbs ensembles as measures of typicality.

When arguing against the individualist view, Wallace (2019, Sec. 4) mentioned as an example that transport coefficients (such as thermal conductivity) can be computed using the two-time correlation function

[TABLE]

where $\langle\cdot\rangle$ means averaging while taking $X(0)$ as random with micro-canonical distribution. Wallace wrote:

since $C(t)$ is an explicitly probabilistic quantity, it is not even defined on the Boltzmannian approach.

Actually, that is not correct. The individualist will be happy as soon as it is shown that for most phase points in $\mathscr{X}_{\mathrm{mc}}$, the rate of heat conduction is practically constant and can be computed from $C(t)$ in the way considered.

Another objection of Wallace’s (2019, Sec. 5) concerns a system $\mathscr{S}$ for which (say) two macro sets, $\Gamma_{\nu_{1}}$ and $\Gamma_{\nu_{2}}$, each comprise nearly half of the volume of the energy shell, so that the thermal equilibrium state is replaced by two macro states $\nu_{1}$ and $\nu_{2}$, as can occur with a ferromagnet (see Footnote 9 in Section 5.1). Wallace asked how an individualist can explain the empirical observation that the system is, in the long run, equally likely to reach $\nu_{1}$ or $\nu_{2}$. To begin with, if a preparation procedure leads to a random phase point with distribution $\rho$, then $\int_{\Phi_{t}^{-1}(\Gamma_{\nu_{1}})}\rho$ is the probability of $\mathscr{S}$ being in $\nu_{1}$ at time $t$. As mentioned in Section 7.4 (penultimate bullet), a broad $\rho$ will lead to approximately equal probabilities for $\nu_{1}$ and $\nu_{2}$. This leads to the question why practical procedures lead to broad $\rho$s, and that comes from typicality as expressed in the development conjecture. Put differently, for a large number of identical ferromagnets it is typical that about half of them are in $\nu_{1}$ and about half of them in $\nu_{2}$.

8 Second Law for the Gibbs Entropy?

To the observation that the Gibbs entropy is constant under the Liouville evolution (6), different authors have reacted in different ways. We describe some reactions along with some comments of ours. (Of course, our basic reservations about the Gibbs entropy are those we expressed in Section 4.)

• •

Khinchin (1941) thought it was simply wrong to say that entropy increases in a closed system. He thought that the second law should merely assert that when a system is in thermal equilibrium and a constraint is lifted, then the new thermal equilibrium state has higher entropy than the previous one.

With this attitude one precludes the explanation of many phenomena.

• •

Ruelle (2007) expressed the view that entropy stays constant at every finite time but increases at $t=\infty$ due to mixing (see Section 7.4 above).

It remains unclear how this view can be applied to any real-world situation, in which all times are finite.

• •

Some authors (e.g., Mackey (1989)) suggested considering a system $\mathscr{S}_{1}$ perturbed by external noise (in the same spirit Kemble (1939)). In the framework of closed Hamiltonian systems of finite size, this situation could be represented by interaction with another system $\mathscr{S}_{2}$. It can be argued that if $\mathscr{S}_{2}$ is large and if initially $\rho=\rho_{1}(x_{1})\,\rho_{2}(x_{2})$ with $\rho_{2}$ a thermal equilibrium distribution, then the marginal distribution

[TABLE]

will follow an evolution for which $S_{\mathrm{G}}(\rho_{1}^{\mathrm{marg}})$ increases with time, and one might want to regard the latter quantity as the entropy of $\mathscr{S}_{1}$.

This attitude has some undesirable consequences: First, if the entropy of $\mathscr{S}_{1}$ increases and that of $\mathscr{S}_{2}$ increases then, since that of $\mathscr{S}_{1}\cup\mathscr{S}_{2}$ is constant, entropy is not additive. Second, since it is not true of all $\rho$ on $\mathscr{X}_{1}\times\mathscr{X}_{2}$ that $S_{\mathrm{G}}(\rho_{1}^{\mathrm{marg}})$ increases with time, the question arises for which $\rho$ it will increase, and why real-world systems should be of that type.

• •

Some authors (e.g., Wallace, 2019; Tolman, 1938, § 51) have considered a partition of (an energy shell in) phase space $\mathscr{X}$, such as the $\Gamma_{\nu}$, and suggested coarse graining any $\rho$ according this partition.111111Tolman (1938) had in mind a partition into cells of equal size, but we will simply use the $\Gamma_{\nu}$ in the following. That is, the coarse-grained $\rho$ is the function $P\rho$ whose value at $x\in\Gamma_{\nu}$ is the average of $\rho$ over $\Gamma_{\nu}$,

[TABLE]

$P$ is a projection to the subspace of $L^{1}(\mathscr{X})$ of functions that are constant on every $\Gamma_{\nu}$. The suggestion is that thermodynamic entropy $S_{t}$ is given by

[TABLE]

with $\rho_{t}$ evolving according to the Liouville equation (6), and that this $S_{t}$ tends to increase with time.

It is plausible that $S_{t}$ indeed tends to increase, provided that the time evolution has good mixing properties (so that balls in phase space quickly become “spaghetti”) and that the scale of variation of $\rho_{0}$ is not small; see also Falcioni et al. (2007). (The argument for the increase of (62) given by Tolman (1938) is without merit. After correctly showing (p. 169) that, for every distribution $\rho$,

[TABLE]

Tolman assumes that $\rho_{0}$ is uniform in each cell, i.e., $P\rho_{0}=\rho_{0}$, and then points out that $S_{\mathrm{G}}(P\rho_{t})\geq S_{\mathrm{G}}(P\rho_{0})$ for all $t$. This is true, but it is merely a trivial consequence of (63) and (5) owed to the special setup for which $P\rho_{0}=\rho_{0}$, and not a general statement of entropy increase, as nothing follows about whether $S_{\mathrm{G}}(P\rho_{t_{2}})>S_{\mathrm{G}}(P\rho_{t_{1}})$ for arbitrary $t_{2}>t_{1}$.)

• •

Some authors have argued that the Liouville evolution (6) is not appropriate for evolving $\rho$.

That depends on what $\rho$ means. If we imagine a preparation procedure that produces a random phase point $X(0)$ with probability distribution $\rho_{0}$, then letting the system evolve up to time $t$ will lead to a random phase point $X(t)$ with distribution $\rho_{t}$ given by the Liouville evolution. However, if we think of $\rho$ as representing an observer’s belief or knowledge, the situation may be different. In the last two bullet points, we consider some alternative ways of evolution.

• •

Let $L$ be the linear differential operator such that $L\rho$ is the right-hand side of the Liouville equation (6); then the Liouville evolution is $\rho_{t}=e^{Lt}\rho_{0}$. The proposal (62) could be re-expressed by saying that when using the Gibbs entropy, $S_{t}=S_{\mathrm{G}}(\rho_{t})$, the appropriate evolution for $\rho$ should be

[TABLE]

• •

Instead of (64), some authors (Mackey, 1989) suggested that

[TABLE]

This evolution corresponds to continually coarse graining $\rho$ after each infinitesimal time step of Liouville evolution.

Then the Gibbs entropy $S_{\mathrm{G}}(\rho_{t})$ is indeed non-decreasing with $t$, in fact without exception, as a consequence of (63) and (5). If the macro evolution is deterministic, then it should agree with the Boltzmann entropy. A stochastic macro evolution would yield additional contributions to the Gibbs entropy not present in the Boltzmann entropy (contributions arising from spread over many macro sets), but these contributions are presumably tiny. Furthermore, it seems not implausible that $\rho_{t}$ is often close to $Pe^{Lt}P\rho_{0}$, which implies that $S_{\mathrm{G}}(\rho_{t})$ is close to the expectation of $S_{\mathrm{B}}(X_{t})$ with $X_{0}$ random with distribution $P\rho_{0}$. However, as discussed in Section 4.1, one should distinguish between the entropy and the average entropy, so $S_{\mathrm{G}}(\rho_{t})$ is sometimes not quite the right quantity.

9 Further Aspects of the Quantum Case

Much of what we have discussed in the classical setting re-appears mutatis mutandis in the quantum case, and we will focus now on the differences and specialties of the quantum case. We are considering a quantum system of a macroscopic number $N$ (say, $>10^{23}$) of particles. Its Hilbert space can be taken to be finite-dimensional because we can take the systems we consider to be enclosed in a finite volume of 3-space, which leads to a discrete spectrum of the Hamiltonian and thus only finitely many eigenvalues in every finite energy interval. As a consequence, if we cut off high energies at an arbitrary level, or if we consider an energy shell $\mathscr{H}_{(E-\Delta E,E]}$, we are left with a finite-dimensional Hilbert space of relevant pure states.

9.1 Macro Spaces

The orthogonal decomposition $\mathscr{H}=\oplus_{\nu}\mathscr{H}_{\nu}$ into macro spaces can be thought of as arising from macro observables as follows, following von Neumann (1929). If $\hat{A}_{1},\ldots,\hat{A}_{K}$ are quantum observables (self-adjoint operators on $\mathscr{H}$) that should serve as macro observables, we will want to coarse grain them according to the resolution $\Delta M_{j}$ of macroscopic measurements by introducing $\hat{B}_{j}=[\hat{A}_{j}/\Delta M_{j}]\Delta M_{j}$ (with $[\cdot]$ again the nearest integer), thereby grouping nearby eigenvalues of $\hat{A}_{j}$ together into a single, highly degenerate eigenvalue of $\hat{B}_{j}$. Due to their macroscopic character, the commutators $[\hat{B}_{j},\hat{B}_{k}]$ will be small; as von Neumann (1929) argued and Ogata (2013) justified, a little correction will make them commute exactly, $\hat{M}_{j}\approx\hat{B}_{j}$ and $[\hat{M}_{j},\hat{M}_{k}]=0$. Since a macro description $\nu$ of the system can be given by specifying the eigenvalues $(m_{1},\ldots,m_{K})$ of all $\hat{M}_{1},\ldots,\hat{M}_{K}$, their joint eigenspaces are the macro spaces $\mathscr{H}_{\nu}$. They have dimensions of order $10^{10^{23}}$.

9.2 Some Historical Approaches

Pauli (1928) tried to derive entropy increase from quantum mechanics. In fact, he described several approaches. In §1 of his paper, he developed a modified Boltzmann equation that takes Fermi-Dirac and Bose-Einstein statistics into account and makes sense in the individualist view; in §2 he took an ensemblist attitude and suggested a novel dynamics in the form of a Markov chain on the macro states $\nu$, for which he proved that the (analog of the) Gibbs entropy of the probability distribution increases with time; in §3 and 4, he argued (not convincingly, to our minds) that the distribution over $\nu$’s provided by an evolving quantum state is, to a good degree of approximation, a Markov chain.

Returning to Pauli’s §1, we note that his derivation of the modified Boltzmann equation is far less complete than Boltzmann’s derivation of the Boltzmann equation from classical dynamics of hard spheres, and no validity theorem analogous to Lanford’s has been proven for it, as far as we are aware. So Pauli’s description has rather the status of a conjecture. According to it, the macro evolution $\nu(t)$ of a quantum gas with $N$ particles is deterministic for large $N$ (as with the Boltzmann equation), the expression for the entropy of $\nu$ agrees with the quantum Boltzmann entropy $S_{\mathrm{qB}}(\nu)$ as in (12), and it increases with time as a mathematical consequence of the evolution equation for $\nu(t)$. This approach fits into the framework of an orthogonal decomposition $\mathscr{H}=\oplus_{\nu}\mathscr{H}_{\nu}$ that arises in a way analogous to Boltzmann’s example (41) from an orthogonal decomposition of the 1-particle Hilbert space, $\mathscr{H}_{1}=\oplus_{i}\mathscr{K}_{i}$, by taking $\nu$ to be the list of occupation numbers (within tolerances) of each $\mathscr{K}_{i}$ and $\mathscr{H}_{\nu}$ the corresponding subspaces of $\mathscr{H}$.

Von Neumann famously came up with the von Neumann entropy (10), the quantum analog of the Gibbs entropy. It is less known that in 1929 he defended a different formula because (von Neumann, 1929, Sec. 1.3)

[t]he expressions for entropy [of the form $-k\operatorname{tr}\hat{\rho}\log\hat{\rho}$] are not applicable here in the way they were intended, as they were computed from the perspective of an observer who can carry out all measurements that are possible in principle—i.e., regardless of whether they are macroscopic[.]

We see here a mix of ensemblist and individualist thoughts in von Neumann’s words. Be that as it may, the formula that von Neumann defended was, in our notation,

[TABLE]

Similar expressions were advocated recently by Safranek et al. (2017; 2018). Note that this expression has a stronger individualist character than $-k\operatorname{tr}\hat{\rho}\log\hat{\rho}$, as it ascribes a non-trivial entropy value also to a system in a pure state. The problem with the expression is that it tends to average different entropy values where no averaging is appropriate. For example, suppose that $\psi$ has contributions from many but not too many macro spaces; since $\dim\mathscr{H}_{\nu}$ is very large, $\|\hat{P}_{\nu}\psi\|^{2}\log\|\hat{P}_{\nu}\psi\|^{2}$ can be neglected in comparison, and then (66) is just the weighted average of the quantum Boltzmann entropies (12) of different macro states, weighted with $\|\hat{P}_{\nu}\psi\|^{2}$. But in many situations, such as Schrödinger’s cat, we would not be inclined to regard this average value as the true entropy value valid in all cases. For comparison, we would also not be inclined to define the body temperature of Schrödinger’s cat as the average over all contributions to the wave function—we would say that the quantum state is a superposition of states with significantly different body temperatures. Likewise, we should say that the quantum state is a superposition of states with significantly different entropies.

In the same paper, von Neumann (1929) proved two theorems that he called the “quantum $H$ theorem” and the “quantum ergodic theorem.” However, they are not at all quantum analogs of either the $H$ theorem or the ergodic theorem. In fact, the two 1929 theorems are very similar to each other (almost reformulations of each other), and they assert that for a typical Hamiltonian or a typical choice of the decomposition $\mathscr{H}=\oplus_{\nu}\mathscr{H}_{\nu}$, all initial pure states $\psi_{0}$ evolve such that for most $t$ in the long run,

[TABLE]

(See also Goldstein et al. (2010b).) This is an interesting observation about the quantum evolution that has no classical analog; but it is far from establishing the increase of entropy in quantum mechanics.

9.3 Entanglement Entropy

Let us come back to option (d) of Section 3.1 above: the possibility that the density matrix $\hat{\rho}$ in $S_{\mathrm{vN}}$ should be the system’s reduced density matrix. This approach has the undesirable consequence that entropy is not extensive, i.e., the entropy of $\mathscr{S}_{1}\cup\mathscr{S}_{2}$ is not the sum of the entropy of $\mathscr{S}_{1}$ and the entropy of $\mathscr{S}_{2}$; after all, often the entropies of $\mathscr{S}_{1}$ and $\mathscr{S}_{2}$ will each increase while that of $\mathscr{S}_{1}\cup\mathscr{S}_{2}$ is constant if that is a closed system (because the evolution is unitary). Moreover, it is not true of all $\hat{\rho}$ on $\mathscr{H}_{1}\otimes\mathscr{H}_{2}$ that $S_{\mathrm{vN}}(\operatorname{tr}_{2}\hat{\rho})$ increases with time (think of time reversal).

A variant of option (d), and another widespread approach to defining the entropy of a macroscopic quantum system $\mathscr{S}$ with (say) a pure state $\Psi$, is to use the entanglement entropy as follows: Divide $\mathscr{S}$ into small subsystems $\mathscr{S}_{1},\ldots,\mathscr{S}_{M}$, let $\hat{\rho}_{j}$ be the reduced density matrix of $\mathscr{S}_{j}$,

[TABLE]

and add up the von Neumann entropies of $\hat{\rho}_{j}$,

[TABLE]

Since the $j$-th term quantifies how strongly $\mathscr{S}_{j}$ is entangled with the rest of $\mathscr{S}$, it is known as the entanglement entropy of $\mathscr{S}_{j}$. We will simply call also their sum (69) the entanglement entropy.

In thermal equilibrium, the entanglement entropy $S_{\mathrm{ent}}$ actually agrees with the thermodynamic entropy. That is a consequence of microscopic thermal equilibrium as described in Section 3.3, and thus of canonical typicality (Gemmer et al., 2004; Goldstein et al., 2006; Popescu et al., 2006): For most $\Psi$ in an energy shell of $\mathscr{S}$, assuming that the $\mathscr{S}_{j}$ are not too large (Goldstein et al., 2017b), $\hat{\rho}_{j}$ is approximately canonical, so its entanglement entropy agrees with its thermodynamic entropy, and extensivity implies the same for $\mathscr{S}$.

This argument also shows that for $\mathscr{S}$ in thermal equilibrium, $S_{\mathrm{ent}}$ does not depend much on how $\mathscr{S}$ is split into subsystems $\mathscr{S}_{j}$. On top of that, it also shows that $S_{\mathrm{ent}}$ still yields the correct value of entropy when $\mathscr{S}$ is in local thermal equilibrium, provided each $\mathscr{S}_{j}$ is so small as to be approximately in thermal equilibrium.

Many practical examples of non-equilibrium systems are in local thermal equilibrium. Still, for general non-equilibrium systems, $S_{\mathrm{ent}}$ may yield wrong values; an artificial example is provided by a product state $\Psi=\otimes_{j}\psi_{j}$ with each $\psi_{j}$ an equilibrium state of $\mathscr{S}_{j}$, so $S_{\mathrm{ent}}=0$ while the thermodynamic entropy, by extensivity, should have its equilibrium value. Artificial or not, this example at least forces us to say that $S_{\mathrm{ent}}$ is not correct for all $\Psi$; that $S_{\mathrm{ent}}$ is correct for most $\Psi$ we knew already since most $\Psi$ are in thermal equilibrium.

Another issue with $S_{\mathrm{ent}}$ arises when $\mathscr{S}$ is in a macroscopic superposition such as Schrödinger’s cat. Then $S_{\mathrm{ent}}$ yields approximately the average of the entropy of a live cat and that of a dead cat, while the correct value would be either that of a live cat or that of a dead cat. In this situation one may want to conditionalize on the case of (say) the cat being alive, and replace $\Psi$ by the part of it corresponding to a live cat. Ultimately, this leads us to replace $\Psi$ by its projection to one $\mathscr{H}_{\nu}$, and if we are so lucky that states in $\mathscr{H}_{\nu}$ are in local thermal equilibrium, then the conditionalized $S_{\mathrm{ent}}$ will agree with $S_{\mathrm{qB}}(\nu)$. So at the end of the day, $S_{\mathrm{qB}}$ seems to be our best choice as the fundamental definition of entropy in quantum mechanics.

9.4 Increase of Quantum Boltzmann Entropy

Just as the classical macro sets $\Gamma_{\nu}$ have vastly different volumes, the quantum macro spaces $\mathscr{H}_{\nu}$ have vastly different dimensions, and as phase space volume is conserved classically, dimensions of subspaces are conserved under unitary evolution.

It follows that if macro states $\nu$ follow an autonomous, deterministic evolution law $\nu\mapsto\nu_{t}$, then quantum Boltzmann entropy increases. In fact,

[TABLE]

More generally, not assuming that macro states evolve deterministically, already at this point it is perhaps not unreasonable to expect that in some sense the unitary evolution will carry a pure state $\psi_{t}$ from smaller to larger subspaces $\mathscr{H}_{\nu}$. In fact, it was proven (Goldstein et al., 2010a) that if one subspace has most of the dimensions in an energy shell, then $\psi_{t}$ will sooner or later come very close to that subspace (i.e., reach thermal equilibrium) and stay close to it for an extraordinarily long time (i.e., for all practical purposes, never leave it).

Proposition 1 below is another observation in the direction of Conjecture 1, concerning only two times instead of an entire history and only two macro states $\nu,\nu^{\prime}$. It is a quantum analog of the classical statement that if

[TABLE]

that is, if $S_{\mathrm{B}}(\nu^{\prime})$ is less than $S_{\mathrm{B}}(\nu)$ by an appreciable amount, then for any $t\neq 0$, most $X_{0}\in\Gamma_{\nu}$ are such that $X_{t}\notin\Gamma_{\nu^{\prime}}$.

Proposition 1**.**

Let $\hat{H}$ and $t\neq 0$ be arbitrary but fixed. If

[TABLE]

(that is, if $S_{\mathrm{qB}}(\nu^{\prime})$ is less than $S_{\mathrm{qB}}(\nu)$ by an appreciable amount), then for most $\psi_{0}\in\mathbb{S}(\mathscr{H}_{\nu})$,

[TABLE]

Proof.

Let $\hat{U}=e^{-i\hat{H}t}$, let $d_{\nu}:=\dim\mathscr{H}_{\nu}$, and let $\Psi$ be a random vector with uniform distribution on $\mathbb{S}(\mathscr{H}_{\nu})$. Then

[TABLE]

so

[TABLE]

If a non-negative quantity is small on average, it must be small in most cases, so the proposition follows. ∎

Note that the proof only used that $\mathscr{H}_{\nu^{\prime}}$ and $\mathscr{H}_{\nu}$ are two mutually orthogonal subspaces of suitable dimensions; as a consequence, the statement remains true if we replace $\mathscr{H}_{\nu^{\prime}}$ by the sum of all macro spaces with dimension less than that of $\mathscr{H}_{\nu}$. That is, if $\mathscr{H}_{\nu}$ is much bigger than all of the smaller macro spaces together (a case that is not unrealistic), then it is atypical for entropy to be lower at time $t$ than at time 0.

10 Conclusions

We have argued that entropy has nothing to do with the knowledge of observers. The Gibbs entropy (1) is an efficient tool for computing entropy values in thermal equilibrium when applied to the Gibbsian equilibrium ensembles $\rho$, but the fundamental definition of entropy is the Boltzmann entropy (2). We have discussed the status of the two notions of entropy and of the corresponding two notions of thermal equilibrium, the “ensemblist” and the “individualist” view. Gibbs’s ensembles are very useful, in particular as they allow the efficient computation of thermodynamic functions (Goldstein, 2019), but their role can only be understood in Boltzmann’s individualist framework. We have also outlined an extension of Boltzmann’s framework to quantum mechanics by formulating a definition of the quantum Boltzmann entropy and, as Conjecture 1, a statement of the second law of thermodynamics for it.

Acknowledgments. We thank Jean Bricmont and David Huse for valuable discussions. The work of JLL was supported by the AFOSR under award number FA9500-16-1-0037. JLL thanks Stanislas Leibler for hospitality at the IAS.