The Free Energy of a General Computation

TL;DR

This paper explores the thermodynamic costs of computation, linking information erasure to free energy, and generalizes Landauer's principle to broader computational processes, while reinterpreting the second law of thermodynamics through a computational lens.

Contribution

It introduces a generalized bound on free energy for any computation and connects thermodynamic principles with the logical reversibility of time evolution.

Findings

Erasure cost equals the length of optimal reversible compression.

Bounds on free energy investment for general computations are established.

Revisiting the second law reveals a logico-informational perspective with simulation resilience.

Abstract

Starting from Landauer's slogan "information is physical," we revise and modify Landauer's principle stating that the erasure of information has a minimal price in the form of a certain quantity of free energy. We establish a direct link between the erasure cost and the work value of a piece of information, and show that the former is essentially the length of the string's best compression by a reversible computation. We generalize the principle by deriving bounds on the free energy to be invested for --- or gained from, for that matter --- a general computation. We then revisit the second law of thermodynamics and compactly rephrase it (assuming the Church/Turing/Deutsch hypothesis that physical reality can be simulated by a universal Turing machine): Time evolutions are logically reversible --- "the future fully remembers the past (but not necessarily vice versa)." We link this view to previous formulations of the second law, and we argue that it has a particular feature that suggests its "logico-informational" nature, namely simulation resilience: If a computation faithfully simulates a physical process violating the law --- then that very computation procedure violates it as well.

Figures

Equations

$$\mathrm{WV}(S)=(\mathrm{len}(S)-K(S))kT\ln 2.$$

$$\mathrm{WV}(S|X):=(\mathrm{len}(S)-\mathrm{len}(P))kT\ln 2$$

$$\mathcal{Z}:\{0,1{\}}^{\ast }\times \{0,1{\}}^{\ast }⟶\{0,1{\}}^{\ast }$$

$$(A,B)\mapsto (\mathcal{Z}(A,B),B)$$

$$\mathrm{WV}(S|X)\ge (\mathrm{len}(S)-\mathrm{len}(\mathcal{Z}(S,X)))kT\ln 2.$$

$$A||B\mapsto \mathcal{Z}(A,B)||0^{\mathrm{len}(A)-\mathrm{len}(\mathcal{Z}(A,B))}||B$$

$$\mathrm{WV}(S|X)\le (\mathrm{len}(S)-K_{\mathcal{U}}(S|X))kT\ln 2,$$

$$\mathrm{WV}(S|X)+\mathrm{EC}(S|X)=NkT\ln 2.$$

$$\mathrm{EC}(S|X)\le NkT\ln 2-\mathrm{WV}(S|X).$$

$$\mathrm{WV}(S|X)\ge NkT\ln 2-\mathrm{EC}(S|X):$$

$$K_{\mathcal{U}}(S|X)kT\ln 2$$

$$\le \mathrm{len}(\mathcal{Z}(S,X))kT\ln 2.$$

$$\displaystyle{\rm Cost}_{\cal U}(A\rightarrow_{\{C_{i}\}}B\,|\,X)\ \geq\ kT\ln 2\cdot\bigg{[}K_{\cal{U}}(A|X)$$

$$\displaystyle-\sum_{i=1}^{n}{\big{(}{\rm len}(\mathcal{Z}(C_{i},X))-K_{\cal{U}}(C_{i}|X)\big{)}}-{\rm len}(\mathcal{Z}(B,X))\bigg{]}\,.$$

$${\mathrm{Cost}}_{\mathcal{U}}$$

$$[K_{\mathcal{U}}(A|X)-\mathrm{len}(\mathcal{Z}(B,X))]\cdot kT\ln 2$$

$${\mathrm{WV}}_{\mathcal{U}}(A\to B|X)\ge [K_{\mathcal{U}}(B|X)-\mathrm{len}(\mathcal{Z}(A,X))]\cdot kT\ln 2.$$

$$A⟶B⟶A.$$

$$[K_{\mathcal{U}}(B|X)-\mathrm{len}(\mathcal{Z}(A,X))]\cdot kT\ln 2,$$

$$K(a):\mathbb{N}\to \mathbb{N},\text{with}n\mapsto K\left(a_{[n]}\right),$$

$$K(a\mid b):\mathbb{N}\to \mathbb{N},\text{with}n\mapsto K\left(a_{[n]}\mid b_{[n]}\right).$$

$$K(a)\approx n:⟺\underset{n\to \infty }{\lim }\frac{K\left(a_{[n]}\right)}{n}=1,$$

$$K(a)\approx 0:⟺\underset{n\to \infty }{\lim }\frac{K\left(a_{[n]}\right)}{n}=0.$$

$$\underset{n\to \infty }{\lim }\frac{f(n)}{g(n)}=1.$$

$$x_i\oplus y_i=a_i{b}_i\text{ for all }i\in \mathbb{N},$$

$$K(x\mid a)\approx K(x\mid a,b),\text{ and}$$

$$K(y\mid b)\approx K(y\mid a,b).$$

$$K(x\mid a)=\Theta (n)\text{ and }K(y\mid b)=\Theta (n).$$

$$\frac{\left(\genfrac{}{}{0ex}{}{n}{(w+\Delta )n}\right)\left(\genfrac{}{}{0ex}{}{n}{(1-w-\Delta )n}\right)}{\left(\genfrac{}{}{0ex}{}{n}{wn}\right)\left(\genfrac{}{}{0ex}{}{n}{(1-w)n}\right)}=2^{-\Theta (n)}.$$

Full text

I Introduction

In 1961, Rolf Landauer famously stated “Information is physical” Landauer (1998): Despite the success of Shannon’s making information an abstract concept (that can be viewed and understood independently of its particular physical realization), Landauer — while not questioning the power of that abstract view — recalls that all information storing, treatment, and transmission is ultimately a physical process and, thus, subject to physical laws. A specific law relevant in this context is the second law of thermodynamics. Its consequence for information processing has been called Landauer’s principle Landauer (1961): “The erasure of $N$ bits of information costs at least an amount of $NkT\ln 2$ ($k$ being Boltzmann’s constant) of free energy that must be dissipated as heat into the environment of temperature $T$.” (Note that this heat dissipation is crucial for the argument: It represents the compensation required for avoiding a violation of the second law despite an entropy decrease in the memory device through the erasure process.) Conversely, erased strings have a work value (see, e.g., Ref. Szilard (1929); Dahlsten et al. (2009)): By, e.g., encoding an erased bit string of length $N$ in the particles position of a gas within a box, where the particle’s position is on the left half for the value [math] and on the right half otherwise, and by placing a piston in the center, $NkT\ln 2$ of free energy can be extracted from the environment, “randomizing” the original string.

In this article, we modify and generalize Landauer’s principle in the following respects: First, it is claimed that the erasure cost is not proportional to the length of the string to be erased, but of its best compression — given the entire knowledge of the erasure device (Section IV). We obtain these results from new bounds on the work value of information (Section III), and a direct connection between erasure cost and work value of any piece of information. Second, we generalize these results to a lower bound on the free-energy cost, or value, of a general computation (Section V). Our findings are modifications of known results (see e.g., Refs. Bennett (1982); Dahlsten et al. (2009, 2011); Faist et al. (2015)) to the constructive setting — where all involved processes are imagined to be carried out by a Turing machine. Furthermore, we give a lower bound on the free-energy gain possible from certain computations — a bound that matches the cost of the inverse computation. We look at the use of the erasure cost as an intrinsic randomness measure in the context of quantum correlations (Section VI). Having these results at hand, we finish by proposing a computational version of the second law of thermodynamics (Section VII). This comes with a speculation about what trait of it is the reason why such a version exists in the first place. Candidates are its “encoding independence” and “simulation resilience:” If a computation simulates a process violating the second law, then that computing procedure cannot be closed but must dissipate “junk” bits onto the other parts of the tape, or heat into the environment. Thus, the violation of the law by a process carries over to its simulation. The reason is that a degree of freedom is represented by — a degree of freedom.

II Work Value: State of the Art

While other results (see, e.g., Refs. Plenio and Vitelli (2001); Sagawa and Ueda (2012); Deffner and Jarzynski (2013); Merhav (2017); Boyd et al. (2018); Frank (2018)) on the work value of information focus on using information reservoirs to generate energy flows, the below described results and this article focus on the work value of information — being in form of random variables or bit strings — per se. As opposed to discussing the role of information in thermodynamic processes, we discuss the thermodynamics processes of information.

II.1 Bennett’s view

Bennett Bennett (1982) claimed the work value of a string $S$, ${\rm WV}(S)$, to be proportional to the difference between its length, ${\rm len}(S)$, and the length of the shortest program that produces $S$. The latter is called the Kolmogorov complexity of $S$, denoted by $K(S)$ Kolmogorov (1965). Expressed mathematically, this amounts to

[TABLE]

Bennett’s argument is that $S$ can be logically, hence, thermodynamically Fredkin and Toffoli (1982) reversibly mapped to the string $P||000\cdots 0$, where the symbol $||$ denotes concatenation, and $P$ is the shortest program generating $S$. The length of the generated [math]-string is ${\rm len}(S)-K(S)$.

It was already pointed out by Zurek Zurek (1989a) that while it is true that the reverse direction exists and is computable by a universal Turing machine, its forward direction, i.e., obtaining $P$ from $S$, is uncomputable. This means that a “demon” that could carry out this work-extraction computation on $S$ does not exist (if the Church/Turing hypothesis is true); the Kolmogorov complexity is an uncomputable value. We will see, however, that Bennett’s value is an upper bound on the work value of $S$. Bennett also links the string’s erasure cost to its probabilistic entropy Bennett (2003).

II.2 Dahlsten et al.’s view

Dahlsten et al. Dahlsten et al. (2009, 2011) follow Szilárd Szilard (1929) in putting the knowledge of the demon extracting the work to the center of their attention. More precisely, they claim ${\rm WV}(S)={\rm len}(S)-D(S)$, where the “defect” $D(S)$ is bounded from above and below by a smooth Rényi entropy of the distribution of $S$ from the demon’s viewpoint, modeling its ignorance. Building on these results and in the same probabilistic spirit, the cost of erasure del Rio et al. (2011) as well as of general computations Faist et al. (2015) have been linked to entropic expressions of (conditional) probability distributions.

III Work Extraction as Data Compression

In the following, we model work extraction to be an algorithm executed by a “demon with knowledge.”

III.1 The Model

We assume the demon to be a universal Turing machine ${\cal U}$ the memory tape of which is sufficiently long for the inputs and tasks in question, but finite. The tape initially contains $S$, the string the work value of which is to be determined, $X$, a finite string modeling the demon’s knowledge about $S$, and [math]s for the rest of the tape. After the extraction computation, the tape contains, at the bit positions initially holding $S$, a (shorter) string $P$ plus $0^{{\rm len}(S)-{\rm len}(P)}$, whereas the rest of the tape is (again) the same as before the work extraction. The operations are logically reversible and can, hence, be carried out thermodynamically reversibly Fredkin and Toffoli (1982). Logical reversibility is the ability of the same demon to carry out the backward computation step by step, i.e., from $P||X$ to $S||X$. We denote by ${\rm WV}(S|X)$ the maximal length of an all-[math]-string extractable logically reversibly from $S$, given the knowledge $X$, times $kT\ln 2$, i.e.,

[TABLE]

if $P$’s length is minimal.

III.2 Lower Bound

We show that every specific data-compression algorithm leads to a lower bound on the extractable work: Let $\mathcal{Z}$ be a computable function

[TABLE]

such that

[TABLE]

is injective.111The set $\{0,1\}^{*}$ is the set of all finite but arbitrarily long bit strings. We call $\mathcal{Z}$ a data-compression algorithm with helper. Then we have

[TABLE]

This can be seen as follows. First, note that the function

[TABLE]

is computable and bijective. From the two (possibly irreversible) circuits which compute the compression and its inverse, one can obtain a reversible circuit for the function such that no further input or output bits are involved: This can be achieved by first implementing all logical operations with Toffoli gates and uncomputing the “junk” Bennett (1973) in both circuits. The resulting two circuits have now still the property that the input is part of the output. As a second step, we can simply combine the two such that the first circuit’s first and second outputs become the second’s second and first inputs, respectively. Roughly speaking, the first computes the compression and the second reversibly uncomputes the raw data (see Figure 1).

The combined circuit has only the compressed data plus the [math]s as the output, sitting on the bit positions carrying the input before. (This circuit is roughly as efficient as the less efficient of the two irreversible circuits for data compression and decompression, respectively.) A typical example for an algorithm that can be used here is universal data compression à la Ziv-Lempel Ziv and Lempel (1978).

III.3 Upper Bound

We have the following upper bound on the extractable work:

[TABLE]

where $K_{\cal U}(S|X)$ is the conditional Kolmogorov complexity (with respect to the universal Turing machine ${\cal U}$) of $S$ given $X$, i.e., the length of the shortest program $P$ for ${\cal U}$ that outputs $S$, given $X$. The reason is that the extraction demon is only able to carry out the computation in question (logically, hence, thermodynamically) reversibly if it is able to carry out the reverse computation as well. Therefore, the string $P$ must be at least as long as the shortest program for ${\cal U}$ generating $S$ if $X$ is given.

Although the same is not true in general, this upper bound is tight if $K_{\cal U}(S|X)\approx 0$. The latter means that $X$ itself can be seen as a program for generating an additional copy of $S$. The demon can then bit-wisely XOR this extra $S$ to the original $S$ (to be work-extracted) on the tape, hereby producing $0^{{\rm len}(S)}$ reversibly to replace the original $S$, at the same time saving the new one, as reversibility demands (see Figure 2).

When Bennett’s “uncomputing trick” is used — allowing to make any computation by a Turing machine logically reversible Bennett (1973) —, then a history string $H$ is written to the tape during the computation of $S$ from $X$ such that after XORing, the demon can, in a (reverse) stepwise manner, uncompute the generated copy of $S$ and end up in the tape’s original state — except that the original $S$ is now replaced by $0^{{\rm len}(S)}$: This results in a maximal work value matching the (in that case trivial) upper bound.222Let us compare our bounds with the entropy-based results of Dahlsten et al. (2009, 2011): According to the latter, a demon knowing $S$ entirely is able to extract maximal work: ${\rm WV}(S)\approx{\rm len}(S)kT\ln 2$. What does it mean to “know $S$”? The knowledge can consist of (a) a copy of $S$, or of (b) the ability to compute such a copy with a given program $P$, or (c) it can determine $S$ uniquely without providing the ability to compute it. The constructive as opposed to the entropic groups of results are in accordance in the cases (a) and (b) but in conflict in case (c): For instance, assume the demon’s knowledge about $S$ is: “$S$ equals the first $N$ bits $\Omega_{N}$ of the binary expansion of $\Omega$,” where, $\Omega$ is the so-called halting probability Chaitin (1975) of a fixed universal Turing machine ${\cal A}$ (e.g., the demon ${\cal U}$ itself). Although there is a short description of $S$ in this case, and $S$ is thus uniquely determined in an entropic sense, it is still incompressible, even given that knowledge: K_{{\cal U}}(\Omega_{n}\,|\,\mbox{``It is bits 1$--$n$of TM${\cal A}'s halting probability''})\approx n: No work is extractable according to our upper bound. Intuitively, this gap opens up whenever the “description complexity” is smaller than the Kolmogorov complexity. (Note that a self-reference argument, called Berry paradox, shows that the notion of “description complexity” is problematic and can never be defined consistently for all strings.)

IV Revising Landauer’s Principle

Here, we revise Landauer’s principle to give a lower and an upper bound on the erasure cost.

IV.1 Connection to Work Value

For a string $S\in\{0,1\}^{N}$, let WV$(S|X)$ and EC$(S|X)$ be its work value and erasure costs, respectively, given an additional string $X$ (a “catalyst” which remains unchanged, as above). Then

[TABLE]

To see this, consider first the combination extract-then-erase. In the extraction process we gain WV$(S|X)$ of free energy, and consequently have to erase $N$ bits. Since this is one specific way of erasing, we have

[TABLE]

If, on the other hand, we consider the combination erase-then-extract, this leads to

[TABLE]

We spend EC$(S|X)$ of free energy to erase the string and use all of the string as “fuel.”

IV.2 Bounds on the Erasure Cost

Given the results on the work value above, as well as the connection between the work value and erasure cost, we obtain the following bounds on the thermodynamic cost of erasing a string $S$ by a demon, modeled as a universal Turing machine ${\cal U}$ with initial tape content $X$.

Landauer’s principle, revisited. *Let $\mathcal{Z}$ be a computable function, $\mathcal{Z}\,:\,\{0,1\}^{*}\times\{0,1\}^{*}\longrightarrow\{0,1\}^{*}$, such that $(A,B)\mapsto(\mathcal{Z}(A,B),B)$ is injective. Then we have *

[TABLE]

The first inequality follows from Equation (8) in combination with the upper bound (7), the second from Equation (8) and the lower bound (5).

V Generalizing Landauer’s Principle

Erasure as well as work extraction can be seen as special cases of a computation with a given input and an output. Here, we generalize Landauer’s principle and discuss the work cost and work value of a general computation, i.e., we generalize the already obtained bounds on the cost (minimal amount of free energy that has to be used) and value (maximal amount of free energy that can be gained) to general computation. Assume that a (universal) Turing machine performs a computation such that the initial content of the tape is $A$ and $X$ (plus a corresponding finite number of [math]s) and the final state is $B$ and $X$ (where $X$ can be seen, again, as a “catalyst”). Depending on $A$, $B$, and $X$, this computation can have a work cost or value, respectively. If it has some work cost, then the party performing the computation has to invest free energy that will be dissipated as heat to the environment during the computation. In the case that the computation has some value, heat from the environment is transformed to free energy.

V.1 The Energy Cost of a General Computation

The following result is an algorithmically constructive modification of entropic results Faist et al. (2015) and a generalization of less constructive but also complexity-based claims Zurek (1989b).

Work cost of a general computation. Let $\mathcal{Z}$ be a computable function, $\mathcal{Z}\,:\,\{0,1\}^{*}\times\{0,1\}^{*}\rightarrow\{0,1\}^{*}$, such that $(V,W)\mapsto(\mathcal{Z}(V,W),W)$ is injective. Assume that the Turing machine ${\cal U}$ carries out a computation such that $A$ is its initial state, $C_{1}$ the first intermediate state, $C_{2}$ the second, etc., up to $C_{n}$, and $B$ is the final state. Then the energy cost of this computation with side information $X$ (always on the tape), ${\rm Cost}_{\cal U}(A\rightarrow_{\{C_{i}\}}B\,|\,X)$, is at least

[TABLE]

Proof. Let us consider the computation from $(A,X)$ to $(C_{1},X)$. According to the above (see expression (11)), the erasure cost of $A$, given $X$, is at least $K_{\cal U}(A|X)\cdot kT\ln 2$. One possibility of realizing this complete erasure of $A$ is to first transform it to $C_{1}$ (given $X$), and then erase $C_{1}$ — at cost at most len$(\mathcal{Z}(C_{1},X))\cdot kT\ln 2$. Therefore, the cost to get from $A$ to $C_{1}$ given $X$ cannot be lower than the difference between $K_{\cal U}(A|X)\cdot kT\ln 2$ and len$(\mathcal{Z}(C_{1},X))\cdot kT\ln 2$. The statement follows by summing all contributions of the individual computing steps. qed.

Note that if no intermediate results are specified, the bound simplifies to

[TABLE]

(see also Ref. Wolf (2018)).

V.2 The Energy Value of a General Computation

We consider the work value of a computation from $A$ to $B$, given $X$. More specifically, this is a computation that starts with $(A,X)$ and finishes with $(B,X)$, where $B$ is freely choosable by the computation among all strings with a given complexity $K_{\cal{U}}(B|X)$. The work value is denoted by ${\rm WV}_{\cal U}(A\rightarrow B\,|\,X)$, and it is bounded from below as follows.

Work value of a general computation. Let $\mathcal{Z}$ be a computable function, $\mathcal{Z}\,:\,\{0,1\}^{*}\times\{0,1\}^{*}\rightarrow\{0,1\}^{*}$, such that $(V,W)\mapsto(\mathcal{Z}(V,W),W)$ is injective. The work value of a computation from $A$ to $B$, given $X$, is bounded from below by

[TABLE]

Proof. The cost of erasing $A$, given $X$, is at most len$(\mathcal{Z}(A,X))\cdot kT\ln 2$ (see expression (11)). We use a stretch of the resulting all-[math]-string of some length $N$ for gaining $NkT\ln 2$ free energy. The resulting string of length $N$ is then used as a program for the universal Turing machine ${\cal U}$, with additional input $X$, and where the computation is made logically reversible using Bennett’s “uncomputing” trick Bennett (1973); let $B$ be the resulting string. Then $K_{\cal{U}}(B|X)\leq N$. qed.

V.3 Combination

Consider the following “circular computation,” given $X$:

[TABLE]

The free-energy gain of computing $B$ from $A$ is at least

[TABLE]

whereas the cost for computing $A$ back from $B$ is at least this same amount. The identity of the two bounds is not very surprising; it implies that the bound on the work value is “at least as tight” as the one for the cost of the inverse computation, since otherwise a perpetuum mobile of the second kind results.

VI “Randomness” and Quantum Correlations

Landauer’s revised principle suggests that the erasure cost of a piece of information is an intrinsic, context-free, physical measure for its randomness independent of probabilities and counterfactual statements (that “some value could just as well have been different”).333Moreover, such a point of view allows one to discuss randomness on operational grounds. This can be tested in a context in which randomness is central: Bell correlations Bell (1964) predicted by quantum theory. In a proof of principle, it was shown Wolf (2015) that in essence, a similar mechanism as in the probabilistic setting arises: If the correlation is non-local, the inputs are incompressible, and non-signaling holds, then the outputs must be highly complex as well.

Before we describe some of the findings of Ref. Wolf (2015) in more detail, we introduce the required notation. For an infinite string $a=(a_{1},a_{2},\dots)$, we define its “truncation” $a_{[n]}:=(a_{1},a_{2},\dots,a_{n},0,0,\dots)$: the string $a$ where all symbols after the $n$-th are set to [math]. The expressions $K(a)$ and $K(a\mid b)$, where $b$ is an infinite string as well, denote the functions

[TABLE]

An incompressible string $a$ has the property

[TABLE]

and a computable string $a$ the property

[TABLE]

Intuitively, the shortest program that prints an incompressible string consists of that very same string, and the shortest program that prints a computable string has a constant length. Moreover, we say two functions $f$ and $g$ mapping natural numbers to natural numbers, where $g\not\approx 0$, satisfy $f\approx g$ if and only if

[TABLE]

Having this notation at hand, the result stated above is the following. Let $(a,b,x,y)$ be a tuple of four infinite binary strings, where

the PR-box condition Popescu and Rohrlich (1994) is satisfied, i.e.,

[TABLE] 2. 2.

both “input” strings $a$ and $b$ are independent and incompressible, i.e., $K(a,b)=K(a)+K(b)\approx 2n$, 3. 3.

the no-signaling condition is satisfied, i.e.,

[TABLE]

It follows from these conditions that the “output” strings are not computable — even if conditioned on the “input:”

[TABLE]

Whilst this stated result assumes the existence of correlations not attainable by quantum means Cirel’son (1980), the same article proves an analogous statement for quantum correlations, e.g., for the quantum violations of the chained Bell inequalities Barrett et al. (2005); Colbeck and Renner (2012).

These results allow for a discussion of quantum correlations without the usual counterfactual arguments used in derivations of Bell inequalities (combining in a single formula results of different measurements that cannot actually be carried out together). Furthermore, this potentially opens the door to novel functionalities, namely complexity amplification and expansion Baumeler et al. (2017). What results is an all-or-nothing flavor of the Church/Turing hypothesis Wolf (2017): Either no physical computer exists that is able to produce non-Turing-computable data — or even a “device” as simple as a single photon can.

VII The Second Law as Logical Reversibility

In Landauer’s principle, the price for the logical irreversibility of the erasure transformation comes in the form of a thermodynamic effort. (Since the amount of the required free energy, and heat dissipation, is proportional to the length of the best compression of the string, the latter can be seen as a quantification of the erasure transformation’s irreversibility.) In an attempt to harmonize this somewhat hybrid picture, we invoke Wheeler’s Wheeler (1989) “It from Bit: Every it — every particle, every field of force, even the spacetime continuum itself — derives its function, its meaning, its very existence entirely […] from the apparatus-elicited answers to yes-or-no questions, binary choices, bits.” This is an anti-thesis to Landauer’s slogan, and we propose the following synthesis of the two: If Wheeler motivates us to look at the environment as being a computation as well, then Landauer’s principle may be read as: The necessary environmental compensation for the logical irreversibility of the erasure of $S$ is such that the overall computation, including the environment, is logically reversible: no information ever gets completely lost.

**Second law, logico-computational version.

***Time evolutions of closed systems are injective:

Nature computes with Toffoli, but no AND or OR gates. *

Note that this fact is a priori asymmetric in time: The future must uniquely determine the past, not necessarily vice versa. In case the condition holds also for the reverse time direction, the computation is called deterministic, and randomized otherwise.

Logical reversibility is a simple computational version of a discretized second law, and it has implications resembling the traditional versions of the law: First of all, it leads to a “Boltzmann-like” form, i.e., the existence of a quantity essentially monotonic in time. More specifically, the logical reversibility of time evolution implies that the Kolmogorov complexity of the global state at time $t$ can be smaller than the one at time [math] only by at most $K(R_{t})+O(1)$ if $R_{t}$ is a string encoding the time span $t$. The reason is that one possibility of describing the state at time [math] is to give the state at time $t$, plus $t$ itself; the rest is exhaustive search using only a constant-length program simulating forward time evolution (including possible randomness).

Similarly, logical reversibility also implies statements resembling the version of the second law due to Clausius: “Heat does not spontaneously flow from cold to hot.” The rationale here is explained with a toy example: If we have a circuit — the time evolution — using only (logically reversible) Toffoli gates, then it is impossible that this circuit computes a transformation mapping a pair of strings to another pair such that the Hamming-heavier of the two becomes even heavier while the lighter gets lighter.444The Hamming weight of a binary string $S$ is the number of $1$s in $S$. A function accentuating imbalance, instead of lessening it, is not injective, as the following counting argument shows.

“Clausius” Toy Example. Let a circuit consisting of only Toffoli gates map an $N(=2n)$-bit string to another. We consider the map separately on the first and second halves and assume the computed function to be conservative, i.e., to leave the Hamming weight of the full string unchanged at $n$ (conservativity can be seen as some kind of first law, i.e., the preservation of a quantity). We look at the excess of $1$s in one of the halves (which equals the deficit of $1$s in the other). We observe that the probability (with respect to the uniform distribution over all strings of some Hamming-weight couple $(wn,(1-w)n)$, where the first half has $wn$ 1s and the second $(1-w)n$) of the imbalance substantially growing is exponentially weak. The key ingredient for the argument is the function’s injectivity. Explicitly, the probability that the weight couple goes from $(wn,(1-w)n)$ to $((w+\Delta)n,(1-w-\Delta)n)$ — or more extremely —, for $1/2\leq w<1$ and $0<\Delta\leq 1-w$, is

[TABLE]

The example suggests that logical reversibility might be the “Church/Turing manifestation” of the second law: If reality is computed by a Turing machine, then physical laws correspond to properties of such computations — as in the case of the second law: logical reversibility. If we assume for a moment that the second law of thermodynamics has indeed such a simple Church/Turing manifestation, it is a natural question inhowfar just this already makes the law special. In fact, the law does have a peculiar related property, encoding independence: Since the second law deals with degrees of freedom, and a degree of freedom will correspond, in another encoding, to a degree of freedom again, it is either respected in both encodings or violated in both. (In comparison: It cannot be decided by just looking at a running program whether the simulated system “violates or respects Kepler’s laws” — that would crucially depend on how masses and their position are represented by the code.) Hand in hand with this comes the property of simulation resilience. Let us take again the example of Kepler’s laws: If a program simulates planets moving on, say, square orbits, this does not mean that the program execution, viewed as a physical process, is by itself problematic — the laws of gravity are not simulation resilient. If, in sharp contrast to this, we simulate a system, e.g., the microstate sequence of a steam engine, then that simulation process violates the second law just as the simulated system does: With respect to the second law of thermodynamics, the simulation of “reality” is just as good as “reality” itself.

VIII Conclusion

We start from Landauer’s principle, stating that the erasure of information requires an amount of free energy, to be dissipated as heat into the environment, proportional to the number of independent binary degrees of freedom of that information. Specifically, the use of reversible data compression, imagined to be carried out by Fredkin and Toffoli’s ballistic computer, implies that the necessary amount is proportional to the length of the best compression of the information into a binary string (and not to the length of the original string, as often stated). We generalize and broaden the scope of the principle, and its converse, to lower bounds on the free-energy cost of — or gain from — a general computation: the bounds on cost versus gain are in accord.

Landauer has derived, in 1961, his principle from the second law of thermodynamics. We close the circle by formulating a simple “Church/Turing version” of that law: Logical reversibility of the overall computation, including the environment. This fact alone implies variants of the historical versions of the second law, due to Boltzmann, Clausius, also Kelvin; it is perhaps equivalent to them, certainly simpler. The arising belief that the law is rather logical than physical in its nature is nourished by two properties of the second law: its encoding independence and its simulation resilience.

Confronted with the relevance of the second law of thermodynamics in computation, and with its simulation resilience, let us close with the (provocative) question whether Landauer’s “Information is physical” should be replaced by

“The second law of thermodynamics is not physical.”

Acknowledgments. We thank Charles Bédard, Claus Beisbart, Sophie Berthelette, Paul Boes, Gilles Brassard, Časlav Brukner, Xavier Coiteux-Roy, Bora Dakic, Paul Erker, Jürg Fröhlich, and Arne Hansen for enlightening discussions, and anonymous reviewers for helpful comments. Ä.B. is supported by the Swiss National Science Foundation (SNSF) under grant 175860, the Erwin Schrödinger Center for Quantum Science & Technology (ESQ), and the Austrian Science Found (FWF): Z3. S.W. is supported by the Swiss National Science Foundation (SNSF), the NCCR QSIT, and the Hasler Foundation.