Energy Dissipation Bounds for Autonomous Thermodynamic Cycles

TL;DR

This paper derives bounds on energy dissipation in autonomous thermodynamic cycles, revealing that even infinitely slow processes can be energetically irreversible due to combined effects of thermodynamic friction and control precision costs.

Contribution

It introduces a stochastic thermodynamics framework to quantify energy costs considering both thermodynamic friction and control precision, challenging traditional adiabatic assumptions.

Findings

Energy dissipation bounds depend on both friction and control precision.

Infinitely slow protocols are not necessarily reversible energetically.

Autonomous control introduces fundamental irreversibility in thermodynamic cycles.

Abstract

How much free energy is irreversibly lost during a thermodynamic process? For deterministic protocols, lower bounds on energy dissipation arise from the thermodynamic friction associated with pushing a system out of equilibrium in finite time. Recent work has also bounded the cost of precisely moving a single degree of freedom. Using stochastic thermodynamics, we compute the total energy cost of an autonomously controlled system by considering both thermodynamic friction and the entropic cost of precisely directing a single control parameter. Our result suggests a challenge to the usual understanding of the adiabatic limit: here, even infinitely slow protocols are energetically irreversible.

Equations

$$⟨\dot{S}⟩\approx \frac{\mathrm{d}{\lambda }^{\alpha }}{\mathrm{d}t}{\tilde{g}}_{\alpha \beta }^{\lambda }\frac{\mathrm{d}{\lambda }^{\beta }}{\mathrm{d}t}$$

$$S\ge \frac{{\tilde{\mathcal{L}}}^2({\mathbf{\lambda }}_i,{\mathbf{\lambda }}_f)}{\Delta t}$$

$${⟨\dot{S}⟩}_{\theta }\approx {⟨\frac{\mathrm{d}{\lambda }^{\alpha }}{\mathrm{d}{t}^{+}}⟩}_{\theta }{\tilde{g}}_{\alpha \beta }^{\lambda }{⟨\frac{\mathrm{d}{\lambda }^{\alpha }}{\mathrm{d}{t}^{-}}⟩}_{\theta }+\frac{v^2}{D}+D{g}^{\theta }$$

$$S\ge 2\mathcal{L}({\mathbf{\lambda }}_i,{\mathbf{\lambda }}_f)+\frac{{\tilde{\mathcal{L}}}^2({\mathbf{\lambda }}_i,{\mathbf{\lambda }}_f)}{\Delta t}(1-ϵ)$$

$$U(\mathbf{y})={\lambda }^{\alpha }(\theta ){\varphi }_{\alpha }(\mathbf{x})\mathbf{y}=(\theta ,\mathbf{x})$$

$$\begin{array}[]{cc}\begin{aligned} \dot{\theta}&=v+\sqrt{2D}\eta\\ \dot{x}^{i}&=D^{ij}F_{j}(\boldsymbol{{y}})+\sqrt{2D^{ij}}\xi_{j}\end{aligned}&\begin{aligned} \langle\eta(t)\eta(t^{\prime})\rangle&=\delta(t-t^{\prime})\\ \langle\xi_{i}(t)\xi_{j}(t^{\prime})\rangle&=\delta_{ij}\delta(t-t^{\prime})\end{aligned}\end{array}$$

$$j^{\theta }\equiv \left[v-D{\partial }_{\theta }\right]$$

$$⟨\dot{S}(t)⟩=\int \mathrm{d}\mathbf{y}\left[\frac{[j^{\theta }(\mathbf{y},t){]}^2}{Dp(\mathbf{y},t)}+\frac{j^i(\mathbf{y},t)[D^{-1}{]}_{ij}{j}^j(\mathbf{y},t)}{p(\mathbf{y},t)}\right]$$

$$⟨\dot{S}{⟩}_{\theta }=\frac{v^2}{D}+D{g}^{\theta }(\theta )+⟨\delta {\varphi }_{\alpha }{⟩}_{\theta }\left[v{\frac{\partial \lambda }{\partial \theta }}^{\alpha }(\theta )+D\frac{{\partial }^2{\lambda }^{\alpha }}{\partial {\theta }^2}(\theta )\right]$$

$$g^{\theta }(\theta )\equiv {\frac{\mathrm{d}\lambda }{\mathrm{d}\theta }}^{\alpha }{g}_{\alpha \beta }^{\lambda }(\lambda ){\frac{\mathrm{d}\lambda }{\mathrm{d}\theta }}^{\beta }\equiv {\frac{\mathrm{d}\lambda }{\mathrm{d}\theta }}^{\alpha }⟨\delta {\varphi }_{\alpha }\delta {\varphi }_{\beta }{⟩}_{\text{eq},\lambda }{\frac{\mathrm{d}\lambda }{\mathrm{d}\theta }}^{\beta }$$

$$⟨\delta {\varphi }_{\alpha }(t){⟩}_{\theta (\tau )}={\int }_{-\infty }^0\mathrm{d}{t}^{\prime }{C}_{\alpha \beta }^{\theta (t)}(t^{\prime })\frac{\mathrm{d}}{\mathrm{d}{t}^{\prime }}\left[{\lambda }^{\beta }(t+t^{\prime })\right]$$

$$⟨\delta {\varphi }_{\alpha }(t){⟩}_{\theta (\tau )}\approx {\frac{\mathrm{d}\lambda }{\mathrm{d}t}}^{\beta }{\tilde{g}}_{\alpha \beta }^{\lambda }{\tilde{g}}_{\alpha \beta }^{\lambda }(\theta )\equiv {\int }_{-\infty }^0\mathrm{d}{t}^{\prime }{C}_{\alpha \beta }^{\theta }(t^{\prime })$$

$${⟨\frac{\mathrm{d}{\lambda }^{\beta }}{\mathrm{d}{t}^{\pm }}⟩}_{{\theta }_0}\equiv \underset{t\to 0^{\pm }}{\lim }\frac{\mathrm{d}}{\mathrm{d}{t}^{\prime }}⟨{\lambda }^{\beta }(t_0+t^{\prime })|{\theta }_0,t_0⟩\approx v\frac{\mathrm{d}{\lambda }^{\beta }}{\mathrm{d}\theta }\pm D\frac{{\mathrm{d}}^2{\lambda }^{\beta }}{\mathrm{d}{\theta }^2}$$

$$⟨\delta \varphi {⟩}_{\theta }\approx {⟨\frac{\mathrm{d}\lambda }{\mathrm{d}{t}^{-}}⟩}_{\theta }{\tilde{g}}^{\lambda }\approx \left(v\frac{\mathrm{d}\lambda }{\mathrm{d}\theta }(\theta )-D\frac{{\mathrm{d}}^2\lambda }{\mathrm{d}{\theta }^2}(\theta )\right){\tilde{g}}^{\lambda }$$

$$v\tau /L\ll 1$$

$$U(\theta ,\mathbf{x})=\frac{1}{2}k(\mathbf{x}-A\mathbf{\lambda }{)}^2\mathbf{\lambda }=(\cos \theta ,\sin \theta )$$

$$⟨\dot{S}⟩=\frac{v^2}{D}(1+g^{\theta }D\tau )+D{g}^{\theta }(1-D\tau )$$

$$D^{(opt)}=\frac{v}{\sqrt{g^{\theta }}}(1+ϵ)$$

$$⟨S⟩\ge 2\mathcal{L}+\frac{{\tilde{\mathcal{L}}}^2}{\Delta t}\left(1-\frac{(1+ϵ{)}^2}{g^{\theta }}\right)$$

$$U(\mathbf{y})={\lambda }^{\alpha }(\theta ){\varphi }_{\alpha }(\mathbf{x})$$

$$\begin{array}{rlr}\dot{\theta }=v+\sqrt{2D}\eta & & {\dot{x}}^i=D^{ij}{F}_j(\mathbf{y})+\sqrt{2D^{ij}}{\xi }_j\\ ⟨\eta (t)\eta (t^{\prime })⟩=\delta (t-t^{\prime })& & ⟨{\xi }_i(t){\xi }_j(t^{\prime })⟩={\delta }_{ij}\delta (t-t^{\prime })\end{array}$$

$$\begin{array}{rl}j(\mathbf{y},t)& =\left[v-D{\partial }_{\theta }\right]p(\mathbf{y},t)\\ & =p(\mathbf{y},t)D\left[v\gamma -{\partial }_{\theta }\log p(\mathbf{y},t)\right]\\ j^i(\mathbf{y},t)& =D^{ij}\left[F_j(\mathbf{y})-{\partial }_j\right]p(\mathbf{y},t)\\ & =p(\mathbf{y},t)D^{ij}\left[F_j(\mathbf{y})-{\partial }_j\log p(\mathbf{y},t)\right]\end{array}$$

$$⟨\dot{S}(t)⟩$$

$$=\int \mathrm{d}\mathbf{y}\left[\frac{j^i(\mathbf{y},t){\gamma }_{ij}{j}^j(\mathbf{y},t)}{p(\mathbf{y},t)}+\frac{j(\mathbf{y},t)\gamma j(\mathbf{y},t)}{p(\mathbf{y},t)}\right]$$

$$\begin{array}{rl}⟨{\dot{S}}^{\theta }⟩& =\int \mathrm{d}\mathbf{y}\frac{j(\mathbf{y},t)\gamma j(\mathbf{y},t)}{p(\mathbf{y},t)}\\ & =\int \mathrm{d}\mathbf{y}\left[v\gamma -{\partial }_{\theta }\log p(\mathbf{y},t)\right]j(\mathbf{y},t)\\ & =v\gamma \int \mathrm{d}\mathbf{y}j(\mathbf{y},t)-\int \mathrm{d}\mathbf{y}j(\mathbf{y},t){\partial }_{\theta }\log p(\mathbf{y},t)\\ & =v^2\gamma -v\int \mathrm{d}\mathbf{y}p(\mathbf{y},t){\partial }_{\theta }\log p(\mathbf{y},t)\\ & -\int \mathrm{d}\mathbf{y}j(\mathbf{y},t){\partial }_{\theta }\log p(\mathbf{y},t)\end{array}$$

$$⟨{\dot{S}}^{\theta }⟩=\frac{v^2}{D}-\int \mathrm{d}\mathbf{y}j(\mathbf{y},t){\partial }_{\theta }\log p(\mathbf{y},t)$$

$$\begin{array}{rl}⟨{\dot{S}}^x⟩& =\int \mathrm{d}\mathbf{y}\frac{j^i(\mathbf{y},t){\gamma }_{ij}{j}^j(\mathbf{y},t)}{p(\mathbf{y},t)}\\ & =\int \mathrm{d}\mathbf{y}{j}^i(\mathbf{y},t)(F_i(\mathbf{y})-{\partial }_i\log p(\mathbf{y},t))\\ & =-\int \mathrm{d}\mathbf{y}{j}^i(\mathbf{y},t){\partial }_{i}U(\mathbf{y})\\ & -\int \mathrm{d}\mathbf{y}{j}^i(\mathbf{y},t){\partial }_i\log p(\mathbf{y},t)\end{array}$$

$$⟨\dot{S}⟩=\frac{v^2}{D}-\int \mathrm{d}\mathbf{y}\left[(j{\partial }_{\theta }+j^i{\partial }_i)\log p+j^i{\partial }_{i}U\right]$$

$$⟨\dot{S}⟩=\frac{v^2}{D}-\int \mathrm{d}\mathbf{y}\left[(j{\partial }_{\theta }+j^i{\partial }_i)\log p-j^i{\partial }_i\log p_{eq}(\mathbf{x}|\theta )\right]$$

$$⟨\dot{S}⟩=\frac{v^2}{D}+\int \mathrm{d}\mathbf{y}\left[({\partial }_{\theta }j+{\partial }_i{j}^i)\log p-({\partial }_i{j}^i\log p_{eq}(\mathbf{x}|\theta )\right]$$

$$⟨\dot{S}⟩=\frac{v^2}{D}+\int \mathrm{d}\mathbf{y}\left[{\partial }_{\theta }j(\mathbf{y})\right]\log p_{eq}(\mathbf{x}|\theta )$$

Full text

Controlling the state of a thermodynamic system requires interacting with and doing work on its degrees of freedom. Given an initial thermodynamic state $\lambda_{i}$ and a final state $\lambda_{f}$, the second law of thermodynamics implies that the average work required to produce the transition between these states is at least the difference in their free energy: $\langle W\rangle\geq F(\lambda_{f})-F(\lambda_{i})$. Recent work has strengthened this inequality, quantifying two distinct sources of energetic dissipation. First, in order to transform a system at a finite rate, energy must be expended to overcome thermodynamic friction Jarzynski (1997); Sivak and Crooks (2012a, b); Large et al. (2018); Burbea and Rao (1982); Crooks (1999); Kirkwood (1946). Second, energy must be expended to bias the motion of $\lambda$ itself in a preferred direction Cao et al. (2015); Feng and Crooks (2008); Brown and Sivak (2017); Parrondo et al. (2009); Costa et al. (2005); Zhang et al. (2019); Barato and Seifert (2016); Verley et al. (2014a). Here we consider both of these energetic costs together. We find that while either cost can be made arbitrarily small, they cannot be minimized simultaneously, as illustrated in Figure 1. This leads to a lower bound on the required energetic cost for controlling a thermodynamic system that remains non-zero even in the limit that the process moves infinitely slowly and imprecisely.

Any process that transforms a system in a finite amount of time must necessarily push the system out of equilibrium. This requires doing non-conservative work which causes the irreversible loss of usable energy, a familiar example of which is the heat dissipated while moving an object through a viscous medium. This dissipative cost, incurred by thermodynamic friction, can be written as: $\langle E_{\textnormal{diss}}\rangle\equiv\langle\Delta S\rangle=\langle W\rangle-\Delta F$, where the dissipation equals $\Delta S$, the change in the entropy of the system and environment111 Since $\Delta F=\Delta E-T\Delta S$ and since the energy of the universe $E$ is constant, changes in the total amount of free energy can be identified with changes in the entropy of the universe.  Seifert (2005). Recent work has explicitly quantified this form of dissipation, finding an elegant geometric interpretation for the non-conservative work done in a finite time process Sivak and Crooks (2012b, a).

In the above analysis, the energetic cost is identified with $\langle W\rangle$, the average work done on the thermodynamic system by an externally defined and deterministic control protocol $\lambda(t)$. However, for an autonomous system which implements its own control protocol, energy must be dissipated in order to move $\lambda$ itself in a directed manner. Physical systems obey microscopically time-reversible dynamics 222Though not all physical process are fully invariant under time reversal, the distinction is not important for out current understanding of how this type of directionality arises in in biological end engineered systems.. Thus to bias the motion of $\lambda$, breaking time-reversal symmetry, forward progress must be accompanied by an increase in entropy, which can be accomplished by dissipating energy into a heat bath. This entropic increase causes forward movement to be more likely than backwards movement, and without this, there can be no “arrow of time” Feng and Crooks (2008). A classic example of this is the consumption of ATP by myosin motors to “walk” along actin polymers Pollard and Korn (1973). Without the irreversible consumption of free energy, the system would be constrained to have equilibrium dynamics, transitioning backward and forward at the same rate. Recently, there has been considerable work done in understanding the tradeoffs between this bias cost and the accuracy and speed of system trajectories Verley et al. (2014b, c, a); Cao et al. (2015); Barato and Seifert (2016, 2017); Pietzonka et al. (2016); Zhang et al. (2019), culminating with the development of so-called thermodynamic uncertainty relations Gingrich et al. (2016); Barato and Seifert (2015); Horowitz and Gingrich (2017).

Here we consider the total dissipative cost of autonomously controlled thermodynamic systems, including contributions from both thermodynamic friction and entropic bias. In particular, we consider thermodynamic systems with state $\boldsymbol{{x}}$ and control parameter $\lambda$, where the dynamics of $\lambda$ are now stochastic due to the fact that the entropic bias required to run a deterministic path is infinite Crooks (1999). Additionally, we suppose that the dynamics of $\boldsymbol{{x}}$ depend on $\lambda$ but the dynamics of $\lambda$ do not receive feedback from $\boldsymbol{{x}}$. This one-way coupling distinguishes $\lambda$ as a physical control parameter rather than as simply a degree of freedom in a larger thermodynamic system. We can think of $\lambda$ as mediating a coupling between $\boldsymbol{{x}}$ and different thermodynamic baths, as in Ref. Machta (2015). This setup is analogous to a control knob changing the amount of power flowing into a thermodynamic system in that the work required to turn the knob333 Since the cost of turning a control knob doesn’t scale with the system size, for any macroscopic system one would naturally ignore this cost. is independent of the state of the system.

Examples of this class of processes are found throughout biology in systems that undergo regulated change. In these systems, energy is expended both in running the regulatory mechanism and in instantiating downstream changes Goldbeter (1996); Pomerening et al. (2003); Rodenfels et al. (2019). In muscles, the release of calcium modulates the pulling force by activating myosin heads which then hydrolyze ATP to produce a step Szent-Györgyi (1975); Eisenberg and Hill (1985). In this case, the bias cost is the energy used to control the amount of calcium that is released through concentration gradients. Energy is also dissipated through the non-conservative work that the muscles themselves do as they consume ATP to contract. Another example is biological cycles, where energy is used both to coordinate phase in the cycle and to run downstream processes. Cyclins, which underlie the cell cycle and KaiABC proteins which implement circadian oscillations, are kinases that phosphorylate each other to implement a cycle as well as downstream targets to instantiate the changes that must occur throughout the cycle Zon et al. (2007); Nakajima et al. (2005); Terauchi et al. (2007); Monti et al. (2018); Rust et al. (2007); Rodenfels et al. (2019); Zhang et al. (2019); Zwicker et al. (2010). Phosphorylating these downstream targets also consumes energy. In both of these examples, the control degrees of freedom (calcium concentration and cyclin phosphostate) receive minimal feedback from their downstream target’s state (contraction state of myosin heads and phosphostate of targets).

Our central result is that thermodynamic processes cannot be controlled in an energetically reversible manner. While each of these two energetic costs can be made small in isolation, they cannot be made small together. The cost of thermodynamic friction can only be minimized by moving the system slowly and precisely, while the bias cost can only be made small by moving imprecisely and with large fluctuations. These competing constraints cannot be satisfied at the same time (see Figure 1). We thus suggest that the notion of an energetically reversible control process exists only as an abstraction for when the control itself is considered external to the object under study. A realistic self-contained thermodynamic machine has no reversible transformations, even in the quasistatic limit.

Our work builds on a framework developed in Ref. Sivak and Crooks (2012b, a) which considered lower bounds on the energy dissipation arising from a deterministic control protocol $\boldsymbol{{\lambda}}(t)$ moving at finite speed. Those authors found that the dissipation rate is given by:

[TABLE]

where $\tilde{g}^{\lambda}$ is the Kirkwood friction tensor Kirkwood (1946) and where we have replaced the energy dissipation $E_{\textnormal{diss}}$ with the equivalent notion of entropy production Seifert (2005, 2012). This rate yields a lower bound on the energy dissipation:

[TABLE]

where $\tilde{\mathcal{L}}$ is the minimum length path in $\boldsymbol{{\lambda}}$ space between $\boldsymbol{{\lambda}}_{i}$ and $\boldsymbol{{\lambda}}_{j}$ with the metric $\tilde{g}^{\lambda}_{\alpha\beta}$ Crooks (2007) and where $\Delta t$ is the total time of the protocol. Energetically optimal control protocols are geodesics in the Riemannian thermodynamic space defined by $\tilde{g}^{\lambda}_{\alpha\beta}$.

Importantly, this result implies that in the quasistatic limit where the protocol moves infinitely slowly ($\Delta t\to\infty$), the excess energy cost of transforming the system becomes negligible (see eq.** 2**). However, in this framework, the breaking of time-reversal symmetry arises from the deterministic trajectory $\boldsymbol{{\lambda}}(t)$, whose cost is neglected from this energetic book-keeping.

When we include the cost of the time-reversal symmetry breaking in the control protocol $\boldsymbol{{\lambda}}(t)$, the optimal control protocols are no longer deterministic. Instead, we consider a protocol $\boldsymbol{{\lambda}}(\theta)$ where $\theta$ is a stochastic variable that moves with net drift velocity $v$ and diffusion $D$. When we include the bias cost of the control protocol itself, we find that the total rate of energy dissipation in the near-equilibrium limit is given by:

[TABLE]

where $g^{\theta}$ is the Fisher information metric on $\theta$ (see eq.** 10). The first term of eq. 3** captures the thermodynamic friction associated with pushing the system out of equilibrium. We will see that this term is the natural generalization of eq.** 1** to a stochastic system. The second term captures the dissipation associated with breaking time reversal symmetry, moving the control protocol forward in time. The third term is the result of the redundant thermodynamic friction caused by the stochastic system trajectory.

From this rate, we show that for a simple system, the total energetic cost of moving the system from $\boldsymbol{{\lambda}}_{i}$ to $\boldsymbol{{\lambda}}_{f}$ has a lower bound given by:

[TABLE]

where $\mathcal{L}$ and $\tilde{\mathcal{L}}$ are the lengths traversed in thermodynamic space with metrics $\tilde{g}$ and $g$ and where $\epsilon$ is a small correction that disappears as $\Delta t\to\infty$, or in the limit of large protocol size.

This result is significant because the first term defies the quasistatic limit, remaining non-zero even in the limit of an infinitely slow protocol ($\Delta t\to\infty$). The result we obtain from the simple system analyzed here suggests a more general principle: every thermodynamic transformation has a minimal energy requirement that cannot be made arbitrarily small by using a slow control protocol. This energy requirement is determined by the geometry of thermodynamic space.

Our result generalizes and clarifies a bound anticipated in Ref. Machta (2015), which considered coupling a thermodynamic system to a succession of particle reservoirs. Here we explicitly take a continuum limit, and consider protocols which move at finite rate, connecting this to other bounds in the literature Sivak and Crooks (2012b). Our results clarify that this bound is far more general, showing that the minimum energy bound is not specific to that class of systems but is rather a fundamental property of thermodynamic machines. In particular, it holds for any thermodynamic system in which a single degree of freedom obeying the microscopic laws of thermodynamics mediates a coupling to a generalized thermodynamic reservoir.

Derivation

Consider a system of particles interacting through an energy of the form:

[TABLE]

where $\boldsymbol{{x}}$ denotes the microstate of the system and $\theta$ parametrizes the path of the control functions around a cycle. Here we use greek letters $\alpha$,$\beta$,$\gamma$, etc to index the controlling potentials $\boldsymbol{{\lambda}}$ and their conjugate variables $\boldsymbol{{\phi}}$ and we use roman letters $i$, $j$, $k$ to index the microstate $\boldsymbol{{x}}$. We also use natural units and unit temperature such that $\beta=T=1$.

The control functions $\boldsymbol{{\lambda}}(\theta)$ are all $2\pi$ periodic in $\theta$, allowing the system to reach a steady state where the system cycles are fully self-contained. We suppose that the control parameter $\theta$ is driven stochastically with net drift velocity $v$ and diffusion constant $D$. We describe both $\theta$ and $\boldsymbol{{x}}$ using overdamped Langevin equations Seifert (2012):

[TABLE]

The terms $\eta$ and $\xi^{i}$ represent thermal noise. The overdamped force on $x^{i}$ due to the interaction energy $U$ is given by $F_{i}(\boldsymbol{{y}})=-\partial_{x^{i}}U(\boldsymbol{{y}})$. As discussed in the introduction, by control parameter we mean precisely that the term $F_{\theta}(\boldsymbol{{y}})=-\partial_{\theta}U(\boldsymbol{{y}})$ is absent from the stochastic equation for $\dot{\theta}$; the dynamics of $\theta$ don’t receive feedback from the state of the system.

The Langevin dynamics of the system give rise to probability currents:

[TABLE]

Using stochastic thermodynamics, we can derive the average rate of energy dissipation by equivalently calculating the average rate of total entropy production Seifert (2012):

[TABLE]

Because the system is periodic and the Langevin dynamics (eq.** 6) are time-independent, the system will eventually reach a steady-state that can be described by the joint probability distribution $p(\boldsymbol{{y}})$. To compute the energy dissipation, we first compute the average rate of energy dissipation conditioned on the the control parameter being in the known state $\theta$. We can then compute the total energy dissipation by averaging over a full cycle in $\theta$. Denote $p(\boldsymbol{{x}}|\theta)$ to be the non-equilibrium steady-state probability of finding the system in the microstate $\boldsymbol{{x}}$ given that the system is at $\theta$. Denote $\langle\cdots\rangle_{\theta}$ to mean an average over the non-equilibrium distribution $p(\boldsymbol{{x}}|\theta)$ for a fixed $\theta$. Finally, denote $\langle\cdots\rangle_{\theta,\textnormal{eq}}$ to mean an average over the equilibrium Boltzmann weight $p_{eq}(\boldsymbol{{x}}|\theta)\propto e^{-U(\theta,\boldsymbol{{x}})}$. We find the rate of entropy production when the system is at $\theta$ is given by (for the full details of the calculation see eq. 20-eq. 40** in the Supplement):

[TABLE]

where $\langle\delta\phi_{\alpha}\rangle_{\theta}=\langle\phi_{\alpha}\rangle_{\theta}-\langle\phi_{\alpha}\rangle_{eq,\theta}$ is the deviance of the conjugate force $\phi_{\alpha}$ at $\theta$ from its equilibrium value and and $g^{\theta}$ is the Fisher information metric with respect to the $\theta$ basis:

[TABLE]

The Fisher information metric can be thought of as a measure of how strongly the distribution $p_{\textnormal{eq}}(\boldsymbol{{x}}|\theta)$ changes as the value of $\theta$ changes as determined by the Kullback-Leibler divergence.

So far this is an exact result. In the next section we will use linear response to approximate $\langle\delta\phi_{\alpha}\rangle_{\theta}$, assuming that $D$ and $v$ are small.

Linear Response for Stochastic Control

Since we are interested in a lower energetic bound, we consider the low dissipation regime where the $\boldsymbol{{x}}$ system is driven near-equilibrium. In this regime, we can use linear response to compute the thermodynamic friction of the $\boldsymbol{{x}}$ system. To evaluate the term $\langle\delta\phi_{\alpha}\rangle_{\theta}$, we use a linear response approximation. For a fixed control parameter path $\theta(\tau)$, the average linear response over all possible microstate paths is well understood Sivak and Crooks (2012b). Here we extend this result to an average over all possible microstate and control parameter paths.

If an ensemble of systems all undergo the same deterministic protocol $\theta(\tau)$, then at time $t$, the average linear response of $\phi_{\alpha}$ over this ensemble is given by Zwanzig (2001):

[TABLE]

where $C^{\theta}_{\alpha\beta}(t^{\prime})=\langle\delta\phi_{\alpha}(0)\delta\phi_{\beta}(t^{\prime})\rangle_{eq,\,\theta}$ is the equilibrium time correlation between the conjugate forces $\phi_{\alpha}$ and $\phi_{\beta}$ and where we have written $\lambda(t)\equiv\lambda(\theta(t))$ for shorthand.

Assuming the protocol speed is much slower than the timescale of system relaxation, by integrating by parts we find Sivak and Crooks (2012b):

[TABLE]

We now have to extend this result to an ensemble of stochastic protocols which are all located at the same point $\theta_{0}$ at time $t_{0}$ (to emphasize that we are referring to a specific point, we have switched from using the generic $\theta$ and $t$ to $\theta_{0}$ and $t_{0}$). This process is a bit more challenging (see eq.** 41-eq. 55** in the Supplement for the exact details). However the form of the result is manifestly the same: we only need to replace $\mathrm{d}\lambda^{\beta}/\mathrm{d}t$ with the ensemble average of $\mathrm{d}\lambda^{\beta}/\mathrm{d}t$, where the ensemble is taken over all control parameter trajectories $\theta(\tau)$ such that $\theta(t_{0})=\theta_{0}$. As illustrated in Fig 2, this quantity is discontinuous at time $t_{0}$. Its left and right-sided limits are:

[TABLE]

which correspond to ensemble velocities under the reverse-Ito and Ito conventions Gardiner (2004). Here we have dropped terms quadratic in $v$ and $D$. Since the domain of eq.** 11** is $t^{\prime}<0$, it the left-sided limit that is relevant in our calculation, and thus we obtain:

[TABLE]

To obtain this result we have required that both the control parameter velocity and the diffusion rate are small with respect to the system relaxation timescale at equilibrium, $\tau$. Explicitly, we demand:

[TABLE]

where $L$ is related to the length scale associated with the control function $\lambda(\theta)$. Finally, by using the result given by equation 14 in equation 9, we obtain our central result, equation 3.

Discussion

The first term in eq.** 3** represents the frictional dissipation arising from pushing the system out of equilibrium. This term is a generalization of the dissipation rate found in the deterministically controlled coupled system studied in Ref. Sivak and Crooks (2012b). To obtain that result, the authors made the assumption that the system moves much slower than the relaxation timescale of the system. Concretely, this is the assumption that $v\tau/L\ll 1$. In moving from eq.** 11** to eq.** 14**, we make essentially the same assumption, but we also require that $D\tau/L^{2}\ll 1$. This is merely the statement that the random movements of the control parameter cannot be large enough to take us away from the linear response regime. Since we are searching for a lower energetic bound, this is certainly true in the limiting behavior. The system of Ref. Sivak and Crooks (2012b) can be seen as a special case where $D\to 0$ and the $v^{2}/D$ term is ignored.

The other two terms in eq.** 3** are analogous to dissipation terms found in an earlier paper Machta (2015) investigating a discrete system. The first of these, $v^{2}/D$, is the energy required to break time reversal symmetry in the control parameter, i.e. the energy required for “constantcy” in the control clock Pietzonka and Seifert (2018); Cao et al. (2015). The final term $Dg^{\theta}$ can be thought of the energetic cost of straying from the optimal protocol, a geodesic Crooks (2007).

Figure 1a plots the contribution to the total energy cost of a control protocol from the two sources of dissipation: the nonconservative work done by the control parameter on the $\boldsymbol{{x}}$ system ($W^{NC}=W^{on}-\Delta F$ shown in blue) and the work required to break time reversal symmetry and direct the movement of $\theta$ forward in time ($W^{TR}=\int\frac{v^{2}}{D}\mathrm{d}t$ shown in red). We see that they are minimized on opposite ends of the stochasticity spectrum.

In equation 3 there is an implicit energetic tradeoff. A control protocol that is very precise ($D\ll v$) (e.g. the red curve in Figure 1b) may minimize the dissipation due to thermodynamic friction done by the control parameter on the system, however, such a protocol pays a high energetic cost for strongly breaking time-reversal symmetry. On the other hand, a control protocol that only weakly breaks time-reversal symmetry ($D\gg v$) (e.g. the blue curve in Figure 1b), pays a high energetic cost for undergoing suboptimal trajectories and performing redundant thermodynamic transitions. Ref. Sivak and Crooks (2012b) investigated the energetically optimal control path $\lambda(t)$. This work shows that there is also the question of the energetically optimal “diffusive tuning” between $v$ and $D$ which minimizes this tradeoff (e.g. the purple mark and curve in Figure 1a, b). In particular, the optimal control protocol is not deterministic $(D\neq 0)$.

As an example, consider a two-dimensional harmonic oscillator where the center is moved in a circle of radius $A$:

[TABLE]

Here $\boldsymbol{{\lambda}}(\theta)=(\cos\theta,\sin\theta)$ is the control function and $\phi(\boldsymbol{{x}})=-kA\boldsymbol{{x}}$ is the conjugate force. The Fisher information metric is given by $g^{\lambda}=kA^{2}\boldsymbol{{I}}$ where $\boldsymbol{{I}}$ is the two-dimensional identity matrix and in the $\theta$ basis we have $g^{\theta}=\frac{\mathrm{d}\lambda^{x}}{\mathrm{d}\theta}g^{\lambda}_{xx}\frac{\mathrm{d}\lambda^{x}}{\mathrm{d}\theta}+\frac{\mathrm{d}\lambda^{y}}{\mathrm{d}\theta}g^{\lambda}_{yy}\frac{\mathrm{d}\lambda^{y}}{\mathrm{d}\theta}=kA^{2}$. Using eq.** 3**, we find the average dissipation rate:

[TABLE]

If we require that the protocol takes an average of $\Delta t$ time per cycle, this fixes the net drift velocity $v=2\pi/\Delta t$. To find the minimum dissipation, we then optimize $S$ with respect to $D$. This yields:

[TABLE]

where $|\epsilon|\ll 1$ is a small order correction due to the small $D\tau$ term. Plugging this in yields a total dissipation per cycle of:

[TABLE]

where $\mathcal{L}=2\pi\sqrt{g^{\theta}}$ and $\tilde{\mathcal{L}}=2\pi\sqrt{g^{\theta}\tau}$ are the thermodynamic lengths of paths under the metrics $g^{\lambda}$ and $\tilde{g}^{\lambda}$. In particular, we note that the dissipation remains bounded by $2\mathcal{L}$ in the limit of an infinitely long protocol $\Delta t\to\infty$.

In units where $\beta\neq 1$, the Fisher information metric is $g^{\theta}=kA^{2}\beta$. Thus the correction to the Sivak and Crooks bound can be neglected whenever the energy scale of the control is greater than the average thermal fluctuation.

The non-vanishing bound $2\mathcal{L}$ scales with $\sqrt{g^{\lambda}}$, the size of an average fluctuation in the system. Thus it is subextensive and disappears in the macroscopic limit. Its contribution is also dwarfed by thermodynamic friction when the control protocol is fast. Therefore it is expected that this bound should only become relevant in slow microscopic systems.

We also note that this analysis only applies to autonomous thermodynamic machines: those whose control is independent of environmental signals. In cases where the source of time-reversal symmetry breaking occurs externally, e.g. a system reacting to cyclic changes in heat from the sun, this bound does not necessarily apply, though other bounds likely do. This is because the origin of time-reversal symmetry breaking in such networks is environmental and thus no energy must be expended to drive the system in a particular direction. Such reactive systems would be more appropriately characterized by Ref. Large et al. (2018), which likewise addressed the question of optimality and energetic bounds in stochastically controlled systems, only without taking into account the cost of breaking time reversal symmetry. Those authors found a minimum bound on the energy of control which cannot be made arbitrarily small in the presence of a noisy control protocol. However, their bound is proportional to the magnitude of the control noise, and thus can be made arbitrarily small in the limit of noiseless protocols.

This work elucidates new constraints in the design of optimal thermodynamic machines. In previous studies, it has been found that systems must dissipate more energy to increase the accuracy of their output Barato and Seifert (2015); Horowitz and Gingrich (2017); Gingrich et al. (2016); Lan et al. (2012). The optimal diffusive tuning found here indicates that below a certain level of accuracy, increasing precision actually decreases the dissipation cost. In addition, the lower dissipation bound indicates that for very slow microscopic thermodynamic transformations, the dissipation cost no longer scales inversely with time. The diminishing energetic returns from increasing the length of control protocols perhaps sets a characteristic timescale for optimal microscopic machines without time constraints.

Acknowledgements

Thanks to Nicholas Read and Daniel Seara for useful discussions about our project. This work was supported in part by a Simons Investigator award in MMLS (B.B.M.) and by NSF DMR-1905621 (S.J.B. and B.B.M.).

Supplemental Material

Main Derivation

Here we derive the main results (eq. 5 to eq. 14 in the main text) in greater detail. The system’s energy is given by:

[TABLE]

Here $\boldsymbol{{\lambda}}$, $\boldsymbol{{\phi}}$ are indexed by $\alpha$,$\beta$,$\gamma$ and $\boldsymbol{{x}}$ is indexed by $i$,$j$,$k$. We use natural units and unit temperature so that $\beta=T=1$ and $\mu=D=\gamma^{-1}$, where $\mu$ is the mobility, $D$ is the diffusion, and $\gamma$ is the resistance. When $\mu$, $D$, $\gamma$ and things like probability currents $j$ appear without indices, they refer to $\theta$. In particular we use $\partial_{i}$ and $\partial_{j}$ as shorthand for $\partial_{x^{i}}$ and $\partial_{x^{j}}$. When they appear with roman indices ($i$,$j$,$k$), they refer to $\boldsymbol{{x}}$. The Langevin equations are:

[TABLE]

where $\eta$ and $\xi$ are white noise functions and $F_{j}(\boldsymbol{{y}})=-\partial_{j}U(\boldsymbol{{y}})$. We denote $p(\boldsymbol{{y}},t)$ to mean the probability of finding the system at state $\boldsymbol{{y}}=(\theta,\boldsymbol{{x}})$ at time $t$. The probability currents are given by:

[TABLE]

The total averaged entropy production rate is the sum of the production rate for the two individual variables:

[TABLE]

Starting with the $\theta$ contribution:

[TABLE]

Since $p(\boldsymbol{{y}},t)\partial_{\theta}\log p(\boldsymbol{{y}},t)=\partial_{\theta}p(\boldsymbol{{y}},t)$ and $\theta$ is periodic, after integrating over $\theta$, the middle term disappears leaving:

[TABLE]

Next we consider the $\boldsymbol{{x}}$ contribution:

[TABLE]

Giving us:

[TABLE]

We can rewrite $U=\mathcal{F}-\log p_{eq}$ where $\mathcal{F}(\theta)$ is the free energy for the system for a fixed $\theta$ and $p_{eq}(\boldsymbol{{x}}|\theta)=e^{\mathcal{F}(\theta)-U(\theta,\boldsymbol{{x}})}$ is the equilibrium Boltzmann distribution for fixed $\theta$. Then $\partial_{i}U=-\partial_{i}\log p_{eq}$ since $\mathcal{F}$ is a function of $\theta$ only:

[TABLE]

Now we may integrate by parts on each of the terms in the integral:

[TABLE]

where the boundary terms disappear because $p(\boldsymbol{{y}},t)$ goes to zero at the boundaries of $x$.

Now we assume that the system reaches a steady state so all time dependences drop out of the problem. In particular, the Fokker-Plank equation yields $\partial_{\theta}j(\boldsymbol{{y}})+\partial_{i}j^{i}(\boldsymbol{{y}})=-\partial_{t}p=0$, allowing us to remove the first term in the integral and swap $\partial_{i}j^{i}(\boldsymbol{{y}})=-\partial_{\theta}j(\boldsymbol{{y}})$ in the second:

[TABLE]

Next we expand $\partial_{\theta}j(\boldsymbol{{y}})=v\partial_{\theta}p(\boldsymbol{{y}})-D\partial_{\theta}^{2}p(\boldsymbol{{y}})$ and $\log p_{eq}(\boldsymbol{{x}}|\theta)=\mathcal{F}(\theta)-U(\boldsymbol{{y}})$, giving:

[TABLE]

Now we integrate by parts to move the $\partial_{\theta}$, $\partial_{\theta}^{2}$ to the other term. Since $\theta$ is periodic, we can always neglect the boundary terms:

[TABLE]

We now need to compute the derivatives of $U$ and $\mathcal{F}$ with respect to $\theta$. Recall we denote $\langle\cdots\rangle_{\theta}$ as an average over all microstates $\boldsymbol{{x}}$ with non-equilibrium weight $p(\boldsymbol{{x}}|\theta)$ for a fixed value of $\theta$. We also denote $\langle\cdots\rangle_{\textnormal{eq},\theta}$ to indicate an average over all $\boldsymbol{{x}}$ with equilibrium Boltzmann weight $p_{\textnormal{eq}}(\boldsymbol{{x}}|\theta)$ for a fixed value of $\theta$:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

where $g^{\lambda}_{\alpha\beta}=-\frac{\partial}{\partial\lambda^{\beta}}\langle\phi_{\alpha}\rangle_{\textnormal{eq},\theta}$ is the equilibrium Fisher information metric in the $\lambda$ basis. Plugging these in gives:

[TABLE]

Where $\delta\phi_{\alpha}=\phi_{\alpha}(\boldsymbol{{x}})-\langle\phi_{\alpha}\rangle_{\textnormal{eq},\theta}$ is the deviation of $\phi$ from its equilibrium value for fixed system state $\theta$. If we identify $g^{\theta}\equiv\frac{\partial\lambda^{\alpha}}{\partial\theta}g^{\lambda}_{\alpha\beta}\frac{\partial\lambda^{\beta}}{\partial\theta}$ as the Fisher information metric on $\theta$ inherited from $\lambda$, we get:

[TABLE]

To find the average dissipation rate when the system is at the point $\theta$, we average over all $\boldsymbol{{x}}$ with weight $p(x|\theta)$. This immediately yields the multidimensional analog of $(13)$:

[TABLE]

Again, this is an exact expression. Next we use linear response to approximate $\langle\delta\phi_{\alpha}\rangle_{\theta}$ assuming that $v$ and $D$ are small.

Linear Response Approximation

Our starting point is the following expression Zwanzig (2001) which gives the average linear response of $\phi$ at time $t_{0}$ to a specific control trajectory $\theta(\tau)$:

[TABLE]

with the multidimensional autocorrelation function for $\boldsymbol{{\phi}}$:

[TABLE]

This is the linear response function to a single path.

The expression we need is $\langle\delta\phi_{\alpha}(t_{0})\rangle_{\theta_{0}}$, the average of the above expression over all paths $\theta(\tau)$ such that $\theta(t_{0})=\theta_{0}$ for a specific point $(\theta_{0},t_{0})$. The important point is that only $\boldsymbol{{\lambda}}(t+t^{\prime})$ is trajectory dependent, the other parts of the expression only depend on the value of the trajectory at the moment $t_{0}$. Therefore we may write:

[TABLE]

where here $\langle\cdots|\theta_{0},t_{0}\rangle$ represents an average over all possible control paths $\theta(\tau)$ such that $\theta(t_{0})=\theta_{0}$ weighted by their probability as given by the Langevin dynamics of $\theta$. By integrating by parts we are left with:

[TABLE]

To compute this, we need an expression for the expectation value of $\lambda^{\beta}$ for this set of trajectories, which we may formally write as:

[TABLE]

Note that despite the fact that $\theta$ is periodic, the integration bounds here are not. This is because $\theta^{\prime}=2\pi$ in this context refers to the control parameter making a full cycle in time $t^{\prime}$ which is not the same as it not moving ($\theta^{\prime}=0$).

Since the stochasticity of $\theta$ is driven by Gaussian noise, it’s trivial to write down $p$:

[TABLE]

which is a Gaussian that diffuses away from a Dirac delta function as $|t^{\prime}|>0$ (see Fig 2 from the main text). We also Taylor expand $\lambda^{\beta}(\theta_{0}+\theta^{\prime})$ about $\theta_{0}$:

[TABLE]

Putting these together gives:

[TABLE]

Solving this integral for each value of $k$ will yield terms of the form:

[TABLE]

An underlying assumption of this work is that we are in the near-equilibrium regime where the perturbation on the system is weak. We treat $v$ and $D$ as small parameters and only keep terms of order $m+n=1$ in $v$, $D$:

[TABLE]

We can see that to leading order, the time-derivative of this expression will be time-independent. Thus, to leading order we can replace $\frac{\mathrm{d}}{\mathrm{d}t^{\prime}}\langle\lambda^{\beta}(t+t^{\prime})|\theta_{0},t_{0}\rangle$ with its value at $t^{\prime}=0$. However, due to the presence of the absolute value terms $|t^{\prime}|$, the time derivative at $t^{\prime}=0$ is discontinuous. This is more than a minor detail; it’s connected with the choice of whether to define derivatives in stochastic calculus using left sided limits or right sided limits analogous to choices in Riemann sums Gardiner (2004).

We introduce the following notation to distinguish between the left and right-sided limits of the time derivative of the conditional expectation value of $\lambda$:

[TABLE]

As one may expect, this is exactly the formula for the net drift of $\lambda^{\beta}$ at $t^{\prime}=0$ given by the Ito formula under the Ito ($+$) and reverse-Ito ($-$) conventions Gardiner (2004).

Plugging this result into eq.** 46** and using the definition of thermodynamic friction:

[TABLE]

yields the time-independent expression for the average linear response:

[TABLE]

which in turn gives an average entropy production rate for the system when it’s at the state $\theta_{0}$:

[TABLE]

What about the higher order terms in ${eq{.\bf~{}\ref{eq:sup_lambda_approx}}}$? We can neglect them under the assumption that the speed of control is small compared to the excitation timescale $\tau$ of the system at equilibrium. However, an explicit mathematical statement of this requirement is challenging because of the unknown form of $\lambda(\theta)$. Since $C$ is a decay function, we expect roughly that:

[TABLE]

Thus in keeping terms of order $m+n>1$ we would generate additional contributions to eq.** 55** of the form:

[TABLE]

Comparing these to the leading order terms $(D\tau)g^{\lambda}_{\alpha\beta}\partial_{\theta}^{2}\lambda^{\beta}$ and $(v\tau)g^{\lambda}_{\alpha\beta}\partial_{\theta}\lambda^{\beta}$, we can see that for a reasonably behaved function $\lambda(\theta)$ with a characteristic length scale $L$ we can summarize our assumption via the requirements:

[TABLE]

Essentially this is the requirement that $\left\langle\frac{\mathrm{d}\lambda^{\alpha}}{\mathrm{d}t}(t)\right\rangle$ (under the reverse-Ito convention) remains relatively constant over the system relaxation timescale $\tau$. This is the natural stochastic generalization of the constraint imposed by [3].

However, because we have not explicitly given a definition for $L$, this requirement is admittedly a little vague. One major complication in doing so arises from the fact that because we only care about the total dissipation bounds, we only want to keep the terms in eq.** 56** that contribute to the leading order behavior of the integral of eq.** 56. Thus, while it may be the case that for a specific point $\theta_{0}$, the second order term dominates: $(v\tau)^{2}\partial_{\theta}^{2}\lambda(\theta_{0})\gg(v\tau)\partial_{\theta}\lambda(\theta_{0})$, we still want to drop the higher order term because its contribution to the total integral is subleading. The actual formal constraints dictating when this approximation is appropriate is further complicated by the unconstrained behavior of $\tilde{g}(\theta)$ and $\lambda(\theta)$. However, it should be clear that as we approach equilibrium behavior ($D$, $v\to 0$), the kept terms dominate over the dropped terms. We feel that the constraint given in eq. 59** satisfactorily captures this idea.