Stochastic thermodynamics in the strong coupling regime: An unambiguous approach based on coarse-graining

TL;DR

This paper develops an unambiguous stochastic thermodynamics framework for strongly coupled systems by coarse-graining, valid when the auxiliary system evolves faster than the primary system, extending previous results to time-dependent couplings.

Contribution

It introduces a coarse-graining approach that unambiguously defines thermodynamic quantities in strongly coupled systems, including time-dependent interactions and instantaneous rates.

Findings

The framework matches strong coupling results in the fast auxiliary limit.

Numerical examples show previous approaches fail outside the fast auxiliary limit.

The method extends thermodynamic analysis to non-Markovian reservoirs with time-dependent couplings.

Abstract

We consider a classical and possibly driven composite system $X \otimes Y$ weakly coupled to a Markovian thermal reservoir $R$ so that an unambiguous stochastic thermodynamics ensues for $X \otimes Y$. This setup can be equivalently seen as a system $X$ strongly coupled to a non-Markovian reservoir $Y \otimes R$. We demonstrate that only in the limit where the dynamics of $Y$ is much faster then $X$, our unambiguous expressions for thermodynamic quantities such as heat, entropy or internal energy, are equivalent to the strong coupling expressions recently obtained in the literature using the Hamiltonian of mean force. By doing so, we also significantly extend these results by formulating them at the level of instantaneous rates and by allowing for time-dependent couplings between $X$ and its environment. Away from the limit where $Y$ evolves much faster than $X$, previous approaches fail to reproduce the correct results from the original unambiguous formulation, as we illustrate numerically for an underdamped Brownian particle coupled strongly to a non-Markovian reservoir.

Figures

Equations

$$E_{xy}({\lambda }_t)=E_x({\lambda }_t)+V_{xy}({\lambda }_t)+E_y.$$

$$d_t{p}_{xy}(t)=\sum _{x^{\prime },y^{\prime }}{R}_{xy,x^{\prime }{y}^{\prime }}({\lambda }_t)p_{x^{\prime }{y}^{\prime }}(t).$$

$$\frac{R_{xy,x^{\prime }{y}^{\prime }}({\lambda }_t)}{R_{x^{\prime }{y}^{\prime },xy}({\lambda }_t)}=e^{-\beta [E_{xy}({\lambda }_t)-E_{x^{\prime }{y}^{\prime }}({\lambda }_t)]},$$

$$U_{XY}(t)$$

$$\dot{W}(t)$$

$$\dot{Q}(t)$$

$$S_{XY}(t)$$

$$d_t{U}_{XY}(t)=\dot{W}(t)+\dot{Q}(t)$$

$$\dot{\Sigma }(t)\equiv d_t{S}_{XY}(t)-\beta \dot{Q}(t)\ge 0.$$

$$\dot{\Sigma }(t)$$

$$=\sum _{x,x^{\prime },y,y^{\prime }}{R}_{xy,x^{\prime }{y}^{\prime }}({\lambda }_t)p_{x^{\prime }{y}^{\prime }}(t)\ln \frac{R_{xy,x^{\prime }{y}^{\prime }}({\lambda }_t)p_{x^{\prime }{y}^{\prime }}(t)}{R_{x^{\prime }{y}^{\prime },xy}({\lambda }_t)p_{xy}(t)}$$

$$\dot{\Sigma }(t)=-{\partial }_t{|}_{\lambda (t)}D[p_{xy}(t)\Vert p_{xy}^{\text{eq}}({\lambda }_t)]\ge 0,$$

$$p_{xy}^{\text{eq}}({\lambda }_t)\equiv \frac{e^{-\beta E_{xy}({\lambda }_t)}}{{\mathcal{Z}}_{XY}({\lambda }_t)},$$

$$D[p_x\Vert q_x]\equiv \sum _x{p}_x(\ln p_x-\ln q_x)\ge 0,$$

$$\dot{\Sigma }(t)=\beta [\dot{W}(t)-d_t{F}_{XY}(t)]\ge 0.$$

$${\mathcal{F}}_{\alpha }({\lambda }_t)$$

$${\mathcal{U}}_{\alpha }({\lambda }_t)$$

$${\mathcal{S}}_{\alpha }({\lambda }_t)$$

$$p_{xy}(t)=p_{y|x}(t)p_x(t)$$

$$d_t{p}_x(t)=\sum _{x^{\prime }}{R}_{x,x^{\prime }}{p}_{x^{\prime }}(t)$$

$$R_{x,x^{\prime }}=R_{x,x^{\prime }}({\lambda }_t,t)=\sum _{y,y^{\prime }}{R}_{xy,x^{\prime }{y}^{\prime }}({\lambda }_t)p_{y^{\prime }|x^{\prime }}(t).$$

$${\dot{\Sigma }}^{(1)}(t)=\sum _{x,x^{\prime }}{R}_{x,x^{\prime }}{p}_{x^{\prime }}(t)\ln \frac{R_{x,x^{\prime }}{p}_{x^{\prime }}(t)}{R_{x^{\prime },x}{p}_x(t)}\ge 0,$$

$${\dot{\Sigma }}^{(1)}(t)=d_t{S}_{XY}(t)-\beta {\dot{Q}}^{(1)}(t)\ge 0.$$

$${\dot{Q}}^{(1)}(t)\equiv -\frac{1}{\beta }\sum _{x,x^{\prime }}{R}_{x,x^{\prime }}{p}_{x^{\prime }}(t)\ln \frac{R_{x,x^{\prime }}}{R_{x^{\prime },x}}+\frac{1}{\beta }{d}_t{S}_{Y|X}(t)$$

$$S_{Y|X}(t)=-\sum _x{p}_x(t)\sum _y{p}_{y|x}(t)\ln p_{y|x}(t),$$

$$R_{xy,x{y}^{\prime }}\gg R_{xy,x^{\prime }{y}^{\prime }}$$

$${\bar{p}}_{y|x}={\bar{p}}_{y|x}({\lambda }_t)=e^{-\beta (E_{xy}-F_{Y|x})}.$$

$$F_{Y|x}$$

$$=E_x-{\beta }^{-1}\ln {⟨e^{-\beta V_{xy}}⟩}_Y^{\text{eq}}+{\mathcal{F}}_Y,$$

$$\frac{{\bar{R}}_{x,x^{\prime }}}{{\bar{R}}_{x^{\prime },x}}=e^{-\beta (F_{Y|x}-F_{Y|x^{\prime }})},$$

$${\dot{Q}}^{(1)}(t)=\sum _{x,x^{\prime }}{F}_{Y|x}{\bar{R}}_{x,x^{\prime }}{p}_{x^{\prime }}(t)+\frac{1}{\beta }{d}_t{S}_{Y|X}(t),$$

$$\dot{Q}(t)={\dot{Q}}^{(1)}(t).$$

$$\begin{array}{rl}\frac{1}{\beta }{d}_t{S}_{Y|X}(t)=& -\frac{1}{\beta }\sum _{x,y}[d_t{p}_x(t){\bar{p}}_{y|x}({\lambda }_t)]\ln {\bar{p}}_{y|x}({\lambda }_t)\\ =& \sum _{x,y}(E_{xy}-F_{Y|x})[d_t{p}_x(t){\bar{p}}_{y|x}({\lambda }_t)],\end{array}$$

$$\begin{array}{rl}& \frac{1}{\beta }{d}_t{S}_{Y|X}(t)\\ & =\sum _{x,y}{E}_{xy}{d}_t[{\bar{p}}_{y|x}({\lambda }_t)p_x(t)]+\sum _x{F}_{Y|x}{d}_t{p}_x(t).\end{array}$$

Full text

I Introduction

Establishing the laws of thermodynamics for a given setup is not only beneficial for practical purposes, but also provides an important consistency check for the validity of the model and provides much deeper insights into the structure of the problem. Yet, establishing these laws for small-scale systems away from the well-established weak-coupling and Markovian limit can be very challenging.

This paper focuses on the case of a small, driven classical system in strong contact with a single environment. This case has attracted a lot of attention recently and was mostly tackled by introducing a Hamiltonian of mean force (HMF), classically Jarzynski (2004); Gelin and Thoss (2009); Seifert (2016); Talkner and Hänggi (2016); Philbin and Anders (2016); Jarzynski (2017); Miller and Anders (2017) as well as quantum-mechanically Gelin and Thoss (2009); Campisi et al. (2009a, b); Hilt et al. (2011); Pucci et al. (2013); Philbin and Anders (2016). However, the question of what exactly are the correct definitions for heat, internal energy and other quantities causes already controversies at the classical level Gelin and Thoss (2009); Seifert (2016); Talkner and Hänggi (2016); Jarzynski (2017).

We present an enlightening perspective on this problem by considering two coupled systems $X\otimes Y$ which are in weak contact with a large thermal reservoir $R$ and obey standard stochastic thermodynamics. By realizing that the system $X$ can be strongly coupled to the system $Y$, we see that the situation is equivalent to a system $X$ in strong contact with an environment $E=Y\otimes R$. In fact, for many relevant scenarios it makes sense that the system $X$ only couples strongly to a subpart $Y\subset E$ of the environment, but not to each degree of freedom of $E$. This picture is also supported by our example at the end of the paper.

The benefit of our approach is that we start from a well-defined thermodynamics with unambiguous definitions and we can then compare under which conditions previous approaches based on the HMF coincide with them. Our framework can be seen as an applications of the laws of thermodynamics under coarse graining as detailed in Ref. Esposito (2012), also see Refs. Seifert (2011); Bo and Celani (2014).

*Outline: * We start by presenting the thermodynamic description of the combined system $X\otimes Y$ in contact with $R$ in Sec. II. We show what changes if we coarse-grain $Y$ and consider the important limit where $Y$ evolves much faster than $X$ such that it can be adiabatically eliminated and a closed thermodynamic description for $X$ alone emerges. In Sec. III we turn the situation around and start from a description of $X$ coupled to $E$ and derive an exact inequality. Based on this we recapitulate the thermodynamics previously established using the HMF and we show that we are able to rederive and greatly extend these results if $Y$ evolves much faster than $X$. Beyond that we explicitly quantify the difference in the two proposed definitions of entropy production and we use an example in Sec. IV to illustrate generic features in the thermodynamics of strongly coupled systems demonstrating that knowledge of the HMF alone does not suffice to reproduce the original thermodynamics. It also provides a strategy to identify a system $Y$ if the initial setup is described at the level of $X\otimes E$. Sec. V summarizes the main implications of our findings.

II Setup

II.1 Basic quantities

A sketch of the setup is shown in Fig. 1. Two systems $X$ and $Y$ interact with each other and their joint Hamiltonian is assumed to be of the form

[TABLE]

Here, the energy $E_{x}(\lambda_{t})$ of system $X$ as well as the interaction energy $V_{xy}(\lambda_{t})$ can be time-dependent due to some externally controlled parameters $\lambda_{t}$. The energy $E_{y}$ of system $Y$ is assumed to be time-independent.

Since the joint system $X\otimes Y$ is weakly coupled to a large thermal reservoir at inverse temperature $\beta$, we assume that its dynamics can be modeled by a Markovian master equation (ME) of the form

[TABLE]

Here, $p_{xy}(t)$ denotes the probability to find the system in state $xy$ at time $t$. The rate matrix obeys $\sum_{x,y}R_{xy,x^{\prime}y^{\prime}}(\lambda_{t})=0$ which ensures conservation of probability [$\sum_{x,y}p_{xy}(t)=1$]. Furthermore, we assume local detailed balance

[TABLE]

which allows a physical interpretation of the ME and especially a consistent thermodynamic description. The rest in this section then follows standard stochastic thermodynamics Schnakenberg (1976); Seifert (2012); Van den Broeck and Esposito (2015).

For this purpose we introduce the following quantities:

[TABLE]

Here, we have denoted the ensemble average with respect to any solution $p_{xy}(t)$ of the ME above by ${\left\langle{f_{xy}(t)}\right\rangle}\equiv\sum_{x,y}p_{xy}(t)f_{xy}(t)$. Furthermore, the thermodynamic entropy $S$ coincides in the weak coupling regime with the definition of Shannon entropy.

Based on the definitions above, it is straightforward to derive the first law:

[TABLE]

(we define heat and work positive if they increase the energy of $X\otimes Y$). Furthermore, the second law states that the overall entropy production rate is postive:

[TABLE]

Its positity can be proven by noting the identity

[TABLE]

and using $-\ln x\geq 1-x$. Another useful identity is

[TABLE]

where the partial derivative $\partial_{t}|_{\lambda(t)}$ indicates that the change of $D[p_{xy}(t)\|p_{xy}^{\text{eq}}(\lambda_{t})]$ is evaluated at fixed $\lambda_{t}$. Here, we introduced the equilibrium (Gibbs, thermal) state

[TABLE]

which depends parametrically on time and ${\cal{Z}}_{XY}(\lambda_{t})$ denotes the equilibrium partition function. Also the concept of relative entropy,

[TABLE]

which is always positive for any two probability distributions $p_{x}$ and $q_{x}$, will be used later on.

Finally, let us introduce the concept of a non-equilibrium free energy $F_{XY}(t)=U_{XY}(t)-\beta^{-1}S_{XY}(t)$, which is defined for any state $p_{xy}(t)$. Using this, we can reformulate the second law as

[TABLE]

Whenever a system $\alpha$ (where $\alpha$ could stand for $XY,X,E,\dots$ depending on the situation) is at equilibrium, it is useful to note the relations

[TABLE]

for the equilibrium free energy, internal energy and entropy. Note that we use calligraphic letters ${\cal{F}},{\cal{Z}},{\cal{U}},{\cal{S}}$ to denote thermodynamic quantities at equilibrium.

Below, to keep a compact notation, we will often omit the dependence on $\lambda_{t}$ in the notation.

II.2 Coarse-graining

We now shift our attention to system $X$ alone and mostly follow Ref. Esposito (2012) for the rest of this section. For this purpose we split the joint probability into a conditional and marginal probability as

[TABLE]

with $p_{x}(t)=\sum_{y}p_{xy}(t)$ and $\sum_{y}p_{y|x}(t)=1$. It is not hard to deduce that $p_{x}(t)$ evolves according to the ME

[TABLE]

where the new effective rate matrix is given by

[TABLE]

In general, it depends explicitly on time due to the time-dependence of $p_{y^{\prime}|x^{\prime}}(t)$ and solving (19) is equally hard as solving the original ME unless further assumptions are made.

Nevertheless, there is an apparent second law related to the reduced dynamics of $X$ Esposito (2012):

[TABLE]

which can be rewritten as

[TABLE]

Here, we introduced the apparent heat flow

[TABLE]

and $S_{Y|X}$ denotes the conditional Shannon entropy

[TABLE]

which fulfills $S_{XY}=S_{X}+S_{Y|X}$ with $S_{X}\equiv-\sum_{x}p_{x}(t)\ln p_{x}(t)$. Unfortunately, at this general level there is no relation between $\dot{Q}^{(1)}$ and the real heat flow $\dot{Q}$ making it hard to establish a local version of the first law. Furthermore, note that $\dot{\Sigma}^{(1)}$ always underestimates the true entropy production ${\dot{\Sigma}\geq\dot{\Sigma}^{(1)}\geq 0}$.

II.3 Time-scale separation

There is an important limit, in which $Y$ evolves much faster than $X$ and can be adiabatically eliminated. We will refer to this as time-scale separation (TSS). Within TSS one assumes that

[TABLE]

for $x\neq x^{\prime}$. To get a simple description we also assume that for each given $x$ all states $y$ are connected, i.e., $R_{xy,xy^{\prime}}\neq 0$ for all $y,y^{\prime}$.

Under these conditions one can show that the conditional probabilities $p_{y|x}(t)$ equilibrate and can be written as111Of course, there are also alternative parametrizations possible. For instance, $\bar{p}_{y|x}=e^{-\beta[E_{y}+V_{xy}(\lambda_{t})-F^{\prime}_{Y|x}(\lambda_{t})]}$ with $F^{\prime}_{Y|x}(\lambda_{t})=F_{Y|x}(\lambda_{t})-E_{x}(\lambda_{t})$. This does not affect the resulting thermodynamics at the end.

[TABLE]

Normalization is ensured by choosing

[TABLE]

where ${\left\langle{\dots}\right\rangle}_{Y}^{\text{eq}}$ denotes the ensemble average with respect to $p_{y}^{\text{eq}}=e^{-\beta E_{y}}/{\cal{Z}}_{Y}$. Note that $F_{Y|x}=F_{Y|x}(\lambda_{t})$ depends parametrically on time. In contrast, the equilibrium free energy ${\cal{F}}_{Y}$ has no time-dependence. Although it appears in the definition of $F_{Y|x}$, we remark that the reduced state of $Y$ is not given by the equilibrium state $p_{y}^{\text{eq}}$.

Within TSS we denote the rate matrix $R_{x,x^{\prime}}$ by $\bar{R}_{x,x^{\prime}}=\bar{R}_{x,x^{\prime}}(\lambda_{t})$, which now depends only parametrically on time and greatly simplifies the solution of Eq. (19). Furthermore, it fulfills an effective local detailed balance relation of the form

[TABLE]

which makes the meaning of $F_{Y|x}$ as a free energy landscape for system $X$ transparent. Using this, we can express the apparent heat flow (23) as

[TABLE]

which now coincides with the real heat flow:

[TABLE]

To prove Eq. (30) it is useful to note that

[TABLE]

where we used (26). Then, after realizing that ${\sum_{y}\bar{p}_{y|x}(\lambda_{t})=1}$ and $\sum_{y}d_{t}\bar{p}_{y|x}(\lambda_{t})=0$, we get

[TABLE]

Plugging this result into Eq. (29), we finally obtain

[TABLE]

which equals our original definition (6) within TSS.

Furthermore, it makes sense to rewrite the internal energy as $U_{XY}(t)=\sum_{x}U_{XY|x}p_{x}(t)$ where we introduced the average internal energy conditioned on the state $x$:

[TABLE]

Formally, the first law remains the same as before

[TABLE]

In contrast to the general case, however, the time-dependence of all quantities comes only from the dynamical time-dependence of the system $X$ alone and the parametric dependence on $\lambda_{t}$. The same observation holds true for the second law of thermodynamics which can be expressed as

[TABLE]

Thus, within TSS we have indeed $\dot{\Sigma}=\dot{\Sigma}^{(1)}$. For later purposes it will be also convenient to note the following two identities

[TABLE]

which look remarkably similar to Eqs. (16) and (17). Proving them follows from straightforward though tidious algebraic manipulations, which we will not display here.

Finally, we briefly mention how to extend the results above to the stochastic level following the well-established procedure Seifert (2012); Van den Broeck and Esposito (2015). This will also underline the fact that any information about $Y$ enters only statically in the description. If the system starts at time $t_{0}$ in state $x_{0}$, jumps at time $t_{1}>t_{0}$ to $x_{1}$ and stays in that state until it jumps at time $t_{2}>t_{1}$ to $x_{2}$, etc., we denote this trajectory by $\textbf{x}_{t}\equiv(x_{0},t_{0};x_{1},t_{1};\dots)$. Then, the fluctuating internal energy at each instant $t$ is given by

[TABLE]

The work along the trajectory $\textbf{x}_{t}$ becomes

[TABLE]

and the stochastic entropy is defined as

[TABLE]

Finally, the heat can be decomposed as

[TABLE]

where the sum indexed by $j$ runs over all jumps which have happened from $t_{0}$ to $t$. Since $p_{x}(t)$ obeys a Markovian ME with rates that fulfill the local detailed balance relation (28), it is clear that the integral and detailed fluctuation theorems are also obeyed, e.g.,

[TABLE]

where $\Sigma[\textbf{x}_{t}]=S_{XY}[x_{t}]-S_{XY}[x_{0}]-\beta Q[\textbf{x}_{t}]$ and ${\left\langle{{\left\langle{\dots}\right\rangle}}\right\rangle}$ denotes an ensemble average over all trajectries $\textbf{x}_{t}$.

III The Hamiltonian of mean force

III.1 Exact identities

In this section we turn the situation around and consider a system $X$ in contact with an environment $E$ as shown on the right hand side of Fig. 1 and we only use in Sec. III.2 the decomposition $E=Y\otimes R$. We assume that the combined system $X\otimes E$ is isolated and obeys an Hamiltonian dynamics with an Hamiltonian

[TABLE]

where $e$ denotes a microstate of the environment. Note that we will in general denote thermodynamic quantities in this section by a “tilde” to distinguish them from previously introduced quantities. Their relation will be clarified in Sec. III.2.

As in Ref. Seifert (2016); Talkner and Hänggi (2016), we assume that the initial state of $X\otimes E$ reads

[TABLE]

where $p_{x}(0)$ is an arbitrary initial system state and $\bar{p}_{e|x}(\lambda_{0})$ denotes the equilibrium state of $E$ conditioned on a microstate $x$ of the system. Clearly, $F_{E|x}$ is defined as in Eq. (27) with $Y$ replaced by $E$. The state $\bar{p}_{e|x}$ can be more elegantly expressed by introducing the HMF,

[TABLE]

which has been successfully used for a long time in thermostatics Kirkwood (1935); Roux and Simonson (1999). Using this, we find

[TABLE]

and also the important relation

[TABLE]

Given the initial state (45), we follow Ref. Esposito et al. (2010) and define an entropy production

[TABLE]

which measures the deviation of the true state $p_{xe}(t)$ from an idealized reference state $p_{x}(t)\bar{p}_{e|x}(\lambda_{t})$. Note that definition (49) differs from Ref. Esposito et al. (2010) only in the choice of the reference state We discuss in Appendix A how both are related. We will now show that Eq. (49) coincides with the definition used in Ref. Seifert (2016) as was independently and simultaneously noted in Ref. Miller and Anders (2017).

It is convenient to rewrite Eq. (49) as

[TABLE]

where we used that the Shannon entropy of the global system $X\otimes E$ remains constant under Hamiltonian dynamics, $S_{XE}(t)=S_{XE}(0)$. Furthermore, we use the notation $\Delta f(t)\equiv f(t)-f(0)$ for any time-dependent function $f(t)$. Now, in accordance with phenomenological nonequilibrium thermodynamics, we would like to split $\tilde{\Sigma}(t)$ into two parts:

[TABLE]

Without additional information, there is obviously no unique splitting of these two quantities at this formal level, which essentially translates the results of Ref. Talkner and Hänggi (2016) into our framework. For the moment we will use the following definitions, which comply with the suggestions of Ref. Seifert (2016):

[TABLE]

The latter can be also rewritten as

[TABLE]

if we use the generally accepted definition for work $W(t)={\left\langle{E_{xe}(\lambda_{t})}\right\rangle}(t)-{\left\langle{E_{xe}(\lambda_{0})}\right\rangle}(0)$. Assuming the first law of thermodynamics to be valid in the strong coupling case, this then implies a definition for internal energy:

[TABLE]

Introducing the non-equilibrium free energy

[TABLE]

we can alternatively write Eq. (51) as

[TABLE]

The definitions above of $\tilde{\Sigma}$, $\tilde{S}_{X}$, $\tilde{Q}$, $\tilde{U}_{X}$ and $\tilde{F}_{X}$ seem to provide a satisfactory extension of thermodynamics to the strong-coupling case and they coincide with the definitions used by Seifert, who further motivates them by arguments of equilibrium statistical mechanics Seifert (2016). In addition to Ref. Seifert (2016), we have seen that the framework can be even extended by allowing for a time-dependence in the coupling $V_{xe}(t)$, too.

Nevertheless, the approach above should be taken with care because it is ambiguous Talkner and Hänggi (2016), and is not formulated at the level of instantaneous rates implying that the positivity of entropy production (51) crucially relies on the choice of initial state (45).

III.2 Reduced thermodynamics description in $X\otimes Y$

We now clarify this situation by returning to our previous results in Sec II.3. where we assumed that the environment is made of two parts, $E=Y\otimes R$. The first part $Y$ is strongly coupled to the system $X$ and is explicitly described. The second part $R$ is an ideal weakly coupled and Markovian thermal reservoir. Under these assumptions the ME (2) and the full Hamiltonian dynamics give rise to the same description in the reduced space $X\otimes Y$. This implies, e.g., that the work computed within the ME framework [see Eq. (5)] coincides with the work computed using the exact Hamiltonian dynamics as in Sec. III.1.

In the limit of TSS, $Y$ instantaneously equilibrates with respect to a given microstate of $X$. Thus, the initial requirement (45) is not only fulfilled initially but at any time $t$. This implicitly means that for any fixed value of $\lambda_{t}$, the global equilibrium steady state reads

[TABLE]

where $E_{r}$ is the bare Hamiltonian of $R$. As a result, the HMF introduced in Eq. (46) coincides with

[TABLE]

which can be regarded as the HMF of $X\otimes Y$ only. Using the last equation together with (37) and (38), it is not hard to deduce the following two relations:

[TABLE]

Thus, apart from a time-independent additive constant the definitions for internal energy and system entropy within TSS coincide with the definitions based on the HMF. Furthermore, since both approaches agree on the definition of work, we can show for the heat flow that

[TABLE]

Thus, within TSS we agree on this definition too and are able to derive the first law at the level of instantaneous rates. Likewise, we can also prove the positivity of the entropy production rate by noting that

[TABLE]

As a preliminary summary we have thus shown that within TSS, the framework introduced in Ref. Seifert (2016) is thermodynamically consistent and can be greatly extended. Furthermore, no ambiguity is left within our approach which allows us to refute the criticism raised in Ref. Talkner and Hänggi (2016) for our setup.

It is interesting to ask what happens away from TSS when $Y$ does not instantaneously conditionally equilibrate and $p_{y|x}$ is thus dynamically evolving. It is then possible to show that the framework of Sec. III.1 does not coincide with the original thermodynamic description of Sec. II anymore. For instance, we prove in Appendix B that the difference in entropy production can be expressed as

[TABLE]

Thus, $\tilde{\Sigma}(t)$ overestimates $\Sigma(t)=\int_{0}^{t}ds\dot{\Sigma}(s)$ by the relative entropy between the true state of $X\otimes Y$ and an idealized state of the form (45). Also, the rate of change of $\tilde{\Sigma}(t)$ can be negative. This and other features are explicitly demonstrated in the next section with the help of an example where the ME description in $X\otimes Y$ exactly coincides with the reduced Hamiltonian dynamics.

To conclude, when it is possible to separate out the strongly coupled and non-Markovian degrees of freedom $Y$ from the environment $E$, then the following hierarchy of inequalities holds,

[TABLE]

The equality $\tilde{\Sigma}(t)=\Sigma(t)=\Sigma^{(1)}(t)$ holds in the limit of TSS. Each of the entropy production in Eq. (65) corresponds to a different layer of the description. $\Sigma^{(1)}(t)$ assumes $Y$ to be conditionally (versus $X$) equilibrated and, of course, implicitly $R$ to be equilibrated. $\Sigma(t)$ assumes only an ideal reservoir $R$ and $\tilde{\Sigma}(t)$ is an exact result which can be applied to any Hamiltonian dynamics (44) as long as the initial condition is of the form (45).

IV Discrepancy in the non-Markovian regime

Within TSS, i.e., whenever the environment behaves Markovian by instantaneously adapting to the microstate of the system $X$, we have proven the equivalence of the coarse-grained thermodynamic framework from Sec. II.3 with the approach based on the HMF. In principle, both frameworks can be also applied beyond TSS and we will now provide a counterexample showing that the HMF-approach then no longer coincides with the standard framework of Sec. II.

We consider the example of driven Brownian motion thereby demonstrating that our main results above do not only hold for dynamics on discrete states but also for continuous variables. The global Hamiltonian with mass-weighted coordinates is specified by Weiss (2008); Sekimoto (2010)

[TABLE]

and we identify $\lambda_{t}=t$ in the following and use $\omega(t)=\omega_{0}+g\sin(\omega_{L}t)$. We relaxed the notation meaning with $E_{x}(t)$ the energy associated to the microstate $(x,p_{x})$ and a microstate $e$ of the bath is given by specifying $(x_{k},p_{k})$ for all $k$. Furthermore, the spectral density (SD) of the bath is defined as and parametrized by

[TABLE]

Here, $\lambda_{0}$ controls the overall coupling strength between the system and the environment and $\gamma$ changes the shape of the SD from a pronounced peak around $\omega_{1}$ for small $\gamma$ to a rather unstructured and flat SD for large $\gamma$.

The corresponding Langevin equation for this setup reads Weiss (2008); Sekimoto (2010)

[TABLE]

with the friction kernel

[TABLE]

and the noise $\xi(t)$, which obeys the statistics

[TABLE]

We see that for an Ohmic SD $J(\omega)=\eta\omega$ (times a high-frequency cutoff as usual) we obtain $\Gamma(t)=2\eta\delta(t)$ and this gives the standard Langevin equation with Gaussian white noise. Unfortunately, our SD (69) is not Ohmic unless we scale $\lambda_{0}=\sqrt{\alpha_{1}\alpha_{2}\gamma}$, $\omega_{1}=\sqrt{\alpha_{2}\gamma}$ and send $\gamma\rightarrow\infty$. This implies an Ohmic SD for sufficiently large $\alpha_{2}$:

[TABLE]

Establishing a consistent thermodynamic framework for the general Langevin Eq. (70) cannot be done using standard tools from stochastic thermodynamics. One route, however, could be to take the definitions from Sec. III and to apply them here. Application of these definitions is facilitated by the fact that for a Brownian motion Hamiltonian the HMF coincides with the bare system Hamiltonian, i.e., $E_{x}^{*}=E_{x}$, which can be directly checked by evaluating the Gaussian integrals. Thus, the change in internal energy and system entropy read $\Delta\tilde{U}_{X}={\left\langle{E_{x}(t)}\right\rangle}(t)-{\left\langle{E_{x}(0)}\right\rangle}(0)$ and $\Delta\tilde{S}_{X}={\left\langle{-\ln p_{x}(t)}\right\rangle}(t)-{\left\langle{-\ln p_{x}(0)}\right\rangle}(0)$. That is to say, the HMF-approach uses for this examples the standard weak-coupling definitions irrespective of the spectral properties of the bath. Furthermore, since work can be computed using $W(t)=\int_{0}^{t}ds{\left\langle{d_{t}E_{x}(s)}\right\rangle}(s)$, we obtain $\tilde{Q}$ and $\tilde{\Sigma}$, too. However, to access the dynamics of the system, we would have to simulate the non-Markovian Langevin equation (70), which is numerically demanding.

We therefore follow a different strategy and identify a subsystem $Y\subset E$, which transforms the non-Markovian system $X$ to a Markovian system $X\otimes Y$. This is most conveniently done by identifying a collective degree of freedom in the environment defined via

[TABLE]

In this context, $y$ is also known as a reaction coordinate. It has been shown to successfully model the dynamics of non-Markovian open quantum systems (see, e.g., Refs. Prior et al. (2010); Martinazzo et al. (2011); Woods et al. (2014); Iles-Smith et al. (2014)) and has been recently proposed as a method to establish a consistent thermodynamic framework beyond the Markovian and weak-coupling approximation Strasberg et al. (2016); Newman et al. (2017).

We skip the details of the derivation, which can be looked up in the literature Prior et al. (2010); Martinazzo et al. (2011); Woods et al. (2014); Iles-Smith et al. (2014); Strasberg et al. (2016); Newman et al. (2017), and only state the main result. After the transformation, the Hamiltonian becomes

[TABLE]

where the new SD of the “residual environment” $R$ is defined and for the choice (69) given by

[TABLE]

This SD immediately yields the coupled set of Markovian Langevin equations

[TABLE]

with Gaussian white noise $\xi(t)$.

Following standard procedures Sekimoto (2010), we can associate a Fokker-Planck equation for the probability distribution $P(t)=P(x,p_{x},y,p_{y};t)$ to the set of Langevin equations above. It reads

[TABLE]

where we defined $\nabla\equiv(\partial_{x},\partial_{p_{x}},\partial_{y},\partial_{p_{y}})^{T}$, ${\textbf{x}\equiv(x,p_{x},y,p_{y})^{T}}$ and introduced the matrices

[TABLE]

and $B$ whose only non-zero component is $B_{44}=2\gamma/\beta$. We emphasize that Eq. (80) describes the exact dynamics in $X\otimes Y$. No approximation has been made in any of the steps above (apart from assuming an initially equilibrated reservoir state).

An advantage of this Fokker-Planck equation is that the dynamics of the first and second cumulants are closed. In fact, the equations of motion for the first cumulants ${\left\langle{z}\right\rangle}$ (with $z\in\{x,p_{x},y,p_{y}\}$) couple only to themselves and the same is true for the second cumulants $C_{zz^{\prime}}\equiv{\left\langle{zz^{\prime}}\right\rangle}-{\left\langle{z}\right\rangle}{\left\langle{z^{\prime}}\right\rangle}$. Thus, an initially Gaussian state will stay Gaussian for all times. Computing the time-evolution of the first two cumulants based on an initial condition of the form (45) can then be easily done numerically. Because standard stochastic thermodynamics applies to Eqs. (79) or (80), we have direct access to averaged thermodynamic quantities for $X\otimes Y$ introduced in Sec. II, also see Ref. Sekimoto (2010). Furthermore, because we have the exact dynamics in $X\otimes Y$, we also get the exact reduced dynamics of $X$ by tracing over $Y$, consequently giving direct access to the time evolution of $\tilde{U}_{X},\tilde{S}_{X},\tilde{Q}$ and $\tilde{\Sigma}$. Thus, our Markovian embedding strategy has allowed us to circumvent the difficulty to simulate the non-Markovian Langevin equation (70).

Results of the simulation are shown in Fig. 2. We vary the SD from a strongly non-Markovian situation (shown on the left) to a Markovian but strong-coupling situation (on the right) by changing $\gamma$ and $\alpha_{1}$. For each $\gamma$ we compare the integrated heat flows $Q$ and $\tilde{Q}$ (upper panel) and the integrated entropy production $\Sigma$ and $\tilde{\Sigma}$ (lower panel). The following main features are observable: for large $\gamma$ the assumption of TSS is justified and quantities defined in Sec. II.3 and III agree perfectly. In fact, $\gamma$ is directly linked to the rate of relaxation of the reaction coordinate $(y,p_{y})$, but does not directly couple to the system degrees of freedom $(x,p_{x})$. Thus, a large $\gamma$ corresponds to the limit of TSS as introduced in Eq. (25). Away from that limit, however, we observe that $\tilde{Q}$ differs significantly from $Q$ and the same observation is true for the different definitions of entropy production, too. Also, although Eq. (51) is always obeyed, the rate of $\tilde{\Sigma}$ can become negative. Furthermore, we can also confirm the validity of Eq. (64), $\tilde{\Sigma}(t)\geq\Sigma(t)$.

V Summary

We clarified important questions in the framework of strong-coupling thermodynamics. Our main achievements are the following:

1) Justification of the HMF within TSS. Within the limit of TSS, the framework provided in Ref. Seifert (2016) is thermodynamically consistent for arbitrary system states $p_{x}(t)$ and the HMF is a legitimate tool to investigate the thermodynamics of systems in strong contact with a single environment.

*2) No ambiguity. * Any ambiguity is removed in our framework and the concerns put forward in Ref. Talkner and Hänggi (2016) do not apply. The reason for this is that we start from a well-defined weak coupling framework. Especially and contrary to previous attempts, we do not use the first law to define heat, but have an alternative and unambiguous definition for it.

*3) Extension of previous results. * Thanks to the TSS, we were able to significantly extend previous results by formulating them at the level of instantaneous rates instead of integrated quantities and by allowing also for a time-dependence in the system-environment coupling.

*4) Difficulties in the non-Markovian regime. * Away from TSS, the framework of Ref. Seifert (2016) does not match the original thermodynamic picture though Eq. (49) is always obeyed. Thus, we observe that in order to establish the original laws of thermodynamics in the non-Markovian regime (where the environment is also dynamically evolving), one is forced to fully take into account the (thermo)dynamics of $X$ and $Y$. Any effective description at this stage will in general miss important pieces in the first or second law. This complies with the point of view put forward in Ref. Strasberg et al. (2016); Newman et al. (2017).

Recently, two alternative approaches were put forward in Ref. Jarzynski (2017) by starting from the isothermal-isobaric ensemble and by taking pressure and volume effects into account. These approaches correctly reproduce the macroscopic limit by introducing the notion of “thermodynamic volume” for a microscopic system. The “bare representation” in Ref. Jarzynski (2017) shows that it is possible to retain the original weak coupling definitions of internal energy and entropy by shifting our attention to enthalphy and Gibbs free energy instead. If the isobaric $PV$-contribution is absent or blindly ignored, then the “partial molar representation” in Ref. Jarzynski (2017) coincides with the approach in Sec. III.222To compare notation, we have without $PV$-terms that the Gibbs free energies in Ref. Jarzynski (2017) are related to our free energies via $G_{0}^{{\cal{E}}}={\cal{F}}_{E}$, $G_{x}^{{\cal{E}}}=F_{E|x}-E_{x}$ and the solvation Hamiltonian of mean force becomes $\phi(x)=E_{x}^{*}-E_{x}$. However, note that the $PV$-term in Ref. Jarzynski (2017) is actually only negligible at weak coupling. Then, $E_{x}^{*}\approx E_{x}$ and all the different frameworks coincide with the weak coupling limit. These alternative approaches should be therefore also derivable within TSS, but we expect that outside the limit of TSS they will mismatch again.

Finally, we would like to mention that the framework of Sec. III can be used in principle also beyond TSS, for instance, if it is impossible to find a splitting $E=Y\otimes R$ or if the dynamical simulation of the environment becomes unfeasible. It then nevertheless has to be treated with care and further consistency checks still need to be carried out such as, for instance, the implication of the correct thermodynamic laws in the limit of reversible transformations as investigated in Ref. Esposito et al. (2015).

*Acknowledgements. * Financial support by the National Research Fund Luxembourg (project FNR/A11/02) and by the European Research Council (project 681456) is acknowledged.

Appendix A Relation between the entropy production in Eq. (49) and Ref. Esposito et al. (2010)

In Ref. Esposito et al. (2010), the entropy production of an Hamiltonian dynamics (44) with an initial condition of the form $p_{x}(0)p_{e}^{\text{eq}}$ was defined as

[TABLE]

This result is very close in spirit to the entropy production (49) that we derived in this paper for the same Hamiltonian dynamics but with an initial condition of the form (45). It measures the deviation of the true state from an idealized product state where the environment is always at equilibrium instead of conditionally at equilibrium.

The only meaningful comparison between the two expressions requires to consider situations where the two classes of initial conditions coincide, namely when $V_{xe}(\lambda_{0})=0$ and the interaction is only turned on afterwards. In this case, we find that

[TABLE]

This relation can be rewritten as a difference between the non-equilibrium free energy (56) and the non-equilibrium free energy corresponding to the scheme of Ref. Esposito et al. (2010)

[TABLE]

Explicitly,

[TABLE]

In general, there is no bound for this difference.

Appendix B Proof of Eq. (64)

From Eq. (14) we deduce that $\Sigma(t)=\beta[W(t)-\Delta F_{XY}]$. Thus, with Eq. (57) we immediately get the first line of Eq. (64),

[TABLE]

To prove the second, we look at $F_{XY}(t)-\tilde{F}_{X}(t)$ and $F_{XY}(0)-\tilde{F}_{X}(0)$ in detail. Since the formalism using the HMF in Sec. III assumes that the environment starts in a conditionally equilibrated state $p_{xy}(0)=p_{x}(0)\bar{p}_{y|x}(\lambda_{0})$, Eqs. (60) and (61) are valid at $t=0$. Straightforward algebra then gives

[TABLE]

At later times, using the definitions (52) and (55), we find that

[TABLE]

Next, from Eq. (59) together with (26) we obtain

[TABLE]

Thus, we have explicitly

[TABLE]

and consequently,

[TABLE]

which proves the second line of Eq. (64).