Repeated interactions and quantum stochastic thermodynamics at strong coupling

TL;DR

This paper develops a generalized thermodynamic framework for open quantum systems under repeated interactions and measurements, enabling experimental tests and insights into quantum stochastic processes, causal models, and the effects of strong coupling.

Contribution

It extends quantum thermodynamics to include arbitrary measurements and strong coupling, providing a consistent framework for quantum stochastic processes and causal models.

Findings

Framework applicable to strong coupling regimes

Enables experimental testing of quantum thermodynamics

Links non-Markovianity with nonequilibrium thermodynamics

Abstract

The thermodynamic framework of repeated interactions is generalized to an arbitrary open quantum system in contact with a heat bath. Based on these findings the theory is then extended to arbitrary measurements performed on the system. This constitutes a direct experimentally testable framework in strong coupling quantum thermodynamics. By construction, it provides many quantum stochastic processes and quantum causal models with a consistent thermodynamic interpretation. The setting can be further used, for instance, to rigorously investigate the interplay between non-Markovianity and nonequilibrium thermodynamics.

Equations

$$H_{\text{tot}}({\lambda }_t)=H_{SB}({\lambda }_t)+\sum _{k=0}^n{V}_{SU(k)}({\lambda }_t).$$

$${\rho }_{\text{tot}}(t_0^{-})={\pi }_{SB}({\lambda }_0)\otimes {\rho }_{U(0)}\otimes \cdots \otimes {\rho }_{U(n)}.$$

$${\pi }_X^{\ast }\equiv \text{mbox}{tr}_B\{{\pi }_{XB}\}\equiv \frac{e^{-\beta H_X^{\ast }}}{{\mathcal{Z}}_X^{\ast }},{\mathcal{Z}}_X^{\ast }\equiv \frac{{\mathcal{Z}}_{XB}}{{\mathcal{Z}}_B}.$$

$${\dot{W}}_S(t)$$

$${\dot{W}}_{SU(k)}(t)$$

$$W(t)$$

$$=⟨H_{\text{tot}}({\lambda }_t)⟩(t)-⟨H_{\text{tot}}({\lambda }_0^{-})⟩(t_0^{-}).$$

$$F_X(t)\equiv \text{mbox}{tr}_X\{{\rho }_X(t)[H_X^{\ast }({\lambda }_t)+{\beta }^{-1}\ln {\rho }_X(t)]\}.$$

$$\Sigma (t)\equiv \beta [W(t)-\Delta F_{SU(\text{n})}(t)]\ge 0.$$

$$\Sigma (t)=D[{\rho }_{\text{tot}}(t)\Vert {\pi }_{\text{tot}}({\lambda }_t)]-D[{\rho }_{SU(\text{n})}(t)\Vert {\pi }_{SU(\text{n})}^{\ast }({\lambda }_t)],$$

$${\Sigma }_S(t_{n+1}^{-})$$

$$\ge \Sigma (t_{n+1}^{-})\ge 0.$$

$$\begin{array}{rl}{\Sigma }_S& (t_{n+1}^{-})-{\Sigma }_S(t_n^{-})=\\ & \beta [W(t_{n+1}^{-})-W(t_n^{-})-F_S(t_{n+1}^{-})+F_S(t_n^{-})]\\ & +S[{\rho }_{U(n)}(t_{n+1}^{-})]-S[{\rho }_{U(n)}(t_n^{-})].\end{array}$$

$${\tilde{\rho }}_{\text{tot}}({\text{r}}_n,t_{n+1}^{-})=\left(\prod _{k=0}^n{\mathcal{P}}_{r_k}{\mathcal{U}}_{k+1,k}\right){\rho }_{\text{tot}}(t_0^{-}).$$

$${\tilde{\rho }}_{\text{tot}}({\text{r}}_n,t_{n+1}^{-})=\left(\prod _{k=0}^n{\mathcal{P}}_{r_k}\right){\rho }_{\text{tot}}(t_{n+1}^{-}),$$

$$\sum _{{\text{r}}_n}{\tilde{\rho }}_{SB}({\text{r}}_n,t_{n+1}^{-})={\rho }_{SB}(t_{n+1}^{-}),$$

$$\begin{array}{rl}{f}_{SU(\text{n})}({\text{r}}_n,t)\equiv & ⟨H_{SU(\text{n})}^{\ast }({\lambda }_t)⟩({\text{r}}_n,t)\\ & +T\left\{\ln p({\text{r}}_n)-S[{\rho }_{SU(\text{n})}({\text{r}}_n,t)]\right\},\end{array}$$

$$\sigma ({\text{r}}_n,t)\equiv \beta [w({\text{r}}_n,t)-\Delta f_{SU(\text{n})}({\text{r}}_n,t)].$$

$$\sum _{{\text{r}}_n}p({\text{r}}_n)\sigma ({\text{r}}_n,t)-\Sigma (t)=$$

$$\sum _{{\text{r}}_n}p({\text{r}}_n)\left\{S[{\rho }_{SU(\text{n})}({\text{r}}_n,t)]-\ln p({\text{r}}_n)\right\}-S[{\rho }_{SU(\text{n})}(t)].$$

$$\sum _{{\text{r}}_n}p({\text{r}}_n)\sigma ({\text{r}}_n,t)\ge \Sigma (t)\ge 0.$$

$${\tilde{\rho }}_{SB}({\text{r}}_n,t_{n+1}^{-})=\left(\prod _{k=0}^n{\mathcal{U}}_{k+1,k}^{SB}{\mathcal{A}}_{r_k}\right){\rho }_{SB}(t_0^{-}).$$

$$\begin{array}{rl}{\Sigma }_S(t_2)-{\Sigma }_S(t_1)& =-\beta [F_S(t_2)-F_S(t_1)]\\ & =D[{\rho }_S(t_1)\Vert {\pi }_S^{\ast }]-D[{\rho }_S(t_2)\Vert {\pi }_S^{\ast }].\end{array}$$

$$\rho (t_n^{+}|{\mathcal{A}}_n,\dots ,{\mathcal{A}}_1,{\mathcal{A}}_0)={\mathcal{A}}_n\Lambda (t_n,t_{n-1})\dots {\mathcal{A}}_1\Lambda (t_1,t_0){\mathcal{A}}_0{\rho }_{\mathcal{S}}(t_0^{-}).$$

$$\Lambda (t_k,t_0){\pi }_S^{\ast }={\pi }_S^{\ast }$$

$$D[{\rho }_S(t_1)\Vert {\pi }_S^{\ast }]-D[{\rho }_S(t_2)\Vert {\pi }_S^{\ast }]=D[{\rho }_S(t_1)\Vert {\pi }_S^{\ast }]-D[\Lambda (t_2,t_1){\rho }_S(t_1)\Vert \Lambda (t_2,t_1){\pi }_S^{\ast }].$$

$$E_{SU(\text{n})}^{\ast }(t)$$

$$Q_{SU(\text{n})}(t)$$

$$S_{SU(\text{n})}(t)$$

$$\Sigma (t)=\beta [W(t)-\Delta F_{SU(\text{n})}(t)]=\Delta S_{SU(\text{n})}(t)-\beta Q_{SU(\text{n})}(t)\ge 0.$$

$$e_{SU(\text{n})}^{\ast }({\text{r}}_n,t)$$

$$q_{SU(\text{n})}({\text{r}}_n,t)$$

$$s_{SU(\text{n})}({\text{r}}_n,t)$$

$$\sigma ({\text{r}}_n,t)=\beta [w({\text{r}}_n,t)-\Delta f_{SU(\text{n})}({\text{r}}_n,t)]=\Delta s_{SU(\text{n})}({\text{r}}_n,t)-\beta q_{SU(\text{n})}({\text{r}}_n,t).$$

Full text

Introduction.— Formulating the laws of quantum thermodynamics forces us to rethink many assumptions, which are traditionally taken for granted. In particular, small systems are dominated by fluctuations and in general they do not interact weakly with a Markovian heat bath. Also the desire to monitor and manipulate quantum systems adds another layer of complexity due to the non-trivial effect of quantum measurements.

In this Letter we present a unified thermodynamic framework, which overcomes the assumption of a weakly coupled, Markovian heat bath and which allows to include nonequilibrium resources and quantum measurements. These nonequilibrium resources are a set of small, externally prepared systems – called ‘units’ in the following – which are sequentially put into contact with the system under study. This setup is known as the ‘repeated interaction framework’ or ‘collisional model’ and it has recently attracted much attention in quantum thermodynamics Bruneau et al. (2010); Horowitz (2012); Horowitz and Parrondo (2013); Barra (2015); Uzdin et al. (2016); Pezzutto et al. (2016); Strasberg et al. (2017); Benoist et al. (2018); Manzano et al. (2018); Chiara et al. (2018); Cresser (2019); Seah et al. (2019); Bäumer et al. (2019). However, the coupling to an additional external heat bath (typically present in an experiment) was mostly ignored, a weakly coupled Markovian one was only treated in Refs. Bruneau et al. (2010); Strasberg et al. (2017); Cresser (2019). Based on recent progress in strong coupling thermodynamics Seifert (2016); Strasberg and Esposito (2019), we will show that even the assumption of a weakly coupled macroscopic heat bath can be completely overcome.

Afterwards, following the operational approach to quantum stochastic thermodynamics Strasberg (2019); Strasberg and Winter (2019), we will show how to explicitly take into account measurements into the thermodynamic description. This constitutes a crucial step in strong coupling quantum thermodynamics where different strategies were used to arrive at many interesting conclusions Campisi et al. (2009); Esposito et al. (2010); Takara et al. (2010); Schaller et al. (2013); Gallego et al. (2014); Esposito et al. (2015a, b); Gelbwaser-Klimovsky and Aspuru-Guzik (2015); Strasberg et al. (2016); Bruch et al. (2016); Katz and Kosloff (2016); Ludovico et al. (2016); Newman et al. (2017); Mu et al. (2017); Ludovico et al. (2018); Schaller et al. (2018); Strasberg et al. (2018); Perarnau-Llobet et al. (2018); Bruch et al. (2018); Whitney (2018); Restrepo et al. (2018); Dou et al. (2018); Schaller and A (2018); Guarnieri et al. (2019). However, all strategies rely on a formalism without any explicit measurements, thus making them hard to test and compare 111Some exceptions rely on projective measurements of the bath degrees of freedom Campisi et al. (2009); Schaller et al. (2013). There is currently no technology which allows to carry out such measurements. Another notable exception is Ref. Ludovico et al. (2018). . In contrast, our theory is in principle immediately testable in a lab as it only requires to measure the system. Finally, we rigorously connect our thermodynamic framework to the field of quantum non-Markovianity.

Setting.— We start by considering a system $S$ coupled to a bath $B$ described by the Hamiltonian $H_{SB}(\lambda_{t})=H_{S}(\lambda_{t})+V_{SB}+H_{B}$, where $\lambda_{t}$ denotes an externally specified driving protocol (e.g., a laser field) and $V_{SB}$ denotes the system-bath interaction Hamiltonian. To this setup we add the framework of repeated interactions specified by the following global Hamiltonian:

[TABLE]

Here, $V_{SU(k)}(\lambda_{t})$ describes the time-dependent coupling between the system and unit $U(k)$, $k\in\{0,\dots,n\}$, which is designed in such a way that at most one unit interacts with the system at a given time. Specifically, if we denote the interaction interval between the system and the $k$’th unit by $I_{k}\equiv[t_{k},t_{k+1})$, then $V_{SU(k)}(\lambda_{t})=0$ for all $t\notin I_{k}$. Within $I_{k}$ the time-dependence as specified by $\lambda_{t}$ is arbitrary. Furthermore, we temporarily assume the bare unit Hamiltonian to be degenerate, i.e., $H_{U(k)}\sim 1_{U(k)}$. The problem is completely specified by fixing the global initial state, which is assumed to be of the form

[TABLE]

Here and in general we use the notation $t^{\pm}$ to denote the time $t\pm\epsilon$ in the limit where $\epsilon>0$ becomes immeasurably small. Furthermore, $\pi_{X}=e^{-\beta H_{X}}/{\cal{Z}}_{X}$ denotes the equilibrium Gibbs state of some system $X$ at inverse temperature $\beta$ (perhaps depending on the value of some driving protocol). Finally, the initial state of the units is arbitrary but uncorrelated. A sketch of the present setup is shown in Fig. 1. We remark that various extensions are possible as discussed at the end of this Letter.

Below we will need the notion of the ‘Hamiltonian of mean force’, an old concept Kirkwood (1935) (see also Refs. Seifert (2016); Strasberg and Esposito (2019); Campisi et al. (2009)), which is defined via the reduced equilibrium state of a bipartite system $XB$. Specifically,

[TABLE]

Note that $\pi_{X}^{*}\neq\pi_{X}$ in general. In addition, $H_{X}^{*}$ depends on the inverse temperature $\beta$ and (possibly) a control parameter. Notice that the Hamiltonian of mean force for the system and all units simplifies as at any given time at most one unit is physically coupled to the system, e.g., for $t\in I_{k}$, $H_{SU(\textbf{n})}^{*}(\lambda_{t})=H_{SU(k)}^{*}(\lambda_{t})$. Here and in general we use $U(\textbf{n})$ to denote the entire sequence of units from $U(0)$ to $U(n)$.

The average rate of injected work (the power) has two contributions. For $t\in I_{k}$ we define

[TABLE]

where ${\left\langle{\dots}\right\rangle}(t)$ denotes a quantum statistical average at time $t$. It follows that the total mechanical work performed on the system up to time $t$ is

[TABLE]

Note that this definition of average mechanical work is widely accepted even in the strong coupling regime Strasberg and Esposito (2019); Esposito et al. (2010); Takara et al. (2010); Esposito et al. (2015a, b); Strasberg et al. (2016); Ludovico et al. (2016); Strasberg et al. (2018); Perarnau-Llobet et al. (2018); Whitney (2018); Dou et al. (2018) as it is directly related to the change in internal energy of the universe (i.e., the system, the bath and all units all together).

Strong coupling repeated interactions framework.— We start by introducing the basic concept of a nonequilibrium free energy adapted to the strong coupling regime Seifert (2016); Strasberg and Esposito (2019),

[TABLE]

In the weak coupling limit, where $H_{X}^{*}(\lambda_{t})\approx H_{X}(\lambda_{t})$, this definition reduces to the conventional one. The slight modification allows us to express the second law even at strong coupling and even in presence of the system-unit interactions in the conventional way ($k_{B}\equiv 1$):

[TABLE]

Here, $\Sigma(t)$ denotes the entropy production and $\Delta F_{SU(\textbf{n})}(t)\equiv F_{SU(\textbf{n})}(t)-F_{SU(\textbf{n})}(t_{0}^{-})$. Positivity of the second law follows by confirming that

[TABLE]

where $D[\rho\|\sigma]=\mbox{tr}\{\rho(\ln\rho-\ln\sigma)\}\geq 0$ is the quantum relative entropy. Hence, $\Sigma(t)$ is positive by monotonicity of relative entropy Uhlmann (1977); Ohya and Petz (1993). The derivation uses only Eq. (2) and the unitary dynamics, which implies for the von Neumann entropy $S[\rho_{\text{tot}}(0)]\equiv-\mbox{tr}\{\rho_{\text{tot}}(0)\ln\rho_{\text{tot}}(0)\}=S[\rho_{\text{tot}}(t)]$. It is lengthy but straightforward and hence not displayed here.

Equation (8) corresponds to the second law if we regard the system and all units as one big system and explicitly keep their correlations in the description. In practice it often turns out that keeping the information about all units and all their correlations is superfluous (compare also with the discussion in Ref. Strasberg et al. (2017)). Thus, at time $t_{n+1}^{-}$ after the $n$’th interaction but before the $(n+1)$’th interaction, where the system is decoupled from all units, the following second law is practically more meaningful,

[TABLE]

Here, we added a subscript $S$ to indicate that this is the entropy production from the system point of view ignoring superfluous information about the units. To arrive at Eq. (10), we used subadditivity of entropy and $H_{U(k)}\sim 1_{U(k)}$. In contrast to Eq. (8) it contains only the change in the marginal von Neumann entropy of the units. The entropy production per interaction interval $I_{n}$ is then given by

[TABLE]

It is the strong coupling generalization of the second law in the repeated interaction framework, see Eq. (49) in Ref. Strasberg et al. (2017). Interestingly, in contrast to the Markovian weak coupling situation, we cannot ensure the positivity of this expression. This is similar to the classical case Strasberg and Esposito (2019) and we will connect it to the notion of non-Markovianity later on. But first we will advance conceptually by introducing explicit measurements in the description.

Quantum stochastic thermodynamics at strong coupling.— We consider the case where the experimenter measures the state of the unit after the interaction with the system as indicated in Fig. 1 (i.e., $V_{SU(n)}(\lambda_{t})=0$ at the time $t$ of the measurement). By doing so, she can gather valuable information about the state of the system. In a moment we will also show that this allows her to implement arbitrary generalized measurements on the system and that the resulting theory can be fruitfully linked to the study of quantum stochastic processes and quantum causal models.

Mathematically, we denote the measurement result of the $k$’th unit by $r_{k}$ and associate a positive operator $P_{r_{k}}$ to it, which fulfills the normalization condition $\sum_{r_{k}}P_{r_{k}}^{2}=1_{U(k)}$. The state of the unit then changes according to the map ${\cal{P}}_{r_{k}}\rho_{U(k)}\equiv P_{r_{k}}\rho_{U(k)}P_{r_{k}}\equiv\tilde{\rho}_{U(k)}(r_{k})$. Notice that $\tilde{\rho}_{U(k)}(r_{k})$ is a subnormalized state with the probability $p(r_{k})=\mbox{tr}_{U(k)}\{\tilde{\rho}_{U(k)}(r_{k})\}$ as its norm. After multiple units were subjected to their respective measurements giving results $\textbf{r}_{n}\equiv(r_{n},\dots,r_{1},r_{0})$, the global subnormalized state reads

[TABLE]

Here, ${\cal{U}}_{k+1,k}$ describes the global unitary evolution from $t_{k}^{-}$ (shortly before the $k$’th unit starts interacting with the system) to $t_{k+1}^{-}$. Due to the fact that the measurement always acts after the interaction we can also write

[TABLE]

where $\rho_{\text{tot}}(t_{n+1}^{-})={\cal{U}}_{n+1,0}\rho_{\text{tot}}(t_{0}^{-})$ is the global time-evolved state without any measurements. This allows us to confirm the useful relation

[TABLE]

i.e., the average system-bath state does not change due to the measurements. This is not true for the units.

Inspired by Ref. Strasberg (2019), we now introduce the following thermodynamic definitions along a single trajectory characterized by the measurement results $\textbf{r}_{n}$. First, for $t\in I_{n}$ the stochastic power $\dot{w}_{S}(\textbf{r}_{n},t)$ and $\dot{w}_{SU(n)}(\textbf{r}_{n-1},t)$ are simply obtained from Eqs. (4) and (5) by replacing the average over the unconditional state $\rho_{SU(\textbf{n})}(t)$ with an average over the conditional state $\rho_{SU(\textbf{n})}(\textbf{r}_{n},t)$. Notice that $\dot{w}_{SU(n)}(\textbf{r}_{n-1},t)$ does not depend on the last measurement outcome $r_{n}$ because by construction the measurement ${\cal{P}}_{r_{n}}$ acts after the $n$th unit has interacted with the system. Therefore, together with Eq. (14) we immediately obtain the relations $\sum_{\textbf{r}_{n}}p(\textbf{r}_{n})\dot{w}_{S}(\textbf{r}_{n},t)=\dot{W}_{S}(t)$ and $\sum_{\textbf{r}_{n}}p(\textbf{r}_{n})\dot{w}_{SU(n)}(\textbf{r}_{n-1},t)=\dot{W}_{SU(n)}(t)$. Second, we have to generalize the nonequilibrium free energy to the stochastic case, which becomes

[TABLE]

where ${\left\langle{\dots}\right\rangle}(\textbf{r}_{n},t)$ denotes an average with respect to the conditional state. An essential difference compared to definition (7) is the appearence of the stochastic entropy $-\ln p(\textbf{r}_{n})$ associated to the measurement results obtained with probability $p(\textbf{r}_{n})=\mbox{tr}\{\tilde{\rho}_{\text{tot}}(\textbf{r}_{n},t)\}$. A similar but not identical construction is used in classical stochastic thermodynamics Seifert (2005), compare with the discussion in Refs. Strasberg (2019); Strasberg and Winter (2019). In contrast to the stochastic work, we have in general $\sum_{\textbf{r}_{n}}p(\textbf{r}_{n})f_{SU(\textbf{n})}(\textbf{r}_{n},t)\neq F_{SU(\textbf{n})}(t)$. Finally, we introduce the stochastic entropy production

[TABLE]

As in classical stochastic thermodynamics, it can be negative along a single trajectory Seifert (2012); Van den Broeck and Esposito (2015). However, we will now prove that on average $\sum_{\textbf{r}_{n}}p(\textbf{r}_{n})\sigma(\textbf{r}_{n},t)\geq 0$, which demonstrates the thermodynamic consistency of our strong coupling quantum stochastic framework.

As a consequence of Eq. (14) and our previously derived second law (8), we confirm that

[TABLE]

This quantifies the change in informational entropy of all constituents (system, units and the classical memory) due to the big joint measurement (13). Its positivity follows from the Lemma in Ref. Strasberg (2019), which simply combines Theorem 11 of Ref. Jacobs (2014) and Theorem 11.10 of Ref. Nielsen and Chuang (2000) and which can be interpreted as the second law for a quantum measurement. Hence, we conclude

[TABLE]

As before, the (averaged) stochastic entropy production $\sigma(\textbf{r}_{n},t)$ contains the information about all the correlations in the units, which is typically not needed. Using subadditivity of entropy, it is again possible to arrive at expressions similar to Eq. (10). Note that, depending on the experimental situation, one could decide to not only discard information about the unit correlations, but also about the measurement results $\textbf{r}_{n}$.

This concludes the formal part of the Letter, where we have introduced a consistent notion of work, nonequilibrium free energy and entropy production along a single run of an experiment regardless of any details of the system-bath coupling. It is instructive to connect the present picture to the theory of quantum causal models and quantum stochastic processes. If we consider the limit of an instantaneous system-unit interaction, ideally described by a coupling of the form $V_{SU(k)}(\lambda_{t})=v_{k}\delta(t-t_{k})$, we can write the system-bath dynamics as

[TABLE]

Here, ${\cal{U}}^{SB}_{k+1,k}$ is the unitary time evolution generated by $H_{SB}(\lambda_{t})$ and the completely positive map ${\cal{A}}_{r_{k}}$ is defined via its action ${\cal{A}}_{r_{k}}\rho_{S}=\mbox{tr}_{U}\{{\cal{P}}_{r_{k}}[e^{-iv_{k}\hbar}\rho_{S}\rho_{U(k)}e^{iv_{k}/\hbar}]\}$ on an arbitrary system state $\rho_{S}$. In this context ${\cal{A}}_{r_{k}}$ is also known as an ‘instrument’ describing the most general state transformation possible in quantum mechanics Holevo (2001); Wiseman and Milburn (2010). The application of a set of instruments ${\cal{A}}_{r_{0}},\dots,{\cal{A}}_{r_{n}}$ to an open quantum system defines a general quantum stochastic process (or quantum causal model), which can be formally represented by a ‘quantum comb’ or ‘process tensor’ Chiribella et al. (2008, 2009); Costa and Shrapnel (2016); Oreshkov and Giarmatzi (2016); Pollock et al. (2018a); Milz et al. (2017). The sole difference compared to the most general case is that we do not allow for real-time feedback control, i.e., the instruments ${\cal{A}}_{r_{k}}$ are not allowed to depend on the previous results $\textbf{r}_{k-1}$, otherwise Eq. (14) would no longer be true. Whether the present framework can be extended to arbitrary real-time feedback control as in the Markovian case Strasberg (2019); Strasberg and Winter (2019) remains an open question.

Thermodynamic signatures of non-Markovianity.— We now turn towards an important application linking the field of quantum thermodynamics and quantum non-Markovianity Rivas et al. (2014); Breuer et al. (2016) in a rigorous way. As recognized below Eq. (11), at strong coupling we cannot ensure that the entropy production is positive in every time-interval $[t_{\ell},t_{k})$ for $t_{\ell}>t_{k}>t_{0}$. There have been repeated claims in the literature that negative entropy production rates indicate non-Markovianity Argentieri et al. (2014); Bhattacharya et al. (2017); Marcantoni et al. (2017); Popovic et al. (2018); Thomas et al. (2018); Bhattacharya et al. (2019). Doubts were raised in Refs. Strasberg and Esposito (2019) since the definitions for entropy production rates used in Refs. Argentieri et al. (2014); Bhattacharya et al. (2017); Marcantoni et al. (2017); Popovic et al. (2018); Thomas et al. (2018); Bhattacharya et al. (2019) do not yield an overall positive entropy production when integrated from the initial time $t_{0}$ to any final time $t>t_{0}$. Moreover, they can be even negative for Markovian dynamics Strasberg and Esposito (2019).

On the other hand, for a suitable notion of entropy production based on the Hamiltonian of mean force progress was achieved for classical dynamics Strasberg and Esposito (2019). In there, the situation of a strongly coupled system prepared in an arbitrary nonequilibrium state was considered (no repeated interactions were present). Then, it was shown that Markovian dynamics necessarily imply a positive entropy production rate if the system is undriven (i.e., $\lambda_{t}=$ constant). In our quantum formalism the system is initially in equilibrium such that its state does not change when undriven and left on its own. However, we can link the present picture to the classical case by realizing that we can use the very first unit $U(0)$ to prepare the system in an arbitrary nonequilibrium state via a short control operation. This preparation procedure has a thermodynamic cost captured by the always positive entropy production $\Sigma_{S}(t)=\beta W_{SU}(t)-\beta\Delta F_{S}(t)-\beta\Delta S[\rho_{U(0)}]\geq 0$, compare with Eq. (10). After the system-unit interaction, the system is left on its own and the entropy production in between any two times $t_{2}>t_{1}>t_{0}$ reads

[TABLE]

This quantifies the dissipation associated with the relaxation dynamics of the system. Equation (20) is positive if the dynamics are Markovian and if $\pi_{S}^{*}$ is a steady state of the dynamics at any time $t$. Interestingly, the latter point can be shown rigorously based on the definition of Markovianity from Ref. Pollock et al. (2018b), which is adapted to the situation of a general quantum stochastic process as used here. This is proven in the Appendix A. Thus, $\Sigma_{S}(t_{2})-\Sigma_{S}(t_{1})\geq 0$ for a Markov process in complete analogy to the classical result Strasberg and Esposito (2019). This opens up the door to investigate the interplay between entropy production and non-Markovianity in a mathematically and thermodynamically rigorous sense for quantum systems.

Further applications.— The ability to analyse general non-Markovian quantum processes from a thermodynamic perspective will find applications in various areas. One example is sequential quantum metrology Giovanneti et al. (2011). Specifically, a particular intriguing parameter to estimate is the temperature of a system. Much progress has been achieved to understand it from the perspective of metrology Mehboudi et al. (2019), but the thermodynamic costs of thermometry were not yet explored. With the recent progress in the design of optimal quantum probes Correa et al. (2015) and strong-coupling thermometry Correa et al. (2017), the present Letter opens up the possibility to thermodynamically analyse many scenarios in metrology and thermometry. Furthermore, recent progress shows how to unambigously detect quantum features in quantum stochastic processes Smirne et al. (2018); Strasberg and Díaz (2019); Milz et al. (2019). Since quantum thermodynamics is still in the search for clear observable quantum effects induced by coherence González et al. (2019), the present letter will allow to rigorously address such questions. Furthermore, originally used to understand Nobelprize-winning experiments Sayrin et al. (2011); Zhou et al. (2012) in quantum optics from a thermodynamic perspective Strasberg (2019), the present framework can be used to explore more general cases where the system is not a high-quality cavity and has substantial losses or is coupled to other cavities. Also the units do not have to be identical, which opens up the possibility to, e.g., thermodynamically analyse single-photon distillation experiments Daiss et al. (2019) where an atom in a cavity is first probed by a weak coherent pulse (unit 1) followed by a measurement (modeled by unit 2) to herald the photon distillation. Quite generally, the present setup is even relevant for experiments in the Markovian regime, if detailed control about all system parts is not possible. Finally, the present framework can be combined with the traditional picture of scattering theory and, following Ref. Strasberg et al. (2017), it allows to investigate Maxwell’s demon and Landauer’s principle at strong coupling.

Extensions.— As detailed in Appendix B, the present framework can be extended into various directions: the Hamiltonian of the units does not need to be degenerate, the initial state of the units can be correlated, and the general identities (8), (10) and (18) still hold in the case where the system-bath coupling is time-dependent. In the last case, however, the theory is no longer ‘operational’ in the sense that explicit knowledge about the state of the bath is necessary, which is hardly accessible. Similarly, we show in Appendix B how to treat multiple heat baths. Unfortunately, also in that case the state of the system and units does not suffice to have access to all thermodynamic quantities 222This problem, however, is already present in classical stochastic thermodynamics Seifert (2012); Van den Broeck and Esposito (2015), see also the discussion in Ref. Strasberg (2019)..

Concluding remarks.— The present contribution establishes a consistent thermodynamic framework – even along a single trajectory recorded in an experiment – for a system in contact with an arbitrary bath and additionally subjected to arbitrary nonequilibrium resources interacting one by one with the system. This pushes the applicability of nonequilibrium thermodynamics far beyond its traditional scope. Furthermore, the present work also demonstrates how quantum stochastic thermodynamics departs from its classical version in the strong coupling regime Seifert (2016); Jarzynski (2017); Miller and Anders (2017); Strasberg and Esposito (2017, 2019). While the basic concepts at the unmeasured level are similar, any possible measurement strategy has a non-trivial influence on the description in the quantum regime, even on average. For instance, in general there is a strict inequality on the left hand side of Eq. (18). This is not a deficiency of our theory, but a necessary ingredient, which can be already recognized at the level of the work statistics Perarnau-Llobet et al. (2017). Quantum stochastic thermodynamics is more than a mere extension of its classical counterpart. The present operational approach is, however, flexible enough to reproduce the unmeasured picture: it is recovered by choosing the trivial but legitimiate measurement operator $P_{r_{k}}=1_{U(k)}$, i.e., the identity. Then, the stochastic entropy production $\sigma(\textbf{r}_{n})$ reduces to $\Sigma$.

Acknowledgments.— I thank Kavan Modi for illuminating initial discussions. This research was financially supported by the DFG (project STR 1505/2-1) and also the Spanish MINECO FIS2016-80681-P (AEI-FEDER, UE).

Appendix A Steady state of an undriven quantum Markov process

According to Ref. Pollock et al. (2018b), a Markov process is characterized by a set of completely positive and trace-preserving maps $\{\Lambda(t_{\ell},t_{k})|t_{\ell}>t_{k}\}$ such that the normalized state conditioned on an arbitrary sequence of control operations ${\cal{A}}_{0},{\cal{A}}_{1},\dots,{\cal{A}}_{n}$ can be written as

[TABLE]

Notice that the composition law $\Lambda(t_{3},t_{1})=\Lambda(t_{3},t_{2})\Lambda(t_{2},t_{1})$ for $t_{3}>t_{2}>t_{1}$ (the ‘quantum Chapman-Kolmogorov equation’) automatically follows from that by realizing that the identity operation ${\cal{I}}$ is a legitimate control operation too.

By applying this to our problem, we immediately realize that we obtain the relation

[TABLE]

for any $t_{k}>t_{0}$ and hence, the same holds for any intermediate map $\Lambda(t_{\ell},t_{k})$ ($t_{\ell}>t_{k}$). Note that the assumption of no driving as well as the particular form of the initial system-bath state [$\rho_{SB}(t_{0}^{-})=\pi_{SB}$] is crucial to arrive at this conclusion. Equation (20) of the main text can therefore be written as

[TABLE]

Now, we can use that for a Markov process relative entropy is contractive Rivas et al. (2014); Breuer et al. (2016) to conclude that Eq. (20) of the main text is positive.

It is, of course, worth to point out that in general, for an initially correlated state of the form $\pi_{SB}$, there exists no such set of completely positive and trace-preserving maps $\{\Lambda(t_{\ell},t_{k})|t_{\ell}>t_{k}\}$. Obviously, this simply shows that one should not typically expect Markovian behaviour in strongly coupled systems. Nevertheless, in particular – and not necessarily uninteresting – limiting cases this can still be the case, e.g., in the limit of time-scale separation Strasberg and Esposito (2017), which allows in the quantum case to derive a Markovian master equation in a Polaron frame Schaller et al. (2013); Gelbwaser-Klimovsky and Aspuru-Guzik (2015); Strasberg et al. (2016).

Appendix B Extensions of strong coupling quantum stochastic thermodynamics

For later comparison, we start by listing the definitions of internal energy, heat flow and system entropy complementary to the definition of work and nonequilibrium free energy based on the assumptions of the main text (only one heat bath, energy degenerate units, no driving in the system-bath interaction). At the unmeasured level we have

[TABLE]

Remember that $W(t)$ is the total work as defined in Eq. (6) of the main text and the letter $S$ without subscripts always denotes the von Neumann entropy $S(\rho)=-\mbox{tr}\{\rho\ln\rho\}$. Hence, we see that the thermodynamic entropy $S_{SU(\textbf{n})}(t)$ is not identical to the von-Neumann entropy in the strong coupling regime. Also note that $F_{SU(\textbf{n})}(t)=E^{*}_{SU(\textbf{n})}(t)-TS_{SU(\textbf{n})}(t)$ reproduces the definition of nonequilibrium free energy of the main text. The above definitions have been already discussed for a single system strongly coupled to a bath in the classical Seifert (2016); Jarzynski (2017); Miller and Anders (2017); Strasberg and Esposito (2017, 2019) and quantum Strasberg and Esposito (2019) case in the absence of any units. The second law of nonequilibrium thermodynamics can be expressed in the two alternative forms

[TABLE]

To extend the above definitions to the trajectory level, we have to replace the state $\rho_{SU(\textbf{n})}(t)$ by the conditional state $\rho_{SU(\textbf{n})}(\textbf{r}_{n},t)$ and we have to add the stochastic entropy of the detector $[-\ln p(\textbf{r}_{n})]$ to the definition of entropy. We denote this as

[TABLE]

Note that we naturally assume the notation $(\textbf{r}_{n},t)$ to imply a time $t$ where we have already measured the state of the $n$th unit (giving result $r_{n}$) but where the next $(n+1)$th unit was not yet measured. The stochastic entropy production can be again expressed in the two alternative forms

[TABLE]

On average, $\sum_{\textbf{r}_{n}}p(\textbf{r}_{n})\sigma(\textbf{r}_{n},t)\geq 0$.

We will now investigate various extensions of the framework presented in the main text. We will be very explicit with the steps in Extension 4 (multiple heat baths) as it is the most general setting implying all the others (including the one of the main text). Therefore, for the Extensions 1 to 3 we limit ourselves to only pointing out some essential observations.

B.1 Extension 1: Driven system-bath interaction

In this case we allow that the system-bath interaction $V_{SB}=V_{SB}(\lambda_{t})$ depends on some externally prescribed time-dependent protocol $\lambda_{t}$. Such a setting is important, e.g., to model how a system is put into contact with a bath. Including this case only changes the definition of power [Eq. (4) in the main text] in an obvious way to

[TABLE]

Despite being only a little formal change, this scenario implies that the power can no longer be computed based only on the knowledge of the system state $\rho_{S}(t)$, but in general requires to know the entire state $\rho_{SB}(t)$. This is problematic from an experimental point of view as the theory is no longer operational and also from a computational point of view it is very challenging. The stochastic generalization $\dot{w}_{S}(\textbf{r}_{n},t)$ of Eq. (32) simply uses the conditional state $\rho_{SB}(\textbf{r}_{n},t)$.

B.2 Extension 2: Correlated initial unit state

As will become clear in the detailed derivation below, the initial state of the units $\rho_{U(\textbf{n})}(t_{0}^{-})$ can indeed be arbitrary and does not need to be decorrelated. However, in practical applications this is often unnecessary and only complicates the theoretical treatment. Furthermore, deriving a simplified second law for the system alone [as in Eqs. (10) and (11) of the main text] becomes non-trivial then.

B.3 Extension 3: Energetic units

This generalization is slightly more subtle as the previous two when it comes to the trajectory level. The reason is that for energetically non-degenerate units the act of measurement, which requires to couple the microscopic unit to some macroscopic (not further specified) detection apparatus, can change the energy of the unit even on average unless the measurement operators $P_{r_{k}}$ commute with the unit Hamiltonian or the final unit state. While this is often the case in practice, see, e.g., Refs. Sayrin et al. (2011); Zhou et al. (2012), we are here interested in the most general scenario. Note that these subtleties were already discussed in greater detail in Ref. Strasberg (2019), compare also with the related (but not identical) concept of “quantum heat” introduced in Ref. Elouard et al. (2017).

To start with, nothing changes at the unmeasured level: Eqs. (24) to (27) remain true as usual. Now, to isolate the contribution steming solely from the energetic measurements of the unit due to the final measurement we define

[TABLE]

Here, $E_{U(\textbf{n})}(t)=\mbox{tr}_{U(\textbf{n})}\{H_{U(\textbf{n})}\rho_{U(\textbf{n})}(t)\}$ is the energy of all units evaluated with respect to the unmeasured state $\rho_{U(\textbf{n})}(t)=\mbox{tr}_{SB}\{{\cal{U}}_{t,t_{0}}\rho_{\text{tot}}(t_{0}^{-})\}$ of the units, whereas $e_{U(\textbf{n})}(t)=\mbox{tr}_{U(\textbf{n})}\{H_{U(\textbf{n})}\rho_{U(\textbf{n})}(\textbf{r}_{n},t)\}$ involves the measured state, compare also with Eq. (13) in the main text. Note that at the time of the measurement of the $n$th unit giving result $r_{n}$, the unit is physically decoupled from the system, i.e., we have $V_{SU(n)}(\lambda_{t})=0$ as indicated in Fig. 1 of the main text. We also remark that it is debated whether Eq. (33) should be counted as ‘heat’ or ‘work’. In fact, we will now split off this term from the total stochastic heat (29) as follows

[TABLE]

This defines $q(\textbf{r}_{n},t)$ as the essential part of the stochastic heat flow, which contains two contributions: first, all the heat flow affecting the system and unit during the time without measurements and second, any stochastic change of the system energy due to the measurement, which causes an update of our state of knowledge about the system. Consequently, using Eq. (14) of the main text we can confirm that on average

[TABLE]

We then define the stochastic entropy production by including only the heat flow $q$ and we exlude $q_{\text{meas}}$, i.e.,

[TABLE]

This definition allows us to conclude that $\sum_{\textbf{r}_{n}}p(\textbf{r}_{n})\sigma(\textbf{r}_{n},t)\geq 0$, see the next Section for all the details. Note that in case of energetically degenerate units we always have $q_{\text{meas}}(\textbf{r}_{n},t)=0$ and $q_{SU(\textbf{n})}(\textbf{r}_{n},t)=q(\textbf{r}_{n},t)$.

B.4 Extension 4: Multiple heat baths

The setup we are here assuming is specified by the following Hamiltonian

[TABLE]

and initial state

[TABLE]

Physically, it describes a system coupled to two heat baths $B_{1}$ and $B_{2}$, initialized at inverse temperatures $\beta_{1}$ and $\beta_{2}$ respectively, which additionally interacts with a stream of units as specified in the main text. The initial state is chosen such that the system is in a joint equilibrium state with $B_{1}$ decoupled from the equilibrium state of $B_{2}$. We thus assume that the coupling with $B_{2}$ is suddenly switched on at the initial time $t_{0}$. Note that decorrelated system-bath states are commonly assumed in the theory of open quantum systems. The initial state of the units is arbitrary but decorrelated from the system and the baths.

We start with the thermodynamic description at the unmeasured level, which essentially combines the tools of Refs. Esposito et al. (2010); Takara et al. (2010); Seifert (2016); Strasberg and Esposito (2019). We therefore use the following basic definitions:

[TABLE]

Note that the internal energy differs from the previous definition (24) by explicitly including the interaction $V_{SB_{2}}$ between the system and $B_{2}$ Esposito et al. (2010). Furthermore, the heat flow from $B_{2}$ is defined as (minus) the change in the expectation value of $H_{B_{2}}$. Finally, remember that the Hamiltonian of mean force depends explicitly on the inverse temperature $\beta_{1}$ of $B_{1}$, which we suppress in the notation. We will now show that the second law as known from phenomenological nonequilibrium thermodynamics holds:

[TABLE]

For this purpose we notice that the heat flux from $B_{1}$ can be microscopically expressed as

[TABLE]

where the delta-notation means $\Delta f(\lambda_{t},t)=f(\lambda_{t},t)-f(\lambda_{0}^{-},t_{0}^{-})$ for any $f$ and where we defined $H_{SB_{1}U(\textbf{n})}(\lambda_{t})\equiv H_{SU(\textbf{n})}(\lambda_{t})+V_{SB_{1}}(\lambda_{t})+H_{B_{1}}$. The entropy production therefore reads

[TABLE]

Next, we use the standard trick $X=\ln e^{X}$ to write

[TABLE]

Now, using the defining porperty of the Hamiltonian of mean force [Eq. (3) in the main text], we confirm that the term involving the partition functions cancels. We now sort the terms in the entropy production into terms depending on the final time $t$ and terms depending on the initial time $t_{0}^{-}$:

[TABLE]

Next, we use that the initial time $t_{0}^{-}$ prior to the first system-unit interaction is chosen such that $V_{SU(0)}(\lambda_{0}^{-})=0$, which implies $\pi^{*}_{SU(\textbf{n})}(\beta_{1},\lambda_{0}^{-})=\pi^{*}_{S}(\beta_{1},\lambda_{0}^{-})\otimes\pi_{U(\textbf{n})}(\beta_{1})$ and $\pi_{SB_{1}U(\textbf{n})}(\beta_{1},\lambda_{0}^{-})=\pi_{SB_{1}}(\beta_{1},\lambda_{0}^{-})\otimes\pi_{U(\textbf{n})}(\beta_{1})$. This means that we can replace $\pi_{U(\textbf{n})}(\beta_{1})$ by $\rho_{U(\textbf{n})}(t_{0}^{-})$, which cancels out in any case:

[TABLE]

Now, by comparing with Eq. (38), we see that the very last term is nothing else than (minus) the initial von Neumann entropy of the universe, which is preserved during the unitary time-evolution. Hence,

[TABLE]

Next, again by using the particular form (38) of the initial state, we notice that the last line cancels. By using the quantum relative entropy, we are hence left with

[TABLE]

By monotonicity of relative entropy this term is evidently positive.

We now turn to the trajectory representation defined by a certain sequence of measurement results $\textbf{r}_{n}$. They are obtained in the same way as for a single heat bath by measuring each unit after the interaction with the system, see the main text. We use the following definitions which extend our previous ones to the case of two heat baths:

[TABLE]

Note that the definition of stochastic measurement heat is the same as in Eq. (33) describing the random changes in the unit energy due to the measurement backaction. Furthermore, due to the fact that we measure only the units and perform no real-time feedback control, we have similar to Eq. (14) of the main text

[TABLE]

where $\rho_{SB_{1}B_{2}}(t)=\mbox{tr}_{U(\textbf{n})}\{{\cal{U}}_{t,t_{0}}\rho_{\text{tot}}(t_{0}^{-})\}$ is the unmeasured state of the system and baths without any measurements. This allows us to confirm the following two essential properties:

[TABLE]

Thus, the stochastic entropy production $\sigma(\textbf{r}_{n},t)=\Delta s_{SU(\textbf{n})}(\textbf{r}_{n},t)-\beta_{1}q_{1}(\textbf{r}_{n},t)-\beta_{2}q_{2}(\textbf{r}_{n},t)$ becomes on average

[TABLE]

Hence, we obtain

[TABLE]

This term is formally identical to Eq. (17) of the main text and its positivity follows by virtue of the same arguments. Hence,

[TABLE]

Thus, we showed how to extend quantum stochastic thermodynamics to multiple heat bath. Unfortunately, even if we do not drive the system-bath interaction $V_{SB_{i}}$, the definitions proposed here are not fully operational in the sense that they cannot be computed by knowing solely the state of the system and the units. To evaluate the heat flow $q_{2}$ knowledge about the state of $B_{2}$ is necessary. It should be emphasized, however, that this does not indicate a particular shortcoming of our approach. It is known that in case of multiple heat bath already classical stochastic thermodynamics (even in the weak coupling and Markovian regime) is not operational in the sense above unless additional assumptions are made. For further details see the discussion in Sec. VII A in Ref. Strasberg (2019).