Integral Fluctuation Relations for Entropy Production at Stopping Times

TL;DR

This paper derives a universal fluctuation relation for entropy production at stopping times in stochastic systems, leading to a second law that bounds heat, work, and efficiency in nonequilibrium processes.

Contribution

It introduces a new integral fluctuation relation for entropy at stopping times, revealing universal properties and bounds on thermodynamic quantities in stochastic systems.

Findings

Entropy production at stopping times satisfies a universal fluctuation relation.

Systems can extract heat from the environment when stopped optimally.

Efficiency of stochastic heat engines can surpass Carnot bound at stopping times.

Abstract

A stopping time $T$ is the first time when a trajectory of a stochastic process satisfies a specific criterion. In this paper, we use martingale theory to derive the integral fluctuation relation $\langle e^{-S_{\rm tot}(T)}\rangle=1$ for the stochastic entropy production $S_{\rm tot}$ in a stationary physical system at stochastic stopping times $T$. This fluctuation relation implies the law $\langle S_{\rm tot}(T)\rangle\geq 0$, which states that it is not possible to reduce entropy on average, even by stopping a stochastic process at a stopping time, and which we call the second law of thermodynamics at stopping times. This law implies bounds on the average amount of heat and work a system can extract from its environment when stopped at a random time. Furthermore, the integral fluctuation relation implies that certain fluctuations of entropy production are universal or are bounded by universal functions. These universal properties descend from the integral fluctuation relation by selecting appropriate stopping times: for example, when $T$ is a first-passage time for entropy production, then we obtain a bound on the statistics of negative records of entropy production. We illustrate these results on simple models of nonequilibrium systems described by Langevin equations and reveal two interesting phenomena. First, we demonstrate that isothermal mesoscopic systems can extract on average heat from their environment when stopped at a cleverly chosen moment and the second law at stopping times provides a bound on the average extracted heat. Second, we demonstrate that the average efficiency at stopping times of an autonomous stochastic heat engines, such as Feymann's ratchet, can be larger than the Carnot efficiency and the second law of thermodynamics at stopping times provides a bound on the average efficiency at stopping times.

Equations

$$S_{\mathrm{tot}}(t)=\log \frac{p(\vec{X}(1),\vec{X}(2),\dots ,\vec{X}(t))}{p(\vec{X}(t),\vec{X}(t-1),\dots ,\vec{X}(1))},$$

$$⟨e^{-S_{\mathrm{tot}}(T)}⟩=1,$$

$$\displaystyle\Big{\langle}e^{-S_{\rm tot}(T)}|\vec{X}^{T^{\prime}}_{0}\Big{\rangle}=e^{-S_{\rm tot}(T^{\prime})},$$

$$⟨S_{\mathrm{tot}}(T)⟩\ge 0,$$

$$⟨Q(T)⟩\le {\mathsf{T}}_{\mathrm{env}}⟨\Delta S_{\mathrm{sys}}(T)⟩.$$

$${\eta }_T:=-\frac{⟨W(T)⟩}{⟨Q_{\mathrm{h}}(T)⟩},$$

$${\eta }_T\le {\eta }_{\mathrm{C}}-\frac{⟨\Delta {\mathcal{F}}_{\mathrm{c}}(T)⟩}{⟨Q_{\mathrm{h}}(T)⟩},$$

$$\mathbb{P}\left(S_{\mathrm{tot}}(t)\le -s\right)\le e^{-s},s\ge 0,$$

$$p_{+}\ge 1-\frac{1}{e^{s_{-}}-e^{-s_{+}}},p_{-}\le \frac{1}{e^{s_{-}}-e^{-s_{+}}}.$$

$$p_{+}=\frac{e^{s_{-}}-1}{e^{s_{-}}-e^{-s_{+}}},p_{-}=\frac{1-e^{-s_{+}}}{e^{s_{-}}-e^{-s_{+}}}.$$

$$\mathbb{P}\left[S_{\inf }\le -s\right]\le e^{-s},s\ge 0,$$

$$p_{S_{\inf }}(-s)=e^{-s},s\ge 0.$$

$$p_{\mathrm{sur}}(t)=\frac{e^{-s_0}-1}{e^{-s_0}-⟨e^{-S_{\mathrm{tot}}(t)}{⟩}_{\mathrm{sur}}},$$

$$\mathbb{P}(N_{\times }=0;\Delta )$$

$$\mathbb{P}(N_{\times }>n)$$

$$\mathbb{P}(N_{\times}=n)=\left\{\begin{array}[]{lr}1-e^{-\Delta}&n=0,\\ 2\sinh(\Delta)e^{-2n\Delta}&n\geq 1.\end{array}\right.$$

$$p_{X(t)}(x)=\frac{18}{37}{\delta }_{x,1}+\frac{18}{37}{\delta }_{x,-1}+\frac{1}{37}{\delta }_{x,0},$$

$$F(t)=n+b\sum _{s=1}^t(Y(s)X(s)+|X(s)|-1),$$

$$p_{\mathrm{ruin}}(n)=\mathbb{P}\left[F(T_{\mathrm{play}})\le 0\right]$$

$$E\left[F(t)|X(0),X(1),\dots ,X(s)\right]\le F(s),$$

$$p_{\mathrm{ruin}}(n)=q{p}_{\mathrm{ruin}}(n-b)+(1-q)p_{\mathrm{ruin}}(n+b),n\in [0,m],$$

$$M(t)={\left(\frac{q}{1-q}\right)}^{F(t)/b},$$

$$E\left[M(t)|X(0),X(1),\dots ,X(s)\right]=M(s),$$

$$E\left[M(T)\right]=E[M(0)].$$

$$E\left[M(0)\right]={\left(\frac{q}{1-q}\right)}^{n/b},$$

$$p_{\mathrm{ruin}}(n)+(1-p_{\mathrm{ruin}}(n)){\left(\frac{q}{1-q}\right)}^{m/b+1}\ge E\left[M(T_{\mathrm{play}})\right]\ge p_{\mathrm{ruin}}(n)+(1-p_{\mathrm{ruin}}(n)){\left(\frac{q}{1-q}\right)}^{m/b},$$

$$\frac{{\left(\frac{q}{1-q}\right)}^{m/b+1}-{\left(\frac{q}{1-q}\right)}^{n/b}}{{\left(\frac{q}{1-q}\right)}^{m/b+1}-1}\ge p_{\mathrm{ruin}}(n)\ge \frac{{\left(\frac{q}{1-q}\right)}^{m/b}-{\left(\frac{q}{1-q}\right)}^{n/b}}{{\left(\frac{q}{1-q}\right)}^{m/b}-1}.$$

$$E\left[|M(t)|\right]<\infty ,\forall t\ge 0;$$

$$E[M(t)|{\mathcal{F}}_s]=M(s),\forall s\le t,\mathrm{and}\forall t\ge 0.$$

$$E\left[M(T)\right]=E\left[M(0)\right].$$

$$\underset{m\to \infty }{\lim }{\sup }_{t\ge 0}\int |M(t)|I_{|M(t)|\ge m}\mathrm{d}\mathbb{P}=0,$$

Full text

1 Introduction and statement of the main results

Stochastic thermodynamics is a thermodynamics theory for the slow degrees of freedom $\vec{X}(t)$ of a mesoscopic system that is weakly coupled to an environment in equilibrium [1, 2, 3, 4, 5]. Examples of systems to which stochastic thermodynamics applies are molecular motors [6, 7], biopolymers [8], self-propelled Brownian particles [9], micro-manipulation experiments on colloidal particles [10, 11, 12], and electronic circuits [13, 14, 15].

The stochastic entropy production $S_{\rm tot}(t)$ is a key variable in stochastic thermodynamics. It is defined as the sum of the entropy change of the environment $S_{\rm env}(t)$ and a system entropy change $\Delta S_{\rm sys}(t)$ [16]. In stochastic thermodynamics entropy production is a functional of the trajectories of the slow degrees of freedom in the system. If time is discrete and $\vec{X}(t)$ is a variable of even parity with respect to time reversal, then the entropy production $S_{\rm tot}(t)$ associated with a trajectory of a nonequilibrium stationary process $\vec{X}(t)$ is the logarithm of the ratio between the stationary probability density of that trajectory $p(\vec{X}(1),\vec{X}(2),\ldots,\vec{X}(t))$ and the probability density of the same trajectory but in time-reversed order [4, 17, 18, 19],

[TABLE]

where $\log$ denotes natural logarithm. Here and throughout the paper we use dimensionless units for which Boltzmann’s constant $k_{\rm B}=1$. Equation (1) is a particular case of the general expression of stochastic entropy production in terms of probability measures that we will discuss below in Eq. (35). The functional $S_{\rm tot}(t)$ is exactly equal to zero at all times for systems in equilibrium. For nonequilibrium systems, entropy production fluctuates with expected value larger than zero, $\langle S_{\rm tot}(t)\rangle\geq 0$.

An interesting consequence of definition (1) is that the exponential of the negative entropy production $e^{-S_{\rm tot}(t)}$ is a martingale associated with the process $\vec{X}(t)$ [20, 21, 22]. Historically the concept of martingales has been introduced to understand fundamental questions in betting strategies and gambling [23]. Martingale theory [24, 25, 26], and in particular Doob’s optional stopping theorem, provides an elegant resolution to the question whether it is possible to make profit in a fair game of chance by leaving the game at a cleverly chosen moment. We can distinguish unfair games of chances, where the expected values of the gambler’s fortune decreases (or increases) with time, from fair ones, where such expected values are constant in time on average. In probability theory, these categories correspond to supermartingales (or submartingales) and martingales, respectively. In a nutshell, the optional stopping theorem for martingales states that a gambler cannot make profit on average in a fair game of chance by quitting the game at an intelligent chosen moment. The optional stopping theorem holds as long as the total amount of available money is finite. A gambler with access to an infinite budget of money could indeed devise a betting strategy that makes profit out of a fair game; the St. Petersburg game provides an example of a such a strategy, see [27] and chapter 6 of [28]. Nowadays martingales have various applications, for example, they model stock prices in efficient capital markets [29].

In this paper, we study universal properties of entropy production in nonequilibrium stationary states using martingale theory and Doob’s optional stopping theorem. In an analogy with gambling, the negative entropy production $-S_{\rm tot}(t)$ of a stationary process $\vec{X}(t)$ is equivalent to a gambler’s fortune in an unfair game of chance $\vec{X}(t)$ and the exponentiated negative entropy production $e^{-S_{\rm tot}(t)}$ is a martingale associated with the gambler’s fortune. In stochastic thermodynamics, Doob’s optional stopping theorem implies

[TABLE]

where the expected value $\langle\cdot\rangle$ is over many realizations of the physical process $\vec{X}(t)$, and where $T$ is a stopping time. A stopping time $T$ is the first time when a trajectory of $\vec{X}$ satisfies a specific criterium; it is thus a stochastic time. This criterium must obey causality and cannot depend on the future. The relation (2) holds under the condition that either $T$ acts in a finite-time window, i.e., $T\in[0,\tau]$ with $\tau$ a positive number, or that $S_{\rm tot}(t)$ is bounded for all times $t\in[0,T]$. We call (2) the integral fluctuation relation for entropy production at stopping times because it is an integral relation, $\langle e^{-S_{\rm tot}(T)}\rangle=\int{\rm d}\mathbb{P}\>e^{-S_{\rm tot}(T)}=1$, that characterises the fluctuations of entropy production. Here, $\mathbb{P}$ is the probability measure associated with $\vec{X}(t)$.

The fluctuation relation at stopping times (2) can be extended into a fluctuation relation conditioned on trajectories $\left\{\vec{X}(t)\right\}_{t\in[0,T^{\prime}]}$ of random duration $[0,T^{\prime}]$, namely,

[TABLE]

with $T^{\prime}$ a stopping time for which $T^{\prime}\leq T$, and where $\vec{X}^{s}_{0}=\left\{\vec{X}(t^{\prime})\right\}_{t^{\prime}\in[0,s]}$ denotes a trajectory in a finite-time window. Notice that for $T^{\prime}=0$ we obtain $\langle e^{-S_{\rm tot}(T)}|\vec{X}(0)\rangle=e^{-S_{\rm tot}(0)}=1$, since in our definitions $S_{\rm tot}(0)=0$. The fluctuation relation (3) implies thus (2).

There are two important implications of the integral fluctuation relations (2) and (3). First, it holds that

[TABLE]

or in other words, it is not possible to reduce the average entropy production by stopping a stochastic process $\vec{X}(t)$ at a cleverly chosen moment $T$ that can be different for each realisation. The relation (4) is reminiscent of the second law of thermodynamics, and therefore we call it the second law of thermodynamics at stopping times. A second implication of the integral fluctuation relations (2) and (3) is that certain fluctuation properties of entropy production are universal. In what follows, we discuss in more detail these two consequences of the integral fluctuation relations.

We first discuss the second law at stopping times Eq. (4). Remarkably, this second law holds for any stopping time $T$ defined by a (arbitrary) stopping criterium that obeys causality and does not use information about the future of the physical process; first-passage times are canonical examples of stopping times. Interestingly, the inequality (4) bounds the average amount of heat and work a system can perform on or extract from its surroundings at a stopping time $T$. For isothermal systems, Eq. (4) implies

[TABLE]

Relation (5) states that the average amount of heat $\langle Q(T)\rangle$ a system can on average extract at a stopping time $T$ from a thermal reservoir at temperature $\mathsf{T}_{\rm env}$ is smaller or equal than the average system entropy difference $\langle\Delta S_{\rm sys}(T)\rangle$ between the initial state and the final state at the stopping time. Similar considerations allow us to derive bounds on the average amount of work that a stationary heat engine, e.g. Feynman’s ratchet, can extract from its surrounding when stopped at a cleverly chosen moment. Consider a system in contact with two thermal reservoirs at temperatures $\mathsf{T}_{\rm h}$ and $\mathsf{T}_{\rm c}$ with $\mathsf{T}_{\rm h}\geq\mathsf{T}_{\rm c}$. We define the stopping-time efficiency $\eta_{T}$ associated with the stopping time $T$ as

[TABLE]

where $\langle W(T)\rangle$ is the average work exerted on the system in the time interval $[0,T]$, and $\langle Q_{\rm h}(T)\rangle$ is the average heat absorbed by the system from the hot reservoir within the same time interval. If $\langle Q_{\rm h}(T)\rangle>0$, then the second law at stopping times (4) implies that

[TABLE]

where $\eta_{\rm C}=1-(\mathsf{T}_{\rm c}/\mathsf{T}_{\rm h})$ is the Carnot efficiency, $\langle\Delta\mathcal{F}_{\rm c}(t)\rangle=\langle\Delta v(t)\rangle-\mathsf{T}_{\rm c}\langle\Delta S_{\rm sys}(t)\rangle$ is the generalised free energy change of the system at the stopping time $T$, and $\Delta v(t)=v(X(t))-v(X(0))$ is change of the internal energy of the system. Note that the second term in the right-hand side of (7) can be positive, and thus efficiencies at stopping times of stationary heat engines can be greater than the Carnot efficiency. This is because using a stopping time the system is, in general, no longer cyclic.

We now discuss universal properties of the fluctuations of entropy production. By applying the integral fluctuation relations (2) and (3) to different examples of stopping times $T$, we will derive the following generic relations for the fluctuations of entropy production:

• •

In the simple case $T=t$, where $t$ is a deterministic fixed time, relation (4) reads $\langle S_{\rm tot}(t)\rangle\geq 0$, which is a well-known second-law like relation derived in stochastic thermodynamics [16, 4], and the relation (2) is the stationary integral fluctuation relation $\langle e^{-S_{\rm tot}(t)}\rangle=1$ [16, 4]. The integral fluctuation relation at fixed times implies that in a nonequilibrium process events of negative entropy production must exist and their likelihood is bounded by [4]

[TABLE]

where $\mathbb{P}\left(\cdot\right)$ denotes the probability of an event.

• •

Our second choice of stopping times are first-passage times $T_{\rm fp}={\rm inf}\left\{t:S_{\rm tot}(t)\notin(-s_{-},s_{+})\right\}$ for entropy production with two absorbing boundaries at $-s_{-}\leq 0$ and $s_{+}\geq 0$. As we show in this paper, the integral fluctuation relation Eq. (2) implies that the splitting probabilities $p_{-}=\mathbb{P}\left[S_{\rm tot}(T)\leq-s_{-}\right]$ and $p_{+}=\mathbb{P}\left[S_{\rm tot}(T)\geq s_{+}\right]$ are bounded by

[TABLE]

If the trajectories of entropy production are continuous, then [21]

[TABLE]

• •

Global infima of entropy production, $S_{\rm inf}={\rm inf}_{t\geq 0}\>S_{\rm tot}(t)$, quantify fluctuations of negative entropy production. The cumulative distribution $\mathbb{P}\left[S_{\rm inf}\leq-s\right]$ is equal to the splitting probability $p_{-}$ in the limit $s_{+}\rightarrow\infty$. Using (9) we obtain [21]

[TABLE]

which implies the infimum law $\langle S_{\rm inf}\rangle\geq-1$ [21]. It is insightful to compare the two relations (8) and (11). Since $S_{\rm inf}\leq S_{\rm tot}(t)$, the inequality (11) implies the inequality (8), and (11) is thus a stronger result. Remarkably, the bound (11) is tight for continuous stochastic processes. Indeed, using (10) we obtain the probability density function for global infima of the entropy production in continuous processes [21],

[TABLE]

The mean global infimum is thus $\langle S_{\rm inf}\rangle=-1$.

• •

The survival probability $p_{\rm sur}(t)$ of the entropy production is the probability that entropy production has not reached a value $s_{0}$ in the time interval $[0,t]$. For continuous stochastic processes we obtain the generic expression

[TABLE]

where $\langle\dots\rangle_{\rm sur}$ is an average over trajectories that have not reached the absorbing state in the interval $[0,t]$.

• •

We consider the statistics of the number of times, $N_{\times}$, that entropy production crosses the interval $[-\Delta,\Delta]$ from $-\Delta$ towards $\Delta$ in one realisation of the process $\vec{X}(t)$. The probability of $N_{\times}$ is bounded by

[TABLE]

In other words, the probability of observing a large number of crossing decays at least exponentially in $N_{\times}$. For continuous stochastic processes we obtain a generic expression for the probability of $N_{\times}$ [22], given by

[TABLE]

Remarkably, all these results on universal fluctuation properties are direct consequences of the integral fluctuation relation for entropy production at stopping times Eq. (2) and its conditional version, Eq. (3).

Some of the results in this paper have already appeared before in the literature or are closely related to existing results. A fluctuation relation analogous to (2) has been derived for the exponential of the negative housekeeping heat, see Eq. (6) in [30]. Since for stationary processes the housekeeping heat is equal to the entropy production, the relation (6) in [30] implies the relation (2) in this paper. The relations (10), (11), (12) and (15) have been derived before in [21] and [22]. Instead, the relations (3), (5), (7), (9), (13), and (14) are, to the best of our knowledge, derived here for the first time. Moreover, we demonstrate that all the results (3), (5), (7), (9), (10), (11), (12), (13) and (15) descend from the integral fluctuation relation fluctuation relations (2) and (3) in a few simple steps, and we discuss the physical meaning of the results derived in this paper on examples of simple nonequilibrium systems.

The paper is organised as follows. Section 2 introduces the notation used in the paper. In Section 3, we revisit the theory of martingales in the context of gambling. In Section 4, we briefly recall the theory of stochastic thermodynamics, focusing on the aspects we will use in this paper. These two sections only contain review material, and can be skipped by readers who want to directly read the new results of this paper. In Section 5, we derive the first important results of this paper: the integral fluctuation relations at stopping times (2) and (3). In Section 6, we derive the second law of thermodynamics at stopping times (4), and we discuss the physical implications of this law. In Section 7, we use the integral fluctuation relation at stopping times to derive universal properties for the fluctuations of entropy production in nonequilibrium stationary states, including the relations (9)-(15). In Section 8, we discuss the effect of finite statistics on the integral fluctuation relation at stopping times, which is relevant for experimental validation. In Section 9, we illustrate the second law at stopping times and the integral fluctuation relation at stopping times in paradigmatic examples of nonequilibrium stationary states. We conclude the paper with a discussion in Section 10. In the Appendices, we provide details on important proofs and derivations.

2 Preliminaries and notation

In this paper, we will consider stochastic processes described by $d$ degrees of freedom $\vec{X}(t)=(X_{1}(t),X_{2}(t),\ldots,X_{d}(t))$. The time index can be discrete, $t\in\mathbb{Z}$, or continuous, $t\in\mathbb{R}$. We denote the full trajectory of $\vec{X}(t)$ by $\omega=\left\{\vec{X}(t)\right\}_{t\in(-\infty,\infty)}$ and the set of all trajectories by $\Omega$. We call a subset $\Phi$ of $\Omega$ a measurable set, or an event, if we can measure the probability $\mathbb{P}\left[\Phi\right]$ to observe a trajectory $\omega$ in $\Phi$. The $\sigma$-algebra $\mathscr{F}$ is the set of all subsets of $\Omega$ that are measurable.

The triple $(\Omega,\mathscr{F},\mathbb{P})$ is a probability space. We denote random variables on this probability space in upper case, e.g., $X,Y,Z$, whereas deterministic variables are written in lower case letters, e.g., $x,y,z$. An exception is the temperature $\mathsf{T}_{\rm env}$, which is a deterministic variable. Random variables are functions defined on $\Omega$, i,e, $X:\omega\rightarrow X(\omega)$. For simplicity we often omit the $\omega$-dependency in our notation of random variables, i.e., we write $X=X(\omega)$, $Y=Y(\omega)$, etc. For stochastic process, $\vec{X}(t)=\vec{X}(t,\omega)$ is a function on $\Omega$ that returns the value of $\vec{X}$ at time $t$ in the trajectory $\omega$. The expected value of a random variable $X$ is denoted by $\langle X\rangle$ or $E[X]$ and is defined as the integral $\langle X\rangle=E[X]=\int_{\Omega}d\mathbb{P}\>X(\omega)$ in the probability space $(\Omega,\mathscr{F},\mathbb{P})$. We write $p_{X}(x)$ for the probability density function or probability mass function of $X$, if it exists. We denote vectors by $\vec{x}=(x_{1},x_{2},\ldots,x_{d})$ and we use the notation $t\wedge\tau={\rm min}\left\{t,\tau\right\}$.

We will consider situations where an experimental observer does not have instantaneously access to the complete trajectory $\omega$ but rather tracks a trajectory $\left\{\vec{X}(s)\right\}_{s\in[0,t]}=\vec{X}^{t}_{0}$ in a finite time interval. In this case, the set of measurable events gradually expands as time progresses and a larger part of the trajectory $\omega$ becomes visible. Mathematically this situation is described by an increasing sequence of $\sigma$-algebras $\left\{\mathscr{F}_{t}\right\}_{t\geq 0}$ where $\mathscr{F}_{t}$ contains all the measurable events $\Phi$ associated with finite-time trajectories $\vec{X}^{s}_{0}$. The sequence of sub $\sigma$-algebras $\left\{\mathscr{F}_{t}\right\}_{t\geq 0}$ of $\mathscr{F}$ is called the filtration generated by the stochastic process $X(t)$ and $(\Omega,\mathscr{F},\left\{\mathscr{F}_{t}\right\}_{t\geq 0}\mathbb{P})$ is a filtered probability space. If time is continuous, then we assume that $\left\{\mathscr{F}_{t}\right\}_{t\geq 0}$ is right-continuous, i.e., $\mathscr{F}_{t}=\cap_{s>t}\mathscr{F}_{s}$; this implies that the process $\vec{X}(t)$ consists of continuous trajectories intercepted by a discrete number of jumps.

We denote by $E[M(t)|\mathscr{F}_{s}](\omega)$ the conditional expectation of a random variable $M(t)$ given a sub-$\sigma$-algebra $\mathscr{F}_{s}$ of $\mathscr{F}$ [26, 25]. Note that conditional expectations $E[M(t)|\mathscr{F}_{s}](\omega)$ are random variables on the measurable space $(\Omega,\mathscr{F}_{s})$. Since $E[M(t)|\mathscr{F}_{s}](\omega)$ is an expectation value we also use the physics notations $E[M(t)|\mathscr{F}_{s}]=\langle M(t)|\mathscr{F}_{s}\rangle$ and $E[M(t)|\mathscr{F}_{s}]=\langle M(t)|\vec{X}^{s}_{0}\rangle$.

3 Martingales

In a first subsection, we introduce martingales within the example of games of chance, to illustrate how fluctuations of a stochastic process can be studied with martingale theory. The calculations in this subsection are similar to those for the stochastic entropy production presented in Sections 5 and 7, with the difference that game of chances are simpler to analyse, since they consist of sequences of independent and identically distributed random variables. In a second subsection, we present a definition of martingales that applies to stochastic processes in continuous and discrete time, and we discuss the optional stopping theorem.

3.1 Gambling and martingales

Games of chance have inspired mathematicians as far back as the 17th century and have laid the foundation for probability theory [31]. A question that has often been studied is the gambler’s ruin problem: Consider a gambler that enters a casino and tries his/her luck at the roulette. The gambler plays until he/she has either won a predetermined amount of money or until all the initial stake is lost. We are interested in the probability of success, or equivalently in the ruin probability of the gambler.

The roulette is a game of chance that consists of a ball that rolls at the edge of a spinning wheel and falls in one of 37 coloured pockets on the wheel: 18 pockets are coloured in red, 18 pockets are coloured in black, and one is coloured in green. Before each round of the game, the gambler puts his/her bet on whether the ball will fall in either a red or a black pocket. If the gambler’s call is correct, then he/she wins an amount of chips equal to the bet size, otherwise he/she looses the betted chips. The gambler cannot bet for the green pocket. The presence of the green pocket biases the game in favour of the casino: if the ball falls in the green pocket, then the casino wins all the bets. We are interested in the gambler’s ruin problem: what is the probability that the gambler loses all of his/her initial stakes before reaching a certain amount of profit?

A gambling sequence at the roulette can be formalised as a stochastic process $X(t)$ in discrete time, $t=1,2,3,\ldots$ We define $X(t)=1$ if the ball falls in a red pocket and $X(t)=-1$ if the ball falls in a black pocket in the $t$-th round of the game. If the ball falls in a green pocket we set $X(t)=0$. We denote the bets of the gambler by the process $Y(t)$: if the gambler calls for red we set $Y(t)=1$ and if the gambler calls for black we set $Y(t)=-1$. The gambler does not bet on green. Finally, we assume the bet size of the gambler $b$ is constant.

For an ideal roulette, the random variables $X(t)$ are independently drawn from the distribution

[TABLE]

where $\delta_{x,y}$ is the Kronecker’s delta. The gambler’s bet $Y(t)=y(X(0),X(1),\ldots,X(t-1))$ with $y$ a function that defines the gambler’s betting system. The gambler’s fortune at time $t$ is the process

[TABLE]

where $F(0)=n$ is the initial stake.

The duration of the game is random. The gambler plays until a time $T_{\rm play}$ when the gambler is either ruined, i.e., $F(T_{\rm play})\leq 0$, or the gambler’s fortune has surpassed for the first time a certain amount $m$, i.e., $F(T_{\rm play})\geq m$. Clearly we require that $m>n=F(0)$, since otherwise $T_{\rm play}=0$. The ruin probability

[TABLE]

is the probability that the gambler loses the game.

The gambler’s fortune is a supermartingale because it is a bounded stochastic process satisfying

[TABLE]

for all $s\leq t$ and all $t\geq 0$. Relation (19) means that on average the gambler will inevitably lose money, irrespective of the betting system he/she adopts.

A gambler whose fortune is expected to decrease may still be tempted to play if the probability of winning is high enough. The probabilities to win or loose the game depend on the fluctuations of $X(t)$. In the roulette game, the gambler’s fortune $F(t)$ can be represented as a biased random walk on the real line $[0,m]$, which starts at the position $n$, and moves each time step either a distance $-b$ to the left or a distance $b$ to the right. The probability to make a step to the left is $q=19/37\approx 0.51$ and the probability to make a step to the right is $1-q=18/37\approx 0.49$. Hence, the gambler’s fortune is slightly biased to move towards the left where $F(t)<0$. The ruin probability $p_{\rm ruin}(n)=\mathbb{P}\left[F(T_{\rm play})\leq 0\right]$ solves the recursive equation

[TABLE]

with boundary conditions $p_{\rm ruin}(0)=1$ and $p_{\rm ruin}(m)=0$. Instead of solving the relations (20) we bound the ruin probability $p_{\rm ruin}(n)$ using the theory of martingales [32]. We define the process

[TABLE]

The processes $M(t)$ is a martingale relative to the process $X(t)$ [26, 25]. Indeed, we say that a bounded process $M$ is a martingale if

[TABLE]

for all $s<t$.

An important property of martingales is that their expected value evaluated at a stopping time $T$ of the process $X$ equals their expected value at the initial time [26, 25, 24],

[TABLE]

Eq. (23) is known as Doob’s optional stopping theorem and will constitute the main tool in this paper to derive fluctuation properties of stochastic processes. In the present example, since

[TABLE]

and since for $q\geq 0.5$

[TABLE]

Doob’s optional stopping theorem (23) implies that

[TABLE]

Hence, Doob’s optional stopping theorem bounds the gambler’s ruin probability.

The formula (26) provides useful information for the gambler. If we start the game with an initial fortune of $10\pounds$, if we play until our fortune reaches $50\pounds$ and if we bet each game $1\pounds$, then the chance of loosing our initial stake is in between $94.8$ and $95.2$ percent. If, on the other hand, we bet each game $10\pounds$, then the ruin probability is in between $82$ and $85.6$ percent. Hence, the probability of winning increases as a function of the betting size. Indeed, since the game is biased in favour of the casino, the best strategy is to reduce the number of betting rounds to the minimal possible and hope that luck plays in our favour. After all, the outcome of a single game is almost fair, since the odds of winning a single game are $q=19/37\approx 0.51$.

3.2 Martingales and the optional stopping theorem

We now discuss martingales and the optional stopping theorem for generic stochastic processes $\vec{X}(t)$ in discrete or continuous time. A martingale process $M(t)$ with respect to another process $\vec{X}(t)$ is a real-valued stochastic process that satisfies the following three properties [26, 25]:

• •

$M(t)$ is $\mathscr{F}_{t}$-adapted, which means that $M(t)$ is a function on trajectories $\vec{X}(0,\ldots,t)$;

• •

$M(t)$ is integrable,

[TABLE]

• •

the conditional expectation of $M(t)$ given the $\sigma$-algebra $\mathscr{F}_{s}$ satisfies the property

[TABLE]

The conditional expectation of a random variable $M(t)$ given a sub-$\sigma$-algebra $\mathscr{F}_{s}$ of $\mathscr{F}$ is defined as a $\mathscr{F}_{s}$-measurable random variable $E[M(t)|\mathscr{F}_{s}]$ for which $\int_{\omega\in\Phi}E[M(t)|\mathscr{F}_{s}]\>{\rm d}\mathbb{P}=\int_{\omega\in\Phi}M(t)\>{\rm d}\mathbb{P}$ for all $\Phi\in\mathscr{F}_{s}$ [26].

If instead of the equality (28) we have an inequality $E[M(t)|\mathscr{F}_{s}]\ \geq M(s)$, then we call the process a submartingale. If $E[M(t)|\mathscr{F}_{s}]\leq M(s)$, then we call the process a supermartingale.

Fluctuations of a martingale $M(t)$ can be studied with stopping times. Stopping times are the random times when the stochastic process $\vec{X}(t)$ satisfies for the first time a given criterion. Stopping times do not allow for clairvoyance (the stopping criterion cannot anticipate the future) and do not allow for cheating (the criterion does not have access to side information). Aside these constraints, the stopping rule can be an arbitrary function of the trajectories of the stochastic process $\vec{X}(t)$.

Formally, a stopping time $T(\omega)\in[0,\infty]$ of a stochastic process $\vec{X}$ is defined as a random variable for which $\left\{\omega:T(\omega)\leq t\right\}\in\mathscr{F}_{t}$ for all $t\geq 0$. Alternatively, we can also define stopping times as functions on trajectories $\omega=\vec{X}^{\infty}_{0}$ with the property that the function $T(\omega)$ does not depend on what happens after the stopping time $T$.

An important result in martingale theory is Doob’s optional stopping theorem. There exist different versions of the optional stopping theorem, which differ in the conditions assumed for the martingale process $M(t)$ and the stopping time $T$. We discuss the version of the theorem presented as Theorem 3.6 in the book of Liptser and Shiryayev [26].

Let $M(t)$ be a martingale relative to the process $\vec{X}$ and let $T$ be a stopping time on the process $\vec{X}$. If $M$ is uniformly integrable, then

[TABLE]

If time is continuous we also require that $M(t)$ is rightcontinuous. A stochastic process $M(t)$ is called uniformly integrable if

[TABLE]

where $I_{\Phi}(\omega)$ is the indicator function defined by

[TABLE]

for all $\omega\in\Omega$ and $\Phi\in\mathscr{F}$. If $M(t)$ is not uniformly integrable, then (29) may not hold. For example if $M(t)=W(t)$ with $W(t)$ a Wiener process on $[0,\infty)$ and $T={\rm inf}\left\{t\geq 0:M(t)=m\right\}$, then $E\left[W(T)\right]=m\neq E\left[W(0)\right]=0$, where we have used the convention that $0\cdot\infty=0$ [33].

An extended version of the optional stopping theorem holds for two stopping times $T_{1}$ and $T_{2}$ with the property $\mathbb{P}[T_{2}\leq T_{1}]=1$,

[TABLE]

where the $\sigma$-algebra $\mathscr{F}_{T_{2}}$ consists of all sets $\Phi\in\mathscr{F}$ such that $\Phi\cap\left\{\omega:T_{2}(\omega)\leq t\right\}\in\mathscr{F}_{t}$.

4 Stochastic thermodynamics for stationary processes

In this section, we briefly introduce the formalism of stochastic thermodynamics in nonequilibrium stationary states; for reviews see Refs. [1, 2, 3, 4]. We use a probability-theoretic approach [34, 35, 21], which has the advantage of dealing with Markov chains, Markov jump processes, and Langevin process in one unified framework. It is moreover the natural language to deal with martingales.

The stochastic entropy production $S_{\rm tot}(t)$ is defined in terms of a probability measure $\mathbb{P}$ of a stationary stochastic process and its time-reversed measure $\mathbb{P}\circ\Theta$. The time-reversal map $\Theta$, with respect to the origin $t=0$, is a measurable involution on trajectories $\omega$ with the property that $X_{i}(t,\Theta(\omega))=X_{i}(-t,\omega)$ for variables of even parity with respect to time reversal and $X_{i}(t,\Theta(\omega))=-X_{i}(-t,\omega)$ for variables of odd parity with respect to time reversal. We say that the measure $\mathbb{P}$ is stationary if $\mathbb{P}=\mathbb{P}\circ\mathcal{T}_{t}$ for all $t\in\mathbb{R}$, with $\mathcal{T}_{t}$ the time-translation map, i.e., $\vec{X}(t^{\prime},\mathcal{T}_{t}(\omega))=\vec{X}(t^{\prime}+t,\omega)$ for all $t^{\prime}\in\mathbb{R}$. In order to define an entropy production we require that the process $\vec{X}(t,\omega)$ is reversible. This means that for all finite $t\geq 0$ and for all $\Phi\in\mathscr{F}_{t}$ it holds that $\mathbb{P}\left[\Phi\right]=0$ if and only if $(\mathbb{P}\circ\Theta)\left[\Phi\right]=0$. In other words, if an event happens with zero probability in the forward dynamics, then this event also occurs with zero probability in the time-reversed dynamics. In probability theory, one says that $\mathbb{P}$ and $\mathbb{P}\circ\Theta$ are locally mutually absolute continuous.

Given the above assumptions, we define the entropy production in a stationary process $\vec{X}$ by [17, 34, 35, 21]

[TABLE]

where we have used the Radon-Nikodym derivative of the restricted measures $\left.\mathbb{P}\right|_{\mathscr{F}_{t}}$ and $\left.(\mathbb{P}\circ\Theta)\right|_{\mathscr{F}_{t}}$ on $\mathscr{F}$. The restriction of a measure $\mathbb{P}$ on a sub-$\sigma$-algebra $\mathscr{F}_{t}$ of $\mathscr{F}$ is defined by $\left.\mathbb{P}\right|_{\mathscr{F}_{t}}[\Phi]=\mathbb{P}[\Phi]$ for all $\Phi\in\mathscr{F}_{t}$. If $t$ is continuous, then $S_{\rm tot}(t,\omega)$ is rightcontinuous, since we have assumed the rightcontinuity of the filtration $\left\{\mathscr{F}_{t}\right\}_{t\geq 0}$. Local mutual absolute continuity of the two measures $\mathbb{P}$ and $\mathbb{P}\circ\Theta$ implies that the Radon-Nikodym derivative in (35) exists and is almost everywhere uniquely defined. The definition (35) states that entropy production is the probability density of the measure $\mathbb{P}$ with respect to the time-reversed measure $\mathbb{P}\,\circ\,\Theta$; it is a functional of trajectories $\omega$ of the stochastic process $X$ and characterises their time-irreversibility.

The definition (35) of the stochastic entropy production is general. It applies to Markov chains, Markov jump processes, diffusion processes, etc. For Markov chains, the relation (35) is equivalent to the expression (1) for entropy production in terms of probability density functions of trajectories. Consider for example the case of $\vec{X}(t)\in\mathbb{R}^{d}$ and $t\in\mathbb{Z}$ and let us assume for simplicity that all degrees of freedom are of even parity with respect to time reversal. Using $\left.{\rm d}\mathbb{P}\right|_{\mathscr{F}_{t}}=p(\vec{X}(1),\vec{X}(2),\ldots,\vec{X}(t))\left.{\rm d}\lambda\right|_{\mathscr{F}_{t}}$, with $\left.\lambda\right|_{\mathscr{F}_{t}}$ the Lebesgue measure on $\mathbb{R}^{td}$, the entropy production is indeed of the form given by relation (1). However, formula (35) is more general than (1) because it also applies to cases where the path probability density with respect to a Lebesgue measure does not exist, as is the case with stochastic processes in continuous time.

For systems that are weakly coupled to one or more environments in equilibrium, the entropy production (35) is equal to [4]

[TABLE]

where $S_{\rm env}(t)$ is the entropy change of the environment, and where

[TABLE]

is the system entropy change associated with the stationary probability density function $p_{\rm ss}(\vec{X}(t))$ of $\vec{X}(t)$.

5 Integral fluctuation relations at stopping times

In this section we initiate the study of the fluctuations of the entropy production in stationary processes. We follow an approach similar to the one presented in Section 3 for the fluctuations of a gambler’s fortune, namely, we first identify a martingale process related to the entropy production, which is the exponentiated negative entropy $e^{-S_{\rm tot}(t)}$, and we then apply Doob’s optional stopping theorem (29) to this martingale process. Since $e^{-S_{\rm tot}(t)}$ is unbounded, we require uniform integrability of $e^{-S_{\rm tot}(t)}$ in order to apply Doob’s optional stopping theorem (29). Therefore, we obtain two versions of the integral fluctuation relation at stopping times: a first version holds within finite-time windows and a second version holds when $S_{\rm tot}(t)$ is bounded for all times $t\leq T$. These two versions represent two different ways to ensure that the total available entropy in the system and environment is finite. Note that when we applied Doob’s optional stopping theorem in the gambling problem, we also required that the gambler’s fortune is finite. Finally, we obtain conditional integral fluctuation relations by applying the conditional version (34) of Doob’s optional stopping theorem to $e^{-S_{\rm tot}(t)}$.

5.1 The martingale structure of exponential entropy production

The exponentiated negative entropy production $e^{-S_{\rm tot}(t)}$ associated with a stationary stochastic process $\vec{X}(t)$ is a martingale process relative to $\vec{X}(t)$. Indeed, $S_{\rm tot}(t)$ is a $\mathscr{F}(t)$-adapted process, $E\left[e^{-S_{\rm tot}(t)}\right]=1$, and in A we show that [20, 21]

[TABLE]

As a consequence, entropy production is a submartingale:

[TABLE]

Notice that we can draw an analogy between thermodynamics and gambling by identifying the negative entropy production with a gambler’s fortune and by identifying the exponential $e^{-S_{\rm tot}(t)}$ with the martingale (21).

5.2 Fluctuation relation at stopping times within a finite-time window

We apply Doob’s optional stopping theorem (29) to the martingale $e^{-S_{\rm tot}(t)}$. We consider first the case when an experimental observer measures a stationary stochastic processes $\vec{X}(t)$ within a finite-time window $t\in[0,\tau]$. In this case, the experimental observer measures in fact the process $e^{-S_{\rm tot}(t\wedge\tau)}$, where we have used the notation

[TABLE]

The process $e^{-S_{\rm tot}(t\wedge\tau)}$ is uniformly integrable, as we show in the A, and therefore

[TABLE]

holds for all stopping times $T$ of $\vec{X}(t)$.

5.3 Fluctuation relation at stopping times within an infinite-time window

We discuss an integral fluctuation relation for stopping times within an infinite-time window, i.e., $T\in[0,\infty]$. In the B we prove that if the conditions

1. (i)

$e^{-S_{\rm tot}}$ converges $\mathbb{P}$-almost surely to [math] in the limit $t\rightarrow\infty$ 2. (ii)

$S_{\rm tot}(t)$ is bounded for all $t\leq T$

are met, then

[TABLE]

The condition (i) is a reasonable assumption for nonequilibrium stationary states, since $\langle S_{\rm tot}(t)\rangle$ grows extensive in time as $\langle S_{\rm tot}(t)\rangle=\sigma t$ with $\sigma$ a positive number. The condition (ii) can be imposed on $T$ by considering a stopping time $T\wedge T_{\rm fp}$ where $T_{\rm fp}={\rm inf}\left\{t:S_{\rm tot}(t)\notin(-s_{-},s_{+})\right\}$ is a first-passage time with two thresholds $s_{-},s_{+}\gg 1$, which can be considered large compared to the typical values of entropy production at the stopping time $T$.

5.4 Conditional integral fluctuation relation at stopping times

We can also apply the conditional optional stopping-time theorem (34) to $e^{-S_{\rm tot}(t)}$. We then obtain the conditional integral fluctuation relation (3) for stopping times $T_{2}\leq T_{1}$, viz.,

[TABLE]

The relation (43) is valid either for finite stopping times $T_{1}\in[0,\tau]$ or for stopping times $T_{1}$ and $T_{2}$ for which $S_{\rm tot}(t)$ is bounded for all $t\in[0,T_{1}]$. The integral fluctuation relations at stopping times (41), (42) and (43) imply that certain stochastic properties of entropy production are universal. This will be discussed in the next section.

6 Second law of thermodynamics at stopping times

Jensen’s inequality $\langle e^{-S_{\rm tot}(T\wedge\tau)}\rangle\geq e^{-\langle S_{\rm tot}(T\wedge\tau)\rangle}$ together with the integral fluctuation relation (41) imply that

[TABLE]

The relation (44) states that on average entropy production always increases, even when we stop the process at a random time $T$ chosen according to a given protocol. This law is akin to the relation (19) describing that a gambler cannot make profit out of a fair game of chance, even when he/she quits the game in an intelligent manner. Analogously, (42) implies the law $\langle S_{\rm tot}(T)\rangle\geq 0$ for unbounded stopping times $T$. We have thus derived the second law (4) of thermodynamics at stopping times.

When applying this second law to examples of physical processes one obtains interesting bounds on heat and work in nonequilibrium stationary states. Below we first discuss bounds on the average dissipated heat in isothermal processes and then bounds on the average work in stationary stochastic heat engines.

6.1 Bounds on heat absorption in isothermal processes

For systems that are in contact with one thermal reservoir at temperature $\mathsf{T}_{\rm env}$ and for which the entropy of hidden internal degrees of freedom is negligible, the entropy production is given by (36-37), and the environment entropy takes the form [4, 36]

[TABLE]

where $Q$ is the heat transferred from the environment to the system. Relation (45) relates the stochastic entropy production (36) to the stochastic heat that enters into the first law of thermodynamics. Therefore, for isothermal systems the second law (44) reads

[TABLE]

Analogously, we obtain the relation (5) for unbounded stopping times $T$.

The relation (46) implies that it is not possible to extract on average heat from a thermal reservoir when the system state is invariant. Indeed, when $X(T\wedge\tau)=X(0)$, then $\langle Q(T\wedge\tau)\rangle\leq 0$. However, if the system entropy at the stopping time $T$ is different than the entropy in the stationary state, then it is possible to extract on average at most an amount \mathsf{T}_{\rm env}\Big{\langle}\log\frac{p_{\rm ss}(X(0))}{p_{\rm ss}(X(T\wedge\tau))}\Big{\rangle} of heat from the thermal reservoir.

For systems in equilibrium $p_{\rm ss}(x)\sim e^{-v(x)/\mathsf{T}_{\rm env}}$, such that the bound (46) reads $\langle Q(T\wedge\tau)\rangle\leq\langle\Delta v(T\wedge\tau)\rangle$. Moreover, according to the first law of thermodynamics $\langle Q(T\wedge\tau)\rangle=\langle\Delta v(T\wedge\tau)\rangle$, such that for systems in equilibrium the bound (46) is tight.

Notice that the bound on the right hand side of (46) is maximal for stopping times of the form

[TABLE]

If $X(t)$ is a recurrent process, i.e. $\mathbb{P}(T^{\dagger}<\infty)=1$, and if $\tau\rightarrow\infty$, then

[TABLE]

6.2 Efficiency of heat engines at stopping times: the case of Feynman’s ratchet

We consider stochastic heat engines in contact with two thermal baths at temperatures $\mathsf{T}_{\rm c}$ and $\mathsf{T}_{\rm h}$ with $\mathsf{T}_{h}\geq\mathsf{T}_{c}$. A paradigmatic example is Feynman’s ratchet [37, 38, 39, 40, 41], which is composed of a ratchet wheel with a pawl that is mechanically linked by an axle to a vane. The ratchet wheel and the pawl are immersed in a hot thermal reservoir, and the vane is immersed in a cold thermal reservoir. An external mass is connected to the axle of the Feynman ratchet and follows the movement of the ratchet wheel. If the wheel turns in the clockwise direction then the axle performs work on the mass, whereas if the wheel turns in the counterclockwise direction then the mass performs work on the axle .

We now perform an analysis of the Feynman ratchet at stopping times. For example, we ask the question what are the efficiency and the power of the ratchet when the system is stopped right before or after the ”main event”, i.e., the passage of the pawl over the peak of the ratchet wheel. The first law of thermodynamics implies that

[TABLE]

where $Q_{\rm h}$ is the heat absorbed by the ratchet from the hot reservoir, $Q_{\rm c}$ is heat absorbed by the ratchet from the cold reservoir, $v$ is the mechanical energy stored in the pawl, and $W$ is the work performed on the external mass. For the Feynman’s ratchet, the second law of thermodynamics at stopping times $T$ reads

[TABLE]

where $S_{\rm sys}$ is the entropy of the ratchet. If $\langle Q_{\rm h}(T)\rangle>0$, the first and second law of thermodynamics at stopping times imply the inequality (7), i.e.,

[TABLE]

where we have introduced the efficiency at stopping times

[TABLE]

the Carnot efficiency

[TABLE]

and the system free energy

[TABLE]

For $T=t$, $\langle\Delta F_{\rm c}(T)\rangle=0$ and we obtain the classical Carnot bound $\eta_{t}\leq\eta_{\rm C}$. Moreover, if $X(T)=X(0)$, which implies that the process stops when it returns to its original state, then $\langle\Delta F_{\rm c}(T)\rangle=0$ and we obtain again the classical Carnot bound $\eta_{t}\leq\eta_{\rm C}$. Hence, it is not possible to exceed on average the Carnot efficiency when the final state equals the initial state and thus when the heat engine is a cyclic process in phase space.

However, for general stopping times $T$, $\langle\Delta\mathcal{F}_{\rm c}(T)\rangle$ is different than zero. Interestingly, for stopping times $T$ for which $\langle\Delta\mathcal{F}_{c}(T)\rangle/\langle Q_{\rm h}(T)\rangle$ is negative, the second law of thermodynamics at stopping times implies that $\eta_{T}$ is bounded by a constant that is larger than the Carnot efficiency. Note that the stopping-time efficiency $\eta_{T}$ is defined as the ratio of averages, and not as the average of the ratios. In general $\langle W(T)\rangle/\langle Q_{\rm h}(T)\rangle\neq\langle W(T)/Q_{\rm h}(T)\rangle$, and the latter corresponds to the average of an unbounded random variable whose value at fixed times $T=t$ has been previously studied in [42, 43, 44, 45, 12].

Another interesting property of thermodynamic observables at stopping times is that they can take a different sign with respect to their stationary averages. For example, it is possible that at the stopping time $T$ the fluxes of the Feynman ratchet have the same sign as those in a refrigerator, namely, $\langle W(T)\rangle>0$, $\langle Q_{\rm c}(T)\rangle>0$ and $\langle Q_{\rm h}(T)\rangle<0$. To evaluate the performance of this process, we introduce the coefficient $\epsilon_{T}:=-\langle Q_{\rm c}(T)\rangle/\langle W(T)\rangle>0$, for which the second law of thermodynamics at stopping times reads $\epsilon_{T}\leq\left[\left(\frac{\mathsf{T}_{\rm h}}{\mathsf{T}_{\rm c}}-1\right)+\frac{\langle\Delta\mathcal{F}_{\rm h}(T)\rangle}{\langle Q_{\rm c}(T)\rangle}\right]^{-1}$, with $\langle\Delta\mathcal{F}_{\rm h}(T)\rangle:=\langle\Delta v(T)\rangle-\mathsf{T}_{\rm h}\langle\Delta S_{\rm sys}(T)\rangle$. For fixed times $T=t$ and for stopping times $T$ with $X(T)=X(0)$, we recover the classical bound $\epsilon_{T}\leq\mathsf{T}_{\rm c}/(\mathsf{T}_{\rm h}-\mathsf{T}_{\rm c})$ [46, 47, 48].

In Section 9, we illustrate the bounds (5) and (7) on simple physical models.

7 Universal properties of stochastic entropy production

We use the fluctuation relations (41), (42) and (43) to derive universal relations about the stochastic properties of entropy production in stationary processes.

All our results hold for nonequilibrium stationary states for which $\lim_{t\rightarrow\infty}e^{-S_{\rm tot}(t)}=0$ ($\mathbb{P}$-almost surely).

If entropy production is bounded for $t<T$, then we will use the optional stopping theorem (42) for stopping times within infinite-time windows , and if entropy production is unbounded for $t<T$, then we will use the optional stopping theorem (41) for stopping times within finite-time windows.

7.1 Fluctuation properties at fixed time $T=t$

We first consider the case where the stopping time $T$ is a fixed non-fluctuating time $t$, i.e., $T=t$. In this case the fluctuation relation (41) is the integral fluctuation theorem derived in [16],

[TABLE]

and the second law inequality (44) yields the second law of stochastic thermodynamics [4]

[TABLE]

The relation (53) provides a bound on negative fluctuations of entropy production, and for isothermal systems bounds on the fluctuations of work [3]. Since $e^{-S_{\rm tot}(t)}$ is a positive random variable, we can use Markov’s inequality, see e.g. chapter 1 of [49], to bound events of large $e^{-S_{\rm tot}(t)}$, namely,

[TABLE]

Using the integral fluctuation relation (53) together with (55) we obtain

[TABLE]

which is a well-known bound on the probability of negative entropy production, see Equation (54) in [4].

The relations (41) and (44) are more general than the relations (53) and (54), since the former concern an average over an ensemble of trajectories of variable length $x^{T}_{0}$ whereas the latter concern an average over an ensemble of trajectories of fixed length $x^{t}_{0}$. Therefore, we expect that evaluating (41) at fluctuating stopping times $T$ it is possible to derive stronger constraints on the probability of negative entropy production than (56). This is the program we will pursue in the following sections.

7.2 Splitting probabilities of entropy production

We consider the first-passage time

[TABLE]

for entropy production with two absorbing thresholds at $-s_{-}<0$ and $s_{+}>0$. If the set $\left\{t:S_{\rm tot}(t)\notin(-s_{-},s_{+})\right\}$ is empty, then $T_{\rm fp}=\infty$. Since entropy production is bounded for all values $t<T_{\rm fp}$ we can use the optional stopping theorem (42).

We split the ensemble of trajectories $\Omega$ into two sets $\Omega_{-}=\left\{S_{\rm tot}(T_{\rm fp})\leq-s_{-}\right\}$ and $\Omega_{+}=\left\{S_{\rm tot}(T_{\rm fp})\geq s_{+}\right\}$. Since $S_{\rm tot}(T_{\rm fp})\in(\infty,-s_{-}]\cup[s_{+},\infty)$ and $\lim_{t\rightarrow\infty}e^{-S_{\rm tot}(t)}=0$ ($\mathbb{P}$-almost surely), the splitting probabilities $p_{-}=\mathbb{P}\left[\Omega_{-}\right]$ and $p_{+}=\mathbb{P}\left[\Omega_{+}\right]$ have a total probability of one,

[TABLE]

We apply the integral fluctuation relation (42) to the stopping time $T_{\rm fp}$:

[TABLE]

where

[TABLE]

The relations (58) and (59) imply that

[TABLE]

Moreover, since $S_{\rm tot}(T_{\rm fp})\in(\infty,-s_{-}]\cup[s_{+},\infty)$ we have that $\langle e^{-S_{\rm tot}(T)}\rangle_{-}\geq e^{s_{-}}$ and $\langle e^{-S_{\rm tot}(T)}\rangle_{+}\leq e^{-s_{+}}$ . Using these two inequalities in (62) and (63) we obtain the universal inequalities (9) for the splitting probabilities, viz.,

[TABLE]

which hold for first-passage times with $s_{-},s_{+}>0$.

In the case where $\vec{X}(t)$ is a continuous stochastic process $S_{\rm tot}(T_{\rm fp})\in\left\{-s_{-},s_{+}\right\}$ holds with probability one. Using in (62) and (63) that $\langle e^{-S_{\rm tot}(T)}\rangle_{-}=e^{s_{-}}$ and $\langle e^{-S_{\rm tot}(T)}\rangle_{+}=e^{-s_{+}}$, we obtain

[TABLE]

which are the relations (10). Hence, the splitting probabilities of entropy production are universal for continuous processes.

The bounds (9) apply not only to first-passage times but hold more generally for stopping times $T$ of $\vec{X}(t)$ for which $S_{\rm tot}(T)\in(\infty,-s_{-}]\cup[s_{+},\infty)$ holds with probability one and for which $S_{\rm tot}(t)$ is bounded for all $t\in[0,T]$. Analogously, the equalities (10) apply not only to first-passage times but hold more generally for stopping times $T$ of $\vec{X}(t)$ for which $S_{\rm tot}(T)\in\left\{-s_{-},s_{+}\right\}$ holds with probability one and for which $S_{\rm tot}(t)$ is bounded for all $t\in[0,T]$.

7.3 Infima of entropy production

We can use the results of the previous subsection to derive universal bounds and equalities for the statistics of infima of entropy production. The global infimum of entropy production is defined by

[TABLE]

and denotes the most negative value of entropy production.

Consider the first-passage time $T_{\rm fp}$, denoting the first time when entropy production either goes below $-s_{-}$ or goes above $s_{+}$, with $s_{-},s_{+}\geq 0$ and its associated splitting probability $p_{-}$ The cumulative distribution of $S_{\rm inf}$ is given by

[TABLE]

Using the inequalities (9) we obtain the bound (11) for the cumulative distribution of infima of entropy production, i..e.,

[TABLE]

The inequality (11) bears a strong similarity with the inequality (56). However, since by definition $S_{\rm inf}\leq S_{\rm tot}(t)$ for any value of $t$, we also have that $\mathbb{P}\left[S_{\rm tot}\leq-s_{-}\right]\leq\mathbb{P}\left[S_{\rm inf}\leq-s_{-}\right]$ and therefore the inequality (11) is stronger.

For continuous processes $\vec{X}$, the inequality (11) becomes an equality. Indeed, if the stochastic process $\vec{X}$ is continuous, then with probability one $S_{\rm tot}(T_{\rm fp})\in\left\{-s_{-},s_{+}\right\}$, and therefore $p_{-}$ is given by (10). Using the relation (65), we obtain

[TABLE]

As a consequence, for continuous stochastic processes the global infimum $S_{\rm inf}$ of entropy production is characterised by an exponential probability density (12) with mean value mean value $\langle S_{\rm inf}\rangle=-1$.

7.4 Survival probability of entropy production

We analyse the survival probability

[TABLE]

of the first-passage time with one absorbing boundary,

[TABLE]

for continuous stochastic processes $\vec{X}(t)$. If the set $\left\{t:S_{\rm tot}(t)=s_{0}\right\}$ is empty, then $\tilde{T}=\infty$.

We use the fluctuation relation (41) since for first-passage times with one absorbing boundary $S_{\rm tot}(t)$ is unbounded for $t\in[0,T]$. Applying (41) to $\tilde{T}$ we obtain

[TABLE]

and thus also relation (13), i.e.,

[TABLE]

where

[TABLE]

For positive values of $s_{0}$ we expect that $\lim_{t\rightarrow\infty}p_{\rm sur}(t)=0$. This implies that $\lim_{t\rightarrow\infty}\langle e^{-S_{\rm tot}(t)}\rangle_{\rm sur}=\infty$. For negative values of $s_{0}$ we expect that $\lim_{t\rightarrow\infty}p_{\rm sur}(t)=1-e^{s_{0}}=\mathbb{P}\left[S_{\rm inf}\geq s_{0}\right]$ which holds if \lim_{t\rightarrow\infty}\Big{\langle}e^{-S_{\rm tot}(t)}I_{\tilde{T}>t}/\langle I_{\tilde{T}>t}\rangle\Big{\rangle}=0.

7.5 Number of crossings

We consider the number of times $N_{\times}$ entropy production crosses an interval $[-\Delta,\Delta]$ from the negative side to the positive side, i.e., in the direction $-\Delta\rightarrow\Delta$. We can bound the distribution of $N_{\times}$ using a sequence of stopping times.

The probability that $N_{\times}>0$ is equal to the probability that the infimum is smaller or equal than $-\Delta$, and therefore using (11) we obtain

[TABLE]

Applying the conditional fluctuation relation (43) on two sequences of stopping times, we derive in the C the inequality

[TABLE]

and therefore

[TABLE]

which is the inequality (14). The probability of observing a large number of crossing decays thus at least exponentially in $N_{\times}$. For continuous processes $\vec{X}(t)$ the probability mass function $\mathbb{P}(N_{\times})$ is a universal statistic given by

[TABLE]

which is the same relation as derived in [22] for overdamped Langevin processes.

8 The influence of finite statistics on the integral fluctuation relation at stopping times

In empirical situations we may want to use the integral fluctuation relation at stopping times to test whether a given process is the stochastic entropy production [50, 51, 52]. In these cases we have to deal with finite statistics. It is useful to know how many experiments are required to verify the integral fluctuation relation at stopping times with a certain accuracy. Therefore, in this section we discuss the influence of finite statistics on tests of the integral fluctuation relation at stopping times.

We consider the case where $T$ is a two-boundary first-passage time for the entropy production of a continuous stationary process, i.e., $T=T_{\rm fp}={\rm inf}\left\{t\geq 0:S_{\rm tot}(t)\notin(-s_{-},s_{+})\right\}$. We imagine to estimate in an experiment the average $\langle e^{-S_{\rm tot}(T_{\rm fp})}\rangle$ using an empirical average over $m_{\rm s}$ realisations,

[TABLE]

where the $T_{j}$ are the different outcomes of the first passage time $T_{\rm fp}$ and where $N_{-}$ is the number of trajectories that have terminated at the negative boundary, i.e., for which $S_{\rm tot}(T_{j})=-s_{-}$. The expected value of the sample mean is thus

[TABLE]

and the variance of the sample mean is

[TABLE]

Hence, for small enough values of $s_{-}$ a few samples ${m_{\rm s}}$ will be enough to test the stopping-time fluctuation relation. For large enough $s_{-}$, we obtain $\langle A^{2}\rangle-\langle A\rangle^{2}\sim e^{s_{-}}/m_{\rm s}$. The number of required samples ${m_{\rm s}}$ scales exponentially in the value of the negative threshold $s_{-}$. The full distribution of the empirical estimate $A$ of $\langle e^{-S_{\rm tot}(T_{\rm fp})}\rangle$ is given by

[TABLE]

where we have again used $\delta$ for the Kronecker delta.

Since we know the full distribution of the empirical average $A$, the integral fluctuation relation $\langle e^{-S_{\rm tot}(T_{\rm fp})}\rangle=1$ can be tested in experiments: given a certain observed value of $A\neq 1$ we can use the distribution (81) to compute its $p$-value, i.e., the probability to observe a deviation from $1$ larger or equal than the empirically observed $|A-1|$.

9 Examples

We illustrate the bounds (5) and (7) on two simple examples of systems described by Langevin equations. We demonstrate a randomly stopped process can extract work from a stationary isothermal process and we show that heat engines can surpass the Carnot efficiency at stopping times. Moreover, the mean of the extracted heat and the performed work are bounded by the second law of thermodynamics at stopping times.

9.1 Heat extraction in an isothermal system

We illustrate the second law (5), for the average heat at stopping times, on the case of a colloidal particle on a ring that is driven by an external force $f$ and moves in a potential $v(y)$, where $y\in[0,2\pi\ell]$ denotes the position on the ring and where $\ell$ is the radius of the ring. We ask how much heat $\langle Q(T^{\ast})\rangle$ a colloidal particle can extract on average from its environment at the time $T^{\ast}$ when the particle reaches for the first time the highest peak of the landscape.

We assume that the dynamics of the colloidal particle is governed by an overdamped Langevin equation of the form

[TABLE]

where $X(t)\in\mathbb{R}$ , $\mu$ is the mobility coefficient, $d$ is the diffusion coefficient, and $\zeta(t)$ is a Gaussian white noise with $\langle\zeta(t)\zeta(t^{\prime})\rangle=\delta(t^{\prime}-t)$. We assume that the environment surrounding the particle is in equilibrium at a temperature $\mathsf{T}_{\rm env}$, so that Einstein relation $d=\mathsf{T}_{\rm env}\mu$ holds. The potential $v(x)$ is a periodic function with period $2\pi\ell$, i.e., $v(x)=v(x+2\pi\ell)$. The actual position of the particle is given by the variable $Y(t)=X(t)-N(t)2\pi\ell\in[0,2\pi\ell)$, where $N(t)\in\mathbb{Z}$ is the winding number, i.e., the net number of times the particle has traversed the ring. The heat $Q$ can be expressed as [53, 2]

[TABLE]

and the stationary distribution is [7]

[TABLE]

We simulate the model (82) for a periodic potential of the form [54],

[TABLE]

which is illustrated in figure 1(a) for $\ell=1$ and $\mathsf{T}_{\rm env}=1$. In this case, the time $T^{\ast}$ when the particle reaches for the first time the highest peak of the landscape is

[TABLE]

and the stationary distribution of $X(t)$ is given by [54]

[TABLE]

Hence, the bound (82) reads

[TABLE]

In figure 1(b) we illustrate the bound (88) for $\langle Q(T^{\ast})\rangle$ as a function of the nonequilibrium driving $f$. We find that heat absorption at stopping times is significant at small values of $f$ and in the linear-response limit of small $f$ the bound (88) is tight; the tightness of the bound (82) holds in general for recurrent Markov processes in the linear response limit. For comparison we also plot the mean heat dissipation $\langle Q(t)\rangle$ at a fixed time $t=1$. While $\langle Q(T^{\ast})\rangle$ is positive at small values of $f$, the dissipation $\langle Q(t)\rangle$ at fixed times is always negative. Note that at intermediate times $\langle Q(1)\rangle>\langle Q(T^{\ast})\rangle$, but if we increase $f$ furthermore then eventually $\langle Q(1)\rangle<\langle Q(T^{\ast})\rangle$ (not shown in the figure).

9.2 Illustration of an empirical test of the integral fluctuation relation at stopping times

We use the the model described in Subsection 9.1 to illustrate how the integral fluctuation relation at stopping times (2) with $T=T_{\rm fp}={\rm inf}\left\{t\geq 0:S_{\rm tot}(t)\notin(-s_{-},s_{+})\right\}$ can be tested in an experimental setup. To this aim, we will verify whether the quantity $A$ defined in  (76), which is the empirical average of $e^{-S_{\rm tot}(T)}$, converges for $m_{s}\rightarrow\infty$ to one. We use the results on the statistics of the empirical average of $A$ described in Section 8 to validate the statistical significance of the experimental results.

In Figure 2 we plot the empirical average (76) as a function of the number of samples $m_{s}$ for ten simulation runs. We also plot the theoretical curves $1+\sigma_{A}$ and $1-\sigma_{A}$, with $\sigma_{A}$ the standard deviation of $A$, see (78). We observe in Figure 2 that all test runs lie within the $1\pm\sigma_{A}$ confidence intervals, and we can thus conclude that numerical experiments are in agreement with the integral fluctuation relation at stopping times and thus also with the fact that $e^{-S_{\rm tot}(t)}$ is martingale.

9.3 Super Carnot efficiency for heat engines at stopping times

We illustrate the bound (7) for the efficiency of stationary stochastic heat engines at stopping times with a Brownian gyrator [55], which is arguably one of the simplest models of a Feynman ratchet. This system is described by two degrees of freedom $x_{1},x_{2}$ that are driven by an external force field $f(x_{1},x_{2})$ and interact via a potential $v(x_{1},x_{2})$. Two thermal reservoirs at temperatures $\mathsf{T}_{\rm h}$ and $\mathsf{T}_{\rm c}$, with $\mathsf{T}_{\rm h}>\mathsf{T}_{\rm c}$, interact independently with the coordinates $x_{1}$ and $x_{2}$ of the system, respectively. We are specifically interested in the efficiency $\eta_{T}=-\langle W(T)\rangle/\langle Q_{\rm h}(T)\rangle$, which is the ratio between the work $-W(T)$ the gyrator performs on its suroundings in a time interval $[0,T]$, and the heat $Q_{\rm h}(T)$ absorbed by the gyrator in the same time interval, with $T$ the stopping time at which a specific criterion is first satisfied. The efficiency $\eta_{T}$ is a measure of the average amount of work the gyrator performs on its environment.

We consider a Brownian gyrator described by the two coupled stochastic differential equations [55, 56, 57, 58, 59]

[TABLE]

Here $\mu$ is the mobility coefficient, $d_{1}=\mu\mathsf{T}_{\rm h}$ and $d_{2}=\mu\mathsf{T}_{\rm c}$ are the diffusion coefficients of the two degrees of freedom, $v$ is a potential, and $f_{1}$ and $f_{2}$ are two external nonconservative forces, whose functional form we specify below. We use the model from Ref. [57], for which the potential is

[TABLE]

with $u_{1},u_{2},c>0$ and $c<\sqrt{u_{1}u_{2}}$ and for which the two components of the external nonconservative force are

[TABLE]

The two thermal reservoirs induce two stochastic forces with amplitudes $d_{1}$ and $d_{2}$, which appear in Eqs.(89-90) as two independent Gaussian white noises $\zeta_{1}$ and $\zeta_{2}$ with zero mean and autocorrelation ($i,j=1,2$)

[TABLE]

Because of the external driving forces $f_{1}$ and $f_{2}$ and the presence of two thermal reservoirs at different temperatures, the gyrator develops a nonequilibrium stationary state characterised by a current in the clockwise direction, see Fig. 3, and a non-zero entropy production. At stationarity, we measure the work $W$ that the external driving force exerts on the gyrator and the net heat $Q_{\rm h}$ and $Q_{\rm c}$ that the system absorbs from the hot and cold reservoirs, respectively. Following Sekimoto [39, 53], these quantities are, respectively,

[TABLE]

where $\circ$ denotes that the stochastic integrals are interpreted in the Stratonovich sense. When $k<k_{\rm s}$ this system operates as an engine [57], i.e., $\langle W(t)\rangle<0$, $\langle Q_{\rm h}(t)\rangle>0$ and $\langle Q_{\rm c}(t)\rangle>0$, with

[TABLE]

the stall parameter and $\eta_{\rm C}$ Carnot’s efficiency (51). The efficiency of the engine in the nonequilibrium stationary state satisfies

[TABLE]

Note that when $k\rightarrow k_{\rm s}$, then $\eta\rightarrow\eta_{\rm C}$.

We now investigate the efficiency of the Brownian gyrator at stopping times, $\eta_{T}=-\langle W(T)\rangle/\langle Q_{\rm h}(T)\rangle$. The simplest example of stopping times are the trivial stopping times $T=t$, with $t$ a fixed time, for which $\eta_{t}\leq\eta_{\rm C}$. A more interesting example is the time $T_{\rm me}$ of the main event, i.e. the gyrator crosses the positive $x_{2}$ axis while moving in the clockwise direction, occurs for the first time. Mathematically, $T_{\rm me}$ can be defined as follows: let $z(t)=x_{1}(t)+ix_{2}(t)$ be a complex number whose real and imaginary parts are $x_{1}(t)$ and $x_{2}(t)$ respectively; we define $z(t)=r(t)e^{i\varphi(t)}$, with $r(t)=\sqrt{x_{1}^{2}(t)+x_{2}^{2}(t)}$ its modulus and with $\varphi(t)=\tan^{-1}(x_{2}(t)/x_{1}(t))\in[-\pi,\pi]$ its phase; the stopping time at the main event is defined as

[TABLE]

Figure 4(a) illustrates the stopping strategy defined by Eq. (99) with numerical simulations. In Fig. 4(b), we compare the values of the efficiency $\eta_{t}$ at a fixed time $t=5$ with the efficiency $\eta_{T_{\rm me}}$ at the main event, both as a function of the driving parameter $k$. We observe that $\eta_{t}$ is well described by Eq. (98) and thus is smaller than the Carnot efficiency, whereas $\eta_{T_{\rm me}}$ can surpass the Carnot efficiency if the strength of the driving force $k$ is large enough (Fig. 4(b) red circles). Interestingly, the observed super Carnot efficiencies at stopping times are in agreement with the bound (7), $\eta_{T_{\rm me}}\ \leq\eta_{\rm C}+\langle\Delta\mathcal{F}_{c}(T_{\rm me})\rangle/\langle Q_{\rm h}(T_{\rm me})\rangle$ and thus compatible with the second law at stopping times (4). Moreover, the bound (7) becomes tight when $k$ is large, which corresponds to a close-to-equilibrium limit [cf. Fig. 1(b)]. Hence, efficiencies of stopped engines can surpass the Carnot bound if $\langle Q_{\rm h}(T)\rangle>0$ and $\langle\Delta S_{\rm sys}(T)\rangle>\mathsf{T}_{\rm c}\langle\Delta v(T)\rangle$, which is consistent with the bound (7).

Notice that efficiencies of stopped heat engines can be larger than one because internal system energy can be converted into useful work. To better understand this feature, we plot in Fig. 5 the average energetic fluxes at stopping times, namely, $\langle W(T)\rangle$, $\langle Q_{\rm h}(T)\rangle$ and $\langle Q_{\rm c}(T)\rangle$, together with the change of the internal system energy $\langle\Delta v(T)\rangle$ and the internal system entropy change $\langle\Delta S_{\rm sys}(T)\rangle$. At fixed times $T=5$, we observe the well-known features of a cyclic heat engine for which $\langle\Delta v(t)\rangle\simeq\langle\Delta S_{\rm sys}(t)\rangle\simeq 0$ (Fig. 5(a)). Although the heat and work fluxes of stopped engines have the same sign as those of cyclic heat engines, the $\langle\Delta v(T_{\rm me})\rangle<0$ and $\langle\Delta S_{\rm sys}(T_{\rm me})\rangle<0$, which indicates that on average the energy and entropy of the gyrator are at $T_{\rm me}$ smaller than their initial values (Fig. 5(b)).

This result suggests a recipe in the quest of super Carnot efficiencies at stopping times, namely by designing stopping strategies that lead to a reduction of the energy of the system. In our example, $\langle\Delta S_{\rm sys}(T_{\rm me})\rangle>\langle\Delta v(T_{\rm me})\rangle$, which enables the appearance of super-Carnot stopping-time efficiencies which are nevertheless compatible with the second law at stopping times. This result motivates further research on so-called type-II efficiencies at stopping times defined as the ratio between the average input and output fluxes of entropy production [60, 44, 61].

In Fig. 6 we illustrate the bound provided by Eq. (7) for a wider range of parameters. We observe that when $k$ exceeds the stall parameter $k_{\rm s}$ the thermodynamic fluxes obey $\langle W(T_{\rm me})\rangle<0$, $\langle Q_{\rm h}(T_{\rm me})\rangle<0$ and $\langle Q_{\rm c}(T_{\rm me})\rangle<0$. This behaviour is still compatible with the second law of thermodynamics at stopping times. Indeed, for this range of parameters $\eta_{T_{\rm me}}<0$ and the bound (7) becomes $\eta_{T_{\rm me}}\ \geq\eta_{\rm C}-\frac{\langle\Delta\mathcal{F}_{c}(T)\rangle}{\langle Q_{\rm h}(T)\rangle}$. This bound is corroborated by numerical simulations in Fig. 6(a) and Fig. 6(b). We define this type of operation as a ”type III heater”, which is different than the type I and type II heaters [62, 48], that appear in stochastic thermodynamics at fixed times.

10 Discussion

We have derived fluctuation relations at stopping times for the entropy production of stationary processes. These fluctuation relations imply a second law of thermodynamics at stopping times — which states that on average entropy production in a nonequilibrium stationary state always increases, even when the experimental observer measures the entropy production at a stopping time— and imply that certain fluctuation properties of entropy production are universal.

We have shown that the second law of thermodynamics at stopping times has important consequences for the nonequilibrium thermodynamics of small systems. For instance, the second law at stopping times implies that it is possible to extract on average heat from an isothermal environment by applying stopping strategies to a physical system in a nonequilibrium stationary state; heat is thus extracted from the environment without using a feedback control, as is the case with Maxwell demons [63, 64, 65]. Furthermore, we have demonstrated, using numerical simulations with a Brownian gyrator, that the average efficiency of a stationary stochastic heat engine can surpass Carnot’s efficiency when the engine is stopped at a cleverly chosen moment. This result is compatible with the second law at stopping times, which provides a bound on the efficiency of stochastically stopped engines. Note that the heat engines described in this paper are non cyclic devices since they are stopped at the stopping time $T$. It would be interesting to explore how stochastically stopped engines can be implemented in experimental systems such as, electrical circuits [66], autonomous single-electron heat engines [67], feedback traps [68], colloidal heat engines [11], and Brownian gyrators [56].

Integral fluctuation relations at stopping times imply bounds on the probability of events of negative entropy production that are stronger than those obtained with the integral fluctuation relation at fixed times. For example, the integral fluctuation relation at stopping times implies that the cumulative distribution of infima of entropy production is bounded by an exponential distribution with mean $-1$. Moreover, for continuous processes the integral fluctuation relation at stopping times implies that the cumulative distribution of the global infimum of entropy production is equal to an exponential distribution. A reason why the integral fluctuation relation at stopping times is more powerful than the integral fluctuation relation at fixed times — in the sense of bounding the likelihood of events of negative entropy production — is because with stopping times we can describe fluctuations of entropy production that are not accessible with fixed-time arguments, such as fluctuation of infima of entropy production.

The integral fluctuation relation at stopping times implies also bounds on other fluctuation properties of entropy production, not necessarily related to events of negative entropy production. For example, we have used the integral fluctuation relation to derive bounds on splitting probabilities of entropy production and on the number of times entropy production crosses a certain interval. For continuous processes these fluctuation properties of entropy production are universal, and we have obtained generic expressions for these fluctuation properties of entropy production.

Since martingale theory has proven to be very useful to derive generic results about the fluctuations of entropy production, the question arises what other physical processes are martingales. In this context, the exponential of the housekeeping heat has been demonstrated to be a martingale [30, 69]. The housekeeping heat is an extension of entropy production to the case of non-stationary processes. We expect that all formulas presented in this paper extend in a straightforward manner to the case of housekeeping heat. Recently also a martingale related to the quenched dynamics of spin models [70] and a martingale in quantum systems [71] have been discovered.

There exist universal fluctuation properties that are implied by the martingale property of $e^{-S_{\rm tot}(t)}$, but are not discussed in this paper. For example, martingale theory implies a symmetry relation for the distribution of conditional stopping times [72, 73, 21, 74]. These symmetry relations are also proved by using the optional stopping theorem, but they cannot be seen as a straightforward consequence of the integral fluctuation relation at stopping times.

Since the integral fluctuation relation at stopping times (41) is a direct consequence of the martingale property $e^{-S_{\rm tot}(t)}$, testing the fluctuation relation (41) in experiments could serve as a method to demonstrate that $e^{-S_{\rm tot}(t)}$ is a martingale. It is not so easy to show in an experiment that a stochastic process is a martingale: it is a herculean task to verify the condition (28). A recent experiment [75] shows that the entropy production of biased transport of single charges in a double electronic dot behaves as a martingale. The inequality (66) for the infima of entropy production was shown to be valid in this experiment. The integral fluctuation relation at stopping times (41) provides an interesting alternative to test martingality of $e^{-S_{\rm tot}(t)}$, because the integral fluctuation relation at stopping times is an equality. Hence, the integral fluctuation relation at stopping times could serve as a proxy for the martingale structure of $e^{-S_{\rm tot}(t)}$ in experiments.

Testing the integral fluctuation relation at stopping times in experimental setting may also be advantageous with respect to testing the standard fluctuation relation at fixed times. The number of samples required to test the standard fluctuation relation increases exponentially with time, since events of negative entropy are rare. This makes it difficult to test the conditions of stochastic thermodynamics at large time scales with the standard integral fluctuation relation. Moreover, at fixed times the distribution of the empirical mean of the exponentiated negative entropy production is not known. The integral fluctuation relation at stopping times does not have these issues, since negative fluctuations of entropy can be capped at a fixed value $-s_{-}$, which is independent of time $t$, and these negative values can be reached at any time. Moreover, we have derived an exact universal expression for the distribution of the sample mean of the exponentiated negative entropy production at stopping times, which can be used to determine the statistical significance of empirical tests of the integral fluctuation relation at stopping times. The integral fluctuation relation at stopping times is thus a useful relation to test the conditions of stochastic thermodynamics in a certain experimental setup.

Appendix A The exponential of the negative entropy production is a (uniformly integrable) martingale

We prove that $e^{-S_{\rm tot}(t)}$ is a martingale. To this aim, we have to verify the two conditions (27) and (28) for $M(t)=e^{-S_{\rm tot}(t)}$. We present two proofs, one for processes in discrete time using the expression (1) for the entropy production, and a general proof for reversible right-continuous processes $\vec{X}(t)$ using the expression (35) for the entropy production.

A.1 Reversible processes in discrete time

In discrete time

[TABLE]

with $\tilde{p}$ the probability density function associated with the time-reversed dynamics. We have simplified the notation a bit and used that $\vec{x}(t)=\vec{x}_{t}$.

The condition (27) follows from $e^{-S_{\rm tot}(t,\omega)}\geq 0$ and $E\left[e^{-S_{\rm tot}(t,\omega)}\right]=1$ for all $t\geq 0$. The martingale condition (28) also holds,

[TABLE]

A.2 Reversible stationary processes that are right-continuous

We assume that $\vec{X}(t)$ is rightcontinuous and that the two measures $\mathbb{P}$ and $\mathbb{P}\circ\Theta$ are locally mutually absolutely continuous, such that the entropy production (35) can be defined. Because of the definition of entropy production,

[TABLE]

The condition (27) follows from $e^{-S_{\rm tot}(t,\omega)}\geq 0$ and $E\left[e^{-S_{\rm tot}(t,\omega)}\right]=1$ for all $t\geq 0$.

The martingale condition (28) follows readily from (102):

[TABLE]

The relation (103) is a direct consequence of (102) and the definition of the Radon-Nikodym derivative: The Radon-Nikodym derivative $e^{-S_{\rm tot}(s,\omega)}=\frac{\left.{\rm d}(\mathbb{P}\circ\Theta)\right|_{\mathscr{F}_{s}}}{\left.{\rm d}\mathbb{P}\right|_{\mathscr{F}_{s}}}$ is by definition a $\mathscr{F}_{s}$-measurable random variable for which

[TABLE]

for all $\Phi\in\mathscr{F}_{s}$. We show that $E\left[e^{-S_{\rm tot}(t,\omega)}|\mathscr{F}_{s}\right]$ is a random variable with this property, and therefore (103) is valid. Indeed, for $\Phi\in\mathscr{F}_{s}$ it holds that:

[TABLE]

Note that the martingale condition (28) is consistent with the tower property of conditional expectations: $E\left[E\left[X|\mathscr{F}_{t}\right]|\mathscr{F}_{s}\right]=E\left[X|\mathscr{F}_{s}\right]$.

A.3 Uniform integrability

We prove that the stochastic process $e^{-S_{\rm tot}(t\wedge\tau)}$, with $t\in[0,\infty)$ and $\tau$ a fixed positive number, is uniformly integrable. The process $e^{-S_{\rm tot}(t\wedge\tau)}$ can be written as

[TABLE]

Since a stochastic process $Y(t)$ of the form $Y(t)=E[Z|\mathscr{F}_{t}]$ is uniformly integrable [26], we obtain that $e^{-S_{\rm tot}(t\wedge\tau)}$ is uniformly integrable.

Appendix B Integral fluctuation relation for entropy production within infinite-time windows

We derive two corollaries of the optional stopping theorem, which is Theorem 3.6 in [26] and equation (29) in this paper.

Corollary 1**.**

Let $T$ be a stopping time of a stationary process $\vec{X}(t)$ and let $S_{\rm tot}(t)$ be the stochastic entropy production of $\vec{X}(t)$ as defined in (35), with the two measures $\mathbb{P}$ and $\mathbb{P}\circ\Theta$ locally mutually absolutely continuous. If $t$ is continuous, then $S_{\rm tot}(t)$ is assumed to be rightcontinuous. If the two conditions

1. (i)

$\mathbb{P}(T<\infty)=1$, 2. (ii)

$\lim_{t\rightarrow\infty}E\left[e^{-S_{\rm tot}(t)}I_{T>t}\right]=0$,

are met, then

[TABLE]

Recall that $I_{\Phi}(\omega)$ is the indicator function define in (33). We use a proof analogous to the discrete time proof of Theorem 8.3.5 on page 222 of reference [76].

Proof.

We decompose $\langle e^{-S_{\rm tot}(T)}\rangle$ into three terms,

[TABLE]

Since the right-hand side holds for arbitrary values of $t$ we can take $\lim_{t\rightarrow\infty}$. Using (41) and the condition (ii) we obtain

[TABLE]

Because of condition (i), it holds that $\lim_{t\rightarrow\infty}e^{-S_{\rm tot}(T)}I_{T>t}=0$ in the $\mathbb{P}$-almost sure sense. Because $e^{-S_{\rm tot}(T)}I_{T>t}$ is a nonnegative monotonic decreasing sequence we can apply the monotone convergence theorem, see e.g. [33], and we obtain

[TABLE]

∎

Corollary 2**.**

Let $T$ be a stopping time of a stationary process $\vec{X}(t)$ and let $S_{\rm tot}(t)$ be the stochastic entropy production of $\vec{X}(t)$ as defined in (35), with the two measures $\mathbb{P}$ and $\mathbb{P}\circ\Theta$ locally mutually absolutely continuous. If $t$ is continuous, then $S_{\rm tot}(t)$ is assumed to be rightcontinuous. If the two conditions

1. (i)

$\lim_{t\rightarrow\infty}e^{-S_{\rm tot}(t)}=0$* in the $\mathbb{P}$-almost sure sense,* 2. (ii)

there exist two positive numbers $s_{-}$ and $s_{+}$ such that $S_{\rm tot}(t)\in(-s_{-},s_{+})$ for all $t\leq T$,

are met, then

[TABLE]

Proof.

We show that if the conditions of the present corollary are met, then also the conditions of corollary 1 are met, and therefore (110) holds.

Because of condition (i), the stopping time $T$ is almost surely finite: $1=\lim_{t\rightarrow\infty}\mathbb{P}\left[S_{\rm tot}(t)>s_{+}\right]\leq\mathbb{P}\left[T<\infty\right]$ and $\mathbb{P}\left[T<\infty\right]\leq 1$, therefore $\mathbb{P}\left[T<\infty\right]=1$.

Because of condition (i) and (ii) we obtain $\lim_{t\rightarrow\infty}E\left[e^{-S_{\rm tot}(t)}I_{T>t}\right]=0$: $e^{-S_{\rm tot}(t)}I(T>t)\leq e^{s_{-}}I(T>t)$ is a positive variable that is bounded from above. Hence, the dominated convergence theorem applies and $\lim_{t\rightarrow\infty}E\left[e^{-S_{\rm tot}(t)}I_{T>t}\right]=E\left[\lim_{t\rightarrow\infty}e^{-S_{\rm tot}(t)}I_{T>t}\right]=0$. ∎

Appendix C Derivation of the bound (73) on the statistics of $N_{\times}$

We denote by $M_{\times}(t)$ the number of times entropy production has crossed the interval $[-\Delta,\Delta]$ in the direction $-\Delta\rightarrow\Delta$ within the time interval $[0,t]$, and therefore $N_{\times}=\lim_{t\rightarrow\infty}M_{\times}(t)$.

We define two sequences of stopping times $T_{n}$ and $\tilde{T}_{n}$, with $n\in[0,N_{\times}]\cap\mathbb{Z}$, namely,

[TABLE]

and

[TABLE]

where $s_{+}$ is considered to be a very large positive number.

We apply the fluctuation relation (43) to the two stopping times $\tilde{T}_{n}$ and $T_{n}$. The integral fluctuation relation (43) implies that

[TABLE]

Since $e^{-S_{\rm tot}(T_{n})}\leq e^{-\Delta}$, the right-hand side of (113) is bounded by

[TABLE]

The left-hand side of (113) can be decomposed into two terms,

[TABLE]

where $I$ is the indicator function defined in (33). We take the limit $s_{+}\rightarrow\infty$ and obtain

[TABLE]

Since

[TABLE]

we obtain the inequality

[TABLE]

The relation (113) together with the two inequalities (114) and (117) imply

[TABLE]

This is the formula (73) which we were meant to prove.

Acknowledgements

We thank A. C. Barato, R. Belousov, S. Bo, R. Chétrite, R. Fazio, S. Gupta, D. Hartich, C. Jarzynski, R. Johal, C. Maes, G. Manzano, K. Netočný, P. Pietzonka, M. Polettini, K. Proesmans, S. Sasa, F. Severino, and K. Sekimoto for fruitful discussions.