Entropy production in systems with unidirectional transitions

TL;DR

This paper develops a framework for defining and analyzing entropy production in systems with a mixture of unidirectional and bidirectional transitions, extending thermodynamic concepts to more complex stochastic systems.

Contribution

It introduces a method to define average entropy production in systems with unidirectional links, preserving fluxes and providing physical interpretation.

Findings

The framework is consistent with thermodynamics.

Systems with unidirectional cycles behave pseudo-deterministically.

Application to stochastic resetting reveals links between time arrow and entropy production.

Abstract

The entropy production is one of the most essential features for systems operating out of equilibrium. The formulation for discrete-state systems goes back to the celebrated Schnakenberg's work and hitherto can be carried out when for each transition between two states also the reverse one is allowed. Nevertheless, several physical systems may exhibit a mixture of both unidirectional and bidirectional transitions, and how to properly define the entropy production in this case is still an open question. Here, we present a solution to such a challenging problem. The average entropy production can be consistently defined, employing a mapping that preserves the average fluxes, and its physical interpretation is provided. We describe a class of stochastic systems composed of unidirectional links forming cycles and detailed-balanced bidirectional links, showing that they behave in a pseudo-deterministic fashion. This approach is applied to a system with time-dependent stochastic resetting. Our framework is consistent with thermodynamics and leads to some intriguing observations on the relation between the arrow of time and the average entropy production for resetting events.

Equations

$$\displaystyle\dfrac{dp_{i}{\color[rgb]{0,0,0}(t)}}{dt}=\sum_{j=1}^{N}[W_{j\to i}p_{j}(t)-W_{i\to j}p_{i}(t)],$$

$${\dot{S}}_{tot}=\frac{1}{2}\sum _{i,j}\left(W_{j\to i}{p}_j-W_{i\to j}{p}_i\right)\log \frac{W_{j\to i}{p}_j}{W_{i\to j}{p}_i}\ge 0,$$

$${\dot{S}}_{env}$$

$${\dot{S}}_{sys}$$

$$\frac{d{p}_i}{dt}=\sum _{j=1}^N(w_{j\to i}{p}_j-w_{i\to j}{p}_i)+\sum _{j=1}^N(y_{j\to i}{p}_j-y_{i\to j}{p}_i),$$

$$S_{sys}=-\sum _{i=1}^N{p}_i\log p_i,$$

$${\dot{S}}_{sys}=\sum _{i,j}{w}_{j\to i}{p}_j\log \frac{p_j}{p_i}+\sum _{i,j}{y}_{j\to i}{p}_j\log \frac{p_j}{p_i},$$

$${\dot{S}}_{sys}$$

$$y_{N1}{p}_N(t)=w_f(t)p_N(t)-w_b{p}_1(t)\equiv J(t),$$

$${\dot{S}}_{env}^{(B)}={\dot{S}}_{env}^{(\text{G})}+J\log \left(\frac{w_f}{w_b}\right),$$

$${\dot{S}}_{env}^{(B)}={\dot{S}}_{env}^{(\text{G})}+J\log \left(\frac{y_{N1}{p}_N+w_b{p}_1}{w_b{p}_N}\right),$$

$$\displaystyle{\color[rgb]{0,0,0}\dot{S}_{env}^{(A)}\equiv\dot{S}_{env}^{(B)}\big{|}_{w_{b}\to\infty}}=\dot{S}_{env}^{(\textit{G})}-J\log\left(\frac{p_{N}}{p_{1}}\right).$$

$$\displaystyle\overbrace{\sum_{i,j}w_{j\to i}p_{j}\log\dfrac{w_{j\to i}p_{j}}{w_{i\to j}p_{i}}}^{{\color[rgb]{0,0,0}\dot{S}^{(R)}_{tot}}}=$$

$$={\dot{S}}_{sys}+\stackrel{{\dot{S}}_{env}}{\overbrace{{\dot{S}}_{env}^{(R)}+\sum _{i,j}{y}_{j\to i}{p}_j\log \frac{p_i}{p_j}}}\equiv {\dot{S}}_{tot}.$$

$$\frac{w_{ij}}{w_{ji}}=e^{(E_i-E_j)/T},$$

$$⟨\dot{E}⟩=-T{\dot{S}}_{env}^{(R)}-\sum _{ij}{y}_{j\to i}{p}_j\left(E_j-E_i\right)$$

$$\displaystyle{\color[rgb]{0,0,0}\Sigma_{sys}(\tau|\{i(\tau)\})=-\log p_{i(\tau)}(\tau)}$$

$${\dot{\Sigma }}_{sys}(\tau |\{i(\tau )\})$$

$${\dot{\Sigma }}_{env}(\tau |\{i(\tau )\})$$

$${\dot{\Sigma }}_{tot}(\tau |\{i(\tau )\})$$

$${\dot{S}}_{sys}$$

$${\dot{S}}_{env}$$

$${\dot{S}}_{tot}$$

$${\dot{S}}_{sys}=\sum _{i,j}{w}_{j\to i}{p}_j\log \frac{p_j}{p_i}+\sum _{i,j}{y}_{j\to i}{p}_j\log \frac{p_j}{p_i},$$

$${\dot{p}}_3(t)=-y_{3\to 1}{p}_3(t)$$

$$\Delta {\Sigma }_{tot}(\{i(\tau )\})=\log \frac{\mathcal{P}(\{i(\tau )\})}{{\mathcal{P}}^{†}(\{i(\tau ){\}}^{†})}={\int }_0^{t}d\tau {\dot{\Sigma }}_{tot}(\tau |\{i(\tau )\}),$$

$$\displaystyle\Delta\Sigma_{tot}(\{i(\tau)\})=\log\bigg{[}\dfrac{p_{\text{start}}}{p_{\text{end}}}\bigg{(}\dfrac{w_{f,1\to 2}}{w_{b}}\bigg{)}^{\mathcal{N}_{1\to 2}-\mathcal{N}_{2\to 1}}\bigg{(}\dfrac{w_{f,2\to 3}}{w_{b}}\bigg{)}^{\mathcal{N}_{2\to 3}-\mathcal{N}_{3\to 2}}\bigg{(}\dfrac{w_{f,3\to 1}}{w_{b}}\bigg{)}^{\mathcal{N}_{3\to 1}-\mathcal{N}_{1\to 3}}\bigg{]},$$

$${\dot{S}}_{sys}$$

$${\dot{S}}_{tot}^{(R)}={\dot{S}}_{tot}\equiv \sum _{i,j}{w}_{j\to i}{p}_j\log \frac{w_{j\to i}}{w_{i\to j}}-\sum _j{r}_j{p}_j\log \frac{p_j}{p_{i_0}}$$

$$r(t)=2r_0{\cos }^2(2\pi \omega t)$$

$$\dot{S}_{reset}(t\to\infty)\bigg{|}_{r=r_{0},k_{f}=k^{+},k_{b}=1}=0$$

$$⟨{\dot{S}}_{reset}(t){⟩}_T=\frac{1}{T}{\int }_T{\dot{S}}_{reset}(t)dt$$

$$⟨{\dot{S}}_{env}^{(R)}(t){⟩}_T=\frac{1}{T}{\int }_T{\dot{S}}_{env}^{(R)}(t)dt$$

Full text

I Introduction

Almost all natural systems operate out of equilibrium, producing entropy in their surroundings. A general situation for a system to show a non-equilibrium dynamics, and to eventually achieve a non-equilibrium stationary state, is to be in contact with different reservoirs (e.g., of energy, volume, matter) not necessarily in a mutual equilibrium schn . The starting point to model a system in this situation is to perform a coarse-graining procedure on some degrees of freedom, ending up with a set of equations describing the system in terms of probabilities and transition rates.

Given a dynamical description, the most interesting features of a system out of equilibrium arise from the study of its thermodynamic properties. In particular, the fingerprint of a non-equilibrium condition is the entropy production. It has been widely investigated and found to be of paramount importance in estimating the accuracy of relevant observables of natural systems from several perspectives dechant ; barato ; horo ; jarz ; Esposito-CG . Moreover, the entropy production plays a leading role in controlling the linear response to external stimuli baiesi and allows determining the non-equilibrium stationary state of the system, in close to equilibrium conditions prigogine ; vanden .

Consider a discrete-state Markovian system whose dynamics can be described by a Master Equation schn . In formulas:

[TABLE]

where $p_{i}(t)$ indicates the probability to be in the state $i$ at time $t$, $W_{i\to j}$ the transition rate to pass from the state $i$ to the state $j$, that can also depend on time, and $N$ the total number of states. The entropy production of such a system can be estimated using the Schnakenberg’s formula schn :

[TABLE]

where $\dot{S}_{tot}=0$ corresponds to the system in the thermodynamic equilibrium.

The quantity $\dot{S}_{tot}$ can be further splitted into an environment contribution, $\dot{S}_{env}$, and a system one, $\dot{S}_{sys}$, this latter vanishing at the stationary state, as follows:

[TABLE]

where $S_{sys}=-\sum_{i}p_{i}\log p_{i}$. The usual assumption is that if $W_{i\to j}\neq 0$, so it is $W_{j\to i}$ so that (2) and (3) are well defined. Nevertheless, there is a vast class of systems for which this assumption is not satisfied. For example, total asymmetric simple exclusion process (TASEP) involves the transition only in one direction while respecting the exclusion mechanism Ronald ; saha . Unidirectional jumps are also used to model biological enzymatic reactions Ting . The lack of microscopic reversibility can be seen in uniform sheared granular matter and driven inelastic Lorentz-gas Chong . Some other examples include directed percolation Masaki , spontaneous decay of an excited atom rahav , and stochastic resetting search1 ; reviewresetting . The onset of this peculiar situation can also be due to the presence of different processes driving the system out of equilibrium. Once we identify each single process, a transition between $i$ and $j$ can be unidirectional for one of them, but possible along a different pathway. The existence of multiple channels allows for a microscopic interpretation of the local detailed balance Paolo . However, as one can easily see from (2), the entropy production cannot be defined when there are some unidirectional transitions hinric ; saha .

In this direction, several efforts have been made in understanding and describing this vast class of systems. For example, Ben-Avraham et al. ben2011entropy tackled the problem employing a coarse-graining of the sampling time and derived the entropy production that diverges as the sampling time approaches zero, i.e., counting each transition. Similar results on the temporal coarse-graining can also be seen in hinric . In contrast to these studies, Murashita et al. Murashita identified the reasoning behind this divergence at a trajectory level by separating regular and irregular trajectories, and obtaining a correction to the integral fluctuation theorem. In contrast, some fluctuation theorems were derived by Ohkubo ohkubo , using a Bayesian approach and defining a posterior probability to describe the reverse process. Recently, Rahav et al. rahav studied a fluctuating quantity similar (but not equal) to the entropy production in a system with unidirectional transitions, finding that it obeys an integral fluctuation theorem. However, none of these methods can be useful in defining the entropy production without observing a divergence absent in experiments lacasta ; Ronald . Then, we present an argument to evaluate the average entropy production even in presence of unidirectional transitions thus generalizing Schnakenberg’s original work.

We apply our derivations to a peculiar situation in which detailed balance is satisfied on bidirectional transitions, leading to a pseudo-deterministic behavior, i.e., on average, a null entropy production emerging from unidirectional cycles.

Further, we consider the well-known problem of resetting. This is a process involving a sudden transition to a single preselected state or region of the system. It has attracted attention in optimization problems search1 ; search2 ; search3 , proofreading proof1 ; proof2 , population dynamics popdyn and information theory info . We derive the results presented in resetting in our general framework.

Finally, we extend the study of systems with resetting to cases in which this latter mechanism is periodic in time. On the one hand, the time-periodic driving has been recently studied as a tool to mimic desired non-equilibrium features (e.g. stationary distribution, entropy production, fluxes) of biological systems in which detailed balance does not hold browne ; hern ; astumian ; raz-subasi ; busiello-raz . Since some of them may exhibit a resetting mechanism or, more in general, unidirectional transitions, then a way to estimate the entropy production both in steady and time-periodic conditions is of paramount importance to efficiently engineering artificial molecular machines. On the other hand, the paradigm of time-periodic resetting presents a much more wide phenomenology than its stationary counterpart. Indeed, we show that it allows switching between a behavior where information is erased from the system to one where, instead, it is added. Interestingly, a signature of the arrow of time emerges by inspecting the average resetting entropy production. Even if presented on a simple toy model, this could serve as a starting point to elucidate the response of biological systems to time-periodic stimuli bennett ; mettetal ; tu ; timeperiodic .

The plan of this paper is as follows. In Sec. II, we give a general discussion on a system involving both bidirectional and unidirectional transitions, and identify the total entropy production for a general system obeying Eq. (1). Sec. III presents the mapping of unidirectional links to bidirectional ones. Here we show that under suitable limiting procedure of the transition rates of these bidirectional links, one is able to derive the total entropy production identified in Sec. II. An interpretation of this limit is given in Subsec. III.1. Using the entropy production along a single trajectory of a non-equilibrium ensemble, the results shown in Sec. III are reobtained in Sec. IV. Moreover, in Subsec. IV.1, we consider some examples to better understand the entropy production along a stochastic trajectory when the system has a mixture of both unidirectional and bidirectional links. Entropy production for a system under stochastic resetting mechanism is treated in Sec. V. Using the general theory we study the entropy production using a constant resetting rate (Subsec. V.1) and the time-periodic variation of the the resetting rate (Subsec. V.2) in a model system. In Sec. VI, we conclude our paper. A detailed derivation for the averaging of entropy on a stochastic trajectory over the path probability is shown in Appendix A. A relation between the rate of total entropy production and the total entropy production along a stochastic trajectory is shown in Appendix B.

II Master Equation with unidirectional transitions

We consider a discrete state Markov system as described by Eq. (1). The system follows a stochastic dynamics and makes transitions from one state to the other. For a selected pair of states (e.g., $i,j$), either (i) the system jumps from a state $i$ to a state $j$ with a transition rate $W_{i\to j}\equiv w_{i\to j}>0$ and there exists a corresponding reverse transition with a rate $W_{j\to i}\equiv w_{j\to i}>0$ (for example, see black dashed links in Fig. 1) or (ii) the system makes a transition from a state $i$ to a state $j$ with a transition rate $W_{i\to j}\equiv y_{i\to j}>0$ in the absence of a conjugate transition, i.e., $W_{j\to i}\equiv y_{j\to i}=0$, (for example, see the red link in Fig. 1). The former process is referred to as bidirectional process whereas the latter is called unidirectional process.

The dynamics of the system is described by a continuous-time Master Equation:

[TABLE]

where on the right hand side, the first term describes the situation (i), and the second term corresponds to the situation (ii) in which the summation is performed over all the couple of transition rates for which the reverse transitions are not allowed, i.e., either $(y_{i\to j}=0,y_{j\to i}\neq 0)$ or $(y_{j\to i}=0,y_{i\to j}\neq 0)$. From now on we will not always explicitly write the time dependence in $p_{i}$ and in the transition rates.

The average entropy of the system is given by schn (as it is common in this context we use temperature units so that the Boltzmann constant can be set equal to $1$)

[TABLE]

where $p_{i}(t)$ is the solution of the Master Equation (5) subject to the initial condition $p_{i}(0)$.

Differentiating the above equation with respect to time $t$ and substituting (5) yields the system entropy production $\dot{S}_{sys}$:

[TABLE]

where the dot represents the derivative with respect to time. Notice that $\dot{S}_{sys}$ is a finite quantity irrespective of the fact that some of the transitions are not allowed. The above equation can be rewritten as follows

[TABLE]

where the first and second term on the right hand side, respectively, are recognized as the total entropy production $\dot{S}^{(R)}_{tot}$ and the environment entropy production $\dot{S}^{(R)}_{env}$ for the bidirectional processes only. The superscript $R$ refers only to the fact that the corresponding quantities contain only bidirectional transition rates, however, the information regarding both unidirectional and bidirectional links is encoded in $p$’s.

Remark 1: both $\dot{S}_{tot}^{(R)}$ and $\dot{S}_{env}^{(R)}$ depend on the whole set of transition rates, $\{w\}$ and $\{y\}$, through the $p$’s that satisfy Eq. (5).

Remark 2: the contribution of the unidirectional links (i.e., last term on the right hand side) cannot be splitted into a total and an environment entropy production, since both terms will formally lead to divergent (and unphysical) quantities.

We name “unidirectional entropy production”, $\dot{S}^{(\rm u)}$, the last term on the r.h.s. in Eq. (8) explicitly depending on unidirectional transition rates. This extra contribution has to be investigated case by case to understand its physical origin. When, for example, there is an external process inducing unidirectional transitions whose entropy production is negligible, $\dot{S}^{(u)}$ can be understood as an additional contribution to the environment entropy production. In the next section we corroborate the latter physical interpretation with an explicit example.

III Dynamics-preserving irreversible entropy production

Let us consider the discrete-state system in Fig. 2A, say $(A)$. All the transition rates present in the internal network G are bidirectional. The only unidirectional link is the external one, $y_{N1}:=y_{N\to 1}$. This system is physically equivalent to the one depicted in Fig. 2B, say $(B)$, as long as the flux flowing through the unidirectional link (A) is equal to the corresponding bidirectional links (B) for all time $t$. In formulas, this condition is

[TABLE]

where we have set $w_{b}$ to be time-independent. In the above equation (9), in order to obtain $w_{f}(t)$, we substitute $p_{1}(t)$ and $p_{N}(t)$, the solutions obtained from (5) for a given initial condition $[p_{1}(0),p_{2}(0),p_{3}(0),\dots,p_{N}(0)]$ and $w_{b}$. Notice that $w_{b}\neq 0$, since $w_{b}=0$ implies $w_{f}=y_{N\to 1}$, corresponding to the original situation depicted in $(A)$. With this condition the dynamics of the two systems, $(A)$ and $(B)$, leads to the same average entropy production as in Eq. (7).

Let us first evaluate the environment entropy production for the system $(B)$, $\dot{S}_{env}^{(B)}$. By definition, using Eq. (2) we have:

[TABLE]

where the first term is the environment entropy production due to the internal network G. Using Eq. (9), we can express $w_{f}$ as a function of $y_{N1}$ and $w_{b}$ in Eq. (10), obtaining:

[TABLE]

where $p_{i}$ indicates the probability to be in the state $i$, due to all the transition rates, both the bidirectional and the unidirectional ones, as obtained from the solution of Eq. (5).

In order to eliminate the information about the fictitious transition rate $w_{b}$, we can take the limit $w_{b}\rightarrow+\infty$. The physical interpretation of this limit corresponds to the assumption that the external process inducing unidirectional transitions in the system, if any, has a negligible entropy production. In the next subsection we will discuss further this point. We get the following equation:

[TABLE]

Thus we have obtained a (non-divergent) contribution to the environment entropy production due to the unidirectional links only, which we identify with a part of the environment entropy production of the system of interest $(A)$, $\dot{S}_{env}^{(A)}$. Notice that the environment entropy production $\dot{S}_{env}^{(A)}$ explicitly depends on the transition rate of the unidirectional link just through the probability flux $J=y_{N1}p_{N}$.

The generalization to the case of multiple unidirectional links is straightforward and proceed along the same line. In fact, we can imagine a fictitious system where each unidirectional link $y_{i\to j}$ is replaced by a couple of links $w_{f,i\to j}$ and $w_{b,j\to i}$, properly tuned according to Eq. (9). Then, after the evaluation of the entropy production, we can set all the $w_{b}$’s to infinity as in the above example.

Rewriting the total entropy production, from Eq. (8), we obtain:

[TABLE]

We stress again that the probabilities $p$’s depend on the whole network through Eq. (5), and so also $\dot{S}_{tot}^{(R)}$ and $\dot{S}_{env}^{(R)}$. Only the environment entropy production depends explicitly on the unidirectional links $y$’s. Furthermore, $\dot{S}_{tot}$ does not depend on the choice of $w_{b}$.

We present the above results for an illustrative simple example. Consider the network consisting of four nodes depicted in Fig. 3A. In Fig. 3B, we show the equivalence between the dynamics of the network with the unidirectional links (solid lines) and the one with the bidirectional links (dots) for all times, as discussed in (9), for a particular choice of transition rates. In Fig. 3C, all the components of the entropy production are studied. In particular, the system entropy production tends to zero as the system approaches the stationary state. The environment entropy productions due to bidirectional and unidirectional links also reach a stationary value for large times. Moreover, the latter contribution (the last term in (11)) is plotted for different values of $w_{b}$, exhibiting its convergence to a minimum value for all the times as $w_{b}$ goes to infinite. In the Fig. 3D, we show that with an increasing value of $w_{b}$, the derivative of $\dot{S}_{env}$ with respect to $w_{b}$ tends to zero, i.e., $\dot{S}_{env}$ becomes independent of it. Thus, the environment entropy production for the modified system, the one with $w_{f}$’s and $w_{b}$’s, is bounded from below by the environment entropy production obtained in the limit $w_{b}\to+\infty$, i.e., the one that we have identified as the environment entropy production of the original system.

III.1 Interpretation of the $w_{b}\to+\infty$ limit

The physical interpretation of this limit, as discussed in busielloPRE , can be found in parallel to an electrical circuit made only by conductances, $w_{ij}$. The injection of a net current $J$ in a node $k$, and the ejection of the same current from another node $l$ can be introduced into the dynamics as a unidirectional link, $y_{l\to k}$. Analogously, the production of this current can also be seen as the effect of an external battery, producing a difference of potential $\Delta V=JR_{eq}$, where $R_{eq}$ is the equivalent resistance, which can be estimated as a combination of the bidirectional transition rates busielloPRE . It can be shown that, if we map this battery into bidirectional links, with transition rates $w_{f}(t)$ and $w_{b}$, producing the same flux, the entropy production of the system always contains the energy dissipation of the electric circuit, and an additional contribution proportional to $1/w_{b}$. Letting $w_{b}$ go to infinity corresponds to a circuit where the current is generated by a perfect battery, i.e., without dissipation, at its stationary state. Hence, in this limit, $\dot{S}_{tot}$ corresponds to the energy dissipation of the circuit only.

In the context of this work, the limit $w_{b}\to+\infty$ has an analogous meaning: if we consider unidirectional transitions into the system as induced by an external process, characterized by the rates $w_{f}(t)$ and $w_{b}$, letting $w_{b}\to+\infty$ will lead to the correct expression for energy dissipation of the original system, when there are no other sources of dissipation due to external processes.

It is important to stress that, within this framework, the total entropy production is not affected by the limit of an infinite $w_{b}$, since it depends just on bidirectional transition rates and probabilities (see Eq. (13)), which are the same in both systems $(A)$ and $(B)$.

We emphasize that the entropy production computed by mapping each unidirectional link into a bidirectional one and letting each fictitious transition rate $w_{b}\to+\infty$, is equivalent to the direct computation (without mapping) as shown in the Eq. (8), i.e., both of these results are consistent with each other. The aim of introducing such a mapping is to show that the contribution coming only from unidirectional links may have its own physical meaning in some cases. In particular, when there are no external processes dissipating energy, i.e. a perfect battery in the parallel with an electrical circuit, the contribution coming from unidirectional links can be considered as a part of the environment entropy production of the full system. We stress that such a physical identification is crucial to correctly generalize the Schnakenberg’s formula.

The correct interpretation of $\dot{S}^{(\rm u)}$, i.e. whether or not being a part of the environment entropy production (without further corrections), is an interesting question that we leave for future investigations. Indeed, in order to provide a full answer, one needs to build a mapping that preserves the correct statistics on each trajectory (i.e., not only on average level), as we also point out in the next section.

III.2 First law with unidirectional transitions

A thermodynamically consistent choice for the bidirectional transition rates is

[TABLE]

where $k_{B}=1$ for sake of simplicity.

From Eq.(5), multiplying by $E_{i}$, summing over $i$, and using Eq.(14) for the bidirectional transition rates, it is possible to obtain a formulation of the first law of thermodynamics in presence of unidirectional transitions:

[TABLE]

The first term on the r.h.s. of the previous equation is the heat absorbed by the system from the environment without considering the processes involving unidirectional transitions. The second term, without the minus sign, represents the work extracted from the system, at the expenses of heat absorbed from the thermal bath/environment at temperature $T$, due to events triggered by unidirectional transitions. They both contributes to the total energy change in the system, $\langle\dot{E}\rangle$. A simple example, which justifies the latter identification is illustrated in Fig. 4. Notice that the term $-\sum_{i,j}y_{j\to i}p_{j}(\log{p_{j}}-\log{p_{i}})$ in Eq. (13), which we identify as a contribution to the environmental entropy production, has a form similar to the second term in Eq. (15) with $E_{j}-E_{i}$ substituted by the entropy change $\log{p_{j}}-\log{p_{i}}$. This further suggests the correctness of our guess for the environment emtropy production.

IV Trajectory-based approach

In this section, we will compute the average entropy production using the entropy production along a single stochastic trajectory in a non-equilibrium process. So far we have investigated how the average entropy production can be computed when unidirectional transitions are present in a discrete-state system. The average entropy production can also be explicitly derived using a trajectory-based approach. This is the standard procedure to derive the well-known fluctuation theorems seifFT ; crooks ; jarz2 ; GCohen . Here, for completeness, we present in detail this derivation. However, it is important to stress the fact that our framework only preserves average currents, not their fluctuations. As a consequence, the entropy production on a single trajectory may have different statistics between the real and the fictitious system, and only average quantities can be reliably computed through the proposed mapping. We leave for future investigations the problem of defining an entropy production for unidirectional transitions which exhibits the correct statistics on a single trajectory.

Consider a system which involves only bidirectional links (see Sec. I). Suppose $\Sigma_{sys}(\tau|\{i(\tau)\})$ be the system entropy along a trajectory $\{i(\tau)\}$ at time $\tau$ (where $0<\tau<t$), and it is defined as Andr ; seifEP ; eft

[TABLE]

where $p_{i}(\tau)$ is the solution of the Master equation (1) given the initial condition $p_{i}(0)$. In the above equation, the trajectory note $\{i(\tau)\}=(i_{0},i_{1},i_{2},\dots,i_{M-1},i_{M})$, in which $i_{k}$ is the state of the system from time $\tau_{k}$ to $\tau_{k+1}$, where $\tau_{k}<\tau_{k+1}$, and $\tau_{k}\in(\tau_{0}=0,\tau_{M+1}=t)$, and the jumps occur at times $\tau_{1},\tau_{2},\dots,\tau_{M}$.

Differentiating the above equation with respect to time, we find the rate of entropy of the system along a single trajectory

[TABLE]

where the last two terms are equal to zero as $0=\tau_{0}<\tau<\tau_{M+1}$. From the above equation, we can identify the rate of environment and total entropy as

[TABLE]

In the following, we take the average on (17), (18), and (19) over trajectories, as defined in Appendix A. Notice that averaging the first term on the right hand side in (17), and (19) over $p_{i}(\tau)$ yields zero due to the normalization $\sum_{i}p_{i}(\tau)=1$. Therefore, the rates of average entropy productions are (see Appendix A for more details)

[TABLE]

where $W$’s are the non-zero transitions.

In the case of environment and total entropy productions, when we consider all the transitions are bidirectional, there is no problem with these quantities. However, as already said before, when some transitions are allowed and the reversed ones are not, the above $\dot{S}_{tot}$ and $\dot{S}_{env}$ diverge. Nevertheless, this problem does not arise in the case of system entropy production where one can split $W$ and rewrite the system entropy production as (see Sec. II)

[TABLE]

where, as already said above, the first term involves $w$’s indicating bidirectional transitions whereas the second term involves $y$’s refers to unidirectional transitions.

In order to deal with this divergence and to give it the right physical interpretation, when some transitions are unidirectional, one can separate those links and replace them by the bidirectional links ($w_{f}$ and $w_{b}$ for each unidirectional link) as explained above. Now the system is made by bidirectional links only, then Eqs. (20), (21) and (22) can be obtained. In the end, after the average over many trajectories, one can take the limit $w_{b}\to+\infty$ as shown in Sec. III.

The results obtained following this way, by averaging the trajectory entropy, are consistent with ones derived in the previous sections. In other words, when the limit $w_{b}\to\infty$ is physically meaningful (in the terms discussed above), the extra (non-divergent) contribution in $\dot{S}_{sys}$, Eq. (17), due to the unidirectional transitions, appears to be equal to (minus) the extra term in the environment entropy production depending just on $y_{i\to j}$. Again, Eq. (13) holds.

An important observation is that in our framework there is no need of splitting the trajectories into regular and irregular ones Murashita . One can first consider the system as completely bidirectional, and then the limit is taken just as a final step. Thus, the average entropy production can also be achieved from each single trajectory and averaging over them using the procedure explicitly shown in the Appendix A.

In the next subsection, we obtain the average entropy production for an example where some of the transitions are unidirectional.

IV.1 Partial detailed balance and pseudo-deterministic behaviour

Our framework can be employed to study a particular situation, which we name partial detailed balance.

Consider a system in which the detailed balance holds for all the bidirectional links $w_{i\to j}$, i.e., $w_{i\to j}\pi_{i}=w_{j\to i}\pi_{j}$, where $\pi_{i}$ is the probability to be in the state $i$ at stationarity. This condition differs from global detailed balance, since unidirectional links do not have to satisfy it. We call it partial detailed balance.

Let us start with a simple system, as the one depicted in Fig. 1. In this case, imposing the detailed balance conditions on all the bidirectional (black) links, the probability of the system to be in the state 3 will evolve according to:

[TABLE]

converges to zero as time progresses. The detailed balance condition then implies that all the probabilities have to be zero at stationarity, which is impossible. Therefore, the detailed balance is never reached on the bidirectional links, since it is inconsistent with the dynamical evolution of the whole system. In other words, the detailed balance is not maintained in such systems in contrast to those systems with only bidirectional links. We emphasize that this argument can be generalized as follows: the partial detailed balance cannot be satisfied for networks where the nodes connected by unidirectional links do not form cycles.

We then need to analyze networks in which cycles of unidirectional links do appear. Also for this setting, let us start with the simple example sketched in Fig. 5. Here, when detailed balance holds on bidirectional links, the dynamics still admits a feasible stationary solution for the whole system.

We study this system when the the entropy produced by unidirectional transitions cannot be ascribed to any external process, so that $\dot{S}^{(\rm u)}$ can be identified with an extra contribution to the environment entropy production. As a consequence, the total entropy production is equal to $\dot{S}^{(R)}$, which is the one evaluated taking into account only bidirectional transitions, Eq. (13).

According to this, the total entropy production is equal to zero at stationarity for this particular system, because of the partial detailed balance condition. In other words, in the presence of cycles of unidirectional links, there can be in principle situations in which the system experiences some transitions without producing entropy, even if global detailed balance does not hold.

In order to unveil the apparent paradox of a null average total entropy production, here we detail how to estimate it starting from trajectory-dependent quantities. Given a trajectory$\{i(\tau)\}$, the total entropy change associated to it is

[TABLE]

where $\mathcal{P}(\{i(\tau)\})$ is the probability of observing a given trajectory $\{i(\tau)\}$ and $\mathcal{P}^{\dagger}(\{i(\tau)\}^{\dagger})$ is the probability of observing the time reversed trajectory $\{i(\tau)\}^{\dagger}$. The explicit structure of $\mathcal{P}(\{i(\tau)\})$ is given in Eq. 37. A proof of the last equality in Eq. (25) is given in Appendix B.

As highlighted in rahav , since we cannot identify a time reversed trajectory for the unidirectional links, we cannot strightforwardly apply Eq. (25) to identify $\Delta\Sigma_{tot}$. Instead of proposing a different auxiliary dynamics, generating an auxiliary trajectory $\widehat{\{i(\tau)\}}$, which does not correpond to the time-reversed one rahav , and would not also lead to the total entropy production on a single trajectory, we start modeling our system with all the transitions as bidirectional links, as explained in Sec. III.

The equivalent network is shown in Fig. 5B, where each unidirectional links is replaced by the bidirectional links with the transition rates $w_{f}$’s and $w_{b}$ using the procedure presented in Sec. III. For convenience, we assume $w_{b}$ to be equal for each reverse transition. Generalization to different $w_{b}$ is straightforward. In this equivalent network, consider a trajectory $\{i(\tau)\}$, so that:

[TABLE]

where $\mathcal{N}_{i_{k}\to i_{k\pm 1}}$ is the number of times the transition happened from nodes $i_{k}$ to $i_{k\pm 1}$ in the trajectory $\{i(\tau)\}$. In the above equation, $p_{\text{start}}\ (p_{\text{end}})$ is the probability of the system to be in the initial (final) node from (to) where (which) the trajectory $\{i(\tau)\}$ starts (ends). In Eq. (26), we have focused only on the cycle formed by unidirectional links. If also the contribution from other nodes connected by bidirectional links, then obeying detailed balance, and belonging to the trajectory $\{i(\tau)\}$, were considered in Eq. (26), it would yield $1$ in the argument of the logarithm by definition. Upon averaging Eq. (26) over trajectories (which is equivalent to average Eq. (19) and apply (25)), then using (9) and the limit $w_{b}\to+\infty$, we finally get that the system does not produce entropy on average at stationarity.

An intuitive explanation of this result relies on noting that an intrinsic unidirectional cycle (not caused by some external process performing work) exhibits a non-stochastic behaviour. The stochasticity lingers only in the presence of persistence times, which, however, does not contribute to the entropy difference for each trajectory (see Eq. (26)), showing what we name a pseudo-deterministic behaviour. Thus, we can conclude that, in the framework here discussed, unidirectional cycle exhibit null total average entropy production at stationarity. This is a direct consequence of the equivalence derived in Eq. (13). In fact, at an average level, bidirectional links are the only ones contributing to the entropy production, even if the $p$’s still depend on the whole network of transitions.

V Application: stochastic resetting

Stochastic resetting is a mechanism in which the system undergoes a stochastic dynamics in the state space as well as stochastically resets to a prescribed location with a given transition rate (i.e., a unidirectional process) search1 ; search2 ; search3 ; IFT-arnab ; sanjib-relaxation ; Arnab-1 ; pal-time-dep ; shamik-apoorva ; Uttam ; KPZ-power-law ; falcao-inter ; roldan-path-int ; refractory ; under-restart ; Markov-reset ; run-tumble ; gupta ; gupta2019work ; basu2019symmetric . These resetting transitions involve jumps of the system into given locations and can be called the controlled transitions (i.e., these can be tuned from external sources). Notice that corresponding to each controlled transition, there is no reverse transition. Therefore, the whole dynamics has two classes of transition rates: (i) internal or uncontrolled, and (ii) controlled and unidirectional.

Then we can write the entropy production of the system as follows. Suppose that a given system resets to a state $i_{0}$ from a state $j$ with a resetting rate $r_{j}$. Therefore, using Eq. (8), the system entropy production is

[TABLE]

where $w_{i\to j}$ is the transition rate belonging to the uncontrolled process, which we have considered constituted by bidirectional transitions only, and $r_{j}:=y_{j\to i_{0}}$ is the resetting rate (where $y_{j\to i_{0}}y_{i_{0}\to j}=0$, i.e., controlled process) from the $j$th state.

In the steady state, $\dot{S}_{sys}=0$, using Eq. (13) we are led to:

[TABLE]

Hence, using our general framework, it is possible to obtain the same result as presented in resetting .

In what follows, we identify $\dot{S}^{(\rm u)}$ as $\dot{S}_{reset}$, without further speculating on its physical meaning, since in this case it is univocally determined by the system under investigation.

V.1 Constant resetting

In the following, we consider an example of a network of three nodes labelled by 1, 2, and 3 as shown in Fig. 6A. The transition rates from the node 1 to 2 and from 2 to 3 are $k_{f}$ while the reverse ones are $k_{b}$. Moreover, we consider that the system stochastically resets from nodes 2 and 3 to node 1 with a rate $r>0$. For this system, we plot the contours for $\dot{S}_{reset}=0$ (see Fig. 6B) and $\dot{S}^{(R)}_{env}=0$ (see Fig. 6C) for two different initial condition $\pi_{1}=(0.15,0.15,0.7)$ and $\pi_{2}=(0.7,0.15,0.15)$ at different times in $(k_{f}/k_{b},r/k_{b})$ plane. We observe that as the time increases the system approches the stationary state and the contours $\dot{S}_{reset}=0$ approaches their stationary value.

Within our framework, we can go beyond the stationary operative diagram, derived in resetting . In fact, it is possible to obtain how the region of parameters in which the system exhibits a writer(eraser)-like behaviour changes as time increases. However, this latter is a transient effect, since the system evolves toward a non-equilibrium steady state. It is then worth investigating whether persistent complex phenomena can emerge by introducing a time dependence in the transition rates.

V.2 Time-periodic resetting

Time-periodic driving is a non-equilibrium paradigm which can efficiently mimic non-equilibrium features of molecular machines operating at steady state browne ; hern ; astumian ; raz-subasi ; busiello-raz . Moreover, several biological systems constantly integrate and produce time-periodic stimuli, to cope with environmental perturbations bennett ; mettetal ; tu ; timeperiodic . It is then instructive to study what happens when time-dependent unidirectional transitions come into play. Here, we focus on cases where they implement a time-periodic resetting mechanism.

Consider a system analogous to the one depicted in Fig. 6A, where now the resetting rate depends on time through the following rule (as sketched in Fig. 7A):

[TABLE]

such that $\langle r(t)\rangle_{T}=r_{0}$, where $\langle\cdot\rangle_{T}$ indicates the temporal average over one period $T=(2\omega)^{-1}$. Both $k_{f}$ and $k_{b}$ do not depend on time, so that the periodicity is introduced through the resetting mechanism only. After a transient, the system will relax to a time-periodic state, exhibiting time-periodic environment and resetting entropy production.

For sake of simplicity, we set $k_{b}=1$, $k_{f}=k^{+}$, and $r$ equal to its average value $r_{0}$, such that if the system relaxed to a non-equilibrium steady state, the resetting entropy production would be equal to zero, i.e.,

[TABLE]

The above equation gives the relation between $r_{0}$ and $k^{+}$ corresponding to one point on the magenta curve depicted in Fig. 6B. Since we have different rates acting at the same time on the system, we define a reference timescale $\tau=1/k_{\rm min}$, where $k_{\min}=\min\{1,k^{+},r_{0}\}$. When the period of the resetting $T=\tau$, the corresponding frequency will be denoted as $\omega_{c}$, that is, $\omega_{c}=k_{\rm min}/2$. We then investigate the system for various $\omega$, i.e., $\omega<<\omega_{c}$, $\omega>>\omega_{c}$ and $\omega=\omega_{c}$ To this aim, we define $\omega/\omega_{c}=\alpha$, where $\alpha>0$ is a dimensionless parameter.

In Fig. 7 and 8, we present numerical results. Notice that from now on we study the system for time $t$ such as it has already reached its time-periodic state. Moreover, for sake of simplicity, the time $t$ is considered inizialized to [math] after the transient dynamics.

First of all, the system exhibits a time-periodic switching between writer-like and eraser-like behavior, even if the resetting entropy produced in the two phases is not the same. Moreover, there are two bumps in the positive region and just one minima in the negative region, in a period $T$. In Fig. 7B, solid curves represent $\dot{S}_{reset}$ for three different values of $\alpha$ (corresponding to $\omega$ above, below and equal to $\omega_{c}$). Clearly, as the frequency increases, the speed of the switching increases as well, even if it cannot be seen from the picture, since we are plotting the entropy production against a rescaled time $\mathcal{T}=t/T$.

Unexpectedly, going toward higher values of $\alpha$, an asymmetry appears in $\dot{S}_{reset}$. In particular, we observe a difference in the heights of the two maxima evidenced by a blue and a red dot at fixed $\alpha$ in the picture. The smaller maximum takes place before the larger one in each period, at (rescaled) time $\mathcal{T}_{S}<\mathcal{T}_{L}$, where $\mathcal{T}_{L}$ is the (rescaled) time at which the larger maximum appears. As $\alpha$ is further increased, such an asymmetry disappers again, and eventually the resetting entropy production vanishes at all times. This behavior is a consequence of the fact that, for very fast drivings, $r(t)$ can be approximated with its average $r_{0}$, for which $\dot{S}_{reset}$ is zero by construction, Eq. (30).

Remarkably, this observation can be rephrased in a more fashonable way: time-periodic resetting introduces a signature of the arrow of time. It is known that entropy production is a time-antisymmetric quantity whose average over many dissipative trajectories can help in discriminating if a process is proceeding forward or backward, i.e., to identify the arrow of time parrondo . Here we discuss how time-dependent unidirectional links (performing a resetting to a given state) can lead, in principle, to an error-free discrimination between forward and backward processes. In fact, just by looking at the order in which the maxima appear in $\dot{S}_{reset}$, during a period, we can conclude: if the smaller maximum comes first, then time is proceeding, viceversa, time is regressing. However, the bottleneck is represented by the long-standing problem about how to measure entropy production horo . Moreover, we are reporting results from a simple example rather than a general theory, and we leave for future works a more in-depth analysis of this preliminary - and peculiar - observation.

Dashed lines in Fig. 7B represent the environment entropy production due to the internal transitions. It can be seen that both its average and its minimum value increase when $\alpha$ increases at fixed $\mathcal{T}$. Eventually it approaches the stationary value at all times for very fast drivings. A thermodynamic argument can justify this result, since adiabatic (slow) protocols, in general, can attain lower level of dissipation in the surroundings. However, such a consistency between thermodynamics and our results serves as a further endorsement of the framework here introduced.

We introduce four measures to quantify this phenomenology, and to study the system as a function of frequency of $r(t)$:

[TABLE]

Eq. (31) quantifies the temporal average of the entropy production due to the resetting mechanism over one period $T$. In Fig. 8A we show that it is always negative, and approaches zero for large $\alpha$. This means that on average the system erases information.

Eq. (32) is the temporal average of the entropy production in the environment over one period, and, as reported in Fig. 8B, it increases with $\alpha$, consistently with thermodynamics and our previous qualitative observation. An analogous behaviour is displayed by the minimum amount of dissipation in the surroundings, defined in Eq. (33), and shown in Fig. 8C.

As noticed above, the asymmetry between the positive maxima of $\dot{S}_{reset}$ increases with $\alpha$ up to a maximum value, and then goes back to [math] for very fast drivings (see Fig. 8D).

Putting together all these informations, we produce a qualitative graph in Fig. 8E. Here all the quantities have been normalized so to be seen on the same scale. It appears natural to identify a finite window of $\alpha$ in which the system exhibits high sensitivity to the variation of the resetting frequency. In fact, the slope of $\langle\dot{S}_{reset}(t)\rangle_{T},\langle\dot{S}^{(R)}_{env}(t)\rangle_{T}$ and \dot{S}^{(R)}_{env}\big{|}_{\min} is maximized in the this region, and it is also (roughly) centered around the maximum of \Delta\dot{S}_{reset}\big{|}_{\mathcal{T}_{L},\mathcal{T}_{S}}. It is worth noting that this high sensitivity window contains mostly frequencies higher than the reference time-scale of the system, associated to the minimum average rate $k_{\min}$. However, we stress the fact that this study provides just a qualitative analysis of an example, and it does not aim at being exhaustive about the effect of time-periodic unidirectional links. It is true that a rich and complex phenomenology arises even for the simple system analyzed, and we believe that this topic deserves much attention and leaves several questions for future studies.

VI Conclusions

In this work we have proposed a theoretical (and physically meaningful) argument to generalize the Schnakenberg’s entropy production to cases in which unidirectional transitions are allowed. This kind of situation emerges when several different processes are in play to drive the system out of equilibrium, and it is also relevant for various physical and biological applications.

The solution we have adopted has the advantage of preserving the dynamics of the system at all times, and it can also be derived from a trajectory-based approach. Moreover, it provides a direct procedure to deal with the presence of unidirectional transitions.

We have studied the peculiar case in which unidirectional cycles are present, deriving that, from a thermodynamic perspective, they behave like pseudo-deterministic systems.

Within our framework, we have studied the resetting problem, which is one of the most prominent applications in which unidirectional transitions are involved. We have shown that previous results in this context can thus be recovered in a broader perspective. Moreover, we have generalized the analysis to time-periodic resetting, finding that the system can switch between an eraser-like and a writer-like behaviour. Moreover, our framework is consistent with the fact that the faster is the driving, the higher is the entropy production in the environment. On top of this, fast drivings lead to a signature of the arrow of time in the resetting entropy production, an observation which can be a starting point for future investigations.

The presented framework can be nevertheless applied to several different situations, from population dynamics to biological modelizations, leaving a wide panorama of possibilities for further applications.

Acknowledgements

D. Gupta and A. Maritan acknowledge the support from University of Padova through “Excellence Project 2018” of the Cariparo foundation.

Appendix A Average entropy production

In this section, we compute the average entropy production starting from the entropy production given for a single stochastic trajectory.

Consider a system which evolves according to the Master Equation in Eq. (1). Given a function of the trajectory, $F(\{i(\tau)\})$ (e.g. $\int_{0}^{t}g(i(\tau))d\tau$ with $g$ a generic function defined on the ensemble of states), we define the average over trajectories as

[TABLE]

where we introduced the following summation:

[TABLE]

where $\Theta(\cdot)$ is the Heaviside-theta function. This summation runs over all possible trajectories involving $M$ jumps and lasting a time $t$. Then, {\color[rgb]{0,0,0}\mathcal{P}(\{i(\tau)\})} is the probability of observing such a trajectory:

[TABLE]

Notice that $p_{j}(\tau)$, the solution of the master equation can be obtained as $p_{j}(\tau)=\langle\delta_{j,i(\tau)}\rangle_{\mathrm{traj}}$.

In the above equation, $p_{i_{0}}(0)$ is the initial distribution, and

[TABLE]

where $\mathcal{W}_{i_{X}}(t)=\sum_{j_{X}}W_{i_{X}\to j_{X}}$ is the transition rate of the exit from state $i_{X}$.

We are interested in the case $F=\dot{\Sigma}_{sys}$, defined in Eq. (17). Thus, the average entropy production of the system over all the trajectories is given as

[TABLE]

where $f(i_{k},i_{k-1},\tau_{k})=\log\dfrac{p_{i_{k-1}}(\tau_{k})}{p_{i_{k}}(\tau_{k})}$. We stress that averaging the first term on the right hand side of Eq. (17) we get:

[TABLE]

In the last but one step we have used the fact that $\partial_{\tau}p_{i}(\tau)/p_{i}(\tau)$ does not depend anymore on the trajectory.

Using Eqs. (36) and (37), we can explicitly compute this average:

[TABLE]

where $i=i_{M}$ and $t=\tau_{M+1}$.

We first note that

[TABLE]

Since $k$ has to be smaller or equal than $M$, $M$ can go up to $\infty$, and the term $k=0$ does not contribute to the average entropy production.

We can rewrite the expression above as follows:

[TABLE]

In the above equation, we have performed the integral over $\tau_{k}$ while using $\delta(\tau-\tau_{k})$. In the product of $\Theta(\tau_{j}-\tau_{j-1})$ it is implicitly assumed that $\tau_{k}=\tau$. Now, the term in the square bracket can be identified as $p(i_{M},t|i_{k},\tau)$, so that:

[TABLE]

and, the term in the curly brackets is equal to $1$. Thus, renaming $\kappa=k-1$, we obtain:

[TABLE]

The term in the square brackets is nothing but $p_{i_{\kappa}}(\tau)$, then, naming $i=i_{\kappa+1}$ and $j=i_{\kappa}$, which are any two connected states, and substituting the expression of $f(i_{\kappa+1},i_{\kappa},\tau)$, we finally get:

[TABLE]

which is nothing else that Eq. (20).

Similarly, one can do the averaging over environment and total entropy given in (18) and (19), respectively, to obtain respective the entropy productions given in Eqs. (21) and (22).

Appendix B Relation between (19) and (25)

In this section, we aim to find the relation between (19) and (25). Therefore, we consider (19)

[TABLE]

Now we rewrite the above equation as

[TABLE]

where

[TABLE]

Now $\dot{\sigma}_{1}$ is the total derivative of $\Sigma_{sys}(\tau|\{i(\tau)\})=-\log{p_{i(\tau)}}(\tau)$, as shown in Eqs. (16) and (17). We integrate the above equation over time $\tau$ from 0 to $t$ with $i(0)=i_{0}$ and $i(t)=i$ and obtain

[TABLE]

Combining $\sigma_{1}$ and $\sigma_{2}$ yields the total entropy production (25) along a stochastic trajectory.