Information thermodynamics for interacting stochastic systems without bipartite structure

TL;DR

This paper extends information thermodynamics to coupled stochastic systems lacking bipartite structure, providing new measures and relations applicable to complex biochemical networks and non-Markovian processes.

Contribution

It introduces a generalized framework for information and thermodynamic exchanges in non-bipartite, non-Markovian systems, including new measures and an extended second law of thermodynamics.

Findings

Derived an extension of the second law involving transfer entropy rates.

Generalized information measures for non-bipartite systems.

Applied formalism to cellular process influence analysis.

Abstract

Fluctuations in biochemical networks, e.g., in a living cell, have a complex origin that precludes a description of such systems in terms of bipartite or multipartite processes, as is usually done in the framework of stochastic and/or information thermodynamics. This means that fluctuations in each subsystem are not independent: subsystems jump simultaneously if the dynamics is modeled as a Markov jump process, or noises are correlated for diffusion processes. In this paper, we consider information and thermodynamic exchanges between a pair of coupled systems that do not satisfy the bipartite property. The generalization of information-theoretic measures, such as learning rates and transfer entropy rates, to this situation is non-trivial and also involves introducing several additional rates. We describe how this can be achieved in the framework of general continuous-time Markov processes, without restricting the study to the steady-state regime. We illustrate our general formalism on the case of diffusion processes and derive an extension of the second law of information thermodynamics in which the difference of transfer entropy rates in the forward and backward time directions replaces the learning rate. As a side result, we also generalize an important relation linking information theory and estimation theory. To further obtain analytical expressions we treat in detail the case of Ornstein-Uhlenbeck processes, and discuss the ability of the various information measures to detect a directional coupling in the presence of correlated noises. Finally, we apply our formalism to the analysis of the directional influence between cellular processes in a concrete example, which also requires considering the case of a non-bipartite and non-Markovian process.

Tables

Table 1. Table 1: Definitions of information measures for a non-bipartite process 𝐙 t = ( X t , Y t ) subscript 𝐙 𝑡 subscript 𝑋 𝑡 subscript 𝑌 𝑡 {\bf Z}_{t}=(X_{t},Y_{t}) . Similar quantities are defined by exchanging X 𝑋 X and Y 𝑌 Y . In the third column, it is implicit that the limit h → 0 + → ℎ superscript 0 h\to 0^{+} is taken.

Information measures	Name	Definition
$l_X^{+}\left(t\right)$	Forward learning rate (a.k.a. information flow)	${\frac{1}{h}⟨\ln \frac{P\left(Y_t|X_{t+h}\right)}{P\left(Y_t|X_t\right)}⟩|}_{0^{+}}$
$l_X^{-}\left(t\right)$	Backward learning rate	${\frac{1}{h}⟨\ln \frac{P\left(Y_{t+h}|X_{t+h}\right)}{P\left(Y_{t+h}|X_t\right)}⟩|}_{0^{+}}$
$l_X^s\left(t\right)$	Symmetric learning rate	$l_X^s\left(t\right)=\frac{1}{2}\left[l_X^{+}\left(t\right)+l_X^{-}\left(t\right)\right]$
${𝒯}_{X\to Y}\left(t\right)$	Transfer entropy (TE) rate	${\frac{1}{h}⟨\ln \frac{P\left(Y_{t+h}|X_0^t,Y_0^t\right)}{P\left(Y_{t+h}|Y_0^t\right)}⟩|}_{0^{+}}$
${\overline{𝒯}}_{X\to Y}\left(t\right)$	Single-time-step TE rate	${\frac{1}{h}⟨\ln \frac{P\left(Y_{t+h}|X_t,Y_t\right)}{P\left(Y_{t+h}|Y_t\right)}⟩|}_{0^{+}}$
${𝒯}_{X\to Y}^{†}\left(t\right)$	Backward TE rate	${\frac{1}{h}⟨\ln \frac{P\left(Y_t|X_{t+h}^T,Y_{t+h}^T\right)}{P\left(Y_t|Y_{t+h}^T\right)}⟩|}_{0^{+}}$
${\overline{𝒯}}_{X\to Y}^{†}\left(t\right)$	Single-time-step backward TE rate	${\frac{1}{h}⟨\ln \frac{P\left(Y_t|X_{t+h},Y_{t+h}\right)}{P\left(Y_t|Y_{t+h}\right)}⟩|}_{0^{+}}$
${\widehat{𝒯}}_{X\to Y}\left(t\right)$	Filtered TE rate	${\frac{1}{h}⟨\ln \frac{P(X_{t+h},Y_{t+h}|Y_0^t)}{P\left(X_{t+h}|Y_0^t\right)P\left(Y_{t+h}|Y_0^t\right)}⟩|}_{0^{+}}$
${\widehat{\overline{𝒯}}}_{X\to Y}\left(t\right)$	Single-time-step filtered TE rate	${\frac{1}{h}⟨\ln \frac{P(X_{t+h},Y_{t+h}|Y_t)}{P\left(X_{t+h}|Y_t\right)P\left(Y_{t+h}|Y_t\right)}⟩|}_{0^{+}}$

Table 2. Table 2: Theoretical values of the various TE rates for three IPTG concentrations (given in Table S1 of K2014 ). The time unit is the inverse of the average growth rate μ 0 subscript 𝜇 0 \mu_{0} . The values of 𝒯 ¯ i → j subscript ¯ 𝒯 → 𝑖 𝑗 \overline{{\cal T}}_{i\to j} (numbers in brackets) and l i + superscript subscript 𝑙 𝑖 l_{i}^{+} computed in LNTRL2017 are also reported. The parameters ρ 12 = D 12 / D 11 ​ D 22 subscript 𝜌 12 subscript 𝐷 12 subscript 𝐷 11 subscript 𝐷 22 \rho_{12}=D_{12}/\sqrt{D_{11}D_{22}} and ρ 23 = D 23 / D 22 ​ D 33 subscript 𝜌 23 subscript 𝐷 23 subscript 𝐷 22 subscript 𝐷 33 \rho_{23}=D_{23}/\sqrt{D_{22}D_{33}} quantify the correlations between the white noises ξ 1 subscript 𝜉 1 \xi_{1} , ξ 2 subscript 𝜉 2 \xi_{2} , and ξ 2 subscript 𝜉 2 \xi_{2} , ξ 3 subscript 𝜉 3 \xi_{3} in Eqs. ( VI.2 ).

Conc. of IPTG	Low	Intermediate	High
${\rho }_{12}$	$0.408$	$0.362$	0
${\rho }_{23}$	$0.537$	$0.498$	0.135
${𝒯}_{1\to 2}$ (${\overline{𝒯}}_{1\to 2})$	$0.053\left(0.053\right)$	$0.020\left(0.020\right)$	$0\left(0\right)$
${𝒯}_{2\to 1}$ (${\overline{𝒯}}_{2\to 1})$	$0.008\left(0.008\right)$	$0.001\left(0.001\right)$	$0.036\left(0.036\right)$
${𝒯}_{1\to 2}^{†}$	$0.020$	$0.011$	$0.026$
${𝒯}_{2\to 1}^{†}$	$0.001$		$0.010$
${\overline{𝒯}}_{1\to 2}^{†}$	$0.021$	$0.011$	$0.026$
${\overline{𝒯}}_{2\to 1}^{†}$	$0.002$		$0.010$
$l_1^{+}=-l_2^{-}$	$-0.327$	$-0.219$	$0.026$
$l_2^{+}=-l_1^{-}$	$-0.198$	$-0.177$	$-0.026$

Table 3. Table 3: Differences in the TE rates and symmetric learning rates (the time unit is μ 0 − 1 superscript subscript 𝜇 0 1 \mu_{0}^{-1} ).

Conc. of IPTG	Low	Intermediate	High
${𝒯}_{1\to 2}^{\left(net\right)}={𝒯}_{1\to 2}-{𝒯}_{2\to 1}$	$0.045$	$0.019$	$-0.036$
${𝒯}_{1\to 2}^{†\left(net\right)}={𝒯}_{1\to 2}^{†}-{𝒯}_{2\to 1}^{†}$	$0.019$	$0.011$	$0.016$
${𝒯}_{1\to 2}-{𝒯}_{1\to 2}^{†}$	$0.033$	$0.009$	$-0.026$
${𝒯}_{2\to 1}-{𝒯}_{2\to 1}^{†}$	$0.007$	$0.001$	$0.026$
$l_1^S=-l_2^S$	$-0.064$	$-0.021$	$0.026$

Equations

$$\frac{d}{dt}P({\mathbf{Z}}_t=\mathbf{z}|{\mathbf{Z}}_{t^{\prime }}={\mathbf{z}}^{\prime })=\int d{\mathbf{z}}^{\prime \prime }P({\mathbf{Z}}_t={\mathbf{z}}^{\prime \prime }|{\mathbf{Z}}_{t^{\prime }}={\mathbf{z}}^{\prime })L_t\left({\mathbf{z}}^{\prime \prime },\mathbf{z}\right),$$

$$L_t\left(\mathbf{z},{\mathbf{z}}^{\prime }\right)={\delta }_{\mathbf{z}\ne {\mathbf{z}}^{\prime }}{W}_t\left(\mathbf{z},{\mathbf{z}}^{\prime }\right)-{\delta }_{\mathbf{z}={\mathbf{z}}^{\prime }}\sum _{{\mathbf{z}}^{\prime \prime }\ne \mathbf{z}}{W}_t\left(\mathbf{z},{\mathbf{z}}^{\prime \prime }\right).$$

$$d{X}_t$$

$$d{Y}_t$$

$$D_{XX,t}\left(\mathbf{z}\right)$$

$$D_{XY,t}\left(\mathbf{z}\right)$$

$$L_t(\mathbf{z},{\mathbf{z}}^{\prime })=L_t^{FP}({\mathbf{z}}^{\prime })\left[\delta (\mathbf{z}-{\mathbf{z}}^{\prime })\right],$$

$$L_t^{FP}\left(\mathbf{z}\right)=-{\nabla }_x\circ F_{X,t}\left(\mathbf{z}\right)-{\nabla }_y\circ F_{Y,t}\left(\mathbf{z}\right)+{\nabla }_x\circ {\nabla }_x\circ D_{XX,t}\left(\mathbf{z}\right)+{\nabla }_y\circ {\nabla }_y\circ D_{YY,t}\left(\mathbf{z}\right)+2{\nabla }_x\circ {\nabla }_y\circ D_{XY,t}\left(\mathbf{z}\right)$$

$$J_{X,t}(\mathbf{z})$$

$$J_{Y,t}(\mathbf{z})$$

$${\partial }_t{P}_t(\mathbf{z})+{\nabla }_x{J}_{X,t}(\mathbf{z})+{\nabla }_y{J}_{Y,t}(\mathbf{z})=0.$$

$$P({\mathbf{Z}}_{t+h}|{\mathbf{Z}}_t)=P(X_{t+h}|{\mathbf{Z}}_t)P(Y_{t+h}|{\mathbf{Z}}_t)+\mathrm{O}(h^2),$$

$$L_t(\mathbf{z},{\mathbf{z}}^{\prime })=\delta (y-y^{\prime })L_{t,y}(x,x^{\prime })+\delta (x-x^{\prime })L_{t,x}(y,y^{\prime }),$$

$$l_X(t)$$

$$(\mathbf{B})d_{t}I(X_t:Y_t)=l_X(t)+l_Y(t),$$

$${\mathcal{T}}_{X\to Y}(t)$$

$$\displaystyle=\lim_{h\rightarrow 0^{+}}\frac{1}{h}\Bigl{\langle}\ln\frac{P(Y_{t+h}|X_{0}^{t},Y_{0}^{t})}{P(Y_{t+h}|Y_{0}^{t})}\Bigr{\rangle}\ .$$

$$(\mathbf{M}){\mathcal{T}}_{X\to Y}(t)$$

$$d_{t}I(X_0^t:Y_0^t)$$

$$(\mathbf{M})$$

$$(\mathbf{B})$$

$${\overline{\mathcal{T}}}_{X\to Y}(t)$$

$$(\mathbf{B}+\mathbf{S})$$

$$P(X_t|Y_0^t)=P(X_t|Y_t),$$

$${\sigma }_X(t)\equiv d_t{S}_X(t)+{\sigma }_X^B(t),$$

$$\displaystyle({\bf B})\>\>\>\>\Sigma_{X}^{B}=\Bigl{\langle}\ln\frac{\widehat{P}_{Y_{0}^{t}}(X_{0}^{t}|X_{0})}{\widehat{P}_{Y_{t}^{0}}(X_{t}^{0}|X_{t})}\Bigr{\rangle}\ ,$$

$${\sigma }_X^B(t)\equiv \sum _{\mathbf{z},x^{\prime }}{P}_t\left(\mathbf{z}\right)W_{t,y}(x,x^{\prime })\ln \frac{W_{t,y}(x,x^{\prime })}{W_{t,y}(x^{\prime },x)},$$

$${\sigma }_X^B(t)\equiv \int d\mathbf{z}{D}_{XX,t}(\mathbf{z}{)}^{-1}{J}_{X,t}(\mathbf{z}){\hat{F}}_{X,t}(\mathbf{z}),$$

$${\sigma }_X(t)\ge l_X(t)\ge d_{t}I(X_t:Y_0^t)-{\mathcal{T}}_{X\to Y}(t),$$

$${\Sigma }_X\equiv {\int }_0^{t}ds{\sigma }_X(s)\ge {\int }_0^{t}ds{l}_X(s)\ge \Delta I-{\int }_0^{t}ds{\mathcal{T}}_{X\to Y}(s)\ge \Delta \overline{I}-{\int }_0^{t}ds{\mathcal{T}}_{X\to Y}(s)\ge \Delta \overline{I}-{\int }_0^{t}ds{\overline{\mathcal{T}}}_{X\to Y}(s),$$

$$l_X^{-}(t)$$

$$d_{t}I(X_t:Y_t)$$

$$l_X^S(t)$$

Full text

1. I Introduction

2. II Setup and brief reminder of the bipartite case

3. II.1 Setup

4. II.2 Definition of information measures

5. II.3 Inequalities and sufficient statistic

6. II.4 Entropy production and second law

7. III Information measures for non-bipartite processes

8. III.1 Learning rates

9. III.2 Transfer entropy rates

10. III.3 Backward transfer entropy rates

11. III.4 Inequalities

12. IV Markov diffusion processes and second law of information thermodynamics

13. IV.1 Learning rates

14. IV.2 Single-time-step transfer entropy rates

15. IV.3 Multi-time-step transfer entropy rates

16. IV.4 Second law for non-bipartite processes

17. V Stationary bi-dimensional Ornstein-Uhlenbeck process

18. V.1 Expression of the multi-time-step TE rates

19. V.2 Numerical illustration

20. VI Generalization to a non-Markovian process and application to the study of directional influence between cellular processes

21. VI.1 Extension to a bivariate non-Markovian process

22. VI.2 Application to the study of directional influence between cellular processes

23. VII Summary

24. A Expression of the learning rates for a pure jump Markov process in discrete space

25. B Proof of various relations of the main text

26. C Sufficient statistic for a non-bipartite dynamics

27. D Analytical expressions for a stationary bi-dimensional Ornstein-Uhlenbeck process

28. E Multi-time-step TE rates for the three-component process of Sec. VI.2

I Introduction

One recent and important field of application of information theory is biological systems, in particular gene regulation and signal transduction systems. Cells must sense, process and adapt to their environment or their own physiological state, which are noisy processes subjected to fluctuations (see e.g. BS2014 ). In addition, information transfers consume energy, so that there is a competition between the information gain and the energy cost. This fundamental issue is the realm of information thermodynamics, a recent and active field of research, as reviewed in PHS2015 . In this framework, the present work is motivated by the observation that there exist, broadly speaking, two different sources of fluctuations contributing to the stochasticity of biochemical processes, for instance in cell metabolic networks. The first one – commonly called “intrinsic”– is due the small numbers of biomolecules involved in a given reaction. The second one – the “extrinsic” source– arises from the heterogeneity in the physical environment of the cell and the occurrence of (many) other biochemical reactions (see, e.g.,  E2002 ; TWW2006 ; DCLME2008 ; UW2011 ; GW2014 ; K2014 ; HT2016 ; LEM2017 ; LNTRL2017 ). This implies that the noise in the input biochemical signal – to be detected– and the noise of the reactions that form the network are correlated. In short, stochastic noises have a nontrivial structure and fluctuations in each subsystem are not independent.

In contrast, in the context of stochastic and information thermodynamics, the so-called bipartite assumption is usually made (e.g., for modeling Maxwell’s demons): one assumes that subsystems cannot jump simultaneously if the dynamics is modeled as a Markov jump process or that noises are uncorrelated if the dynamics is modeled as diffusion processes.This simplifies the theoretical analysis and allows the contribution of each components of the system to the entropy production to be clearly identified AJM2009 ; BHS2013 ; HE2014 ; IS2013 ; DE2014 ; BHS2014 ; HBS2014 ; HS2014 ; IS2015 ; SS2015 ; H2015 ; HBS2016 ; SLP2016 ; I2016 ; RH2016 ; MS2018 ; I2018 . Although the abandon of the bipartite (or multipartite H2015 ) structure seriously complicates the interpretation of information and thermodynamics exchanges, our objective in the present work is to show that a detailed description is still available. The price to pay is that several information-theoretic measures must be added to those already introduced in the literature (information flow, aka learning rate, and transfer entropy), which characterize how information is exchanged between two interacting systems in the course of their dynamical evolution.

In this paper, we will mostly consider non-equilibrium systems that can be modeled by continuous-time Markov processes (diffusions, jump processes, or both). It turns out however that many definitions or relations are also valid beyond the Markovian description and we will therefore provide a general framework. Moreover, in order to offer a sufficiently general perspective, we assume the presence of multiplicative noises (additive noises being regarded as a only special case) and we do not restrict the study to steady-state situations, as is often done. On the other hand, as far as stochastic thermodynamics is concerned, we only consider averaged quantities and do not derive fluctuation relations. We leave this important issue to future investigations.

The main results of the paper can be summarized as follows:

1. We introduce a set of information-theoretic measures to consistently characterize information exchanges in a generic non-bipartite stochastic system composed of two interacting subsystems. This includes learning rates and transfer entropy rates. In particular, we define a learning rate that contains the footprints of time-irreversibility. To make the reading of the following sections easier, the definitions of all these information measures are listed in Table I.

2. We derive a set of inequalities among these quantities and show under which circumstances some inequalities become equalities, which may signal the presence of a so-called “sufficient statistic” CT2006 ; MS2018 . For Markov processes, we then propose a generalization of the notion of sensory capacity introduced in HBS2016 .

3. We give explicit expressions of the information measures for Markov diffusion processes in terms of probability currents, diffusion tensor, and propagators. In passing, we obtain a generalization to non-bipartite systems of a classical result linking information theory and estimation theory D1970 ; D1971 .

4. We derive a generalized version of the so-called “second law of information thermodynamics” PHS2015 that applies to non-bipartite diffusion processes, showing that the entropy production rate in one of the subsystems is lower bounded by the difference in transfer entropy rates in the forward and backward time directions.

5. We illustrate the behavior of the various information-theoretic measures in the case of a stationary bidimensional Ornstein-Uhlenbeck process and show their evolution as one systematically varies the parameter quantifying the correlation between the noises.

6. We apply our formalism to the study of the directional influence between cellular processes in the metabolism of E. coli K2014 ; LNTRL2017 and exhibit an intriguing feature that may indicate some optimality in the transmission of information.

More specifically, the paper is organized as follows. In Sec. II, we describe our general setup and briefly review the existing results for bipartite processes. To this aim, we first present the formal tools that will be used throughout the paper, in particular those related to continuous-time Markov processes. We then define the information-theoretic measures that are commonly considered in the framework of information thermodynamics and that satisfy some useful inequalities. We then recall the corresponding formulations of the second law. In Sec. III, the bipartite assumption is lifted and we introduce the relevant information measures. The usual inequalities are then generalized. In Sec. IV, to make all of the introduced definitions and relations more explicit, we focus on Markov diffusion processes. In Sec. V as a special case, we consider a stationary bidimensional Ornstein-Uhlenbeck process with additive noises for which a full analytical study can be carried out. This allows us to illustrate on a simple example the ability of the various information measures to infer a directional coupling in the underlying dynamics. Finally, in Sec. VI, we generalize the formalism to a class of non-Markovian processes and apply it to cellular processes in the metabolism of E. coli. This complements the previous experimental and theoretical investigations of Refs. K2014 ; LNTRL2017 . A brief summary is given in Sec. VII and some demonstrations and technical details are presented in Appendices.

II Setup and brief reminder of the bipartite case

II.1 Setup

We are interested in the information and thermodynamic exchanges between two subsystems, denoted by $X$ and $Y$, of a stochastic system ${\bf Z}$ whose microscopic states at time $t$ are denoted by ${\bf Z}_{t}=(X_{t},Y_{t})$. The random variables $X$ and $Y$ may be multivariate ($X$ and $Y$ are then vectors), continuous or discrete, and live in arbitrary, and not necessarily identical, spaces. In the following, the full process ${\bf Z}_{t}$ can be Markovian or non-Markovian, but even in the former case the individual dynamics of $X_{t}$ and $Y_{t}$, viewed as coarse-grained descriptions of ${\bf Z}_{t}$, are in general non-Markovian.

When ${\bf Z}_{t}$ is a continuous-time Markovian process  R1989 ; G2004 ; C2005 ; EK2005 ; RW2005 , which may involve a combination of drift, diffusion, and jump, the building block of its description is the transition probability $P(\mathbf{Z}_{t}=\mathbf{z}|\mathbf{Z}_{t^{\prime}}=\mathbf{z^{\prime}})$, for $t^{\prime}\leq t$ and all $\bf z,\bf{z^{\prime}}$. Such an object is generated by a kernel $L_{t}({\bf z},{\bf z}^{\prime})$, called the Markovian generator, according to the forward Kolmogorov equation,

[TABLE]

where $d\mathbf{z^{\prime\prime}}$ is the appropriate measure for either continuous or discrete space. For pure jump processes in a discrete space the Markovian generator is a matrix involving the transition rates $W_{t}\left(\bf{z},\bf{z^{\prime}}\right)$ (where by convention the transition is from $\bf{z}$ to $\bf{z^{\prime}}$)

[TABLE]

On the other hand, pure diffusion processes in continuous space are usually described by the stochastic differential equations

[TABLE]

where $F_{X,t}$, $F_{Y,t}$, $\sigma_{X,j,t}$, and $\sigma_{Y,j,t}$ are time-dependent vector fields, and the $W_{j,t}$’s are independent Brownian motions. The non-negative covariance (diffusion) matrix $D_{t}\left(\mathbf{z}\right)$ is the $2\times 2$ block matrix with components

[TABLE]

where the symbol $\otimes$ applied to two vectors $U$ and $V$ means the matrix construction $(U\otimes V)^{ij}\equiv U^{i}V^{j}$. The associated Markovian generator is then obtained as

[TABLE]

where $L_{t}^{FP}$ is the (Fokker-Planck) second-order differential operator

[TABLE]

obtained by interpreting Eqs. (3) with Ito convention. (In the above expression the last term is a priori ambiguous because $D_{XY,t}$ is not necessarily symmetric. The notation $\nabla_{x}\circ\nabla_{y}\circ D_{XY,t}$ should thus be interpreted as $\nabla_{x_{i}}\circ\nabla_{y_{j}}\circ(D_{XY,t})^{i,j}$, using Einstein summation convention for repeated indices.) As usual, one can introduce the probability currents

[TABLE]

and recast the Fokker-Planck (FP) equation as the continuity equation

[TABLE]

Note that the currents are defined up to a divergence-free vector. We will use the definitions in Eq. (7) in the following, which means that correction terms must be added in all expressions involving the currents if another decomposition is adopted.

In continuous time, the Markov process ${\bf Z}$ is called bipartite if the transition probability $P({\bf Z}_{t+h}|{\bf Z}_{t})$ satisfies the property

[TABLE]

when $h\to 0^{+}$. From the forward Kolmogorov equation (1), the above condition is equivalent to assuming that the Markovian generator $L_{t}({\bf z},{\bf z}^{\prime})$ can be written as

[TABLE]

where $L_{t,y}(x,x^{\prime})$ and $L_{t,x}(y,y^{\prime})$ are called partial generators. The delta functions become Kronecker matrices in the case of discrete space. The partial generators must individually satisfy conservation of probability, i.e., $\int dx^{\prime}L_{t,y}(x,x^{\prime})=\int dy^{\prime}L_{t,x}(y,y^{\prime})=0$. In particular, a pure jump process is bipartite if the transition rates have the additive form $W_{t}[\mathbf{z},\mathbf{z^{\prime}}]=\delta_{y=y^{\prime}}W_{t,y}(x,x^{\prime})+\delta_{x=x^{\prime}}W_{t,x}(y,y^{\prime})$, which implies Eq. (10), as can be readily checked. On the other hand, a pure diffusion process is bipartite if $D_{XY,t}=0$, i.e., if the diffusion matrix is block diagonal. From Eqs. (4), a sufficient condition is that $\sigma_{X,j,t}\otimes\sigma_{Y,j,t}=0$ for all $j$, which means that the overall noises affecting $X_{t}$ and $Y_{t}$ are independent.

II.2 Definition of information measures

We start our reminder of information thermodynamics by recalling the definitions of several information-theoretic measures that are usually introduced in this framework. As already stressed, a consequence of the abandon of the bipartite assumption will be a proliferation of information measures. It is thus desirable to use transparent notations as much as possible. (Already in the bipartite case, a given quantity may have different names or be defined with different signs, which is a source of confusion.) It is also important to clearly state under which condition a relation is valid: in the following, the capital letter M on the left of an equation indicates that the joint process is Markovian, the capital letter B indicates that the process is Markovian and bipartite, and the capital letter S indicates that the process is stationary.

1. Information flows, aka learning rates

Information flows quantify how the dynamical evolution of $X_{t}$ or $Y_{t}$ contributes to the change in the mutual information, $I(X_{t}:Y_{t})\equiv\langle\ln(P({\bf Z}_{t})/[P(X_{t})P(Y_{t})]\rangle$, where $P({\bf Z}_{t})$ is the joint probability distribution and $P(X_{t})$, $P(Y_{t})$ its marginals. The latter characterizes the instantaneous correlation between $X$ and $Y$ at time $t$. These information-theoretic measures were first considered in the context of interacting diffusion processes AJM2009 and subsequently introduced in the analysis of the thermodynamics of continuously-coupled, discrete-space stochastic systems HE2014 ; HBS2014 ; SS2015 . Consider for instance the dynamical evolution of $X_{t}$. Introducing the time-shifted mutual information $I(X_{t+h}:Y_{t})$ (with $h>0$) and taking the limit $h\to 0^{+}$, one then defines AJM2009

[TABLE]

where here and in the following we use the bracket symbol for an expectation. Similarly, $l_{Y}(t)$ is defined from $I(Y_{t+h}:X_{t})$. (For brevity, it will be implicit in the following that similar quantities can be defined by exchanging $X$ and $Y$.) One could also introduce Shannon entropies instead of mutual informations, by using $I(X_{t}:Y_{t})=H(X_{t})-H(X_{t}|Y_{t})=H(Y_{t})-H(Y_{t}|X_{t})$, with $H(X_{t})\equiv-\langle\ln P(X_{t})\rangle$ and $H(X_{t}|Y_{t})\equiv-\langle\ln P(X_{t}|Y_{t})\rangle$ CT2006 ; however, we will try to avoid too many equivalent formulations throughout the paper. Note that the definition (11) is not restricted to a steady state. In a steady state the information flow identifies with the so-called learning rate $l_{X}$ defined in BHS2014 ; HBS2016 . Hereafter, we will use the denomination learning rate for $l_{X}(t)$ whether or not the system is in a steady state note00 .

As discussed in AJM2009 ; HE2014 , learning rates have a clear meaning: For instance, $l_{X}(t)>0$ reveals that the dynamical evolution of $X$ increases the mutual information $I(X_{t}:Y_{t})$. In other words, the future of $X$ is more predictable than its present from the viewpoint of $Y$ AJM2009 , or $X$ is “learning about” $Y$ through its dynamics HE2014 .

For a bipartite Markov process, one has the natural decomposition of the time derivative of $I(X_{t}:Y_{t})$ AJM2009 ; HE2014 ; MS2018

[TABLE]

as will be explicitly illustrated below for Markov processes.

2. Transfer entropy

Transfer entropy (TE) is an information-theoretic measure that is used to assess directional dependencies between time series and possibly infer causal interactions S2000 ; PKHS2001 . Instead of $I(X_{t}:Y_{t})$, one considers the change in the mutual information between stochastic trajectories observed during some time interval, say from [math] to $t$, and which are denoted by $X_{0}^{t}$ and $Y_{0}^{t}$ hereafter. Specifically, we define the TE rate from $X$ to $Y$ in continuous time as

[TABLE]

where we have assumed that $Y_{0}^{t+h}\sim(Y_{0}^{t},Y_{t+h})$ for $h$ infinitesimal and used the chain rule for mutual information, $I(A:\{B,C\})=I(A:C)+I(A:B|C)$, where $I(A:B|C)$ is a conditional mutual information CT2006 . Like the learning rate, ${\cal T}_{X\to Y}(t)$ has a clear interpretation in terms of information transfer: It quantifies how much the knowledge of the trajectory $X_{0}^{t}$ reduces the uncertainty about $Y_{t+h}$ (for $h$ infinitesimal) when the trajectory $Y_{0}^{t}$ is already known. As a conditional mutual information, ${\cal T}_{X\to Y}(t)$ is a non-negative quantity, whereas $l_{X}(t)$ has no definite sign. Note that the present definition is more general than the one adopted in HBS2016 or MS2018 since we do not assume at this stage that the joint process is Markovian. When the joint process is Markovian, one has $P(Y_{t+h}|X_{0}^{t},Y_{0}^{t})=P(Y_{t+h}|X_{0}^{t},Y_{t})=P(Y_{t+h}|X_{t},Y_{0}^{t})=P(Y_{t+h}|X_{t},Y_{t})$, and, after some manipulations, Eq. (II.2) can be rewritten as

[TABLE]

which clearly shows the difference with the learning rate $l_{Y}(t)=\lim_{h\to 0^{+}}(1/h)[I(X_{t}:Y_{t+h})-I(X_{t}:Y_{t})]$. Note that the original definition of transfer entropy in discrete time is even more general since the number of time bins in the past of $X_{t}$ and $Y_{t}$ may be different S2000 . This definition can also be extended to continuous time SLP2016 ; BBHL2016 ; SL2018 . Finally, see Ref. WKP2013 for a rigorous definition via a partition of the time interval.

For a Markov bipartite process, in full analogy with Eq. (12), one has the decomposition

[TABLE]

where we have used Eq. (9) and assumed ${\bf Z}_{0}^{t+h}\sim({\bf Z}_{0}^{t},{\bf Z}_{t+h})$ for $h$ infinitesimal. Let us stress that the trajectory mutual information $I(X_{0}^{t}:Y_{0}^{t})$ is a time-extensive quantity, in contrast with $I(X_{t}:Y_{t})$. As a consequence, $d_{t}I(X_{0}^{t}:Y_{0}^{t})$ does not vanish in a steady state and, then, ${\cal T}_{X\to Y}\neq-{\cal T}_{Y\to X}$, while $l_{X}=-l_{Y}$. Here and throughout the paper, quantities without explicit time-dependence refer to a steady state.

Although this is rarely evoked in the stochastic thermodynamic literature, we recall that transfer entropy is essentially a non-linear extension of Granger causality (GC) G1969 , which is a concept widely used in econometrics and neuroscience for analyzing the relationships between time series and inferring causal interactions (see e.g. AM2013 of a review). The general issue is whether the knowledge of one of the variables can help forecast another one. In contrast with transfer entropy, GC is usually identified with a model-based viewpoint, for instance a vector autoregressive modeling of the time series data L2005 . It turns out that linear GC and transfer entropy rate are fully equivalent when the variables are Gaussian distributed, with a simple factor of $2$ relating the two quantities note54 . This equivalence, first shown in the case of discrete-time random processes BBS2009 , can be extended to the continuous-time version BS2017 .

Since the TE rates are conditioned on whole trajectories, they are very hard to compute numerically note55 and one often replaces $X_{0}^{t}$ and $Y_{0}^{t}$ by the states $X_{t}$ and $Y_{t}$ at the latest time $t$. One then defines AJM2009 ; HBS2014 ; HBS2016 ; MS2018

[TABLE]

which is called a “single-time-step” TE rate in MS2018 to contrast with the “multi-time-step” TE rate ${\cal T}_{X\to Y}(t)$. We will adopt this terminology hereafter.

II.3 Inequalities and sufficient statistic

For a Markov bipartite process in a steady state, one has the two inequalities HBS2014 ; HBS2016

[TABLE]

and we do not report the demonstration here since more general inequalities will be derived in the next section. The second inequality expresses the intuitive idea that the instantaneous value of $Y$ is less informative about the instantaneous value of $X$ that the whole past trajectory of $Y$. Within the context of a sensory system, where $X_{t}$ and $Y_{t}$ denote the states of the signal and the sensor, respectively, this prompted the authors of HBS2016 to introduce a so-called “sensory capacity” $C_{Y}=l_{Y}/{\cal T}_{X\to Y}$ as a tool to quantify the performance of the sensor (assuming that $l_{Y}\geq 0$). In particular, $C_{Y}$ reaches its maximal value $1$ when inequality (18) is saturated. As discussed in MS2018 , inequalities (17) and (18) are both saturated when the following condition is satisfied:

[TABLE]

which means that “$Y_{t}$ is a sufficient statistic of $X_{t}$” CT2006 and no more information about $X_{t}$ is contained in the trajectory $Y_{0}^{t}$ than in $Y_{t}$ alone. Interestingly, such an optimization of information transfer may occur in actual biological signaling circuits HSWIH2016 ; MS2018 . By construction, this condition is realized by the Kalman-Bucy filter KB1961 ; KSH2000 ; A2006 , as will be illustrated later on.

As can be expected, things become more complicated when the bipartite assumption is dropped, and we show in the next section that this requires introducing additional information-theoretic measures.

II.4 Entropy production and second law

While the conventional second law of thermodynamics deals with the irreversibility of the whole process ${\bf Z}_{t}$, information measures can be used to formulate modified versions of the second law (which may then be called “second laws of information thermodynamics ”) that assess the irreversibility of one subsystem alone, say $X_{t}$, in the presence of the coupling with the other subsystem. The key quantity is the (fixed-time) entropy production rate $\sigma_{X}(t)$ which is defined by considering $X$ as an open system and $Y$ as just a fictitious external protocol (or idealized work source) SU2012 . On general grounds (see, e.g., VdBE2015 ), $\sigma_{X}(t)$ can be decomposed as

[TABLE]

where $d_{t}S_{X}(t)$, the time derivative of the marginal Shannon entropy $S_{X}(t)=-k_{B}\int dx\>P_{t}(x)\ln P_{t}(x)$, is the rate of change of the entropy of $X$, and $\sigma_{X}^{B}(t)$ is the rate of change of the entropy of the environment or the bath. (From now on the Boltzmann constant $k_{B}$ is set equal to $1$, so that we may use $S$ instead of $H$ as Shannon entropy.) As is now standard in the framework of stochastic thermodynamics (see, e.g., M2003 ), the cumulative entropy change $\Sigma_{X}^{B}=\int_{0}^{t}ds\>\sigma_{X}^{B}(s)$ can be expressed as the mean of the logratio of the probabilities to observe a trajectory in forward and backward “experiments”. As $Y$ is treated as an external protocol, one has

[TABLE]

where $\widehat{P}_{Y_{0}^{t}}(X_{0}^{t}|X_{0})$ is the probability of the trajectory of $X$ for a fixed trajectory of $Y$ and $\widehat{P}_{Y_{t}^{0}}(X_{t}^{0}|X_{t})$ is the corresponding probability of the time-reversed trajectory $X_{t}^{0}$ for the fixed time-reversed trajectory $Y_{t}^{0}$ footnote_probdetached . For a bipartite pure jump process, $\sigma_{X}^{B}(t)$ is then given by

[TABLE]

whereas for a bipartite diffusion process it is equal to

[TABLE]

where the diffusion matrix $D_{XX,t}>0$ and the probability current $J_{X,t}$ have been defined above, and $\widehat{F}_{X,t}({\bf z})$ is the modified drift defined by $\widehat{F}_{X,t}\left(\mathbf{z}\right)\equiv F_{X,t}\left(\mathbf{z}\right)-\nabla_{x}.D_{XX,t}({\bf z})$ CG2008 . In cases where the thermodynamics of subsystem $X$ can be defined and the environment is a single thermal bath at a given inverse temperature $\beta$, $\sigma_{X}^{B}$ identifies with the heat flow $\beta\dot{\cal Q}$ from $X$ to the bath.

Since the two subsystems are coupled, $\sigma_{X}(t)$ may become negative, but a lower bound is provided by including the information shared with $Y$. For a bipartite Markov process, the various second-law-like inequalities proven in the literature AJM2009 ; IS2013 ; HE2014 ; HBS2014 ; IS2015 can be summarized by the following hierarchy of bounds,

[TABLE]

or, for the time-integrated quantities,

[TABLE]

where $\Delta I=I(X_{t}:Y_{0}^{t})-I(X_{0},Y_{0})$ and $\Delta\overline{I}=I(X_{t}:Y_{t})-I(X_{0},Y_{0})$, with $\Delta I\geq\Delta\overline{I}$ by marginalization. The key fact is that the tightest bound is provided by the learning rate.

III Information measures for non-bipartite processes

III.1 Learning rates

We first search for a generalization of Eq. (12). The decomposition of $d_{t}I(X_{t}:Y_{t})$ introduced above in the bipartite case suggests to define the new rate

[TABLE]

in addition to $l_{X}(t)$. Accordingly, $l_{X}(t)$ will be denoted $l^{+}_{X}(t)$ hereafter to make the notations more consistent. Indeed, $l^{-}_{X}(t)$ can be also written as $l^{-}_{X}(t)=-dI(X_{t-h}:Y_{t})/dh|_{h=0^{+}}$ whereas $l^{+}_{X}(t)=dI(X_{t+h}:Y_{t})/dh|_{h=0^{+}}$. This also suggests to call $l_{X}^{-}(t)$ a backward learning rate, in contrast with the forward rate $l^{+}_{X}(t)$. We stress that the definition of $l^{-}_{X}(t)$ as a derivative has nothing to do with stationarity but simply results from a Taylor expansion in $h$: Indeed, for any continuously differentiable function $f(s,t)$, one has $f(t+h,t+h)-f(t,t+h)=h(\partial_{1}f)(t,t)+{\cal O}(h^{2})=f(t,t)-f(t-h,t)+{\cal O}(h^{2})$. Note also that $l^{-}_{X}(t)\neq dI(X_{t}:Y_{t+h})/dh|_{h=0^{+}}$.

Thanks to the introduction of $l^{-}_{X}(t)$, and of the corresponding $l^{-}_{Y}(t)$, Eq. (12) is now replaced by the two relations

[TABLE]

The distinction between the forward and backward learning rates in the general (i.e., non-bipartite) case suggests to introduce the symmetric quantities

[TABLE]

such that Eqs. (27) now yield

[TABLE]

As will be seen below, these symmetric learning rates play a natural role in the thermodynamics since they vanish when the joint system is at equilibrium, i.e., when the condition of detailed balance is satisfied. On the other hand, the non-symmetric rates vanish when the processes $X_{t}$ and $Y_{t}$ are independent. Note that we use the adjective “symmetric” to qualify $l_{X}^{S}(t)$ because it is the half-sum of the forward and backward learning rates, i.e., of right and left derivatives. There must be no confusion with the time-symmetric, kinetic (path-) observables such a “frenesy” which also play a significant role in the study of non-equilibrium phenomena M2019 .

In the general case, but in a steady state, there are only two independent learning rates since $d_{t}I(X_{t}:Y_{t})=0$ and Eqs. (27) yield the “conservation” relations

[TABLE]

and thus

[TABLE]

A basic feature of the learning rates is that they can be expressed in terms of the two-point probability distribution $P({\bf z},t;{\bf z}^{\prime},t^{\prime})\equiv\langle\delta({\bf Z}_{t}-{\bf z})\delta({\bf Z}_{t^{\prime}}-{\bf z}^{\prime})\rangle$, with ${\bf z}\equiv(x,y)$, and the corresponding marginal distributions. (Hereafter, variables with a prime symbol such as ${\bf z}^{\prime},x^{\prime},y^{\prime}$ will always refer to a time $t^{\prime}\leq t$ in the two-point probability distribution functions.) Starting from the definitions (11) and (26), and using the normalization condition $\int dx\>dy^{\prime}\>P(x,t;y^{\prime},t^{\prime})=1$, we get

[TABLE]

where we have used the notation $P_{t}(x,y)\equiv P(x,t;y,t)$ for the joint probability distribution at the same time $t$ (and $P_{t}(x)$, $P_{t}(y)$ for the marginal distributions); similar expressions are obtained for $l^{+}_{Y}(t)$ and $l^{-}_{Y}(t)$. (We recall that we use the same notations for continuous and discrete spaces. In the latter case, integrals must be replaced by sums.) From these equations, we readily see that the learning rates vanish when $P_{t}({\bf z})=P_{t}(x)P_{t}(y)$, which means that the processes $X$ and $Y$ are independent. We stress that these formulas are fully general and do not require the joint process ${\bf Z}$ to be Markovian. However, further simplifications occur in the Markovian case, as one can replace the derivative with respect to $h$ by using the Kolmogorov equation and introducing the Markovian generator $L_{t}({\bf z^{\prime}},{\bf z})$, which leads to

[TABLE]

Furthermore, by using the decomposition in Eq. (27) and the expression of the time derivative of the mutual information,

[TABLE]

$l_{X}^{-}(t)$ is obtained as

[TABLE]

Finally, after using the conservation of probability $\int d{\bf z}L_{t}({\bf z^{\prime}},{\bf z})=0$, we obtain the symmetric learning rates as

[TABLE]

From the above expression, one can immediately see that if $Z_{t}$ is an equilibrium process, such that the probability current $P_{t}({\bf z^{\prime}})L_{t}\left({\bf z^{\prime}},{\bf z}\right)-P_{t}({\bf z})L_{t}\left({\bf z},{\bf z^{\prime}}\right)$ vanishes, the symmetric learning rates both vanish. In Sec. IV.1, we will provide more explicit expressions for these learning rates in the case of a Markovian diffusion process. The case of a Markovian pure jump process in a discrete space is treated in Appendix A.

If we now come back to the special situation of a bipartite process, due to the additive form of the Markovian generator [Eq. (10)] and the conservation of probability, the formulas given in Eqs. (33) and (35) coincide and

[TABLE]

A similar relation holds for $l_{Y}^{+}(t)$ and $l_{Y}^{-}(t)$. There are thus only two independent learning rates instead of four, and Eq. (27) then gives back Eq. (12). Note that the present equalities between learning rates differ from those expressed in Eq. (III.1). More generally, one must carefully distinguish relations valid for a bipartite process from those valid for a non-bipartite process in a steady state note1 . Of course, if the joint process is both bipartite and stationary, Eqs. (III.1) and (37) imply that only one independent learning rate subsists, for instance $l^{S}_{Y}=-l^{S}_{X}=l^{+}_{Y}=l^{-}_{Y}=-l^{+}_{X}=-l^{-}_{X}$.

We conclude this part on the learning rates by briefly discussing their content in terms of information. The various quantities $l_{X}^{+}(t)$, $l_{X}^{-}(t)$, $l_{X}^{S}(t)$, and their counterparts for the $Y$ subsystem, all measure the change of mutual information between $X$ and $Y$ due to different aspects of an infinitesimal dynamical evolution of one subsystem or the other. When both $l_{X}^{+}(t)$ and $l_{X}^{-}(t)$ are strictly positive, and as a direct consequence $l_{X}^{S}(t)>0$, one can plausibly conclude that $X$ is “learning about” $Y$ through its dynamics. However, the non-bipartite structure of the process allows cases with $l_{X}^{+}(t)>0$ and $l_{X}^{-}(t)<0$, which have no manifest interpretation in the context of learning.

III.2 Transfer entropy rates

Transfer entropy rates ${\cal T}_{X\to Y}(t)$ and ${\cal T}_{Y\to X}(t)$ can be defined by the same formulas as in the bipartite case: see Eq. (II.2). They keep the same property of being non-negative and the same meaning as information-theoretic measures. However, whereas the generalization of the decomposition of the variation of mutual information [Eq. (12)] to a non-bipartite process was straightforward, a similar operation for the pathwise mutual information [Eq. (II.2)] turns out to be problematic. Indeed, using the second line of Eq. (II.2), one can write

[TABLE]

where ${\cal T}_{X.Y}(t)\equiv\lim_{h\to 0^{+}}h^{-1}I(X_{t+h}:Y_{t+h}|X_{0}^{t},Y_{0}^{t})$ is a symmetric quantity measuring the “instantaneous” dependence of the two processes (see, e.g., C2011 for the discrete-time version). However, there is a serious obstruction, at least for diffusion processes: ${\cal T}_{X.Y}(t)$ is either zero if ${\bf Z}$ is bipartite [cf. Eq. (II.2) above] or infinite otherwise! In other words, $d_{t}I(X_{0}^{t}:Y_{0}^{t})$ and thus $I(X_{0}^{t}:Y_{0}^{t})$ itself are infinite for non-bipartite diffusions. Indeed, as noticed in N2016 , $X_{t}$ and $Y_{t}$ have a non-zero quadratic variation if the noises are correlated, which results in the singularity of their joint distribution with respect to the product of the corresponding marginals. Such a difficulty does not occur in discrete time, as briefly discussed in note note4 , nor in discrete space.

We thus turn our attention to another class of rates which are well-defined in the non-bipartite case and will allow us to generalize the important inequality (18). These rates are associated with the mutual informations $I(X_{t}:Y_{0}^{t})$ and $I(Y_{t}:X_{0}^{t})$ that appear in filtering theory K1962 ; LS2001 . We introduce the TE rate, called “filtered transfer entropy rate”,

[TABLE]

which quantifies how much the prediction of $X_{t+h}$, for $h$ infinitesimal, is improved by knowing $Y_{t+h}$ in addition to the trajectory $Y_{0}^{t}$. In general there is no simple relation between ${\widehat{\cal T}}_{X\to Y}(t)$ and ${\cal T}_{X\to Y}(t)$, except when the process is Markov bipartite, where

[TABLE]

Indeed, from the definitions (II.2) and (III.2), we have the general equation

[TABLE]

and it can be proven that the right-hand side of this equation is equal to [math] when the process is bipartite. The demonstration for jump processes is in Appendix C of HBS2014 and for diffusion processes it is in given in Appendix B of the present paper.

There is of course a single-time-step TE rate corresponding to ${\widehat{T}}_{X\to Y}(t)$, which is defined as

[TABLE]

Then,

[TABLE]

and

[TABLE]

in the bipartite case, as will be illustrated below for diffusion processes [see Eq. (80)].

As for the learning rates, the single-time-step TE rates can be expressed in terms of the two-point probability distribution $P({\bf z},t;{\bf z}^{\prime},t^{\prime})\equiv\langle\delta({\bf Z}_{t}-{\bf z})\delta({\bf Z}_{t^{\prime}}-{\bf z}^{\prime})\rangle$, with ${\bf z}\equiv(x,y)$, and the corresponding marginal distributions:

[TABLE]

In contrast with the learning rates [Eqs. (32)], one cannot generally introduce derivatives with respect to $h$ in these expressions. On the other hand, we shall derive explicit expressions for Markov diffusion processes: see Sec. IV.2 below.

III.3 Backward transfer entropy rates

Finally, we add to our list of information-theoretic measures another TE rate which can be used to assess the directionality of information transfer (see Sec. VI.2) and which will play an important role in the generalization of the second law (see Sec. IV.4). To this aim, we slightly change our notations by assuming that the trajectories of $X$ and $Y$ are now observed in the time interval $[0,T]$. We then define

[TABLE]

where $X_{t+h}^{T}$ and $Y_{t+h}^{T}$ denote the trajectories of $X$ and $Y$ in the time interval $[t+h,T]$. ${\cal T}^{\dagger}_{X\to Y}(t)$ has clearly the meaning and the properties of a TE rate, but it involves the future trajectories of $X$ and $Y$ instead of their past. It may thus be called a backward TE (BTE) rate and regarded as the continuous-time version of the BTE introduced in I2016 in the discrete-time framework (see also HNMN2013 ; V2015 ; W2016 for the introduction of time-reversed Granger causality). It is actually much simpler to consider continuous time from the outset as this makes the generalization of the relations derived in I2016 to the non-bipartite case a straightforward operation. We draw attention to the fact that the BTE defined in this way has no relation with the time-reversed transfer entropy considered in CS2016 ; SLP2016 ; SLP2018 .

When ${\bf Z}$ is a Markov process, the definition (III.3) can be also rewritten as

[TABLE]

As before, we also define a single-time-step BTE rate as

[TABLE]

which will play a useful role in Sec. IV.4 note5 . Note however that this does not add a new independent measure of information to our list since it can be easily seen from the definitions (26 ) of $l_{Y}^{-}(t)$ and (42) of $\widehat{\overline{\cal T}}_{X\to Y}(t)$ that

[TABLE]

Combining Eq. (49) with Eq. (43) also yields

[TABLE]

which, in the bipartite case, implies that

[TABLE]

This is in agreement with Eq. (18) in I2016 ; note30 .

Furthermore, when the joint process $\bf Z_{t}$ is Markovian, simple manipulations yield

[TABLE]

where we have used $Y_{0}^{t+h}\sim(Y_{0}^{t},Y_{t+h})$ and $Y_{t}^{T}\sim(Y_{t},Y_{t+h}^{T})$ for $h$ infinitesimal to obtain the second equality. By integrating from [math] to $T$, the left-hand side of this equation vanishes since $[S(Y_{0}^{t})+S(Y_{t}^{T})-S(Y_{t})]_{0}^{T}=S(Y_{0}^{T})-S(Y_{0})+S(Y_{T})-S(Y_{0}^{T})-S(Y_{T})+S(Y_{0})=0$.We finally obtain a simple relation (not evident from the outset) involving the forward and backward time-integrated TEs,

[TABLE]

Although this may be regarded as the continuous-time analog of Eq. (17) in I2016 , we stress that the derivation of this relation does not require the joint process to be bipartite. Eq. (53) also leads to the interesting steady-state relation note15

[TABLE]

III.4 Inequalities

We are now in position to generalize the standard inequalities (17) and (18) obtained in the bipartite case.

1. First, one easily obtains from the definitions that the single-time-step TE rate is an upper bound on the forward learning rate,

[TABLE]

and that the single-time-step filtered TE rate is an upper bound on the backward learning rate,

[TABLE]

Indeed, one has ${\overline{T}}_{X\to Y}(t)-l^{+}_{Y}(t)=\lim_{h\rightarrow 0^{+}}h^{-1}[S(X_{t}|Y_{t+h})-S(X_{t}|Y_{t},Y_{t+h})]$ and ${\widehat{\overline{\cal T}}}_{X\to Y}(t)-l^{-}_{Y}(t)=\lim_{h\rightarrow 0^{+}}h^{-1}[S(X_{t+h}|Y_{t+h})-S(X_{t+h}|Y_{t},Y_{t+h})]$, and Shannon entropy never increases by conditioning. These inequalities hold for a general (non-stationary and possibly non-Markovian) process.

2. If the joint process is Markovian, the single-time-step TE rate is an upper bound on the multi-time-step TE rate,

[TABLE]

since ${\overline{\cal T}}_{X\to Y}(t)-{\cal T}_{X\to Y}(t)=\lim_{h\rightarrow 0^{+}}h^{-1}[S(Y_{t+h}|Y_{t})-S(Y_{t+h}|Y_{0}^{t})]\geq 0$. On the other hand, note that ${\widehat{\overline{\cal T}}}_{X\to Y}(t)$ is not an upper bound on ${\widehat{\cal T}}_{X\to Y}(t)$.

3. In a steady state, the second inequality (18) is replaced by

[TABLE]

which is a special case of the more general inequality

[TABLE]

since $l^{+}_{X}=-l^{-}_{Y}$ and $I(X_{t}:Y_{0}^{t})$ is not time extensive, i.e. $\lim_{t\to\infty}t^{-1}I(X_{t}:Y_{0}^{t})=0$ [in contrast with $I(X_{0}^{t}:Y_{0}^{t})$]. Inequality (18) is then recovered for a bipartite process since $\widehat{\cal T}_{X\to Y}={\cal T}_{X\to Y}$ in this case, as we have seen before [Eq. (40)]. We prove the inequality in Eq. (59) in Appendix B.

To summarize all these inequalities and be more concrete, let us give a numerical illustration. Anticipating the calculations performed in Sec. V for a bi-dimensional Ornstein-Uhlenbeck process, we show in Fig. 1 the behavior of the various information measures as a function of the parameter $-1\leq\rho\leq 1$ that quantifies the correlations between the noises affecting the two Langevin subprocesses. The numerical values of the other parameters of the model are chosen in such a way that $X$ may be considered as a source signal measured by $Y$ (see the discussion in Sec. V).

The small arrows in the figure indicate values of $\rho$ for which the system has a non-generic but remarkable behavior.

i) The first one on the left side ($\rho=-0.6$) indicates that the symmetric learning rate $l^{S}_{Y}$ (and thus also $l_{X}^{S}=-l_{Y}^{S}$) vanishes. As seen from Eq. (36), this occurs when the joint system is at equilibrium (in the sense of satisfied detailed balance and zero probability currents).

ii) The next arrow ($\rho=-0.35$) indicates that $l_{Y}^{+}=l_{X}^{+}=l^{S}_{Y}=0$, which occurs when the two subprocesses $X$ and $Y$ become independent: see Eqs. (33) and (35).

iii) The third arrow ($\rho\approx-0.185$) indicates that ${\cal T}_{X\to Y}={\overline{\cal T}}_{X\to Y}$ and $l^{-}_{Y}=\widehat{\cal T}_{X\to Y}={\widehat{\overline{\cal T}}}_{X\to Y}$. Extending the analysis performed in MS2018 and taking the continuous-time limit from the outset, we show in Appendix C that this occurs when $Y_{t}$ is a sufficient statistic of $X_{t}$, as expressed by Eq. (19). Thanks to Eqs. (49) and (54), this also implies that ${\cal T}^{\dagger}_{X\to Y}={\overline{\cal T}}^{\dagger}_{X\to Y}=0$.

iv) Finally, the last arrow on the right side ($\rho=0.4$) indicates that inequality (55) is saturated and $l^{+}_{Y}(t)={\overline{\cal T}}_{X\to Y}(t)$. This generally does not coincide with the saturation of inequalities (57) and (58). Indeed, since ${\overline{T}}_{X\to Y}(t)-l^{+}_{Y}(t)=\partial_{h}I(X_{t}:Y_{t}|Y_{t+h})|_{h=0^{+}}$, equality is obtained when $\lim_{h\rightarrow 0^{+}}h^{-1}\>[P(X_{t}|Y_{t},Y_{t+h})-P(X_{t}|Y_{t+h})]=0$, which differs from condition (19). The case of a bipartite process considered in MS2018 is an exception, as inequalities (55) and (58) then coincide (${\overline{\cal T}}_{X\to Y}={\widehat{\overline{\cal T}}}_{X\to Y}$ and $l^{-}_{Y}=l_{Y}^{+}$).

Although inequality (58), which appears as the generalization of inequality (18), has no intuitive interpretation [in contrast with (18)], the fact that it becomes an equality if $Y_{t}$ is a sufficient statistic of $X_{t}$ may suggest to generalize the concept of a sensory capacity as

[TABLE]

Likewise, we may define a “single-time-step” capacity,

[TABLE]

which is also bounded by $1$ thanks to inequality (56), and equal to $1$ if $Y_{t}$ is a sufficient statistic of $X_{t}$ (see Appendix C). Note that the joint dynamics needs not be Markovian. Moreover, since ${\overline{C}}_{Y}$ involves the single-time-step TE’s ${\overline{\cal T}}_{X\to Y}$ or ${\overline{\cal T}}^{\dagger}_{X\to Y}$ instead of ${\widehat{\cal T}}_{X\to Y}$, it is much simpler to obtain this quantity from experimental time series than $C_{Y}$. Of course, it remains to be seen on specific examples of sensory systems if these quantities are helpful to estimate the performance of the sensor in the presence of correlations between the observation and signal noises, a situation classically treated in the framework of filtering theory K1962 ; LS2001 .

IV Markov diffusion processes and second law of information thermodynamics

To make all the above definitions and relations more explicit and to derive a second law, we now focus on Markov diffusion processes as defined in Eq. (3). To reduce the amount of notation, we consider the case where $X_{t}$ and $Y_{t}$ are unidimensional processes. The vector fields $F_{X,t}$, $F_{Y,t}$ and the matrix fields $D_{XX,t}$, $D_{XY,t}$, and $D_{YY,t}$ are now all scalar fields. The general case can be easily extrapolated from this one.

IV.1 Learning rates

As we have already pointed out, the expressions (32) of the learning rates can be simplified when the joint process ${\bf Z}_{t}$ is Markovian. From the forward Kolmogorov equation (1) and the Markovian generator, we readily obtain

[TABLE]

for $t^{\prime}\leq t$. Integrating over $y$ and using Eq. (6) then yields

[TABLE]

as all terms involving derivatives with respect to $y$ vanish at the boundaries (assuming natural boundary conditions). As a result,

[TABLE]

and, after integration by parts, we transform Eq. (32a) into

[TABLE]

where $P_{t}(y|x)\equiv P_{t}({\bf z})/P_{t}(x)$ is the conditional probability distribution function. Since we only consider Markov processes in this section, we no longer add the bold letter ${\bf M}$ on the left of the equations.

The rate $l_{X}^{-}(t)$ is obtained by using the relation $d_{t}I_{t}(X_{t},Y_{t})=l_{Y}^{+}(t)+l_{X}^{-}(t)$ [Eq. (27)], the expression of the time derivative of the mutual information [Eq. (34)], and the FP equation. It reads

[TABLE]

We immediately see that $l^{+}_{X}(t)=l^{-}_{X}(t)$ and $l^{+}_{Y}(t)=l^{-}_{Y}(t)$ when the process is bipartite (i.e., $D_{XY,t}({\bf z})=0$) in agreement with Eq. (37).

One may also rewrite these expressions in terms of the probability currents defined by Eq. (7). This yields

[TABLE]

and the symmetric learning rate $l^{S}_{X}(t)=(1/2)[l^{+}_{X}(t)+l^{-}_{X}(t)]$ is then simply given by

[TABLE]

As announced before, we observe that the two symmetric rates $l^{S}_{X}(t)$ and $l^{S}_{Y}(t)$ vanish in a steady state when the two probability currents are zero, which corresponds to equilibrium. On the other hand, the non-symmetric rates remain finite. All the learning rates vanish when the two subprocesses $X$ and $Y$ are independent, i.e., when $P_{t}({\bf z})=P_{t}(x)P_{t}(y)$.

The above equations generalize the expressions in the current literature obtained for bipartite processes and additive noises AJM2009 ; HS2014 ; HBS2016 (note that the sign convention may differ). These expressions are immediately recovered by setting $D_{XY,t}=0$ and taking $D_{XX,t}$ and $D_{YY,t}$ independent of $x$ and $y$. Finally, we note that the learning rates obtained above are finite, at least if the integrals in the right-hand sides of (67) and (68) are finite, for all diffusion processes described by Eq. (3). This will not necessarily be the case of the single-time-step transfer entropy rates that we consider in the following.

IV.2 Single-time-step transfer entropy rates

We now derive the expressions of the various single-time-step TE rates. It turns out that it suffices to compute the rate ${\overline{\cal T}}_{X\to Y}(t)$ given by Eq. (45a) (a similar calculation was presented in LNTRL2017 , but it is worth repeating it for completeness). The backward rate ${\overline{\cal T}}^{\dagger}_{X\to Y}(t)$ is then deducible from ${\overline{\cal T}}_{X\to Y}(t)$, and ${\widehat{\overline{\cal T}}}_{X\to Y}(t)$ is finally obtained from Eq. (49).

We start from the expression of the infinitesimal transition probability (or propagator),

[TABLE]

with $L^{FP}_{t}$ the Focker-Planck operator appearing in Eq. (62) and defined by Eq. (6).

Integrating the propagator over $x$ and then integrating $P(y,t+h;{\bf z}^{\prime},t)$ over $x^{\prime}$, we obtain

[TABLE]

where

[TABLE]

and

[TABLE]

To compute the logarithms of the transition probabilities, we need to replace Eqs (70) by their Gaussian small-time expressions, R1989

[TABLE]

when $h\to 0^{+}$, using the convention that the argument of the functions $D_{YY,t}$ and $F_{Y,t}$ is ${\bf z}^{\prime}$. Inserting Eq. (73) into Eq. (45a), we readily see that ${\overline{\mathcal{T}}}_{X\to Y}(t)$ diverges if ${\overline{D}_{YY,t}}(y)\neq D_{YY,t}({\bf z})$, that is if $D_{YY,t}$ is also a function of $x$. We thus recover the conditions specified in D1971 for the mutual information $I(X_{0}^{t},Y_{0}^{t})$ to be finite in the case of multiplicative independent noises. To proceed, we thus assume that $D_{YY,t}$ only depends on $y$. Then,

[TABLE]

and using Eq. (45a), we finally obtain after some manipulations

[TABLE]

This generalizes the expression given in HBS2016 for a bipartite system with additive noises (an explicit calculation is also performed in IS2015 for a bi-dimensional Ornstein-Uhlenbeck model). Note that, contrary to the learning rates, the single-time-step TE rate is infinite when $D_{YY,t}=0$ (or in the multi-dimensional case when the matrix $D_{YY,t}$ is not invertible), which implies for instance that it is not well-suited for underdamped processes. The same is true for the other TE rates considered below.

To obtain the expression of the BTE rate ${\overline{\cal T}}^{\dagger}_{X\to Y}(t)$ defined by Eq. (48), a possible method is to use Bayes theorem to modify the argument of the logarithm and recast Eq. (48) as

[TABLE]

The calculation would then follow the same lines as above. However, it is more instructive to use the fact that ${\overline{\cal T}}^{\dagger}_{X\to Y}(t)$ is the TE rate at time $T-t$ corresponding to the process ${\bf Z}_{T-t}$, which is the time reversal of the process ${\bf Z}_{t}$ (as the state at time $t$ along a forward trajectory is now conditioned on the state at time $t+h$). As is well known, ${\bf Z}_{T-t}$ is also a diffusion process, under some mild conditions (see, e.g., HP1986 ; C2005 ). The covariance (diffusion) matrix and drift coefficients of the time-reversed process are given respectively by $D_{t}^{*}({\bf z})=D_{T-t}({\bf z})$ and

[TABLE]

Denoting the single-time-step TE associated to $D_{t}^{*}$ and $F^{*}_{X,t},F^{*}_{Y,t}$ by ${\overline{\cal T}}^{*}_{X\to Y}$, we have by definition of the single-time-step backward TE rate that

[TABLE]

Of course, the same is true for the multi-time-step TE rate ${\cal T}^{\dagger}_{Y\to X}(t)$ which identifies with ${\cal T}^{*}_{Y\to X}(T-t)$.

After some algebra described in Appendix B, we obtain

[TABLE]

and ${\overline{\cal T}}^{\dagger}_{X\to Y}(t)=\infty$ otherwise. Using Eq. (67), we see that relation in Eq. (51) is recovered in the bipartite case ($D_{XY,t}=0$), as expected. Moreover, if the joint system ${\bf Z}$ is at equilibrium (i.e., if the probability currents vanish), we also have ${\overline{\cal T}}^{\dagger}_{X\to Y}(t)={\overline{\cal T}}_{X\to Y}(t)$, which is not obvious from Eq. (50). From Eq. (54), this also implies that ${\cal T}^{\dagger}_{X\to Y}={\cal T}_{X\to Y}$. In other words, the TE rate is time-symmetric at equilibrium, as it should be.

Finally, after using Eq. (49) and the expression of $l^{-}_{Y}$, i.e., Eq. (66) with $X$ and $Y$ interchanged, we find

[TABLE]

and ${\widehat{\overline{\cal T}}}_{X\to Y}(t)=\infty$ otherwise. This also immediately shows that ${\widehat{\overline{\cal T}}}_{X\to Y}(t)={\overline{\cal T}}_{X\to Y}(t)$ in the bipartite case [Eq. (44)].

We reiterate that for diffusion processes with multiplicative noises, one must have that $D_{YY,t}({\bf z})=D_{YY,t}(y)$, and similarly $D_{XX,t}({\bf z})=D_{XX,t}(x)$, otherwise the various TE rates are infinite, even in the bipartite case: see also Appendix B.

IV.3 Multi-time-step transfer entropy rates

It turns out that an expression somewhat similar to Eq. (75) can also be obtained for the multi-time-step TE rate $\mathcal{T}_{X\to Y}(t)$. To do this one has to generalize the preceding derivation for ${\overline{\cal T}}_{X\to Y}(t)$. A new ingredient is the presence of an infinitesimal propagator $P(y,t+h|y^{\prime}\hskip 0.28453pt_{0}{}^{t})$ that is conditioned by a whole path $y^{\prime}\hskip 0.28453pt_{0}{}^{t}$ from [math] to $t$ instead of a single value at time $t$. By using Bayes theorem and the Markovian property, we rewrite it as

[TABLE]

where by convention $y^{\prime}\hskip 0.28453pt_{0}{}^{t}$ includes the endpoint $y^{\prime}$ at time $t$. From Eq. (70) we then obtain

[TABLE]

with

[TABLE]

The above infinitesimal propagator can also be cast in the form of a Gaussian small-time expression as in Eq. (73).

The derivation of the expression of $\mathcal{T}_{X\to Y}(t)$ from its definition (II.2) then directly follows that of the single-time-step TE rate $\overline{\mathcal{T}}_{X\to Y}(t)$ with the final result

[TABLE]

and $\mathcal{T}_{X\rightarrow Y}(t)=\infty$ otherwise. As found for the single-time-step TE rate, $\mathcal{T}_{X\rightarrow Y}(t)$ is only finite if the diffusion coefficient $D_{YY,t}({\bf z})$ only depends on the variable $y$. As it should be, Eq. (75) is recovered and ${\cal T}_{X\to Y}(t)={\overline{\cal T}}_{X\to Y}(t)$ when $Y_{t}$ is a sufficient statistic of $X_{t}$ [Eq. (19)], as $\widetilde{F}_{Y,t}(y_{0}^{t})={\overline{F}}_{Y,t}(y)$ and $\widetilde{D}_{YY,t}(y_{0}^{t})=\overline{D}_{YY,t}(y)$ in this case. Note also that the above formula has been derived for simplicity for unidimensional processes $X_{t}$ and $Y_{t}$ but mutatis mutandis it is easily extended to multidimensional processes. For instance, the result in Eq. (84) can be generally rewritten as

[TABLE]

which expresses the transfer entropy rate in terms of the minimum mean-square error (MMSE) of the causal estimation. This generalizes the relation obtained in WKP2013 for a bipartite diffusion process with additive noise and extends the classical and beautiful result of Duncan D1970 ; D1971 linking information theory and estimation theory: see MWZ1985 ; AVW2013 for more on this theme.

IV.4 Second law for non-bipartite processes

So far we have discussed learning rates and transfer entropy rates from the strict viewpoint of information exchange between two interacting systems. We now wish to use these concepts to discuss non-equilibrium thermodynamics. In particular, we want to investigate how the second-law inequality involving the learning rate, which provides the tightest lower bound for bipartite processes (see Sec. II.4) is modified when the bipartite assumption is dropped. We stress that we are interested in the average entropy production (EP) during a finite time interval $[0,t]$ and not only in the stationary state or in the limit $t\to\infty$.

Let us again focus on subsystem $X$. At the ensemble level, an entropy balance equation can be obtained as usual by decomposing the time-derivative of the marginal Shannon entropy $S_{X}(t)$,

[TABLE]

which can be rewritten as

[TABLE]

after using Eq. (68) to introduce the symmetric learning rate $l^{S}_{X}(t)$. Inserting the Fokker-Planck equation and performing a few manipulations, we then obtain

[TABLE]

where we have defined the non-negative quantity

[TABLE]

(traditionally referred to as the “irreversible” EP), and the modified drift $\widehat{F}_{X,t}({\bf z})$ is defined by CG2008

[TABLE]

Eq. (88) can be further transformed as

[TABLE]

where $\sigma_{X}(t)$ is defined as in the bipartite case by Eq. (20), i.e., $\sigma_{X}(t)\equiv d_{t}S_{X}(t)+\sigma_{X}^{B}(t)$, with $\sigma_{X}^{B}(t)$ formally defined by note8

[TABLE]

Finally, exchanging $X$ and $Y$ in Eq. (79) and using again Eq. (68), we obtain [provided that $D_{XX,t}({\bf z})=D_{XX,t}({x})$]

[TABLE]

So far, we have been only performing mathematical substitutions. However, as already mentioned in Sec. II.4, connection to physics is possible if the thermodynamics of subsystem $X$ can be defined and $\sigma^{B}_{X}(t)$ identified as the heat flow from $X$ to the environment, e.g., a thermal bath at a given temperature. In this regard, the existence of correlations between the noises is not an obstacle (see also the related discussion in SLP2018 ). For an external observer monitoring only $X$, the quantity $\sigma_{X}(t)$ defined by Eq. (20) would then be interpreted as the EP of the thermodynamic system $X$, ignoring that $X$ also influences the dynamics of $Y$. Note that Eq. (91) implies that $\sigma_{X}=0$ at equilibrium, since $\sigma_{X}^{irr}=0$ from Eq. (89) (as $J_{X}=0$) and $l_{X}^{S}=0$, as pointed out in the preceding section.

Since the two subsystems are coupled, $\sigma_{X}(t)$ may be negative, but thanks to Eqs. (91) and (93) there is a lower bound,

[TABLE]

which is conveniently rewritten as

[TABLE]

This inequality takes a remarkably simple form in the case where $D_{XY,t}({\bf z})=D_{XY,t}(x)$, which includes the important case of additive noises, as it reduces to

[TABLE]

or, after integration from [math] to $T$,

[TABLE]

where we have used Eq. (53) to write the second equality. In practice, the first equality may be more useful since the single-time-step TE rates can be extracted more easily from time series than the multi-time-step TE’s.

Since ${\overline{\cal T}}_{Y\to X}(t)-{\overline{\cal T}}^{\dagger}_{Y\to X}(t)=l_{X}(t)$ in the bipartite case [Eq. (51)], inequality (96) [or its time-integrated version (97)] may be considered as the natural generalization of the second law of information thermodynamics involving the learning rate. As far as we know, it is a new result, which represents a significant outcome of the present work note9 . It can be easily extended to non-bipartite Markov diffusion processes in which the two subprocesses are multidimensional.

Anticipating again the calculations performed in Sec. V for a bi-dimensional Ornstein-Uhlenbeck process, we illustrate in Fig. 2 the generalized second law and show the variations of $\sigma_{X}$ and of the difference ${\cal T}_{T\to X}-{\cal T}^{\dagger}_{Y\to X}={\overline{\cal T}}_{Y\to X}-{\overline{\cal T}}^{\dagger}_{Y\to X}$ with the parameter $\rho$ quantifying the correlations between the noises affecting $X$ and $Y$. The figure also displays the lower bound $-[{\cal T}_{X\to Y}+\Delta\dot{I}^{int}_{X\to Y}]$ recently obtained in Ref. SLP2018 by interpreting the various contributions to the entropy production in terms of computational irreversibilities. The additional term $\Delta\dot{I}^{int}_{X\to Y}$ vanishes for a bipartite dynamics and the bound is then known to be weaker than the one with the learning rate [see Eq. (24)]. For the specific case shown in Fig. 2, we see that this remains true for $\rho\neq 0$ (but this is not a general feature). Note also that ${\cal T}_{X\to Y}+\Delta\dot{I}^{int}_{X\to Y}\neq 0$ when the joint system is at equilibrium, in contrast with $\sigma_{X}$. Incidentally, we see that one may have ${\cal T}_{Y\to X}={\cal T}^{\dagger}_{Y\to X}$ even in a non-equilibrium steady state, which occurs when the second term of Eq. (79) is zero.

V Stationary bi-dimensional Ornstein-Uhlenbeck process

To make one further step to derive explicit analytical expressions and make the discussion of the consequences of dropping the bipartite property even more concrete, we restrict ourselves to a Gaussian Markov process, more specifically a bi-dimensional stationary Ornstein-Uhlenbeck (OU) process with additive noises. This allows us to obtain simple analytical expressions for all the information measures, including the multi-time-step TE rates. In the following, we only explain how these quantities can be computed, the details being given in Appendix D. In this Appendix, we also list the expressions of the learning rates and the single-time-step TE rates, which are easily obtained from the general formulas derived previously. To alleviate the notations we now denote the two (one-dimensional) subprocesses by $X_{1}$ and $X_{2}$ in place of $X$ and $Y$ and the combined process by $\bf X$ instead of $\bf Z$: $l_{X}$ will for instance be replaced by $l_{1}$ and $\mathcal{T}_{X\to Y}$ by $\mathcal{T}_{1\to 2}$.

The stochastic dynamics is governed by the coupled Langevin equations

[TABLE]

where ${\bf A}=[a_{ij}]$ is a $2$ by $2$ matrix and $\boldsymbol{\xi}=\{\xi_{i}\}$ is a vector formed by two Gaussian noises with zero mean and covariances $\langle\xi_{i}(t)\xi_{j}(t^{\prime})\rangle=2D_{ij}\delta(t-t^{\prime})$. We assume that all eigenvalues of ${\bf A}$ have a positive real part so that a stable steady-state solution exists R1989 ; G2004 . Since we only focus on this regime hereafter, we may assume that the process has started at $t=-\infty$ and forget about the initial condition [accordingly, the past history of $X_{i}(t)$ up to time $t$ is now denoted by $X_{i}^{-}(t)\equiv\{X_{i}(s):s\leq t\}$ instead of $(X_{i})_{0}^{t}$]. The solution of Eq. (98) then reads

[TABLE]

where ${\bf H}(t)=[e^{-{\bf A}t}]_{ij}$ is the response (or Green’s or transfer) functions matrix, and the power-spectrum matrix whose elements are the Fourier transform of the stationary correlation functions $\phi_{ij}(t)=\langle X_{i}(t^{\prime})X_{j}(t^{\prime}+t)\rangle$ is given by

[TABLE]

where ${\bf H}(\omega)=\int_{-\infty}^{\infty}dt\>e^{i\omega t}{\bf H}(t)=[{\bf A}(\omega)-i\omega{\bf I}]^{-1}$ and $2{\bf D}$ is the diffusion matrix with elements $2D_{ij}$.

V.1 Expression of the multi-time-step TE rates

An expression of ${\cal T}_{i\to j}$ for a non-bipartite stationary OU process has already been given in SLP2018 , but it turns out that it contains an error, as explained below. Moreover, the derivation is convoluted. We thus believe that it is worth presenting an alternative and much simpler route which has also been recently used for computing the TE rate in the presence of time delay RTM2018 . This is actually a mere application of the formalism presented in BS2017 for computing Granger causality for discrete and continuous-time autoregressive processes. As has already been mentioned, Granger causality and transfer entropy are identical (up to a factor $1/2$) when the random variables are Gaussian distributed BBS2009 , which is the case here.

By definition, ${\cal T}_{i\to j}$ is the slope at $h=0$ of the finite-horizon TE defined by note0

[TABLE]

From the expression of the entropy of Gaussian distributions in terms of their covariance matrix, we then readily obtain

[TABLE]

where

[TABLE]

and

[TABLE]

are the mean of the variances of the conditional probabilities $P(X_{j}(t+h)|{\bf X}^{-}(t))$ and $P(X_{j}(t+h)|X_{j}^{-}(t))$, respectively. In the language of estimation theory, the conditional expectations $\langle X_{j}(t+h)|{\bf X}^{-}(t)\rangle$ and $\langle X_{j}(t+h)|X_{j}^{-}(t)\rangle$ represent the minimum mean-square error (MMSE) estimates of $X_{j}(t+h)$ when the trajectory up to time $t$ of either the full process ${\bf X}$ or of only $X_{j}$ is known: They are the orthogonal projections onto ${\bf X}^{-}(t)$ and $X_{j}^{-}(t)$, respectively. As detailed in Appendix D, the calculation of ${\cal T}_{i\to j}$ amounts to computing first the conditional expectations, then the mean-square errors $\sigma_{jj}(h)$ and $\sigma_{jj,j}(h)$, and finally expanding around $h=0$. In particular, this requires to determine the causal factor of the function $S_{jj}(\omega)$, which is a simple task since it is a rational function A2006 . The final expression of ${\cal T}_{1\to 2}$ is

[TABLE]

where $r_{2}=[a_{11}^{2}+(D_{11}/D_{22})a_{21}^{2}-2(D_{12}/D_{22})a_{11}a_{21}]^{1/2}$; ${\cal T}_{2\to 1}$ is of course obtained by interchanging the roles of $1$ and $2$. As it should be, one can verify that the same result is obtained from Eq. (84), which is actually no more straightforward because the calculation of the effective drift $\widetilde{F}_{2}$ requires to compute the MMSE estimate $\langle X_{1}(t)|X_{2}^{-}(t)\rangle$.

As noticed above, Eq. (105) differs from the expression of ${\cal T}_{1\to 2}$ obtained in SLP2018 after a lengthy calculation (see also SL2018 ). In this expression, $a_{11}$ is replaced by $|a_{11}|$ (cf. Eq. (88) in SLP2018 ), which is erroneous and may even lead to negative values of ${\cal T}_{1\to 2}$ for $a_{11}<0$. (Having $a_{11}<0$ does not preclude the existence of a stable steady state so long as $a_{11}+a_{22}>0$ and $a_{11}a_{22}-a_{12}a_{21}>0$.) More generally, we stress that one must be careful in using a spectral representation of the TE rate, as done for instance in HS2014 in the bipartite case. Indeed, as discussed in RTM2018 , the spectral expression has a limited range of validity (specifically, one must have $a_{11}-(D_{12}/D_{22})a_{21}>0$). Otherwise, it underestimates the actual value of the TE rate, as was already pointed out in G1982 in the case of discrete-time Granger causality.

From Eq. (105), one obtains the expression of the BTE rate ${\cal T}_{1\to 2}^{\dagger}$ by modifying the drifts coefficients according to Eq. (IV.2). For linear Gaussian processes, this simply amounts to changing the matrix ${\bf A}$ into ${\bf A}^{*}=-{\bf A}+2{\bf D}{\boldsymbol{\Sigma}}^{-1}={\boldsymbol{\Sigma}}{\bf A}^{T}{\boldsymbol{\Sigma}}^{-1}$ LK1976 ; A1982 , where ${\boldsymbol{\Sigma}}$ is the stationary covariance matrix, solution of the Lyapunov equation ${\bf A}.{\boldsymbol{\Sigma}}+{\boldsymbol{\Sigma}}.{\bf A}^{T}=2{\bf D}$ R1989 ; G2004 . The quantity $r_{2}$ is invariant under the transformation ${\bf A}\to{\bf A}^{*}$, and we obtain

[TABLE]

with $a^{*}_{11}$ and $a^{*}_{21}$ given by Eqs. (D). It can be checked that this is in agreement with the expression obtained from Eq. (54), with ${\overline{\cal T}}_{1\to 2}$ and ${\overline{\cal T}}^{\dagger}_{1\to 2}$ given by Eqs. (D140) and (D142), respectively.

On the other hand, the calculation of the filtered TE rate ${\widehat{\cal T}}_{1\to 2}$ is more involved. For Gaussian processes, Eq. (III.2) yields

[TABLE]

where \sigma_{11,2}(h)\equiv\langle\big{[}X_{1}(t+h)-\langle X_{1}(t+h)|X_{2}^{-}(t)\rangle\big{]}^{2}\rangle, $\sigma_{22,2}(h)$ is defined above by Eq. (104), and \sigma_{12,2}(h)\equiv\langle\big{[}X_{1}(t+h)-\langle X_{1}(t+h)|X_{2}^{-}(t)\rangle\big{]}\big{[}X_{2}(t+h)-\langle X_{2}(t+h)|X_{2}^{-}(t)\rangle\big{]}\rangle. The calculation, which uses the fact that the conditional expectation $\langle X_{1}(t+h)|X_{2}^{-}(t)\rangle$ is the orthogonal projection of $X_{1}(t+h)$ onto the trajectory $X_{2}^{-}(t)$, is detailed in Appendix D. After some algebra this leads to the expression

[TABLE]

Finally, as also shown in Appendix D, one may recast the non-Markovian Langevin equation for the marginal process $X_{2}$ as

[TABLE]

where $\omega_{\pm}$ are the eigenvalues of the matrix ${\bf A}$ whose expressions are given after Eq. (D150). This formulation is instructive because it shows that a remarkable simplification occurs when $r_{2}=\omega_{\pm}$, that is when

[TABLE]

The second term in the right-hand side of Eq. (109) then vanishes and the equation describes a Markovian dynamics. Accordingly, one has $P(X_{2}(t+h)|X_{2}^{-}(t))=P(X_{2}(t+h)|X_{2}(t))$ and ${\cal T}_{1\to 2}$ becomes equal to its upper bound ${\overline{\cal T}}_{1\to 2}$, as discussed in Appendix C. We show in Appendix D that from the perspective of the Kalman-Bucy filter KB1961 ; KSH2000 ; A2006 , where $X_{1}(t)$ is a state variable that is not fully observed and the equation for $X_{2}(t)$ describes the dynamics of the filter state, this means that the observer gain is optimal. Eq. (110) generalizes the conditions for sufficient statistic discussed in Ref. MS2018 to the non-bipartite case. This may be viewed either as an optimal condition for the set of parameters $a_{ij}$’s for given noise intensities $D_{ij}$’s or as an optimal condition for the noises for a given set of the $a_{ij}$’s. Note that there are in general two distinct solutions when the eigenvalues $\omega_{\pm}$ of the matrix ${\bf A}$ are real. Otherwise, there is no real value of $r_{2}$ that satisfies Eq. (110) and the statistic is never sufficient.

V.2 Numerical illustration

We now give a numerical illustration which will allow us to discuss the behavior of the various information measures in the presence of correlated noises. To this aim, we consider a situation with a quasi-unidirectional coupling between the two random variables $X_{1}$ and $X_{2}$. Indeed, the presence of both bidirectional interactions and correlated noises would make the interpretation of the information exchanges almost impossible. Actually, as far as we know, this case is seldom considered in the literature on Granger causality. Specifically, we choose the parameters of the model (see the caption of Fig. 1) such that the coupling in the direction $1\to 2$ is significantly larger than that in the opposite direction, so that one may consider $X_{1}$ as the source signal (the driver) and $X_{2}$ as the receiver.

On general ground, one is more interested in the net information flow in the system than in the magnitude of the flows in each direction. Inspired by the recent literature on Granger causality W2016 , and replacing Granger causality by transfer entropy, we then consider various differences of the TE rates, such as ${\cal T}^{(net)}_{1\to 2}=-{\cal T}^{(net)}_{2\to 1}\equiv{\cal T}_{1\to 2}-{\cal T}_{2\to 1}$, ${\cal T}^{{\dagger}(net)}_{1\to 2}=-{\cal T}^{{\dagger}(net)}_{2\to 1}\equiv{\cal T}_{1\to 2}^{\dagger}-{\cal T}^{\dagger}_{2\to 1}$, and the individual differences ${\cal T}_{1\to 2}-{\cal T}_{2\to 1}^{\dagger}$, ${\cal T}_{2\to 1}-{\cal T}_{2\to 1}^{\dagger}$. We also compare with the behavior of the symmetric learning rate $l_{2}^{S}=-l_{1}^{S}$. The results are shown in Fig. 3.

In the bipartite case ($\rho=0$), all quantities behave as expected and indicate that the information mainly flows from $1$ to $2$, due to the fact that the interaction $1\to 2$ predominates ($a_{21}/a_{12}=10$). More precisely, we observe in Fig. 3a that ${\cal T}^{(net)}_{1\to 2}>0$ and ${\cal T}^{{\dagger}(net)}_{1\to 2}<0$. The rationale given in the literature (see, e.g., HNMN2013 ; V2015 ; W2016 ) for the second inequality is that the directed information should be reduced (if not reversed) when the temporal order is reversed. We also verify in Fig. 3b that ${\cal T}_{1\to 2}-{\cal T}_{1\to 2}^{\dagger}=-({\cal T}_{2\to 1}-{\cal T}_{2\to 1}^{\dagger})=l_{2}^{S}>0$. As expected, $X_{2}$ is “learning about” $X_{1}$ through its dynamics.

One observes that introducing correlations between the noises has a strong effect on most quantities, even if the properties found for $\rho=0$ remain valid in some interval of $\rho$ around [math] (for instance, ${\cal T}^{(net)}_{1\to 2}$ remains larger than ${\cal T}^{{\dagger}(net)}_{1\to 2}$ in the whole interval $-0.6\lesssim\rho\lesssim 0.3$). What appears robust whatever the correlation of the noises is that the net information flow measured by ${\cal T}^{(net)}_{1\to 2}$ always detects the correct dominant interaction. All of the other differences, as well as the learning rates, wildly vary and change sign as $\rho$ varies. There is clearly a complex competition between the feedback (delayed) effects and the instantaneous influence generated by the correlation of the noises. The downside is that this competition sensitively depends on the quantity under study and on other details of the dynamics (for instance on the relative magnitudes of the intrinsic time scales of each subprocesses, i.e., $a_{11}^{-1}$ and $a_{22}^{-1}$ in the present model). The upside is that one can infer the presence of a strong correlation between the noises if one of these quantities has not the expected sign when ${\cal T}^{(net)}_{1\to 2}$ is positive, which may be a useful piece of information.

It is interesting to draw a comparison with the information about the dynamics of the system that could be extracted from the cross-correlation function $\phi_{12}(t)=\langle X_{1}(0)X_{2}(t)\rangle$. This is indeed a widely used method to infer the directional influence between biological processes DCLME2008 , as will be evoked in the next section. This function is shown in Fig. 4 for several values of the parameter $\rho$.

The two random variables are generally positively correlated, except at short times when the noises are strongly anti-correlated (and for $\rho\gtrsim-0.8$, $\phi_{12}(t)<0$ for all $t>0$). For $\rho\gtrsim-0.3$, the peak at $t>0$ indicates that $X_{2}(t)$ correlates more strongly with the value of $X_{1}$ at an earlier time, suggesting that $X_{1}$ drives $X_{2}$. On the other hand, the presence of a maximum for both $t>0$ and $t<0$ in the interval $-0.3\gtrsim\rho\gtrsim-0.7$ is hard to interpret and could perhaps erroneously suggest the presence of bidirectional coupling. This again illustrates the subtle competition between the correlation in the noises and the actual feedback.

VI Generalization to a non-Markovian process and application to the study of directional influence between cellular processes

We now consider the application of the general formalism for non-bipartite processes presented in Sec. III to a class of non-Markovian processes. We have in mind a situation that is commonly encountered in biological networks in which one is interested in the information exchanges between two random variables, say $X_{1}$ and $X_{2}$ (typically one-dimensional), but other random variables - intrinsic or extrinsic to the system under study - come into play (see, e.g., Fig. 1 in BS2012 ). Within the linear-noise approximation, the network dynamics can be described by a set of chemical Langevin equations where $X_{i}(t)$ is the deviation of the concentration of species $i$ from its mean value (see, e.g., G2000 ; TWW2006 ; HT2014 ; H2016 ). The network dynamics corresponds to a multi-dimensional Markov process, but the (coarse-grained) dynamics of the combined process $(X_{1},X_{2})$ (and not only the dynamics of the individual processes) is non-Markovian. From this coarse-grained perspective, the system therefore corresponds to a process that is non-bipartite and furthermore non-Markovian in general. This is the case on which we focus below.

VI.1 Extension to a bivariate non-Markovian process

For a bivariate non-Markovian process $(X_{1},X_{2})$ obtained as discussed above by coarse-graining a multi-dimensional Markov process over all the extraneous random variables, all the general definitions and relations of information measures introduced in Sec. III directly apply. Furthermore, the Markov character of the underlying network brings a drastic simplification for the derivation of explicit expressions for the information measures, and the latter can be obtained by some form of averaging over the extraneous dynamical variables. Skipping details, and considering only the case of additive noises for simplicity, we find

[TABLE]

[TABLE]

[TABLE]

with ${\bar{J}}_{i,t}(x_{1},x_{2})=\int(\prod_{j\neq 1,2}dx_{j})\>J_{i,t}({\bf x})$ ($i=1,2$), ${\overline{F}}_{2,t}(x_{1},x_{2})=\int(\prod_{i\neq 1,2}dx_{i})\>F_{2,t}({\bf x})P_{t}({\bf x}|x_{1},x_{2})$, and ${\overline{\overline{F}}}_{2,t}(x_{2})=\int dx_{1}\>{\overline{F}}_{2,t}(x_{1},x_{2})P_{t}(x_{1}|x_{2})$, where ${\bf x}$ denotes all the variables of the network, and $F_{i,t}({\bf x})$ and $J_{i,t}({\bf x})$ denote the drift and the probability current for species $i$ in the full multi-dimensional Markov process.

The calculation of the multi-time-step TE rates is more involved and is detailed in Appendix E in the special case of the three-dimensional model considered in Sec. VI.2.

We also do not discuss here the extension of the second law inequalities (95) or (96) as this requires a more extensive and delicate analysis which we defer to future investigations. Indeed, one must first decide which components or processes must be taken into account in the theoretical description, how information is transmitted throughout the network, and identify the sources of stochasticity BS2012 . This is a nontrivial task which is better done on a case by case basis. In particular, the non-bipartite character of the dynamics, i.e., the existence of transitions affecting simultaneously the states of the subsystems, may strongly depend on the level of coarse-graining of the description (see, e.g., LEM2017 for a recent and detailed experimental and theoretical study of a chemical nanomachine).

VI.2 Application to the study of directional influence between cellular processes

In this final section, we apply the previous framework to revisit and complement the study performed in Ref. LNTRL2017 about the information transmission between single cell growth rate and gene expression in the metabolism of E. coli. Understanding how fluctuations in gene expression can affect the growth stability of a cell and, in turn, how the growth noise affects gene expression is an important issue that was initially investigated in Ref. K2014 . The purpose of Ref. LNTRL2017 – which actually prompted our concern for the problem of correlated noises – was to show that transfer entropy is a versatile and model-free tool that can be used to infer directional interactions in biochemical networks, in addition to (or possibly as a substitute for) the standard method based on time-delayed cross-correlation functions DCLME2008 .

A characteristic feature of the biochemical processes studied in K2014 ; LNTRL2017 is the presence of a common extrinsic noise source affecting both the lac enzyme concentration $E(t)$ and the growth rate $\mu(t)$, which makes the stochastic system under study non-bipartite. Specifically, the stochastic model proposed in K2014 to account for the experimental data leads to the following equations within the linear response approximation:

[TABLE]

where $X_{1}(t)\equiv(E(t)-E_{0})/E_{0}$ and $X_{2}(t)\equiv(\mu(t)-\mu_{0})/\mu_{0}$ quantify the deviations of $E(t)$ and $\mu(t)$ from their mean $E_{0}$ and $\mu_{0}$, $N_{l}(t)$ ($l=E,\mu,G$) are three independent Ornstein-Uhlenbeck (OU) noises that are generated by the auxiliary equations $\dot{N}_{l}=-\beta_{l}N_{l}+\xi_{l}$ where the $\xi_{l}$’s are zero-mean Gaussian white noises with amplitudes $\theta_{l}=\eta_{l}\sqrt{2\beta_{l}}$) and $T_{ll^{\prime}}$ are logarithmic gains representing how a variable $l$ responds to the fluctuations of a source $l^{\prime}$ (with $T_{E\mu}=-1$ and $T_{\mu G}=1$. In fact, as shown in LNTRL2017 , the intrinsic noise $N_{E}(t)$ affecting the enzyme concentration may be replaced by a delta-correlated noise with the same intensity $D_{E}=\eta_{E}^{2}/\beta_{E}$ without deteriorating the quality of the fit to the experimental time-dependent correlation functions for $E(t)$ and $\mu(t)$. Then, after some simple manipulations and eliminating the variable $N_{G}$, Eqs. (VI.2) become equivalent to three coupled linear Langevin equations LNTRL2017 ,

[TABLE]

where the third equation describes the dynamics of the OU noise $X_{3}(t)\equiv N_{\mu}(t)$ affecting the growth rate [Eqs (VI.2) correspond to Eqs. (59) in LNTRL2017 where $X_{1},X_{2}$ and $X_{3}$ are denoted $x,y$ and $v$, respectively]. The rate $\mu_{E}=\mu_{0}(1+T_{\mu E}-T_{EE})$ sets the time scale of $E$ fluctuations and the three Gaussian white noises $\xi_{1}\equiv\mu_{0}N_{E},\xi_{2}\equiv\xi_{\mu}+\xi_{G}+\mu_{0}T_{\mu E}N_{E},\xi_{3}\equiv\xi_{\mu}$ have covariances $D_{11}=D_{E}\mu_{0}^{2}$, $D_{22}=\beta_{\mu}\eta_{\mu}^{2}+\beta_{G}\eta_{G}^{2}+D_{E}\mu_{0}^{2}T_{\mu E}^{2}$, $D_{33}=\beta_{\mu}\eta_{\mu}^{2}$, $D_{12}=D_{E}\mu_{0}^{2}T_{\mu E}$, $D_{13}=0$, $D_{23}=\beta_{\mu}\eta_{\mu}^{2}$. The numerical values of all these parameters are given in Table S1 of K2014 .

Eqs. (VI.2) define a Markov process for a set of $3$ interacting random variables, but we are interested in analyzing the information transfer between $X_{1}$ and $X_{2}$ which together form a joint non-Markovian process, as discussed just above. We can thus study the information-theoretic quantities previously introduced. This task was partially accomplished in LNTRL2017 where the single-time-step TE rates $\overline{\cal T}_{1\to 2},\overline{\cal T}_{2\to 1}$ and the learning rates $l_{1}^{+},l_{2}^{+}$ – dubbed information flows – were computed (see note note1 that explains a regrettable confusion in the definition of the learning rates). To complement this analysis, we consider the multi-time-step TE rates ${\cal T}_{1\to 2},{\cal T}_{2\to 1}$ and the backward rates ${\cal T}_{1\to 2}^{\dagger},{\cal T}_{2\to 1}^{\dagger}$ and $\overline{\cal T}_{1\to 2}^{\dagger},{\overline{\cal T}}^{\dagger}_{2\to 1}$. To the best of our knowledge, this is the first time that the backward TE rates are used to infer the direction of information exchanges in a real biochemical network. Whereas $\overline{\cal T}_{1\to 2}^{\dagger}$ and ${\overline{\cal T}}^{\dagger}_{2\to 1}$ are obtained from Eq. (113), the calculation of the multi-time-step TE rates is more involved and is detailed in Appendix E.

The numerical results are presented in Tables 2 and 3 where the inverse of the average growth rate $\mu_{0}$ is taken as the time unit. For comparison, the values of $\overline{\cal T}_{i\to j}$ and $l_{i}^{+}$ computed in LNTRL2017 are also reported in Table 2. As in K2014 ; LNTRL2017 , we consider three different concentrations (low, intermediate, and high) of the inducer IPTG, which allows one to explore different regimes of noise transmission.

Comparing the results for ${\cal T}_{1\to 2}$ and ${\cal T}_{2\to 1}$ in Table 2 to the corresponding results for $\overline{\cal T}_{1\to 2}$ and $\overline{\cal T}_{1\to 2}$ obtained in LNTRL2017 (shown in brackets in the Table), we observe that the numerical values are almost identical. The same is true for the backward rates. In fact, we find numerically that $T_{ij}(h)\approx\overline{T}_{ij}(h)$ for all values of the prediction horizon $h$ (recall that the TE rates are given by the slopes of the finite-horizon curves at the origin). More precisely, $\langle X_{i}(t+h)|X_{1}^{-}(t),X_{2}^{-}(t)\rangle\approx\langle X_{i}(t+h)|X_{1}(t),X_{2}(t)\rangle$ and $\langle X_{i}(t+h)|X_{i}^{-}(t)\rangle\approx\langle X_{i}(t+h)|X_{i}(t)\rangle$ separately, which means that the joint process $(X_{1},X_{2})$ and also the marginal processes $X_{1}$ and $X_{2}$ can be reformulated (in the stationary regime) as Markov processes to a very good approximation. More details are given in Appendix E. This is an intriguing result that suggests some kind of optimization in the transmission of information. Another indication is the behavior of the single-time-step sensory capacity $\overline{C}_{1}$ defined by Eq. (61) as one varies the transmission coefficient $T_{EG}$ describing the response of lac expression to the common noise $N_{G}(t)$ (see Eqs. (VI.2)). As shown in Fig. 5, $\overline{C}_{1}$ at low and intermediate IPTG concentrations reaches the maximal value $1$ for $T_{EG}\approx 1.5$, which is close to the value $T_{EG}=1.3$ used in K2014 to fit the experimental cross-correlation functions (the fit is actually satisfactory for $T_{EG}$ in the range $1.3-1.5$). On the other hand, the maximum of $\overline{C}_{2}$ occurs for a smaller value of $T_{EG}$, significantly below the acceptable range note14 .

Is this a real feature of the metabolic network? Giving a definite answer to this question is difficult because there are many ingredients in the stochastic description and it is not easy to identify those which are responsible for such a behavior. We leave the discussion of this interesting issue to a future investigation.

Finally, we complement the analysis performed in LNTRL2017 by exploiting our determination of the backward TE rates. As suggested in Refs. HNMN2013 ; V2015 ; W2016 and tested on time series generated by multivariate autoregressive processes, time-reversed Granger causality, or backward TE in the present framework, may lead to a better estimate of the directionally of information flows. As in Section V.2, we thus consider the quantities ${\cal T}^{(net)}_{1\to 2}={\cal T}_{1\to 2}-{\cal T}_{2\to 1}$, ${\cal T}^{{\dagger}(net)}_{1\to 2}={\cal T}_{1\to 2}^{\dagger}-{\cal T}_{2\to 1}^{\dagger}$, and the individual differences ${\cal T}_{1\to 2}-{\cal T}_{1\to 2}^{\dagger}$ and ${\cal T}_{2\to 1}-{\cal T}_{2\to 1}^{\dagger}$. (We could also consider the corresponding quantities built from the single-time-step TE rates but they are almost identical.) The results are presented in Table 3 where we also indicate the values of the symmetric learning rates $l_{1}^{S}=-l_{2}^{S}$.

We first observe that ${\cal T}^{(net)}_{1\to 2}>{\cal T}^{{\dagger}(net)}_{1\to 2}$, ${\cal T}_{1\to 2}-{\cal T}_{1\to 2}^{\dagger}>0$ and $l_{2}^{S}>0$ at low and intermediate ITPG concentrations, which strengthens the conclusion of LNTRL2017 (based solely on the positivity of $\overline{\cal T}^{(net)}_{1\to 2}$) that information flows from $X_{1}$ to $X_{2}$ in these two cases. That lac fluctuations propagate through the metabolic network and perturb growth was also the conclusion of K2014 based on the corresponding time-correlation functions. Note however that ${\cal T}_{2\to 1}-{\cal T}_{2\to 1}^{\dagger}$ is also slightly positive. As discussed in Section V.2, this may result from the competition between the correlation in the noises and the direct interactions between the variables $X_{1}$ and $X_{2}$. We must also take into account that the process described by Eqs. (VI.2) is significanltly more complicated to than the one discussed in Section V.2 because of the presence of bidirectional interactions.

At high IPTG concentration, all scores consistently indicate that there is a backward transmission from $X_{2}$ to $X_{1}$, i.e., from growth to expression, which is again in line with the conclusions of K2014 and LNTRL2017 : see Table 1 in LNTRL2017 where ${\overline{\cal T}}^{(net)}_{1\to 2}$ is computed directly from the experimental time series collected in K2014 . Of course, one has ${\cal T}^{(net)}_{1\to 2}<0$ and ${\cal T}_{1\to 2}-{\cal T}_{1\to 2}^{\dagger}<0$ in this case simply because $X_{1}$ no longer influences $X_{2}$ and thus ${\cal T}_{1\to 2}=0$. Indeed, the transmission coefficient $T_{\mu E}$ describing the response of $\mu$ to $E$ fluctuations is taken equal to [math] in the stochastic model (see Table S1 in K2014 ). On the other hand, a less trivial observation is that the difference ${\cal T}_{2\to 1}-{\cal T}_{2\to 1}^{\dagger}$ is significantly more positive than at low and intermediate ITPG concentrations. It would certainly be useful in future investigations to also estimate the backward TE rates directly from the experimental time series.

In passing, note that the numerical results for ${\cal T}_{1\to 2}-{\cal T}_{1\to 2}^{\dagger}$, $-({\cal T}_{2\to 1}-{\cal T}_{2\to 1}^{\dagger})$ and $l_{2}^{S}=-l_{1}^{S}$ are equal in the last column of Table 3. One could think that this directly results from the fact that (i) the white noises $\xi_{1}$ and $\xi_{2}$ are mutually independent in this case ($D_{12}=0$) and (ii) the noise $N_{\mu}\equiv X_{3}$ no longer affects $X_{2}$ (as $\beta_{\mu}=\beta_{G}$ in the model of Ref. K2014 ). The reason turns out to be less straightforward. Indeed, the colored noises $\xi_{1}^{eff}$ and $\xi_{2}^{eff}$ affecting $X_{1}$ and $X_{2}$ are still correlated because $X_{3}$ affects $X_{1}$ and $D_{23}\neq 0$ [see Eqs. (E169) and (E)]. But in the end, when the $3$ coupled Langevin equations (VI.2) are replaced by the equivalent Langevin representation for $X_{1}$ and $X_{2}$ given by Eqs. (E), one discovers that in this case the joint process $(X_{1},X_{2})$ is both bipartite and quasi-Markov. Accordingly, one has ${\cal T}_{1\to 2}-{\cal T}_{1\to 2}^{\dagger}\approx l_{2}^{S}$ and ${\cal T}_{2\to 1}-{\cal T}_{2\to 1}^{\dagger}\approx l_{1}^{S}=-l_{2}^{S}$.

VII Summary

In this paper, we have considered information and thermodynamic exchanges between two coupled stochastic systems that do not satisfy the bipartite property. This should correspond to the generic situation in biochemical networks, in which noises can be correlated (for diffusion processes) or transitions of the two systems can simultaneously take place (for jump processes). The generalization of information quantities, such as learning rates and transfer entropy rates, to the non-bipartite situation is non-trivial and involves introducing several additional rates. We have described how this can be achieved in the framework of continuous-time Markov processes. We have also derived several inequalities that are valid beyond the bipartite assumption and generalized a classical relationship between mutual information and causal estimation error. We have illustrated our general formalism on the case of Markov diffusion processes and obtained a new formulation of the second law of information thermodynamics in which the learning rate is replaced by a difference of transfer entropy rates in the forward and backward time directions. Explicit analytical expressions of all information measures have been derived for the special case of a bivariate Ornstein-Uhlenbeck process, allowing a discussion of the influence of the correlation between the noises on the sign and/or the relative magnitude of the various information measures. Finally, we have applied our formalism to the analysis of the directional influence between cellular processes in a concrete example, which required considering the case of a non-bipartite non-Markovian process. An intriguing “optimal” transmission of information has been observed, which calls for future investigations. More generally, it remains as an interesting future work to extend the present study to describe the fluctuations of pathwise thermodynamic and information-theoretic quantities.

Acknowledgements

T.S. is supported by JSPS KAKENHI Grant Number JP16H02211.

Appendix A Expression of the learning rates for a pure jump Markov process in discrete space

In this Appendix, we give the expression of the learning rate for non-bipartite Markov pure jump processes. In this case, the Markovian generator takes the form (2) in terms of the transition rates $W_{t}\left(z,z^{\prime}\right)$.

After some algebra, the general Markovian expressions (33) and (33) of the learning rates can be cast in the form

[TABLE]

For a bipartite pure jump process, the bipartite transition-rate relation [below Eq. (10)] allows one to show that the two formulas in Eq. A116 coincide, (see also  HBS2014 ; SS2015 )

[TABLE]

Appendix B Proof of various relations of the main text

1. Proof of Eq. (40) for a bipartite Markov diffusion process

To prove Eq. (40) we start with the first line of Eq. (41) and consider $<\ln[P(Y_{t+h}|X_{0}^{t},Y_{0}^{t})/P(Y_{t+h}|Y_{0}^{t},X_{t+h})]>$ which we rewrite as

[TABLE]

where $x^{\prime}\hskip 0.28453pt_{0}{}^{t}$ (resp. $y^{\prime}\hskip 0.28453pt_{0}{}^{t}$) is a short-hand notation to indicate paths of $X$ (resp. $Y$) between [math] and $t$ and we have used the fact that the joint process is Markovian. For convenience of notation, integration with the measure ${\cal D}[x^{\prime}\hskip 0.28453pt_{0}{}^{t}]$ (resp. ${\cal D}[y^{\prime}\hskip 0.28453pt_{0}{}^{t}]$) also includes the integration over the initial and final states, this latter being noted $x^{\prime}$ (resp. $y^{\prime}$). We can now use the properties of a bipartite process, i.e., the factorization of the infinitesimal Markovian propagator (or transition probability) in Eq. (9) and the decomposition of the Markovian generator in Eq. (10), to write

[TABLE]

The specific problem to be handled in the case of diffusion processes in the appearance of singular delta functions in the numerator and denominator of the argument of the logarithm. This can be conveniently bypassed by considering the Gaussian expressions of the infinitesimal propagators, as done in Eq. (73). So, for instance, up to a ${\cal O}(h^{2})$,

[TABLE]

where there was no need to consider the Gaussian expression for $P(x,t+h|x^{\prime\prime},y^{\prime},t)$ because the associated delta function is integrated over.

In the following we consider the case where $D_{YY,t}({\bf z})$ is independent of $x$. Otherwise, the TE rates from $X\to Y$ are infinite, as shown in Sec. IV.3. One therefore has

[TABLE]

and

[TABLE]

so that

[TABLE]

One can check that the above expression reduces to a ${\cal O}(h^{2})$ when $x=x^{\prime}$ and $y=y^{\prime}$. As a consequence when combining Eqs. (B), (B) and (B), it only remains, up to a ${\cal O}(h^{2}$),

[TABLE]

and it is straightforward to see that the above term linear in $h$ exactly vanishes. One can therefore conclude from Eq. (41) that the equality in Eq. (40) is satisfied. Note that we have considered unidimensional processes $X_{t}$ et $Y_{t}$ for simplicity but the demonstration is easily generalized to multidimensional diffusion processes.

*2. Proof of inequality (59)

To prove inequality (59) we consider the difference

[TABLE]

where we have used the definitions of $l_{X}^{+}(t)$, $\widehat{\cal T}_{X\to Y}(t)$, and the derivative of $I(X_{t}:Y_{0}^{t})$. Since $P(X_{t+h}|X_{t},Y_{t})=P(X_{t+h}|X_{t},Y_{0}^{t})$ if the joint process is Markovian, we can write

[TABLE]

Furthermore, the integral analogue of the log-sum inequality applied to the trajectory $Y_{0}^{t-0^{+}}$ yields

[TABLE]

As a result,

[TABLE]

which leads to inequality (59).

3. Proof of the expression (79) for the single-time-step BTE rates for a Markov diffusion process

For the same reason as ${\overline{\cal T}}_{X\to Y}(t)$, the BTE rate ${\overline{\cal T}}^{\dagger}_{X\to Y}(t)$ is infinite when $D_{YY,t}({\bf z})\neq{\overline{D}_{YY,t}(y)}$. We thus consider in the following the case where $D_{YY,t}({\bf z})={\overline{D}_{YY,t}}(y)=D_{YY,t}(y)$.

Inserting the expression of $F^{*}_{Y,t}({\bf z})$ given in Eq. (IV.2) into Eq. (75), we obtain

[TABLE]

where ${\bar{J}}_{Y,t}(y)={\overline{F}}_{Y,t}(y)P_{t}(y)-\partial_{y}[D_{YY,t}(y)P_{t}(y)]$. After combining Eqs. (75), (78) and (B), we arrive at

[TABLE]

where we have used the fact that $D^{*}_{YY,T-t}(y)=D_{YY,t}(y)$ and $P^{*}_{T-t}({\bf z})=P_{t}({\bf z})$. With a few manipulations and the help of the relation $\int dx\,J_{Y,t}({\bf z})=\bar{J}_{Y,t}(y)$, the above equation can be rewritten as

[TABLE]

which corresponds to Eq. (79) of the main text.

Appendix C Sufficient statistic for a non-bipartite dynamics

In this appendix, we show the consequences of the sufficient statistic condition $P(X_{t}|Y_{0}^{t})=P(X_{t}|Y_{t})$ [(Eq. (19) in the main text] for the various inequalities obtained in Sec. III.4. First, this implies that

[TABLE]

Therefore, ${\overline{T}}_{X\to Y}(t)-T_{X\to Y}(t)=\lim_{h\to 0^{+}}h^{-1}\langle\ln[P(Y_{t+h}|Y_{0}^{t})/P(Y_{t+h}|Y_{t})]\rangle=0$ and inequality (57) is saturated.

Since condition (19) also implies that $P(X_{t+h}|Y_{t+h},Y_{0}^{t})=P(X_{t+h}|Y_{t+h})=P(X_{t+h}|Y_{t+h},Y_{t})$, we have

[TABLE]

which yields from Eq. (B)

[TABLE]

Therefore, the steady-state inequality (58) is also saturated.

Finally, since $P(X_{t+h},Y_{t+h}|Y_{0}^{t})=P(X_{t+h},Y_{t+h}|Y_{t})$, one also has ${\widehat{\cal T}}_{X\to Y}={\widehat{\overline{\cal T}}}_{X\to Y}$ and inequality (56) is saturated (note that the joint process does not need to be Markovian in this case).

Appendix D Analytical expressions for a stationary bi-dimensional Ornstein-Uhlenbeck process

In this appendix, we give the analytical expressions of the learning rates and various TE rates for a stationary bi-dimensional OU process.

1. Learning rates and single-time-step TE rates

These rates are obtained from the general formulas for Markov diffusion processes derived in Sec. IV. To this aim, we use the expression of the stationary probability distribution function

[TABLE]

where ${\boldsymbol{\Sigma}}$ is the covariance matrix whose elements $\sigma_{ij}\equiv\langle X_{i}(0)X_{j}(0)\rangle$ are obtained by solving the Lyapunov equation. This gives

[TABLE]

One can check that $\sigma_{11}$ and $\sigma_{22}$ are positive when the two conditions $a_{11}a_{22}-a_{12}a_{21}>0$ and $D_{11}D_{22}-D_{12}^{2}\geq 0$ are satisfied. Some useful relations among these quantities are obtained by multiplying the stationary Fokker-Planck equation by $x_{1}^{2},x_{2}^{2}$ and $x_{1}x_{2}$, respectively, and integrating over ${\bf x}$. This gives

[TABLE]

From Eq. (67), we then find

[TABLE]

where we have used relation (D137a) to go from the first to the second line. Likewise, the symmetric learning rate $l_{1}^{S}=-l_{2}^{S}$ given by Eq. (68) reads

[TABLE]

The TE rate ${\overline{\cal T}}_{1\to 2}$ is obtained from Eq. (75), using $\overline{F}_{2}(x_{2})=-a_{21}\int dx_{1}\>x_{1}\>p(x_{1}|x_{2})-a_{22}x_{2}=-(a_{21}\sigma_{12}/\sigma_{22}+a_{22})x_{2}$, and thus $F_{2}({\bf x})-\overline{F}_{2}(x_{2})=-a_{21}x_{1}+a_{21}(\sigma_{12}/\sigma_{22})x_{2}$. This yields

[TABLE]

and for $D_{12}=0$ one recovers the expressions already given in the literature HBS2016 ; MS2018 .

The BTE rate ${\overline{\cal T}}^{\dagger}_{1\to 2}$ is obtained by modifying the drifts according to Eq. (IV.2), which amounts to replacing ${\bf A}$ by ${\bf A}^{*}={\boldsymbol{\Sigma}}{\bf A}{\boldsymbol{\Sigma}}^{-1}$ A1982 . The modified drifts are

[TABLE]

with $a_{22}^{*}$ and $a_{12}^{*}$ obtained by interchanging $1$ and $2$. Inserting these expressions into Eq. (D140) and noting that the determinant $|\Sigma|$ of the covariance matrix is invariant under the transformation ${\bf A}\to{\bf A}^{*}$, we obtain after some algebra

[TABLE]

Finally, Eq. (49) yields

[TABLE]

2. Multi-time-step TE rates

We now detail the calculation of the multi-time-step TE rates ${\cal T}_{1\to 2}$ and $\widehat{{\cal T}}_{1\to 2}$. As explained in the main text, this requires to compute the conditional expectations $\langle X_{2}(t+h)|{\bf X}^{-}(t)\rangle$, $\langle X_{2}(t+h)|X_{2}^{-}(t)\rangle$, $\langle X_{1}(t+h)|X_{2}^{-}(t)\rangle$, and the corresponding mean-square errors $\sigma_{22}(h)$, $\sigma_{22,2}(h)$, and $\sigma_{11,2}(h)$, $\sigma_{12,2}(h)$.

a) We first compute $\langle X_{2}(t+h)|{\bf X}^{-}(t)\rangle$ and $\sigma_{22}(h)$, which is is straightforward. Starting from

[TABLE]

we readily get

[TABLE]

since the noises are fixed for $s\leq t$ by Eq. (98) and average to zero in the time interval $[t,t+h]$. Eq. (103) then yields

[TABLE]

Of course, since the joint process is Markovian, ${\bf X}^{-}(t)$ can be replaced by ${\bf X}(t)$ in Eqs. (101) and (103), and the same result could be obtained from the well-known expression of the transition probability of the OU process R1989 ; G2004 .

b) The calculation of $\langle X_{2}(t+h)|X_{2}^{-}(t)\rangle$ and thus $\sigma_{22,2}(h)$ is less straightforward because fixing only the past of the process $X_{2}$ up to time $t$ is not sufficient to fix the past of the noises $\xi_{1}$ and $\xi_{2}$. However, this difficulty is bypassed by determining a causal function $H^{\prime}_{22}(t)$ such that

[TABLE]

where $\xi^{\prime}_{2}(t)$ is another Gaussian white noise with variance $2D_{22}$. Then, following the same reasoning as above, we get

[TABLE]

and in turn

[TABLE]

Since Eq. (D147) implies that $S_{22}(\omega)\equiv\langle X_{2}(\omega)X_{2}(-\omega)\rangle=2D_{22}H^{\prime}_{22}(\omega)H^{\prime}_{22}(-\omega)$, the Fourier transform of $H^{\prime}_{22}(t)$ is simply obtained by identifying the component of $S_{22}(\omega)$ that is analytic in the upper-half plane $\mbox{Im}(\omega)>0$. Specifically,

[TABLE]

where $H_{21}(\omega)=-a_{21}/[(\omega_{+}-i\omega)(\omega_{-}-i\omega)]$ and $H_{22}(\omega)=(a_{11}-i\omega)/[(\omega_{+}-i\omega)(\omega_{-}-i\omega)]$ with $\omega_{\pm}=\frac{1}{2}(a_{11}+a_{22}\pm\sqrt{\Delta})$ and $\Delta=(a_{11}-a_{22})^{2}+4a_{12}a_{21}$ ($\omega_{\pm}$ are the eigenvalues of ${\bf A}$). This yields

[TABLE]

where

[TABLE]

It then comes that

[TABLE]

By construction, $H^{\prime}_{22}(\omega)$ has no poles in the upper-half plane $\mbox{Im}(\omega)>0$ and therefore $H^{\prime}_{22}(t)$ vanishes for $t<0$ [specifically, $H^{\prime}_{22}(t)=(u_{+}e^{-\omega_{+}t}-u_{-}e^{-\omega_{-}t})\Theta(t)$ where $\Theta(t)$ is the Heaviside step function and $u_{\pm}=(\omega_{\pm}-r_{2})/\sqrt{\Delta}$]. Moreover, we have chosen $r_{2}=\sqrt{r_{2}^{2}}>0$ so that $H^{\prime}_{22}(\omega)$ is also zero-free in this region. As stressed in RTM2018 , this condition (which corresponds to the so-called “minimum-phase” condition in the language of control theory AM2008 ; B2005 ) ensures that there is a one-to-one correspondence between the process and the corresponding forcing white noise: Fixing the history of the process $X_{2}$ up to time $t$ is equivalent to fixing the history of the noise $\xi^{\prime}_{2}(t)$ and vice versa.

The TE rate ${\cal T}_{1\to 2}$ is finally obtained by expanding $\sigma_{22}(h)$ and $\sigma_{22,2}(h)$ in powers of $h$. Using $H_{21}(t=0^{+})=0$ and $H_{22}(t=0^{+})=H^{\prime}_{22}(t=0^{+})=1$, we get

[TABLE]

and then

[TABLE]

Since $\dot{H}_{22}(t=0^{+})=-a_{22}$, $\dot{H}_{21}(t=0^{+})=-a_{21}$, and $\dot{H}^{\prime}_{22}(t=0^{+})=r_{2}-(\omega_{+}+\omega_{-})=r_{2}-a_{11}-a_{22}$, we finally arrive at Eq. (105) in the main text. Moreover, inserting the expression of $H^{\prime}_{22}(t)$ into Eq. (D147) and performing a few manipulations, we can transform this equation into the (non-Markovian) Langevin equation (109).

c) Finally, we consider the filtered TE rate $\widehat{{\cal T}}_{1\to 2}$. This requires to calculate the conditional mean $\langle X_{1}(t+h)|X_{2}^{-}(t)\rangle$. To this aim, we use the fact that $\langle X_{1}(t+h)|X_{2}^{-}(t)\rangle$ is the orthogonal projection of $X_{1}(t+h)$ onto the trajectory $X_{2}^{-}(t)$. It is a linear functional of $X_{2}^{-}(t)$,

[TABLE]

where $f(h)$ and $g(t,h)$ are unknown functions to be determined by satisfying the orthogonality condition,

[TABLE]

Note that we have singled out the dependence on the value of $X_{2}$ at time $t$ from the dependence on the trajectory up to to time $t$. Plugging Eq. (D156) into Eq. (D157), we then obtain the integral equation

[TABLE]

where $\phi_{ij}(t)\equiv\langle X_{i}(t^{\prime})X_{j}(t^{\prime}+t)\rangle$. This equation is easily solved by using the fact that the stationary correlation functions $\phi_{ij}(t)$ of an OU process are sums of exponentials G2004 . Specifically, for $t\geq 0$,

[TABLE]

where $\omega_{\pm}$ are the eigenvalues of the matrix ${\bf A}$ defined after Eq. (D150). This suggests to introduce the exponential ansatz $g(t,h)=g(h)e^{-\lambda t}$. Then, by comparing the left-hand and right-hand sides of Eq. (D158), we readily find that $\lambda=r_{2}$ (defined by Eq. (D152)), and the remaining unknown functions $f(h)$ and $g(h)$ are obtained by identifying the coefficients of $e^{-\omega_{+}t}$ and $e^{-\omega_{-}t}$. After some algebra, we get

[TABLE]

In particular, when $h=0$,

[TABLE]

From the expression of $\langle X_{1}(t+h)|X_{2}^{-}(t)\rangle$, we can easily compute the mean-square errors $\sigma_{11,2}(h)$ and $\sigma_{12,2}(h)$, and expanding in powers of $h$ finally leads to Eq. (108) in the main text.

Interestingly, the above calculation has a connection to classical linear filtering and stochastic control theory A2006 ; B2005 ; AM2008 . To cast the model described by the coupled Langevin equations (98) as a linear control problem of the state-space form, we introduce two yet unspecified (but non-zero) parameters $G$ and $K$ such that

[TABLE]

and the variable

[TABLE]

The Langevin equation for $X_{1}(t)$ and $X_{2}(t)$ can then be rewritten as

[TABLE]

One realizes that, provided one defines an “observation variable” $Y(t)\equiv-a_{21}X_{1}(t)+\xi_{2}(t)$, the above set of equations can be interpreted as describing the dynamics of a state variable $X_{1}(t)$ (the “signal process”) and of its linear estimator $\widehat{X}_{1}(t)$ given the trajectory $Y^{-}(t)\equiv\{Y(s)\}_{s\in[-\infty,t]}$ (see also SDNM2014 ). In this representation, $G$ and $K$ denote the control gain and the observer (or Kalman) gain, respectively note80 . The first two terms in the r.h.s. of Eq. (D164b) copy the model of the signal process [including the linear control feedback $-G\widehat{X}_{1}(t)$] whereas the third term, which can also be written as $Y(t)+a_{21}\widehat{X}_{1}(t)$, corrects the model by a linear feedback that drives $\widehat{X}_{1}$ closer to $X_{1}$ at time $t+dt$. The process $e(t)\equiv Y(t)+a_{21}\widehat{X}_{1}(t)$ is the so-called “innovation”, which is the part of the measurement that provides new information about the state of the system KSH2000 .

In the standard filtering problem one looks for the linear estimate ${\widehat{X}}_{1}^{*}(t)$ that minimizes the mean-square error $P(t)=\langle[X_{1}(t)-\widehat{X}_{1}(t)]^{2}\rangle$ at every time $t$. This optimal estimate is known to be ${\widehat{X}}_{1}^{*}(t)=\langle X_{1}(t)|Y^{-}(t)\rangle$, the orthogonal projection of $X_{1}(t)$ onto the trajectory $Y^{-}(t)$. The main insight of the Kalman-Bucy theory is to convert the associated Wiener-Hopf integral equation into a nonlinear differential equation by exploiting the state-space formulation KSH2000 . The specification of the optimal Kalman-Bucy filter thus amounts to computing the gain $K^{*}$ that minimizes the estimation error variance $P(t)$. In the present stationary case, one simply has (cf. Eqs. (III) and (IV) in section 7 in KB1961 with $F(t)=-a_{11},H^{\prime}(t)=-a_{21},G(t)=1,Q(t)=2D_{11},R(t)=2D_{22}$)

[TABLE]

and $P^{*}$ is solution of the algebraic Ricatti equation

[TABLE]

The physical solution is

[TABLE]

which can be shown to also be the solution of the Wiener-Kolmogorov causal filter W1949 ; K1992 ; BS1950 (as the two filters are equivalent in the stationary case). Plugging this expression into the quadratic equation $a_{21}K^{2}+(a_{22}-a_{11})K-a_{12}=0$ obtained by eliminating $G$ from Eqs. (D), one then finds that the optimal filter is obtained when $r_{2}$ satisfies Eq. (110) of the main text, i.e., $r_{2}=r_{2}^{*}=\omega_{\pm}$. This condition of course requires specific relations between the parameters of the original bi-dimensional Ornstein-Uhlenbeck model. When this occurs, the individual dynamics of $X_{2}(t)$ [or, equivalently, $\widehat{X}_{1}(t|Y^{-}(t))$] is then Markovian, which is no coincidence. Indeed, as is well known, the innovation $e(t)$ is a white noise process with the same variance $R$ as the measurement noise when the filter is optimal KSH2000 . This fact underscores the property that the Kalman-Bucy filter is so efficient at extracting information from the measurements that one is just left with a white noise afterwards. The filter state $\widehat{X}_{1}^{*}(t|Y^{-}(t))$ is then a sufficient statistic for the conditional distribution of $X_{1}(t)$. In passing, note that $r_{2}^{*}$ and in turn $K^{*}$ do not depend on the parameter $a_{12}$ that defines the feedback from $2$ to $1$. This illustrates the separation principle in optimal control theory that states that the optimal observer gain obtained by minimizing the estimation error is independent of the cost of the control and therefore is independent of the feedback gain $G$.

3. Entropy production rate

In the steady state, the entropy production rate in system $1$ is equal to the rate of entropy change in the environment, and is computed from Eq (92) where $\widehat{F}_{X,t}$, $D_{XX,t}$, and $J_{XX,t}$ are replaced by $F_{1}({\bf x})=-a_{11}x_{1}-a_{12}x_{2}$, $D_{11}$, and $J_{1}({\bf x})=F_{1}({\bf x})P({\bf x})-D_{11}\partial_{x_{1}}P({\bf x})-D_{12}\partial_{x_{2}}P({\bf x})$, respectively. This yields

[TABLE]

Appendix E Multi-time-step TE rates for the three-component process of Sec. VI.2

In this Appendix, we generalize the calculations of Sec. V to the case of a multi-dimensional OU process. Since the complexity of the calculation rapidly increases with the number of components, we only consider the stochastic model studied in section VI.2 which remains simple because the third Langevin equation describing the dynamics of the OU colored noise is decoupled from the two other equations [see Eqs. (VI.2)]. The matrix ${\bf A}$ in Eq. (98) is now a $3\times 3$ matrix with $a_{31}=a_{32}=0$, and $a_{33}>0$.

The main difference with the bi-dimensional Markov case is that the (coarse-grained) dynamics of the joint process of interest, ${\bf X}_{1,2}\equiv(X_{1},X_{2})$, is no longer Markovian because the effective noises affecting the two sub-processes are colored,

[TABLE]

with covariances

[TABLE]

Therefore, ${\bf X}_{1,2}^{-}(t)$ cannot be replaced by ${\bf X}_{1,2}(t)$ in Eqs. (101) and (103), and Eq. (D145) (or the corresponding equation for $\langle X_{1}(t+h)|{\bf X}_{1,2}^{-}(t)\rangle$) is no longer valid because the noises in the time interval $[t,t+h]$ do not average to zero (as they are correlated with the noises for $s\leq t$, which are fixed). To circumvent the problem, we need again to whiten the noises, which amounts to determining four causal functions $\widetilde{H}_{ij}(t)$ such that

[TABLE]

where $\widetilde{\xi}_{1}(t)$ and $\widetilde{\xi}_{2}(t)$ are Gaussian noises with zero-mean and covariances $\langle\widetilde{\xi}_{i}(t)\widetilde{\xi}_{j}(t^{\prime})\rangle=2D_{ij}\delta(t-t^{\prime})$. Then, Eq. (D146) is just replaced by

[TABLE]

and $\sigma_{11}(h)$ is given by the symmetric equation. By construction, the matrix $\widetilde{\bf H}(\omega)=[\widetilde{H}_{ij}(\omega)]$ must satisfy the equation

[TABLE]

where ${\bf S}^{(2)}(\omega)$ is the $2\times 2$ sub-matrix of ${\bf S}(\omega)$ with elements $S_{ij}(\omega)=\langle X_{i}(\omega)X_{j}(-\omega)\rangle$ for $i,j=1,2$. Specifically,

[TABLE]

where $\omega_{\pm}$ are defined in Appendix D after Eq. (D150), $f_{11}=f_{22}=0$, and $f_{12},g_{11},g_{12},g_{22}$ are complicated functions of the model parameters. Note that the numerator and the denominator of $S_{ij}(\omega)$ are only quadratic polynomials despite the fact that the OU process defined by Eqs. (VI.2) is trivariate. This simplification occurs because $a_{31}=a_{32}=0$ and we also set $\beta_{G}=\beta_{\mu}$ (as the two timescales $1/\beta_{G}$ and $1/\beta_{\mu}$ are taken equal to the measured autocorrelation time of $\mu$ when fitting the model parameters to experiments K2014 ).

In order to solve Eq. (E173), we seek a matrix $\widetilde{\bf H}(\omega)=[\widetilde{H}_{ij}(\omega)]$ in the form

[TABLE]

imposing $\lim_{\omega\to\infty}-i\omega\widetilde{\bf H}(\omega)={\bf 1}$ so that the response functions satisfy $\widetilde{\bf H}(t=0^{+})={\bf 1}$. By matching the coefficients of $\omega^{n}$ in the numerators of $S_{ij}(\omega)$, we can then determine the four unknown quantities $\omega_{ij}$. The corresponding non-linear equations are solved numerically and the correct solution is the one that ensures that the elements of the matrix $\widetilde{{\bf H}}^{-1}(\omega)$ have no poles in the upper-half plane $\mbox{Im}(\omega)>0$ (this is the generalization of the minimum-phase condition introduced in Appendix D).

It remains to calculate $\sigma_{22,2}(h)$ from Eq. (D149), where $H^{\prime}_{22}(t)$ is the response function defined by Eq. (D147) and obtained from the Wiener-Hopf factorization of $S_{22}(\omega)$. This yields

[TABLE]

where $\omega^{\prime}$ is the root of the equation $\omega^{2}+g_{22}$ with a positive real part.

Finally, expanding $\sigma_{22}(h)$ and $\sigma_{22,2}(h)$ in powers of $h$, using $H^{\prime}_{22}(t=0^{+})=\widetilde{H}_{22}(t=0^{+})=1$, $\dot{H}^{\prime}_{22}(t=0^{+})=\omega^{\prime}-(a_{11}+a_{22})$, $\dot{\widetilde{H}}_{22}(t=0^{+})=\omega_{22}-(a_{11}+a_{22})$, and $\dot{\widetilde{H}}_{21}(t=0^{+})=\omega_{21}$, we obtain

[TABLE]

and thus

[TABLE]

${\cal T}_{2\to 1}$ is given by the symmetric formula.

In order to compute the backward TE rates, we again have to replace the $a_{ij}$’s by the elements of the matrix ${\bf A}^{*}={\boldsymbol{\Sigma}}{\bf A}^{T}{\boldsymbol{\Sigma}}^{-1}$. However, this simply amounts to repeating the calculation of the forward TE rates after interchanging the power-spectrum functions $S_{12}(\omega)$ and $S_{21}(\omega)$ [as $S_{12}^{\dagger}(\omega)=S_{21}(\omega)$].

It is useful to illustrate the above equations numerically to see why the marginal processes $X_{1}$ and $X_{2}$ have a quasi-Markovian behavior. Let us for instance consider the case of the low-IPTG experiment. The three coupled Eqs. (VI.2) are then equivalent to the two non-Markovian equations

[TABLE]

with

[TABLE]

and $2D_{11}=0.020$, $2D_{22}=0.059$, and $2D_{12}=0.014$. Although the exponential terms seem to only give small corrections, these noises cannot be approximated by their white components. Indeed, this would significantly change the values of the TE rates and even reverse the directionality of the information flow (${\cal T}_{1\to 2}\approx 2.2\times 10^{-3},{\cal T}_{2\to 1}\approx 8.6\times 10^{-3}$, to be compared with the values given in Table 2). On the other hand, by taking the inverse of the response functions $\widetilde{H}_{ij}(\omega)$, we obtain from Eqs. (E171) the equivalent representation

[TABLE]

with $\langle\widetilde{\xi}_{i}(t)\widetilde{\xi}_{j}(t^{\prime})\rangle=2D_{ij}\delta(t-t^{\prime})$. Moreover, by using Eq. (D147) and the similar equation for $X_{1}$, and taking the inverse of the response functions $H^{\prime}_{ii}(\omega)$, the individual processes $X_{1}$ and $X_{2}$ can be represented by

[TABLE]

with $\langle\xi^{\prime}_{i}(t)\xi^{\prime}_{j}(t^{\prime})\rangle=2D_{ij}\delta(t-t^{\prime})$. Let us recall that, by construction, Eqs. (E) and (E) yield the same statistical properties of the stationary processes $X_{1}$ and $X_{2}$ as the original Eqs. (VI.2). Although these are still non-Markovian equations, neglecting the contributions of the memory kernels is now harmless and leads to exactly the same values of the TE rates as the full equations.

Likewise, for the high-IPTG experiment, the three coupled Eqs. (VI.2) are equivalent to the two equations (similar to Eqs. (E))

[TABLE]

In this case the white noises $\widetilde{\xi}_{1}$ and $\widetilde{\xi}_{2}$ are independent, as $D_{12}=0$.